Stream events to local cache automatically, remove sync hook
Major improvement to the local relay caching strategy - events now stream directly into the cache as they arrive from remote relays, eliminating the need for background syncing. Changes: - NostrProvider: Intercept relay.req() to cache events as they stream in - Remove useEventSync hook (no longer needed) - useAuthor: Don't throw/retry when profile not found - Documentation: Updated to reflect streaming cache strategy How it works: - Every remote relay's req() method is wrapped to intercept events - Each event is cached to IndexedDB as it arrives (fire and forget) - Cache builds organically as you browse the app - Posts cached when viewing feed - Profiles cached when viewing author info - Replies cached when opening threads Benefits: - Profiles load instantly on refresh (already cached from previous views) - No background polling/syncing overhead - Cache always reflects what you've actually seen - Zero configuration required Co-authored-by: shakespeare.diy <assistant@shakespeare.diy>
This commit is contained in:
shakespeare.diy
+33
-28
@@ -14,10 +14,11 @@ The local relay is built using three main components:
|
||||
|
||||
### Automatic Caching
|
||||
|
||||
Events are automatically cached to IndexedDB in two ways:
|
||||
Events are automatically cached to IndexedDB as they stream in from the network:
|
||||
|
||||
1. **Query Results** - When you query relays, the local relay is included and returns cached events instantly
|
||||
1. **Incoming Events** - All events received from remote relays are automatically cached as they arrive
|
||||
2. **Published Events** - When you publish events, they're automatically stored locally via the `eventRouter`
|
||||
3. **No Syncing Required** - The cache builds naturally as you use the app, no background syncing needed
|
||||
|
||||
### Query Flow
|
||||
|
||||
@@ -25,38 +26,37 @@ When you query events with `nostr.query()`:
|
||||
|
||||
1. The query is routed to the local relay (`local://indexeddb`) AND remote relays
|
||||
2. Local relay responds immediately with cached events from IndexedDB
|
||||
3. Remote relays respond with fresh events from the network
|
||||
4. Results are deduplicated by event ID
|
||||
5. New events from remote relays are automatically cached locally for future queries
|
||||
3. Remote relays stream fresh events from the network
|
||||
4. **Each event from remote relays is automatically cached as it arrives**
|
||||
5. Results are deduplicated by event ID
|
||||
|
||||
This means you get:
|
||||
- **Instant results** from the local cache
|
||||
- **Fresh data** from remote relays
|
||||
- **Offline access** to previously cached events
|
||||
- **Automatic caching** of all events you encounter
|
||||
- **Offline access** to previously viewed content
|
||||
|
||||
### Event Syncing
|
||||
### Streaming Cache Updates
|
||||
|
||||
You can use the `useEventSync` hook to automatically sync events to the local cache:
|
||||
The NostrProvider intercepts the event stream from each relay and caches events in real-time:
|
||||
|
||||
```typescript
|
||||
import { useEventSync } from '@/hooks/useEventSync';
|
||||
|
||||
function MyComponent() {
|
||||
// Sync posts from users you follow
|
||||
useEventSync({
|
||||
filters: [
|
||||
{ kinds: [1], authors: followingPubkeys, limit: 100 }
|
||||
],
|
||||
interval: 30000, // Sync every 30 seconds
|
||||
onNewEvents: (count) => {
|
||||
console.log(`Synced ${count} new events`);
|
||||
}
|
||||
});
|
||||
|
||||
// ...rest of component
|
||||
}
|
||||
// In NostrProvider - automatically happens for every query
|
||||
relay.req = async function* (filters, opts) {
|
||||
for await (const event of originalReq(filters, opts)) {
|
||||
// Cache each event as it streams in (fire and forget)
|
||||
eventStore.addEvent(event, [url]);
|
||||
yield event;
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
This means:
|
||||
- Posts are cached when you view the feed
|
||||
- Profiles are cached when you view author info
|
||||
- Replies are cached when you open a thread
|
||||
- No manual syncing required - the cache builds as you browse
|
||||
|
||||
## Settings UI
|
||||
|
||||
The local cache is displayed in **Settings > Relays** with:
|
||||
@@ -165,11 +165,16 @@ for await (const event of localRelay.req([{ kinds: [1] }])) {
|
||||
|
||||
## Comparison to mi
|
||||
|
||||
This implementation is inspired by the local event storage system in the `mi` project, with the following differences:
|
||||
This implementation is inspired by the local event storage system in the `mi` project, with the following key differences:
|
||||
|
||||
1. **Automatic integration** - The local relay is transparently integrated into the NPool, so all queries automatically include it
|
||||
2. **Relay interface** - Implements a full relay interface rather than just a storage layer
|
||||
3. **Composite indexes** - Includes a `kind_pubkey` composite index for more efficient queries
|
||||
4. **UI integration** - Displays cache status and management in the relay settings
|
||||
3. **Streaming cache** - Events are cached in real-time as they stream in from relays, not via polling/syncing
|
||||
4. **Composite indexes** - Includes a `kind_pubkey` composite index for more efficient queries
|
||||
5. **UI integration** - Displays cache status and management in the relay settings
|
||||
|
||||
The key advantage is that you don't need to explicitly query the event store - it's automatically queried as part of every `nostr.query()` call, providing instant results from the cache while simultaneously fetching fresh data from remote relays.
|
||||
The key advantages:
|
||||
- **Zero configuration** - You don't need to explicitly query the event store or set up syncing
|
||||
- **Real-time caching** - Events are cached as they arrive, building the cache organically as you use the app
|
||||
- **Instant loads on refresh** - Profiles and posts load instantly from cache while fresh data streams in from relays
|
||||
- **No background syncing** - No polling intervals or sync hooks needed, everything happens naturally
|
||||
|
||||
@@ -53,7 +53,22 @@ const NostrProvider: React.FC<NostrProviderProps> = (props) => {
|
||||
if (url === 'local://indexeddb') {
|
||||
return localRelay as unknown as NRelay1;
|
||||
}
|
||||
return new NRelay1(url);
|
||||
|
||||
const relay = new NRelay1(url);
|
||||
|
||||
// Intercept events as they stream in from remote relays and cache them
|
||||
const originalReq = relay.req.bind(relay);
|
||||
relay.req = async function* (filters: NostrFilter[], opts?: { signal?: AbortSignal }) {
|
||||
for await (const event of originalReq(filters, opts)) {
|
||||
// Cache event from this relay (fire and forget)
|
||||
eventStore.addEvent(event, [url]).catch(error => {
|
||||
console.debug('[NostrProvider] Failed to cache event from relay:', error);
|
||||
});
|
||||
yield event;
|
||||
}
|
||||
};
|
||||
|
||||
return relay;
|
||||
},
|
||||
reqRouter(filters: NostrFilter[]) {
|
||||
const routes = new Map<string, NostrFilter[]>();
|
||||
|
||||
@@ -18,7 +18,8 @@ export function useAuthor(pubkey: string | undefined) {
|
||||
);
|
||||
|
||||
if (!event) {
|
||||
throw new Error('No event found');
|
||||
// Return empty object instead of throwing - profile doesn't exist or isn't cached yet
|
||||
return {};
|
||||
}
|
||||
|
||||
try {
|
||||
@@ -29,6 +30,6 @@ export function useAuthor(pubkey: string | undefined) {
|
||||
}
|
||||
},
|
||||
staleTime: 5 * 60 * 1000, // Keep cached data fresh for 5 minutes
|
||||
retry: 3,
|
||||
retry: false, // Don't retry - if profile isn't found, it just doesn't exist
|
||||
});
|
||||
}
|
||||
|
||||
@@ -1,118 +0,0 @@
|
||||
import { useEffect, useRef, useCallback } from 'react';
|
||||
import { useNostr } from '@nostrify/react';
|
||||
import { useAppContext } from '@/hooks/useAppContext';
|
||||
import { eventStore } from '@/lib/eventStore';
|
||||
import type { NostrEvent, NostrFilter } from '@nostrify/nostrify';
|
||||
|
||||
interface UseEventSyncOptions {
|
||||
/** Filters to sync events for */
|
||||
filters?: NostrFilter[];
|
||||
/** Polling interval in milliseconds (default: 30 seconds) */
|
||||
interval?: number;
|
||||
/** Whether to enable syncing (default: true) */
|
||||
enabled?: boolean;
|
||||
/** Callback when new events are synced */
|
||||
onNewEvents?: (count: number) => void;
|
||||
}
|
||||
|
||||
/**
|
||||
* Hook to automatically sync events from relays to local IndexedDB storage.
|
||||
* Polls relays at regular intervals to fetch new events matching the provided filters.
|
||||
*/
|
||||
export function useEventSync(options: UseEventSyncOptions = {}) {
|
||||
const {
|
||||
filters,
|
||||
interval = 30000, // 30 seconds
|
||||
enabled = true,
|
||||
onNewEvents,
|
||||
} = options;
|
||||
|
||||
const { nostr } = useNostr();
|
||||
const { config } = useAppContext();
|
||||
const lastSyncRef = useRef<number>(Math.floor(Date.now() / 1000));
|
||||
const intervalIdRef = useRef<number | null>(null);
|
||||
|
||||
const syncEvents = useCallback(async (skipCallback = false) => {
|
||||
if (!filters || filters.length === 0) {
|
||||
return;
|
||||
}
|
||||
|
||||
try {
|
||||
const relayUrls = config.relayMetadata.relays.filter(r => r.read).map(r => r.url);
|
||||
|
||||
if (relayUrls.length === 0) {
|
||||
console.debug('[EventSync] No read relays configured, skipping sync');
|
||||
return;
|
||||
}
|
||||
|
||||
// Add 'since' to all filters to only fetch new events
|
||||
const since = lastSyncRef.current;
|
||||
const sinceFilters = filters.map(filter => ({
|
||||
...filter,
|
||||
since: Math.floor(since),
|
||||
limit: filter.limit || 100,
|
||||
}));
|
||||
|
||||
const signal = AbortSignal.timeout(10000); // 10 second timeout
|
||||
|
||||
const events = await nostr.query(sinceFilters, { signal });
|
||||
|
||||
if (events.length > 0) {
|
||||
// Store all events with relay information
|
||||
await eventStore.addEvents(events, relayUrls);
|
||||
|
||||
// Update last sync timestamp to the most recent event
|
||||
const newestEvent = events.reduce((newest, current) =>
|
||||
current.created_at > newest.created_at ? current : newest
|
||||
);
|
||||
lastSyncRef.current = newestEvent.created_at + 1; // +1 to avoid duplicates
|
||||
|
||||
console.debug(`[EventSync] Synced ${events.length} new events`);
|
||||
|
||||
// Only trigger callback if not skipped (skip on initial mount)
|
||||
if (!skipCallback) {
|
||||
onNewEvents?.(events.length);
|
||||
}
|
||||
}
|
||||
|
||||
} catch (error) {
|
||||
console.debug('[EventSync] Sync error:', error);
|
||||
}
|
||||
}, [filters, nostr, config, onNewEvents]);
|
||||
|
||||
useEffect(() => {
|
||||
if (!enabled || !filters || filters.length === 0) {
|
||||
return;
|
||||
}
|
||||
|
||||
// Initialize the event store
|
||||
eventStore.init().catch(error => {
|
||||
console.error('[EventSync] Failed to initialize event store:', error);
|
||||
});
|
||||
|
||||
// Initial sync - skip callback to prevent UI flicker
|
||||
syncEvents(true).catch((error) => {
|
||||
console.debug('[EventSync] Initial sync error:', error);
|
||||
});
|
||||
|
||||
// Set up polling interval
|
||||
intervalIdRef.current = window.setInterval(() => {
|
||||
syncEvents(false).catch((error) => {
|
||||
console.debug('[EventSync] Interval sync error:', error);
|
||||
});
|
||||
}, interval);
|
||||
|
||||
return () => {
|
||||
if (intervalIdRef.current !== null) {
|
||||
window.clearInterval(intervalIdRef.current);
|
||||
}
|
||||
};
|
||||
}, [enabled, filters, interval, syncEvents]);
|
||||
|
||||
return {
|
||||
/** Trigger a manual sync */
|
||||
sync: useCallback(async () => {
|
||||
await syncEvents(false);
|
||||
}, [syncEvents]),
|
||||
};
|
||||
}
|
||||
Reference in New Issue
Block a user