Core

@topgunbuild/core ships the CRDT primitives, Hybrid Logical Clock, and Merkle tree that the client and server share. This page documents the public surface (the “what / how”). For background on why TopGun picks these primitives over alternatives, see Concepts → Data Structures.

The two primary data structures are LWWMap (last-write-wins) and ORMap (observed-remove). Both are exposed on TopGunClient via getMap() and getORMap() — most application code never instantiates them directly.

LWWMap

import { LWWMap, HLC } from '@topgunbuild/core';

const hlc = new HLC('node-1');
const map = new LWWMap<string, User>(hlc);

Last-write-wins semantics: each key holds exactly one value. Concurrent writes from different nodes are resolved by HLC timestamp (highest wins, ties broken by nodeId).

Constructor

new LWWMap<K, V>(hlc: HLC)

Methods

set(key, value, ttlMs?) — LWWRecord<V>

Write a value. Stamps the record with hlc.now(). Pass ttlMs to make the record expire on its own: see Expiring records.

const record = map.set('u1', { name: 'Alice', email: 'alice@example.com' });
// { value: {...}, timestamp: { millis, counter, nodeId } }

// With 1-hour TTL
map.set('session-123', sessionData, 3600000);

get(key) — V | undefined

Read a value. Returns undefined for deleted or expired keys.

getRecord(key) — LWWRecord<V> | undefined

Read the full record (value, timestamp, optional TTL).

remove(key) — LWWRecord<V>

Delete a key by writing a tombstone (value: null). Returns the tombstone record.

merge(key, remoteRecord) — boolean

Merge a remote record. Returns true if local state was updated (i.e., the remote record’s timestamp dominated).

const remote: LWWRecord<User> = {
  value: { name: 'Bob' },
  timestamp: { millis: Date.now(), counter: 0, nodeId: 'node-2' },
};
const updated = map.merge('u1', remote);

subscribe(callback) — () => void

Subscribe to map-change notifications. The callback receives entries: Array<[K, V]>. Returns an unsubscribe function.

const unsubscribe = map.subscribe((entries) => {
  console.log('Map changed,', entries.length, 'live entries');
});

entries() — IterableIterator<[K, V]>

Iterate non-tombstoned entries (drops deleted and expired).

allKeys() — IterableIterator<K>

Iterate every key including tombstones.

size (getter) — number

Count of entries including tombstones.

prune(olderThan) — K[]

Drop tombstones older than the given Timestamp. Returns the removed keys.

pruneExpired(skip?) — K[]

Drop every live record whose TTL has run out (timestamp.millis + ttlMs is before now) from the map and its Merkle tree, and return the removed keys. Keys for which skip(key) returns true are kept (still hidden by get). Like prune, it is local housekeeping: it writes no tombstone and sends nothing. TopGunClient runs it for you (see Expiring records); call it yourself only when you use LWWMap without the client.

clear() — void

Reset to empty state.

getMerkleTree() — MerkleTree

Access the underlying Merkle tree for delta-sync diffing.

LWWRecord shape

interface LWWRecord<V> {
  value: V | null;          // null indicates tombstone
  timestamp: Timestamp;     // HLC stamp
  ttlMs?: number;           // optional TTL
}

ORMap

import { ORMap, HLC } from '@topgunbuild/core';

const hlc = new HLC('node-1');
const map = new ORMap<string, string>(hlc);

Observed-remove semantics: each key holds a set of values. Concurrent adds preserve every value (tagged with unique IDs). A remove operation tombstones only the tags it observed locally, so concurrent add-vs-remove resolves correctly (no spurious deletes).

Use this for tag clouds, comment threads, multi-select fields, presence rosters — anywhere two clients can legitimately contribute different values to the same key.

Constructor

new ORMap<K, V>(hlc: HLC)

Methods

add(key, value, ttlMs?) — ORMapRecord<V>

Add a value under a key. Generates a unique tag. Returns the record. Pass ttlMs to make this value expire on its own: see Expiring records.

const tags = new ORMap<string, string>(hlc);
tags.add('post:123', 'javascript');
tags.add('post:123', 'typescript');
// post:123 -> ['javascript', 'typescript']

remove(key, value) — string[]

Remove a specific value under a key. Returns the tombstone tags created.

get(key) — V[]

Read all live values for a key.

const postTags = tags.get('post:123');
// ['javascript', 'typescript']

getRecords(key) — ORMapRecord<V>[]

Read full records (value + tag + timestamp) for a key.

getTombstones() — string[]

Read all current tombstone tags.

apply(key, record) — boolean

Apply a remote ORMapRecord to local state. Returns true if state changed.

applyTombstone(tag, key?) — void

Apply a remote tombstone tag. The tag is suppressed under every key either way. Pass the key the removal belongs to whenever you know it: the tombstone is then also recorded against that key, which is what lets a removal that emptied the key still be compared with the server during sync. Without key the tombstone is only recorded map-wide.

import { ORMap, HLC } from '@topgunbuild/core';

const remote = new ORMap<string, string>(new HLC('node-2'));
const record = remote.add('post:123', 'draft');

// Replaying a removal that happened elsewhere:
remote.applyTombstone(record.tag, 'post:123');
remote.get('post:123'); // []
remote.getKeyTombstones('post:123'); // Set { record.tag }

getKeyTombstones(key) — Set<string>

Read the tombstone tags recorded against one key. Returns a copy; every tag in it is also in getTombstones(). remove(key, value) records its tombstones against key automatically.

setKeyTombstones(key, tags) — void

Replace the tombstone tags recorded against one key — used to mirror the set an authoritative peer reports for that key. Any record still carrying one of the tags is removed. A tag dropped from the key’s set stays in getTombstones(): the map-wide set only grows here. Passing no tags clears the key’s set.

merge(other) — void

Merge another ORMap of the same key/value types in bulk.

subscribe(callback) — () => void

Subscribe to change events. The callback receives entries: Array<[K, V[]]>.

allKeys() — K[]

All keys that currently hold at least one record. A key left with only tombstones is not listed.

size / totalRecords (getters) — number

size counts distinct keys; totalRecords counts every tagged record (including duplicates per key).

prune(olderThan) — string[]

Drop tombstones older than the timestamp. Returns dropped tags.

clear() — void

Reset to empty.

getMerkleTree() — ORMapMerkleTree

Underlying Merkle tree (a different shape than LWWMap’s tree because ORMap stores multi-value-per-key). A key contributes to it while it holds at least one record or has at least one tombstone recorded against it, so a key emptied by a removal still shows up in a sync comparison.

getSnapshot() — ORMapSnapshot<K, V>

The map’s internal state: items (key → tag → record), tombstones (every removed tag) and keyTombstones (key → the tombstone tags recorded against it). All three fields are required, so code that builds an ORMapSnapshot by hand must supply keyTombstones — an empty Map when there are none.

isTombstoned(tag) — boolean

Check whether a tag is in the tombstone set.

ORMapRecord shape

interface ORMapRecord<V> {
  value: V;
  tag: string;            // unique add ID
  timestamp: Timestamp;   // HLC stamp
  ttlMs?: number;
}

Expiring records

A record written with ttlMs disappears by itself: for sessions, invitations, one-time codes, presence and anything else that should not outlive a deadline. map.set(key, value, ttlMs) and orMap.add(key, value, ttlMs) both take it.

  • When it expires. The TTL counts from the moment the server stores the write, not from when the client made it. Once that moment plus ttlMs has passed, the server and every client hide the record: it is left out of query snapshots, HTTP /sync, search and get. A live query that already shows it drops it when the server deletes it (next point).
  • Then it is deleted. The server deletes the expired record the ordinary way, so live-query subscribers see it leave, the event journal records a delete with reason: "expired", and every client drops it from its local map (an LWWMap by itself, an ORMap when it receives the server’s delete). The delete can come up to the server’s sweep interval (TOPGUN_EXPIRY_SWEEP_INTERVAL_MS, 10 s by default) plus one sweep cycle after the record expired; until then it is already hidden.
  • Clocks. Each reader compares the deadline with its own clock, so readers whose clocks differ can disagree about an expiring record for as long as their clocks differ.
  • Offline writes. A client hides its own entries that the server has not stored yet (unsynced, or written offline) by its local write time, which is earlier than the moment the server stores them. So such an entry can disappear on that client earlier than on the server and on other clients, by up to the time the client was offline. This affects only that client’s own view, and it ends when the client takes the server’s record (LWWMap) or receives the server’s delete (ORMap).
  • Older records. Records written before servers kept the TTL do not expire.

TopGunClient drops expired LWWMap records from memory and from local storage once a second while at least one LWWMap is open, also while offline, and before it compares a map with the server, so its sync state matches the server’s after the server deleted them. A record that still has an unsynced local write is kept until the server has it.

HLC (Hybrid Logical Clock)

import { HLC } from '@topgunbuild/core';

const hlc = new HLC('node-1');
const ts = hlc.now();
// { millis: 1748025600000, counter: 0, nodeId: 'node-1' }

The HLC combines physical wall-clock time (millis) with a logical counter (counter) so total ordering is preserved across nodes even when clocks drift.

Constructor

new HLC(nodeId: string, options?: HLCOptions)

nodeId names this clock and becomes part of every timestamp it issues. It must not contain :, | or # — the constructor throws an Error if it does. Letters, digits, - and _ are always fine, and so is a UUID.

HLCOptions fields:

FieldDefaultDescription
strictModefalseThrow on clock drift exceeding maxDriftMs.
maxDriftMs60000Permitted drift threshold (ms).
clockSourcewall clockCustom ClockSource for deterministic tests.

Methods

now() — Timestamp

Issue a fresh timestamp. Monotonic per node.

update(remote) — void

Update the local clock against a received remote timestamp. Bumps lastMillis/lastCounter as needed to maintain causal ordering.

getClockSource() — ClockSource

Returns the configured clock source.

getNodeId / getStrictMode / getMaxDriftMs (getters)

Inspect configured fields.

Static helpers

HLC.compare(a, b) — number

Total-order comparator. Returns negative if a < b, zero if equal, positive if a > b. Ordered by millis, then counter, then nodeId.

HLC.toString(ts) — string

Canonical string encoding for serialization or sort keys.

HLC.parse(str) — Timestamp

Inverse of toString.

HLC.assertValidNodeId(nodeId) — void

Check a node id without creating a clock. Throws the same Error as the constructor if nodeId contains :, | or #, and returns nothing otherwise. Useful for validating an id that comes from configuration or user input before you build anything with it.

Timestamp shape

interface Timestamp {
  millis: number;
  counter: number;
  nodeId: string;
}

MerkleTree

import { MerkleTree } from '@topgunbuild/core';

const tree = new MerkleTree();

The Merkle tree powers delta sync: clients and servers compare hash trees and exchange only differing buckets instead of full state. LWWMap and ORMap maintain their own trees internally — use this directly only when implementing custom sync providers or diagnostics.

Constructor

new MerkleTree(records?: Map<string, LWWRecord<any>>, depth = 3)

depth controls fan-out (default 3 levels deep).

Methods

update(key, record) — void

Insert or replace the leaf for key and rehash the path to the root.

remove(key) — void

Drop the leaf and rehash.

getRootHash() — number

Top-level hash. Two trees with identical content produce the same root.

getNode(path) — MerkleNode | undefined

Return the node at a given path string (used during diff walks).

getBuckets(path) — Record<string, number>

Per-bucket hashes for the children of the node at path.

getKeysInBucket(path) — string[]

Leaf keys grouped under the given path.

Re-exports

@topgunbuild/core also re-exports types and helpers consumed across the stack. Highlights:

  • Predicates — composable filter builders (equal, gt, lt, contains, and, or, not, match, matchPhrase, matchPrefix). Used by client.query, client.hybridQuery, and the React hooks.
  • SearchOptions — shared FTS options shape used by client.search/searchSubscribe and useSearch.
  • MergeRejection — event shape emitted when the built-in CRDT merge logic rejects a remote change. Observed via useMergeRejections (React) or client.getConflictResolvers().onRejection().
  • WriteConcern / ConsistencyLevel — durability and consistency knobs for cluster writes.
  • IndexedLWWMap / IndexedORMap — indexed variants used by the server’s query engine.
  • Full-Text Search — Tokenizer, InvertedIndex, BM25Scorer, FullTextIndex for embedding FTS directly.
  • Deterministic Simulation Testing — VirtualClock, SeededRNG, VirtualNetwork, InvariantChecker, ScenarioRunner for property-based distributed tests.

The complete re-export list lives in packages/core/src/index.ts.


← Client · React →