Operations
Everything except media blob content, thumbnails, and encryption keys is stored as operations. They form a growing set of immutable CRDT-style units of modification on the library. For example: renaming a photo, creating an album, adding a member. Library state is reconstructed deterministically from the full list of operations.
Operation Group Envelope
Operations are grouped in an operation group. It also contains a unique id, the user who created it, and the parent operation group for ordering.
Contrary to all other UUIDs in the format, operation group IDs are UUIDv7, so when sorting concurrent groups they are tie-broken by date.
Ordering and Reconstruction
State is reconstructed by performing a topological sort of the op group DAG (following parent_op_id links), then replaying operations in that order.
When multiple groups share the same parent, the tie-break is ascending op_id. Because op_id is UUIDv7, this gives "earliest clock wins" semantics on concurrent writes. Every client independently computes the same order, making reconstruction deterministic.
Operation List
Each Operation carries its own timestamp (time of write). This is not used in the logic.
Media
Albums
Groups
Groups are leaf collections. They hold media items directly, cannot have child albums, and each belongs to exactly one parent album set at creation.
File Format
Every operation file is serialized as CBOR and encrypted before being written to the remote. Each op file is immutable. Each file is named by a UUID with a tier-dependent extension:
- A compaction file (
.op{N}_{count}) holds many op groups. Its name is a fresh random UUID assigned to the file itself,Nis the tier, andcountis the total number of operations inside, summed across every op group in the file — readable from the filename without decrypting the file.
Compaction
Even if the operation group set as a whole is growing, and each group immutable, the way they are stored in remotes can sometimes change. We do this to bound the total number of operation files, and the number of files we have to touch to add new groups. To do this we use a Log-Structured Merge-trees (LSM) inspired algorithm.
Local vs Remote
Compaction never happens locally. The two sides store ops differently:
- Local keeps a single append file,
local_state/operations.log, that accumulates op groups as you make changes. It is never compacted. There is also apending.opfile that accumulates groups waiting to be pushed, until it is flushed or grows too large. - Remote stores ops as tiered files and compacts them in the background. The advantage of this is that all compacted states are independent for all remotes, so any change to the procedure can be implemented without problems.
Compaction File Format
Compaction is built on op group identity: every group has a unique op_id, and a compaction file is just the full op groups it covers, each tagged by its op_id.
Compaction Tiers
Files are organized into tiers. Tier 0 holds raw op groups; higher tiers hold compaction files that pack more and more groups together.
Two limits govern the files at each tier: how many operations a single file at that tier may hold (summed across every op group packed into it), and how many files may exist at that tier before compaction is triggered. Because each compaction file encodes its total operation count in its filename, clients can evaluate both limits from the directory listing alone.
| Tier | File extension | Ops limit | Tier files limit |
|---|---|---|---|
| 1 | .op1_{count} | 20 | 10 |
| 2 | .op2_{count} | 200 | 10 |
| N | .opN_{count} | 20×10^(N-1) | 10 |
With this scheme, the maximum number of operations storable with tier i or less:
| Max tier | 1 | 2 | 3 | 4 | 5 | 6 | 7 |
|---|---|---|---|---|---|---|---|
| Max total ops | 200 | 2,200 | 22,200 | 222,200 | 2,222,200 | 22,222,200 | 222,222,200 |
| Max compacted op files | 10 | 20 | 30 | 40 | 50 | 60 | 70 |
Compaction Procedure
When any tier has more files than its file limit, the client responsible for this must compact it. The goal is to completely evict that tier. All its files are merged into a single file at the next tier up, leaving no underpopulated file behind. For now this is done greedily in arbitrary order.
- The client acquires a global operations lock file on the remote, preventing concurrent compactions.
- It reads all files at tier N and merges their op-group lists into a single new compaction file at tier N+1. "All files at tier N" means all tier-N files recorded in the pushing client's own last known state for that remote (see Sync), not a fresh listing of the remote. A client never lists or reads remote op files outside of what it already knows about to decide what to compact.
- The tier-N source files are deleted, still under the same lock.
- If tier N+1 now exceeds its file limit, the procedure is applied again at that tier, cascading upward until no tier is over its limit, all while continuing to hold the same lock acquired in step 1.
- Once no tier is over its limit, the lock is released.
Uploading Operations
When pushing, the client writes new operation groups directly as a single compaction file, at the lowest tier whose ops limit can hold the whole batch's total operation count. A push totaling 200 operations, for example, lands directly at tier 2 rather than as an oversized tier-1 file, regardless of how many op groups those operations are split across.
Local Log Format
operations.log is an append-only binary file that lists all the operation groups. It is a sequence of frames like this:
op_id 16 bytes op group UUID
blob_len 4 bytes unsigned 32-bit integer
blob blob_len CBOR-serialized OperationGroup, encrypted
Rolling out our own format is questionnable, but it's the simplest way I found to write all the operations in append mode without rewriting everything each time.