back

Partial replication sounds easy—just sync the data your app needs, right? But choosing an approach is tricky: logical replication precisely tracks every change, complicating strong consistency, while physical replication avoids that complexity but requires syncing every change, even discarded ones. What if your app could combine the simplicity of physical replication with the efficiency of logical replication? That’s the key idea behind Graft, the open-source transactional storage engine I’m launching today. It’s designed specifically for lazy, partial replication with strong consistency, horizontal scalability, and object storage durability.

Graft is designed with the following use cases in mind:

• Offline-first & mobile apps: Simplify development and improve reliability by offloading replication and storage to Graft.
• Cross-platform sync: Share data smoothly across devices, browsers, and platforms without vendor lock-in.
• Stateless multi-writer replicas: Deploy replicas anywhere, including serverless and embedded environments.
• Any data type: Replicate databases, files, or custom formats—all with strong consistency.

I first discovered the need for Graft while building SQLSync. SQLSync is a frontend optimized database stack built on top of SQLite with a synchronization engine powered by ideas from Git and distributed systems. SQLSync makes multiplayer SQLite databases a reality, powering interactive apps that run directly in your browser.

However, SQLSync replicates the entire log of changes to every client—similar to how some databases implement physical replication. While this approach works fine on servers, it’s poorly suited to the constraints of edge and browser environments.

After shipping SQLSync, I decided to find a replication solution more suited to the edge. I needed something that could:

• Let clients sync at their own pace
• Sync only what they need
• Sync from anywhere, including the edge and offline devices
• Replicate arbitrary data¹
• All while providing strong consistency guarantees.

That didn’t exist. So I built it.

#A different approach to edge replication

If you’ve ever tried to keep data in sync across clients and servers, you know it’s harder than it sounds. Most existing solutions fall into one of two camps:

• Full replication, which syncs the entire dataset to each client—not practical for constrained environments like serverless functions or web apps.
• Schema-aware diffs, like Change Data Capture (CDC) or Conflict-free Replicated Data Types (CRDTs), which track logical changes at the row or field level—but require deep application integration and don’t generalize to arbitrary data.

Graft takes a different path.

Like full replication, Graft is schema-agnostic. It doesn’t know or care what kind of data you’re storing—it just replicates bytes². But instead of sending all the data, it behaves more like logical replication: clients receive a compact description of what’s changed since their last sync.

At the core of this model is the Volume: a sparse, ordered collection of fixed-size Pages. Clients interact with Volumes through a transactional API, reading and writing at specific Snapshots. Under the hood, Graft persists and replicates only what’s necessary—using object storage as a durable, scalable backend.

The result is a system that’s lazy, partial, edge-capable, and consistent.

Want to try the managed version of Graft?

Join the waitlist to get early access: Sign up here →

Each of these properties deserves a closer look—let’s unpack them one by one.

#Lazy: Sync at your own pace

Graft is designed for the real world—where edge clients wake up occasionally, face unreliable networks, and run in short-lived, resource-constrained environments. Instead of relying on continuous replication, clients choose when to sync, and Graft makes it easy to fast forward to the latest snapshot.

That sync starts with a simple question: what changed since my last snapshot?

The server responds with a graft—a compact bitset of page indexes that have changed across all commits since that snapshot³. This is where the project gets its name: a graft attaches new changes to an existing snapshot—like grafting a branch onto a tree. They act as a guide, informing the
client which pages can be reused and which need to be fetched if needed.

Critically, when a client pulls a graft from the server, it doesn’t receive any actual data—only metadata about what changed. This gives the client full control over what to fetch and when, laying the foundation for partial replication.

#Partial: Sync only what’s needed

When you’re building for edge environments—browser tabs, mobile apps, serverless functions—you can’t afford to download the entire dataset just to serve a handful of queries. That’s where partial replication comes in.

After a client pulls a graft, it knows exactly what’s changed. It can use that information to determine precisely which pages are still valid and which pages need to be fetched. Instead of pulling everything, clients selectively retrieve only the pages they’ll actually use—nothing more, nothing less.

To keep things snappy, Graft supports several ways to prefetch pages:

1. General-purpose prefetching: Graft includes a built-in prefetcher based on the Leap algorithm, which predicts future page accesses by identifying patterns⁴.
2. Domain-specific prefetching: Applications can leverage domain knowledge to preemptively fetch relevant pages. For instance, if your app frequently queries a user’s profile, Graft can prefetch pages related to that profile before the data is needed.
3. Proactive fetching: Clients can always fall back to pulling all changes if needed, essentially reverting to full replication. This is particularly useful for Graft workloads running on the server side.

And because Graft hosts pages directly on object storage, they’re naturally durable and scalable, creating a strong foundation for edge-native replication.

#Edge: Sync close to the action

Edge replication isn’t just about choosing what data to sync—it’s about making sure that data is available where it’s actually needed. Graft does this in two key ways.

First, pages are served from object-storage through a global fleet of edge servers, allowing frequently accessed (“hot”) pages to be cached near clients. This keeps latency low and responsiveness high, no matter where in the world your users happen to be.

Second, the Graft client itself is lightweight and designed specifically to be embedded. With minimal dependencies and a tiny runtime, it integrates into constrained environments like browsers, devices, mobile apps, and serverless functions.

The result? Your data is always cached exactly where it’s most valuable—right at the edge and embedded in your application.

But caching data on the edge brings new challenges, particularly around maintaining consistency and safely handling conflicts. That’s where Graft’s robust consistency model comes in.

#Consistency: Sync safely

Strong consistency is critical—especially when syncing data between clients that might occasionally conflict. Graft addresses this by providing a clear and robust consistency model: Serializable Snapshot Isolation.⁵

This model gives clients isolated, consistent views of data at specific snapshots, allowing reads to proceed concurrently without interference. At the same time, it ensures that writes are strictly serialized, so there’s always a clear, globally consistent order for every transaction.

However, because Graft is designed for offline-first, lazy replication, clients sometimes attempt to commit changes based on an outdated snapshot. Accepting these commits blindly would violate strict serializability. Instead, Graft safely rejects the commit and lets the client choose how to resolve the situation. Typically, clients will:

1. Reset and replay, by pulling the latest snapshot, reapplying local transactions, and trying again.
   • Globally, the data remains strictly serializable.
   • Locally, the