Sharding

For datasets too large for one database, ShardRouter partitions data across several shard databases by a shard key (e.g. user_id, region). Each row lives on exactly one shard; a query routes to the shard that owns its key. Unlike read replicas, shards hold disjoint data and scale writes as well as reads.

ShardRouter is a ClientProvider, so it drops into quark.For[T] like any client — the rest of the ORM is unaware of sharding.

shards := map[string]*quark.Client{
    "shard-a": clientA, // each is a normal quark.New(...) client
    "shard-b": clientB,
}
router, err := quark.NewShardRouter(
    shards,
    quark.DefaultShardResolver,            // reads the key set by WithShardKey
    quark.HashShardFunc([]string{"shard-a", "shard-b"}), // FNV-1a hash → shard
)

// The caller supplies the shard key (a string) per operation, via the context.
shardCtx := quark.WithShardKey(ctx, user.Region)        // shard by region
_ = quark.For[User](shardCtx, router).Create(&user)     // → the shard owning user.Region
got, _ := quark.For[User](shardCtx, router).Where("id", "=", user.ID).List()

A complete, self-contained runnable example (two SQLite shards, no Docker) lives in examples/sharding/ — run it with go run ./examples/sharding/main.go. It routes accounts by shard key, proves the data is disjoint per shard, and shows the keyless-query rejection.

Routing writes by the entity (ShardKeyer)

When a model owns its shard key, implement ShardKeyer so the key-deriving logic lives on the model — not repeated at every call site — and route writes with WithShardKeyOf:

type User struct {
    ID       int64  `db:"id" pk:"true"`
    TenantID string `db:"tenant_id"`
    // ...
}

// The model declares which field its partition is keyed on.
func (u User) ShardKey() string { return u.TenantID }

// WithShardKeyOf reads it — no need to repeat user.TenantID at the call site.
ctx = quark.WithShardKeyOf(ctx, user)
_ = quark.For[User](ctx, router).Create(&user) // → the shard owning user.TenantID

WithShardKeyOf(ctx, entity) is exactly WithShardKey(ctx, entity.ShardKey()). It applies to writes, where you hold the entity; reads carry no entity, so they keep passing the key to WithShardKey directly. An entity whose ShardKey() returns the empty string fails to route — like a missing key, never a silent fan-out.

How routing works

1. DefaultShardResolver reads the shard key from the context (WithShardKey); ShardKeyFromContext exposes that value if you need it. You can supply your own ShardResolver to read an existing request value.
2. The ShardFunc maps that key to a shard name. HashShardFunc is the stable hash-mod default; supply your own for range, geo, or lookup-table policies — it is the seam you control for resharding.
3. The query runs on that shard's *Client, unchanged.

A query without a shard key in context errors — there is no implicit cross-shard fan-out (forgetting the key fails loudly rather than silently querying every shard).

Scatter-gather (cross-shard reads)

To read across shards — explicitly, never as a fallback for a forgotten key — use ScatterGather. It runs the same query on every shard concurrently and merges the rows. Supply a comparator and a global limit via ScatterMerge for a correct global top-N:

top, err := quark.ScatterGather(ctx, router,
    func(q *quark.Query[Account]) *quark.Query[Account] {
        return q.OrderBy("balance", "DESC").Limit(10) // each shard's top 10
    },
    quark.ScatterMerge[Account]{
        Less:  func(a, b Account) bool { return a.Balance > b.Balance },
        Limit: 10, // global top 10 across all shards
    },
)

ScatterCount sums per-shard counts (exact, since shards are disjoint):

n, err := quark.ScatterCount[Account](ctx, router, nil) // total across all shards
// => n is the exact total — shards are disjoint, so per-shard counts just sum

If any shard errors, the call returns that error rather than a silently partial result. Aggregates beyond count (AVG/MIN/MAX) and cross-shard GROUP BY are not merged — run those per shard. Merging them correctly needs more than the per-shard result rows (an average needs each shard's count as well as its mean, for instance), so Quark returns nothing rather than a plausible-looking wrong number.

Limits

• No cross-shard joins — a JOIN only sees the resolved shard.
• No cross-shard transactions — a Tx is bound to one shard's client; there is no two-phase commit. Design the model so each operation stays within a shard (choose the shard key well; denormalize where needed).
• Scatter-gather is an explicit opt-in — cross-shard reads happen only through ScatterGather/ScatterCount (above), never as the fallback of a forgotten key. Aggregates beyond count and cross-shard GROUP BY are not merged.
• Resharding (changing the ShardFunc + migrating data) is an operator task; the shard set is fixed at construction.

Multi-tenancy composes orthogonally: a shard's *Client can itself sit behind a TenantRouter. Sharding is a router in front of ordinary clients rather than a mode inside the client, which is what lets the two stack without either knowing about the other.