Skip to content

Commit 5510c8a

Browse files
mysamimiclaude
andcommitted
feat: add memory-authoritative write-behind mode for Redis
Adds an optional "memory-authoritative" mode where Redis is used purely as an async write-behind layer, keeping the hot path fully in-memory after warmup. All changes are backward-compatible (defaults preserve existing read-through behavior). - Single round-trip Redis reads via optional RedisGetTTLClient interface (implemented by v8/v9 adapters with a GET+TTL pipeline); falls back to separate Get+TTL otherwise. - WithWriteThroughOnly(): disables synchronous Redis reads on local miss for Get/getFallback/ModifyNumeric/SetNX; writes still flow async. - WarmFromRedis(keys): loads state from Redis into memory at startup for durability across restarts without runtime synchronous reads. - Local-authoritative SetNX in write-through-only mode: atomicity via the local shard lock, async SETNX to Redis; default behavior preserved. - Fix silent drop-on-full: centralize async sends through enqueueRedis, count drops via DroppedWrites(), and add WithBlockOnFull(d) to block briefly instead of dropping. - Tests for all new behavior (miniredis-backed, no external Redis needed). - README updated with new features and clarified read-through semantics. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 71e73e8 commit 5510c8a

7 files changed

Lines changed: 658 additions & 34 deletions

File tree

README.md

Lines changed: 76 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,13 +10,17 @@ go-cache is an in-memory key:value store/cache similar to memcached that is suit
1010

1111
* **Sharding**: Reduces lock contention for high-concurrency workloads.
1212
* **Redis Integration**: Optional L2 caching and persistence layer using `go-redis` (supports v8 and v9).
13+
* **Memory-Authoritative Mode**: Optional write-behind-only mode where Redis is *never* read synchronously on the hot path — after warmup, reads are fully in-memory while writes still persist asynchronously.
14+
* **Single Round-Trip Reads**: Redis reads fetch the value and its TTL in one round-trip (pipeline / `GETEX`) when the adapter supports it, instead of a separate `GET` + `TTL`.
15+
* **Startup Warmup**: `WarmFromRedis` rebuilds in-memory state from Redis at boot, so durability survives restarts without any synchronous reads at runtime.
1316
* **Capacity Management**: Internal LRU-like eviction when memory limits are reached.
1417
* **Generics**: Type-safe API (Go 1.18+).
15-
* **SetNX**: Atomic "Set if Not Exists" operation, seamlessly synchronized with Redis.
18+
* **SetNX**: Atomic "Set if Not Exists" operation, either synchronized with Redis (default) or local-authoritative with async persistence.
1619
* **Numeric Operations**: Atomic increment/decrement support for numeric types, persisted to Redis.
1720
* **Set Cache**: Track unique members per key, each with its own TTL — ideal for counting active sessions/devices per user.
1821
* **Graceful Shutdown**: Ensures pending Redis operations are completed before exit.
1922
* **Sync**: Force refresh items from Redis.
23+
* **Back-Pressure Control**: Observe dropped async writes via `DroppedWrites`, and optionally block briefly instead of dropping when the write-behind queue is full.
2024
* **Configurable Timeouts**: Fine-tune Redis L2 operation timeouts for all cache types.
2125
* **Performance**: Extremely low latency local operations (see [BENCHMARKS.md](BENCHMARKS.md)).
2226

@@ -28,6 +32,11 @@ go-cache is an in-memory key:value store/cache similar to memcached that is suit
2832

2933
## Recent Updates
3034

35+
* **Memory-Authoritative / Write-Behind-Only Mode**: `WithWriteThroughOnly()` keeps the hot path (`Get`/`ModifyNumeric`/`SetNX`) fully in-memory — Redis is used purely as an async write-behind layer and is never read synchronously on a local miss.
36+
* **Single Round-Trip Redis Reads**: Added the optional `RedisGetTTLClient` interface (implemented by the bundled v8/v9 adapters via a pipeline). When available, a Redis read fetches value + TTL in one round-trip instead of separate `GET` and `TTL` calls.
37+
* **Startup Warmup**: `WarmFromRedis(keys)` loads existing Redis state into local memory at startup, for durability across restarts without runtime synchronous reads.
38+
* **Local-Authoritative SetNX**: In write-through-only mode, `SetNX` guarantees atomicity via the local shard lock and pushes the Redis `SETNX` asynchronously. The original cluster-wide synchronous behavior remains the default.
39+
* **Back-Pressure Metrics & Blocking**: Async write drops (when the queue is full) are now counted and exposed via `DroppedWrites()`. `WithBlockOnFull(d)` blocks briefly instead of dropping immediately.
3140
* **SetNX Support**: Added atomic `SetNX` (Set if Not Exists) operations seamlessly synchronized with Redis.
3241
* **Automatic Redis Fetching**: Enabled automatic Redis fetching for local cache misses and expired items, enhancing multi-instance synchronization.
3342
* **Configurable Timeouts**: Fine-tune Redis L2 operation timeouts for all cache types.
@@ -121,6 +130,72 @@ c.WithRedisTimeout(500 * time.Millisecond)
121130
val, err := c.Sync("key")
122131
```
123132

133+
#### How reads work (default: read-through)
134+
135+
By default the cache is **read-through**: a `Get` first checks local memory, and on a local miss (or a locally-expired item) it transparently falls back to Redis. If the key exists in Redis, the value is pulled back into local memory and returned. This keeps multiple instances loosely synchronized — a value written by one worker becomes visible to others on their next miss.
136+
137+
The Redis read fetches both the value and its remaining TTL. When the configured adapter implements `RedisGetTTLClient` (the bundled `redisv8` / `redisv9` adapters do, via a pipeline), this is a **single round-trip**; otherwise it falls back to a separate `GET` + `TTL`.
138+
139+
---
140+
141+
### Memory-Authoritative Mode (Write-Behind Only)
142+
143+
For latency-critical, high-throughput workloads you can make memory the source of truth and demote Redis to a pure **async write-behind** layer. In this mode the hot path is *never* blocked by a synchronous Redis read.
144+
145+
Enable it with `WithWriteThroughOnly()` (available on `Cache`, `NumericCache`, `ShardedCache`, and `ShardedNumericCache`):
146+
147+
```go
148+
c := cache.NewShardedCache[MyStruct](16, 5*time.Minute, 10*time.Minute)
149+
c.WithRedis(redisv9.New(rdb)).
150+
WithRedisTimeout(500 * time.Millisecond).
151+
WithWriteThroughOnly()
152+
defer c.Close()
153+
```
154+
155+
What changes in this mode:
156+
157+
* **Reads stay in memory.** `Get`, `ModifyNumeric`, `SetNX`, and the `SetCache` getters no longer fall back to Redis on a local miss — a miss is a miss. After warmup, the hot path is 100% in-memory.
158+
* **Writes still persist.** `Set`, `Delete`, `Incr`/`Decr`, etc. are still propagated to Redis asynchronously through the write-behind worker.
159+
* **`SetNX` becomes local-authoritative.** Atomicity is guaranteed by the local shard lock (exactly one winner per process), and the Redis `SETNX` is sent asynchronously. The signature and return value are unchanged. *Note:* cluster-wide uniqueness is no longer enforced synchronously — use the default mode if you need that guarantee.
160+
* **Explicit reads still hit Redis.** `Sync(key)` and `WarmFromRedis(keys)` deliberately bypass this mode, since they are not on the hot path.
161+
162+
#### Warming up from Redis at startup
163+
164+
To get durability across restarts *without* paying for synchronous reads at runtime, load your state from Redis once at boot:
165+
166+
```go
167+
// Rebuild in-memory state for known keys (e.g. after a restart).
168+
loaded, err := c.WarmFromRedis([]string{"user:1", "user:2", "user:3"})
169+
fmt.Printf("warmed %d keys from Redis\n", loaded)
170+
```
171+
172+
Keys missing from Redis are skipped silently; `loaded` is the number of keys actually restored. For `ShardedCache`, keys are automatically routed to the correct shard. An error is returned only if Redis is not configured.
173+
174+
---
175+
176+
### Back-Pressure: Dropped Writes & Blocking
177+
178+
Async Redis writes flow through a buffered queue. If writes are produced faster than the Redis worker can drain them, the queue fills up. By default an overflowing write is **dropped** (to avoid stalling the application), but it is now **counted** rather than silently lost:
179+
180+
```go
181+
// Number of async Redis writes dropped because the queue was full.
182+
// On ShardedCache this is summed across all shards.
183+
dropped := c.DroppedWrites()
184+
if dropped > 0 {
185+
log.Printf("WARNING: %d cache writes were dropped before reaching Redis", dropped)
186+
}
187+
```
188+
189+
If losing a write is unacceptable (for example, an item about to be evicted from memory), configure the producer to **block briefly** instead of dropping:
190+
191+
```go
192+
// When the queue is full, block for up to 50ms waiting for room.
193+
// If still full after the timeout, the write is dropped and counted.
194+
c.WithBlockOnFull(50 * time.Millisecond)
195+
```
196+
197+
> **Trade-off:** blocking applies back-pressure to the calling goroutine (and, for `Set`/`Delete`, briefly while holding the shard lock), so keep the timeout short. A zero or negative duration restores the default drop-immediately behavior.
198+
124199
---
125200

126201
### Capacity & Eviction

0 commit comments

Comments
 (0)