You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
feat: add memory-authoritative write-behind mode for Redis
Adds an optional "memory-authoritative" mode where Redis is used purely as
an async write-behind layer, keeping the hot path fully in-memory after
warmup. All changes are backward-compatible (defaults preserve existing
read-through behavior).
- Single round-trip Redis reads via optional RedisGetTTLClient interface
(implemented by v8/v9 adapters with a GET+TTL pipeline); falls back to
separate Get+TTL otherwise.
- WithWriteThroughOnly(): disables synchronous Redis reads on local miss
for Get/getFallback/ModifyNumeric/SetNX; writes still flow async.
- WarmFromRedis(keys): loads state from Redis into memory at startup for
durability across restarts without runtime synchronous reads.
- Local-authoritative SetNX in write-through-only mode: atomicity via the
local shard lock, async SETNX to Redis; default behavior preserved.
- Fix silent drop-on-full: centralize async sends through enqueueRedis,
count drops via DroppedWrites(), and add WithBlockOnFull(d) to block
briefly instead of dropping.
- Tests for all new behavior (miniredis-backed, no external Redis needed).
- README updated with new features and clarified read-through semantics.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Copy file name to clipboardExpand all lines: README.md
+76-1Lines changed: 76 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -10,13 +10,17 @@ go-cache is an in-memory key:value store/cache similar to memcached that is suit
10
10
11
11
***Sharding**: Reduces lock contention for high-concurrency workloads.
12
12
***Redis Integration**: Optional L2 caching and persistence layer using `go-redis` (supports v8 and v9).
13
+
***Memory-Authoritative Mode**: Optional write-behind-only mode where Redis is *never* read synchronously on the hot path — after warmup, reads are fully in-memory while writes still persist asynchronously.
14
+
***Single Round-Trip Reads**: Redis reads fetch the value and its TTL in one round-trip (pipeline / `GETEX`) when the adapter supports it, instead of a separate `GET` + `TTL`.
15
+
***Startup Warmup**: `WarmFromRedis` rebuilds in-memory state from Redis at boot, so durability survives restarts without any synchronous reads at runtime.
13
16
***Capacity Management**: Internal LRU-like eviction when memory limits are reached.
14
17
***Generics**: Type-safe API (Go 1.18+).
15
-
***SetNX**: Atomic "Set if Not Exists" operation, seamlessly synchronized with Redis.
18
+
***SetNX**: Atomic "Set if Not Exists" operation, either synchronized with Redis (default) or local-authoritative with async persistence.
16
19
***Numeric Operations**: Atomic increment/decrement support for numeric types, persisted to Redis.
17
20
***Set Cache**: Track unique members per key, each with its own TTL — ideal for counting active sessions/devices per user.
18
21
***Graceful Shutdown**: Ensures pending Redis operations are completed before exit.
19
22
***Sync**: Force refresh items from Redis.
23
+
***Back-Pressure Control**: Observe dropped async writes via `DroppedWrites`, and optionally block briefly instead of dropping when the write-behind queue is full.
20
24
***Configurable Timeouts**: Fine-tune Redis L2 operation timeouts for all cache types.
21
25
***Performance**: Extremely low latency local operations (see [BENCHMARKS.md](BENCHMARKS.md)).
22
26
@@ -28,6 +32,11 @@ go-cache is an in-memory key:value store/cache similar to memcached that is suit
28
32
29
33
## Recent Updates
30
34
35
+
***Memory-Authoritative / Write-Behind-Only Mode**: `WithWriteThroughOnly()` keeps the hot path (`Get`/`ModifyNumeric`/`SetNX`) fully in-memory — Redis is used purely as an async write-behind layer and is never read synchronously on a local miss.
36
+
***Single Round-Trip Redis Reads**: Added the optional `RedisGetTTLClient` interface (implemented by the bundled v8/v9 adapters via a pipeline). When available, a Redis read fetches value + TTL in one round-trip instead of separate `GET` and `TTL` calls.
37
+
***Startup Warmup**: `WarmFromRedis(keys)` loads existing Redis state into local memory at startup, for durability across restarts without runtime synchronous reads.
38
+
***Local-Authoritative SetNX**: In write-through-only mode, `SetNX` guarantees atomicity via the local shard lock and pushes the Redis `SETNX` asynchronously. The original cluster-wide synchronous behavior remains the default.
39
+
***Back-Pressure Metrics & Blocking**: Async write drops (when the queue is full) are now counted and exposed via `DroppedWrites()`. `WithBlockOnFull(d)` blocks briefly instead of dropping immediately.
31
40
***SetNX Support**: Added atomic `SetNX` (Set if Not Exists) operations seamlessly synchronized with Redis.
32
41
***Automatic Redis Fetching**: Enabled automatic Redis fetching for local cache misses and expired items, enhancing multi-instance synchronization.
33
42
***Configurable Timeouts**: Fine-tune Redis L2 operation timeouts for all cache types.
By default the cache is **read-through**: a `Get` first checks local memory, and on a local miss (or a locally-expired item) it transparently falls back to Redis. If the key exists in Redis, the value is pulled back into local memory and returned. This keeps multiple instances loosely synchronized — a value written by one worker becomes visible to others on their next miss.
136
+
137
+
The Redis read fetches both the value and its remaining TTL. When the configured adapter implements `RedisGetTTLClient` (the bundled `redisv8` / `redisv9` adapters do, via a pipeline), this is a **single round-trip**; otherwise it falls back to a separate `GET` + `TTL`.
138
+
139
+
---
140
+
141
+
### Memory-Authoritative Mode (Write-Behind Only)
142
+
143
+
For latency-critical, high-throughput workloads you can make memory the source of truth and demote Redis to a pure **async write-behind** layer. In this mode the hot path is *never* blocked by a synchronous Redis read.
144
+
145
+
Enable it with `WithWriteThroughOnly()` (available on `Cache`, `NumericCache`, `ShardedCache`, and `ShardedNumericCache`):
***Reads stay in memory.**`Get`, `ModifyNumeric`, `SetNX`, and the `SetCache` getters no longer fall back to Redis on a local miss — a miss is a miss. After warmup, the hot path is 100% in-memory.
158
+
***Writes still persist.**`Set`, `Delete`, `Incr`/`Decr`, etc. are still propagated to Redis asynchronously through the write-behind worker.
159
+
***`SetNX` becomes local-authoritative.** Atomicity is guaranteed by the local shard lock (exactly one winner per process), and the Redis `SETNX` is sent asynchronously. The signature and return value are unchanged. *Note:* cluster-wide uniqueness is no longer enforced synchronously — use the default mode if you need that guarantee.
160
+
***Explicit reads still hit Redis.**`Sync(key)` and `WarmFromRedis(keys)` deliberately bypass this mode, since they are not on the hot path.
161
+
162
+
#### Warming up from Redis at startup
163
+
164
+
To get durability across restarts *without* paying for synchronous reads at runtime, load your state from Redis once at boot:
165
+
166
+
```go
167
+
// Rebuild in-memory state for known keys (e.g. after a restart).
Keys missing from Redis are skipped silently; `loaded` is the number of keys actually restored. For `ShardedCache`, keys are automatically routed to the correct shard. An error is returned only if Redis is not configured.
173
+
174
+
---
175
+
176
+
### Back-Pressure: Dropped Writes & Blocking
177
+
178
+
Async Redis writes flow through a buffered queue. If writes are produced faster than the Redis worker can drain them, the queue fills up. By default an overflowing write is **dropped** (to avoid stalling the application), but it is now **counted** rather than silently lost:
179
+
180
+
```go
181
+
// Number of async Redis writes dropped because the queue was full.
182
+
// On ShardedCache this is summed across all shards.
183
+
dropped:= c.DroppedWrites()
184
+
if dropped > 0 {
185
+
log.Printf("WARNING: %d cache writes were dropped before reaching Redis", dropped)
186
+
}
187
+
```
188
+
189
+
If losing a write is unacceptable (for example, an item about to be evicted from memory), configure the producer to **block briefly** instead of dropping:
190
+
191
+
```go
192
+
// When the queue is full, block for up to 50ms waiting for room.
193
+
// If still full after the timeout, the write is dropped and counted.
194
+
c.WithBlockOnFull(50 * time.Millisecond)
195
+
```
196
+
197
+
> **Trade-off:** blocking applies back-pressure to the calling goroutine (and, for `Set`/`Delete`, briefly while holding the shard lock), so keep the timeout short. A zero or negative duration restores the default drop-immediately behavior.
0 commit comments