Skip to content

Commit db716dd

Browse files
milosmnsmilosmns
authored
Release Candidate v5.19.3 (#396)
2 parents 73298cf + ecf7624 commit db716dd

39 files changed

Lines changed: 1129 additions & 571 deletions

‎docs/open-api-docs.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@ openapi: 3.0.3
22
info:
33
title: The Agent's user-facing API
44
description: The user-facing parts of The Agent's API service (excluding system-level endpoints, chat completion, maintenance endpoints, etc.)
5-
version: 5.19.2
5+
version: 5.19.3
66
license:
77
name: MIT
88
url: https://opensource.org/licenses/MIT

‎openspec/changes/archive/2026-06-21-migrate-tools-cache-to-repository/.openspec.yaml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
schema: spec-driven
2+
created: 2026-06-20

‎openspec/changes/archive/2026-06-21-migrate-tools-cache-to-repository/design.md‎

Lines changed: 175 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,175 @@
1+
## Context
2+
3+
`tools_cache` still uses the legacy `db/schema` + `db/crud` persistence pattern. `ToolsCacheCRUD` accepts `ToolsCacheSave`, returns `ToolsCacheDB`, and consumers convert rows with `ToolsCache.model_validate(...)`. The legacy schema also contains cache-entry behavior (`is_expired`) and uses `created_at: datetime = datetime.now()`, which evaluates once when the module is imported rather than when each entry is created.
4+
5+
The cache is shared infrastructure used by chat attachment processing, currency exchange-rate fetching, web fetching, HTML cleanup, Twitter fetching, and scheduled cleanup. Cache entries are internal full-state objects rather than partial external snapshots:
6+
7+
```
8+
caller
9+
│
10+
▼
11+
repository ─────▶ ToolsCacheDB
12+
│ ▲
13+
▼ │
14+
domain dataclass ◀──── mapper
15+
```
16+
17+
```
18+
key
19+
required deterministic identity, never DB-generated
20+
21+
value
22+
required full replacement value
23+
24+
created_at
25+
required timestamp, defaulted per domain instance
26+
27+
expires_at
28+
nullable expiration state
29+
None means "never expires"
30+
```
31+
32+
## Goals / Non-Goals
33+
34+
**Goals:**
35+
36+
- Introduce a tools cache domain dataclass, mapper, and repository matching newer persistence units.
37+
- Keep `ToolsCacheDB` as the only SQLAlchemy model for the existing table.
38+
- Preserve deterministic cache keys, cache hit/miss behavior, explicit expiration checks, full upsert behavior, deletion, and expired-entry cleanup.
39+
- Correct `created_at` defaulting so each new domain object receives its own creation timestamp.
40+
- Add domain, mapper, and repository tests beside legacy tests, then migrate production callers in reviewable feature groups.
41+
- Remove legacy CRUD/schema access only after production and test references are gone.
42+
43+
**Non-Goals:**
44+
45+
- No database migration or schema change.
46+
- No external API or OpenAPI change.
47+
- No cache backend replacement, distributed cache, in-memory cache, eviction policy, or serialization redesign.
48+
- No automatic deletion of an expired entry during `get`; callers retain the current explicit expiration check behavior.
49+
- No clock abstraction or timezone migration in this change.
50+
51+
## Decisions
52+
53+
### 1. Keep One SQLAlchemy Model
54+
55+
`ToolsCacheDB` remains the only DB model and table definition for `tools_cache`. The new model is a feature-level dataclass mapped to and from `ToolsCacheDB`.
56+
57+
**Rationale**: The table already has the required shape. A second SQLAlchemy representation would duplicate ownership and create migration risk without improving the domain boundary.
58+
59+
**Alternative considered**: Create a parallel SQLAlchemy model. Rejected because it adds competing metadata for the same table.
60+
61+
### 2. Use One Full-State Domain Model
62+
63+
The domain model should represent both new and persisted entries:
64+
65+
```
66+
ToolsCache(
67+
key: str,
68+
value: str,
69+
created_at: datetime = field(default_factory=datetime.now),
70+
expires_at: datetime | None = None,
71+
)
72+
```
73+
74+
It retains `is_expired()` behavior. `expires_at = None` is meaningful state and means the entry never expires.
75+
76+
**Rationale**: Cache writes are internal and always contain the complete entry state. A separate save/draft model would reproduce the legacy split without adding safety.
77+
78+
**Alternative considered**: Keep `ToolsCacheSave` beside the domain model. Rejected because there is no distinct create-only or partial-update shape.
79+
80+
### 3. Correct `created_at` to a Per-Instance Default
81+
82+
The domain dataclass uses `field(default_factory = datetime.now)`. Repository conversion always writes the domain timestamp explicitly.
83+
84+
On insert, `created_at` records construction time. On update, `save` writes the supplied full object, including `created_at`; callers currently construct a fresh entry when refreshing cache content, so a refresh receives a fresh timestamp.
85+
86+
**Rationale**: The existing Pydantic declaration evaluates `datetime.now()` at import time and can reuse that stale timestamp for every omitted `created_at`. No production consumer reads `created_at`, and expiration cleanup depends only on `expires_at`, so correcting this is isolated and testable.
87+
88+
**Alternative considered**: Preserve the existing row timestamp during upsert. Rejected because `save` is full replacement and callers are replacing the cache entry, not patching individual fields.
89+
90+
### 4. Keep Deterministic Key Creation at the Domain Boundary
91+
92+
The domain model exposes the existing deterministic key creation behavior using the same algorithm:
93+
94+
1. Base64-encode prefix and identifier separately.
95+
2. Join them with `~`.
96+
3. Return the MD5 digest through the existing `digest_md5` helper.
97+
98+
Callers use the domain helper rather than a repository method.
99+
100+
**Rationale**: Key derivation defines cache identity but does not access persistence. Keeping it with the domain model avoids making callers resolve a database dependency for a pure function while preserving the exact stored-key format.
101+
102+
**Alternative considered**: Keep `create_key` on the repository for a one-for-one CRUD replacement. Rejected because it keeps unrelated domain behavior on the persistence boundary.
103+
104+
### 5. Expose a Narrow Repository Surface
105+
106+
The repository provides:
107+
108+
- `get(key) -> ToolsCache | None`
109+
- `get_all(skip, limit) -> list[ToolsCache]`
110+
- `save(entry) -> ToolsCache`
111+
- `delete(key) -> ToolsCache | None`
112+
- `delete_expired() -> int`
113+
114+
`save` performs an exact full-object upsert:
115+
116+
```
117+
┌──────────────────────┐
118+
│ ToolsCache domain │
119+
│ complete entry state │
120+
└──────────┬───────────┘
121+
│
122+
┌──────────────┴──────────────┐
123+
▼ ▼
124+
no existing key existing key found
125+
│ │
126+
▼ ▼
127+
insert all fields replace all fields
128+
```
129+
130+
There are no separate public `create` and `update` methods because production callers use upsert semantics and do not need partial updates.
131+
132+
**Rationale**: A smaller repository API makes the intended cache behavior explicit and avoids carrying CRUD mechanics that only legacy persistence tests use.
133+
134+
**Alternative considered**: Mirror every CRUD method. Rejected because `create` and `update` have no production caller and are both covered by `save` behavior.
135+
136+
### 6. Migrate Consumers by Feature Group
137+
138+
The repository is added to DI while legacy CRUD remains available. Callers then migrate in bounded groups:
139+
140+
1. Domain, mapper, repository, DI, SQL test helper, and focused tests.
141+
2. Currency exchange-rate caching.
142+
3. Web browsing, HTML cleanup, and Twitter caching.
143+
4. Chat attachment caching and related responder/support mocks.
144+
5. Scheduled expired-entry cleanup.
145+
6. Remove legacy DI, CRUD, schema, and tests after all references are gone.
146+
147+
**Rationale**: Cache persistence has many consumers but no single service owner. Feature-group migration keeps failures attributable and preserves manual review checkpoints.
148+
149+
**Alternative considered**: Replace every cache caller in one pass. Rejected because the web-browsing and chat test surfaces are broad enough to warrant separate checkpoints.
150+
151+
## Risks / Trade-offs
152+
153+
- **Deterministic key output changes and existing cache entries become unreachable** -> Keep the exact algorithm and add a golden-output test for the existing prefix/identifier example.
154+
- **`created_at` correction changes stored timestamps on refreshed entries** -> Specify full replacement explicitly and test insert/update timestamps; no current caller reads this field.
155+
- **`expires_at = None` is mistaken for an absent update** -> Mapper/repository tests must cover clearing expiration and never-expiring entries.
156+
- **Expired entries begin disappearing during reads** -> Keep `get` as a raw lookup and retain explicit domain expiration checks in consumers.
157+
- **Mixed DB/Pydantic/domain cache shapes during migration** -> Migrate one feature group at a time and keep focused tests green before removing legacy access.
158+
- **Tests currently return dictionaries or Pydantic cache models from mocks** -> Update mocks to return the domain dataclass as each caller migrates.
159+
160+
## Migration Plan
161+
162+
1. Add the domain model, mapper, repository, DI property, and SQL test helper beside the existing CRUD/schema.
163+
2. Add domain, mapper, and repository tests covering identity, timestamp, expiration, upsert, deletion, and cleanup semantics.
164+
3. Migrate currency cache consumers and tests.
165+
4. Migrate web browsing, HTML cleanup, Twitter cache consumers, and tests.
166+
5. Migrate chat attachment cache consumers and related support mocks/tests.
167+
6. Migrate scheduled cleanup to the repository.
168+
7. Remove legacy DI access, CRUD/schema tests, CRUD/schema files, and SQL test helper after reference searches are clean.
169+
8. Run the full offline test suite, all-files pre-commit, and strict OpenSpec validation.
170+
171+
Rollback remains straightforward because the database schema and deterministic key format do not change, and legacy CRUD remains available until the final cleanup milestone.
172+
173+
## Open Questions
174+
175+
None. Cache writes are internal full-state replacements, and the current caller/test surface defines the required behavior.

‎openspec/changes/archive/2026-06-21-migrate-tools-cache-to-repository/proposal.md‎

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
## Why
2+
3+
Tools cache persistence still uses the legacy `db/schema` + `db/crud` pattern, which exposes Pydantic persistence schemas and SQLAlchemy rows to cache consumers across chat, currencies, web browsing, cleanup, and tests. The completed sponsorship migration provides a proven repository/domain pattern that can remove this leakage while preserving cache behavior.
4+
5+
## What Changes
6+
7+
- Add a feature-level tools cache domain dataclass, mapper, and repository beside the existing `ToolsCacheDB` SQLAlchemy model.
8+
- Keep `ToolsCacheDB` as the single database model and preserve the existing `tools_cache` table, string primary key, stored value, nullable expiration timestamp, and deterministic key format.
9+
- Move cache-entry behavior to the domain boundary:
10+
- `created_at` defaults per instance instead of at Python module import time;
11+
- `expires_at = None` continues to mean the entry never expires;
12+
- expiration checks and deterministic key creation remain available without exposing persistence types.
13+
- Add the new repository to DI, migrate cache consumers in bounded feature groups, then remove legacy `tools_cache_crud` / `db.schema.tools_cache` access after callers and tests are migrated.
14+
- Preserve current cache hit/miss, full upsert, explicit expiration, expired-entry cleanup, and deletion behavior.
15+
- Keep legacy CRUD/schema tests until equivalent domain, mapper, and repository coverage is accepted.
16+
17+
## Capabilities
18+
19+
### New Capabilities
20+
21+
- `tools-cache-persistence`: Domain-model and repository behavior for deterministic cache keys, cache-entry expiration, full create/update persistence, lookup, deletion, and expired-entry cleanup without exposing SQLAlchemy models or legacy Pydantic persistence schemas.
22+
23+
### Modified Capabilities
24+
25+
_(None.)_
26+
27+
## Impact
28+
29+
**Code**
30+
- New feature-level tools cache files under `src/features/tools_cache/` or an equivalent feature-local package.
31+
- `src/di/di.py` gains `tools_cache_repo`; `tools_cache_crud` is removed after production callers no longer use it.
32+
- Cache consumers in chat attachments, currencies, web browsing, Twitter fetching, HTML cleanup, and cleanup migrate to repository domain models.
33+
- `test/db/sql_util.py` gains a `tools_cache_repo()` helper for focused repository tests.
34+
35+
**Database**
36+
- No table, column, primary key, nullability, or migration changes are intended.
37+
- `src/db/model/tools_cache.py` remains the only SQLAlchemy representation for `tools_cache`.
38+
39+
**API**
40+
- No external route, payload, response, or OpenAPI behavior changes are intended.
41+
42+
**Tests**
43+
- New domain/mapper tests cover per-instance creation timestamps, nullable expiration, expiration checks, deterministic key compatibility, and DB/domain round trips.
44+
- New repository tests replace legacy CRUD coverage for create, get, list, full upsert, delete, and expired-entry cleanup behavior.
45+
- Existing consumer tests remain behavior canaries while each feature group migrates.

‎openspec/changes/archive/2026-06-21-migrate-tools-cache-to-repository/specs/tools-cache-persistence/spec.md‎

Lines changed: 105 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,105 @@
1+
## ADDED Requirements
2+
3+
### Requirement: Tools cache domain model represents complete entry state
4+
The system SHALL represent tools cache entries as feature-level domain dataclasses with required key, value, and creation timestamp fields plus nullable expiration state.
5+
6+
#### Scenario: New entry receives per-instance creation timestamp
7+
- **WHEN** a caller creates a tools cache domain entry without specifying `created_at`
8+
- **THEN** the entry receives a creation timestamp at instance construction time
9+
- **THEN** separately created entries do not reuse a module-import timestamp
10+
11+
#### Scenario: Entry without expiration never expires
12+
- **WHEN** a tools cache entry has `expires_at = None`
13+
- **THEN** its expiration check returns false
14+
15+
#### Scenario: Entry with future expiration remains valid
16+
- **WHEN** a tools cache entry has an expiration timestamp later than the current time
17+
- **THEN** its expiration check returns false
18+
19+
#### Scenario: Entry with past expiration is expired
20+
- **WHEN** a tools cache entry has an expiration timestamp earlier than the current time
21+
- **THEN** its expiration check returns true
22+
23+
### Requirement: Tools cache keys remain deterministic and compatible
24+
The system SHALL derive cache keys from prefix and identifier values using the existing deterministic key algorithm.
25+
26+
#### Scenario: Existing key output remains unchanged
27+
- **WHEN** a caller creates a key from prefix `prefix` and identifier `identifier`
28+
- **THEN** the result is `3fffc53e8c62753274ae6ff244f2f4a4`
29+
30+
#### Scenario: Same inputs produce the same key
31+
- **WHEN** a caller creates multiple keys from the same prefix and identifier
32+
- **THEN** every result is identical
33+
34+
### Requirement: Tools cache repository returns domain models
35+
The system SHALL persist through the existing `ToolsCacheDB` table model while accepting and returning feature-level tools cache domain dataclasses.
36+
37+
#### Scenario: Fetch existing key returns domain model
38+
- **WHEN** a cache entry exists for a key
39+
- **THEN** the repository returns a tools cache domain dataclass with all persisted field values
40+
41+
#### Scenario: Missing key returns none
42+
- **WHEN** no cache entry exists for a key
43+
- **THEN** the repository returns `None`
44+
45+
#### Scenario: Fetch all returns domain models
46+
- **WHEN** cache entries exist
47+
- **THEN** the repository returns paginated tools cache domain dataclasses
48+
49+
### Requirement: Tools cache save performs exact full-object upsert
50+
The system SHALL save complete cache-entry domain state through insert-or-replace behavior.
51+
52+
#### Scenario: Save inserts new entry
53+
- **WHEN** the repository saves a domain entry whose key does not exist
54+
- **THEN** it inserts the key, value, creation timestamp, and expiration timestamp exactly as supplied
55+
- **THEN** it returns the persisted domain entry
56+
57+
#### Scenario: Save replaces existing entry
58+
- **WHEN** the repository saves a domain entry whose key already exists
59+
- **THEN** it replaces the stored value, creation timestamp, and expiration timestamp with the supplied domain state
60+
- **THEN** it returns the updated domain entry
61+
62+
#### Scenario: Save can clear expiration
63+
- **WHEN** the repository saves an existing key with `expires_at = None`
64+
- **THEN** it persists null expiration
65+
- **THEN** the returned domain entry never expires
66+
67+
### Requirement: Tools cache deletion behavior is preserved
68+
The system SHALL preserve cache-entry deletion and expired-entry cleanup behavior behind the repository.
69+
70+
#### Scenario: Delete existing key returns deleted snapshot
71+
- **WHEN** the repository deletes an existing cache key
72+
- **THEN** it removes the row
73+
- **THEN** it returns the deleted tools cache domain entry
74+
75+
#### Scenario: Delete missing key returns none
76+
- **WHEN** the repository deletes a missing cache key
77+
- **THEN** it returns `None`
78+
79+
#### Scenario: Delete expired removes only past expirations
80+
- **WHEN** the repository deletes expired entries
81+
- **THEN** it removes entries whose non-null expiration timestamp is earlier than the current time
82+
- **THEN** it keeps future-expiring and never-expiring entries
83+
- **THEN** it returns the number of deleted rows
84+
85+
### Requirement: Production cache consumers use repository domain models
86+
The system SHALL migrate production tools cache consumers from legacy CRUD/schema usage to repository domain models without changing externally visible behavior.
87+
88+
#### Scenario: Cache consumers preserve hit and miss behavior
89+
- **WHEN** chat, currencies, web browsing, HTML cleanup, or Twitter consumers read or write cached data
90+
- **THEN** their cache hit, miss, expiration, value parsing, and refresh behavior remains unchanged
91+
- **THEN** cache persistence uses repository domain models internally
92+
93+
#### Scenario: Scheduled cleanup preserves behavior
94+
- **WHEN** scheduled cleanup removes expired cache entries
95+
- **THEN** it reports the same deleted-entry count behavior
96+
- **THEN** it uses the tools cache repository
97+
98+
#### Scenario: DI exposes only repository access after migration
99+
- **WHEN** production callers no longer use legacy tools cache CRUD
100+
- **THEN** DI exposes `tools_cache_repo` for cache persistence
101+
- **THEN** DI no longer exposes `tools_cache_crud`
102+
103+
#### Scenario: Legacy persistence types are removed only after references are gone
104+
- **WHEN** no production or test code imports `tools_cache_crud`, `ToolsCacheCRUD`, `db.schema.tools_cache`, or `ToolsCacheSave`
105+
- **THEN** the legacy tools cache CRUD/schema files and obsolete tests may be removed

0 commit comments

Comments
 (0)