Description
API v2 index responses and source_conversation_seen event responses can
disagree about whether a source is still visible to the client.
The index path builds its source/item set through eager_query("Source"), which
filters out pending and deleted sources, and only includes items from those
visible sources. The source_conversation_seen event handler uses a direct
Source.query.filter(Source.uuid == event.target.source_uuid).one() lookup
instead, then returns the matched source and seen item metadata in the batch
response.
If a client submits a stale source_conversation_seen event for a source that
the current index omits, the same sync can therefore say "this source and item
are gone" in /api/v2/index and then return the source/item metadata in the
/api/v2/data batch response. Clients that reconcile the index first and then
apply the event response can re-add the source/item state that the index just
removed.
Steps to Reproduce
- Start with a source that has at least one submission and a client-side pending
source_conversation_seen event for that source.
- Mark the source deleted on the server before the next API v2 index response is
built, so the source has deleted_at set.
- Request
/api/v2/index.
- Observe that the source and its items are omitted from the index.
- Submit a
/api/v2/data batch that contains the stale
source_conversation_seen event for the same source.
- Observe that the event handler still finds the deleted source via the direct
source query and returns OK with the source plus seen item metadata in the
batch response.
Expected Behavior
The event response should be consistent with the current API v2 index visibility
rules. If a source is omitted from the index because it is pending or deleted,
event handlers should not return that source or its item metadata in the same API
v2 sync flow.
For source_conversation_seen, a deleted source should probably return Gone,
or the handler should use the same visible-source filtering as the index before
returning source/item data.
Actual Behavior
On current develop, source_conversation_seen can return source and item
metadata for a source that the API v2 index omits:
securedrop/models.py:66-75 filters pending/deleted sources and items in
eager_query().
securedrop/journalist_app/api2/shared.py:92-109 builds the API v2 index
from eager_query("Source").
securedrop/journalist_app/api2/events.py:289-315 handles
source_conversation_seen with a direct source query and returns
sources={source.uuid: source} plus items={item.uuid: item for item in seen_items}.
That lets a stale event response carry source/item metadata that is absent from
the authoritative index for the same sync cycle.
Comments
Suggested server-side fix: make event handlers that return source/item metadata
apply the same visibility constraints as the index, or filter each event result
against the current post-handler API v2 index before serializing the batch
response.
I checked existing issues and PRs for source_conversation_seen, stale event
responses, and reintroduced/omitted sources. I found the merged API v2 event
implementation PRs, but did not find an existing issue tracking this exact index
vs. event-response mismatch.
Description
API v2 index responses and
source_conversation_seenevent responses candisagree about whether a source is still visible to the client.
The index path builds its source/item set through
eager_query("Source"), whichfilters out pending and deleted sources, and only includes items from those
visible sources. The
source_conversation_seenevent handler uses a directSource.query.filter(Source.uuid == event.target.source_uuid).one()lookupinstead, then returns the matched source and seen item metadata in the batch
response.
If a client submits a stale
source_conversation_seenevent for a source thatthe current index omits, the same sync can therefore say "this source and item
are gone" in
/api/v2/indexand then return the source/item metadata in the/api/v2/databatch response. Clients that reconcile the index first and thenapply the event response can re-add the source/item state that the index just
removed.
Steps to Reproduce
source_conversation_seenevent for that source.built, so the source has
deleted_atset./api/v2/index./api/v2/databatch that contains the stalesource_conversation_seenevent for the same source.source query and returns
OKwith the source plus seen item metadata in thebatch response.
Expected Behavior
The event response should be consistent with the current API v2 index visibility
rules. If a source is omitted from the index because it is pending or deleted,
event handlers should not return that source or its item metadata in the same API
v2 sync flow.
For
source_conversation_seen, a deleted source should probably returnGone,or the handler should use the same visible-source filtering as the index before
returning source/item data.
Actual Behavior
On current
develop,source_conversation_seencan return source and itemmetadata for a source that the API v2 index omits:
securedrop/models.py:66-75filters pending/deleted sources and items ineager_query().securedrop/journalist_app/api2/shared.py:92-109builds the API v2 indexfrom
eager_query("Source").securedrop/journalist_app/api2/events.py:289-315handlessource_conversation_seenwith a direct source query and returnssources={source.uuid: source}plusitems={item.uuid: item for item in seen_items}.That lets a stale event response carry source/item metadata that is absent from
the authoritative index for the same sync cycle.
Comments
Suggested server-side fix: make event handlers that return source/item metadata
apply the same visibility constraints as the index, or filter each event result
against the current post-handler API v2 index before serializing the batch
response.
I checked existing issues and PRs for
source_conversation_seen, stale eventresponses, and reintroduced/omitted sources. I found the merged API v2 event
implementation PRs, but did not find an existing issue tracking this exact index
vs. event-response mismatch.