Reimplement gvastreammux by using PTS to cross-stream batch#872
Open
qianlongding wants to merge 10 commits into
Open
Reimplement gvastreammux by using PTS to cross-stream batch#872qianlongding wants to merge 10 commits into
qianlongding wants to merge 10 commits into
Conversation
- Replace GstCollectPads with per-pad queue + dedicated output task, using a PTS anchor + tolerance window to assemble batches and a condition variable for back-pressure when queues fill up. - Coalesce per-pad FLUSH_START / FLUSH_STOP events via a flushing pad counter so the downstream flush, task pause, queue reset and task restart each happen exactly once per flush cycle. - Reject pad indices >= 256 in both mux and demux to bound the per-pad array size and prevent OOM from arbitrary user-supplied sink_N / src_N names; warn on sparse indices at PAUSED. - Drop the strict src/sink pad count match check in demux since pad indices may be sparse.
Different sources can produce buffers whose PTS are on entirely different
timelines (file vs RTSP, multiple cameras with independent clocks, sources
that started at different wall-clock moments). Without normalization, the
batch-anchor + tolerance scheduler either deadlocks (a far-future pad
never enters the window and back-pressures upstream) or oscillates
(anchor jumps between pads each round, producing single-stream batches).
Introduce a 'sync-mode' enum property with five modes:
- none raw PTS, default (preserves existing behavior)
- first-pts subtract each pad's first observed PTS so all pads start at 0
- segment subtract each pad's GST_EVENT_SEGMENT.start
- pipeline overwrite PTS with pipeline running time at arrival, useful
when at least one source is live and source PTS are unreliable
- ntp use GstReferenceTimestampMeta as PTS, for absolute cross-device
time alignment with rtspsrc ntp-sync=true add-reference-
timestamp-meta=true
Per-pad normalization state (first_pts, segment_start) is reset on
FLUSH_STOP and on PAUSED->READY so seek and pipeline restart establish
fresh baselines. The chain function applies normalization before taking
the mux lock to avoid lengthening the critical section.
The element pages and the multi-stream sample were still describing the original GstCollectPads + round-robin design with GstGvaStreammuxMeta and a single max-fps property. Both elements have since been rewritten to use PTS-anchor batch assembly with per-pad queues, GstAnalyticsBatchMeta for source tracking, and a configurable sync-mode for cross-pad PTS normalization. Update the element pages, the elements index, and the sample (README and shell script) to reflect: - the PTS-anchor / max-wait-time / max-queue-size scheduling and back-pressure behavior - the five sync-mode options (none, first-pts, segment, pipeline, ntp) with use cases and a NTP/PTP example pipeline - GstAnalyticsBatchMeta replacing GstGvaStreammuxMeta as the routing metadata between mux and demux - updated property tables, error conditions and tuning notes - a --sync-mode flag on the sample script with corresponding examples
qianlongding
requested review from
a team,
BaoHuiling,
OskarFiedot,
ZiningLi,
dmichalo,
jmotow,
marcin-wadolkowski,
mholowni,
msmiatac,
nszczygl9,
oonyshch,
pbartosik,
tbujewsk,
tjanczak,
walidbarakat,
yangjianfeng1208 and
yunowo
as code owners
Pure formatting cleanup of property descriptions, log lines and a stray blank line. No behavior change.
The struct member was initialized in _init and cleared in _finalize but never locked or passed to GStreamer. The output task is started via the 4-argument gst_pad_start_task(), which uses an internal lock; only gst_pad_start_task_full() would have consumed an external GRecMutex. Remove the field and its init/clear calls.
# Conflicts: # samples/gstreamer/README.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Replaces GstCollectPads with per-pad queues + a dedicated output task that assembles batches around a PTS anchor: pick the earliest PTS across non-EOS pads, then collect buffers whose PTS lies within pts-tolerance of that anchor, waiting at most max-wait-time for late pads. Pads that miss the window do not block the batch — a partial batch is pushed and the loop moves on.
Adds a sync-mode property to normalize per-pad PTS before scheduling: none (default, raw PTS), first-pts (each pad starts at 0), segment (GST_EVENT_SEGMENT.start subtraction), pipeline (overwrite with pipeline running time), ntp (use GstReferenceTimestampMeta).
Fixes # (issue)
Any Newly Introduced Dependencies
Please describe any newly introduced 3rd party dependencies in this change. List their name, license information and how they are used in the project.
How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration
Checklist: