Skip to content

EPICS: Allow reading partial waveforms (first N elements)#1253

Draft
PierreSchnizer wants to merge 2 commits into
bluesky:mainfrom
hz-b:dev-epics-transfer-subvector
Draft

EPICS: Allow reading partial waveforms (first N elements)#1253
PierreSchnizer wants to merge 2 commits into
bluesky:mainfrom
hz-b:dev-epics-transfer-subvector

Conversation

@PierreSchnizer

@PierreSchnizer PierreSchnizer commented Apr 21, 2026

Copy link
Copy Markdown

Large EPICS waveforms (array-valued signals) are currently always transferred in full, even when only a small subset of the data is needed.

For high-rate acquisitions, data is buffered continuously but read out on trigger. In typical use, users select only a small portion of that buffer. Transferring the full array in these cases creates unnecessary network load and does not scale well.

This has a measurable impact in practice: at BESSY II, reading out all ~120 BPMs doubles measurement time due to the volume of data transferred. As a workaround, users are forced to limit the number of devices instead of using all available data sources.

This PR adds support for reading only the first N elements of a waveform (i.e. partial reads of array/spectrum data). It uses the EPICS Channel Access n_elements feature and exposes it via epics_options when defining a signal.

The implementation is currently EPICS-specific.

Discussion: This PR is intended as a starting point to explore partial reads of array-valued signals across backends.
Discussion link: #1254

An initial implementation to support discussion is provided here to make the idea concrete and easier to evaluate.

As an example use case, beam position monitors (e.g. Libera Spark, Brilliance) produce turn-by-turn data at ~1 µs intervals and store it in buffers of up to 65k samples. In many applications, only the first few samples are required, making full data transfer unnecessary.

@PierreSchnizer PierreSchnizer marked this pull request as draft April 21, 2026 09:51
@coretl

coretl commented Apr 22, 2026

Copy link
Copy Markdown
Collaborator

I like the concept, but we need to make sure both CA and PVA can do this. count seems the right approach for CA, but there is no easy concept in PVA. Claude suggests the following approach:

For EPICS 7 IOC waveforms served via QSRV2, array slicing can be requested by the client by appending a subarray field modifier to the channel name in the form PV_NAME.[start:end] (e.g. MY:WF.[0:99]). This is processed server-side by the IOC's dbChannel filter infrastructure, which QSRV2 respects, so no server-side changes are needed — the full waveform is never transmitted. For p4p-based servers such as those created by FastCS, there is no equivalent native mechanism; SharedPV broadcasts the same value to all clients and does not interpret the channel name. Support would need to be added explicitly by registering a DynamicProvider alongside the existing StaticProvider, intercepting channel names with the .[start:end] suffix, and posting sliced numpy arrays to per-slice SharedPV instances whenever the underlying attribute value updates.

Would you be able to try this out for PVA as well please?

@PierreSchnizer

Copy link
Copy Markdown
Author

Yes I will try to get it running with P4P

@PierreSchnizer

Copy link
Copy Markdown
Author

Sorry I was detoured by other tasks.
I will try to implement the comment given above soon

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants