Skip to content

Add live stream memory cleanup hook#903

Open
fastagents wants to merge 1 commit into
livepeer:mainfrom
fastagents:fastagents/ram-stream-cleanup
Open

Add live stream memory cleanup hook#903
fastagents wants to merge 1 commit into
livepeer:mainfrom
fastagents:fastagents/ram-stream-cleanup

Conversation

@fastagents

Copy link
Copy Markdown

Summary

This adds cleanup for live stream resources while preserving warmed model state.

  • add a cleanup_stream() hook for live pipelines
  • call stream cleanup when a live stream returns offline
  • drain StreamDiffusion’s internal frame queue in place
  • release optional per-stream style-image cache
  • run Python GC and malloc_trim() by default after stream cleanup
  • bound the Trickle publish queue to avoid retaining stale output frames under backpressure
  • explicitly release per-frame encoder tensor/image/packet references

Why

The current live pipeline reset clears parent-side multiprocessing queues, but warm live pipelines can still retain stream-scoped objects inside the child pipeline and output publishing path.

For tensor-backed output frames, retained references can also keep shared-memory file descriptors open. Under long streams or publish backpressure, that can lead to unnecessary memory and FD retention even though the model itself should remain warm.

This patch keeps the official live-runner behavior intact while making cleanup explicit and bounded.

Notes

  • does not change runner health policy
  • does not change watchdog behavior
  • does not change schema/parameter compatibility
  • does not unload warmed model weights during stream cleanup
  • CUDA cache clearing is available behind an env flag but remains disabled by default

Testing

  • python3 -m pytest -q runner/tests/live/test_stream_cleanup_source.py
  • Python 3.12 compileall on the edited source/test files

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant