Skip to content

Improve memory efficiency Particle class - #397

Merged
Hendrik1704 merged 4 commits into
sparkx_develfrom
roch/memory_efficiency_particle
Jun 12, 2026
Merged

Improve memory efficiency Particle class#397
Hendrik1704 merged 4 commits into
sparkx_develfrom
roch/memory_efficiency_particle

Conversation

@Hendrik1704

Copy link
Copy Markdown
Collaborator

This PR improves the memory efficiency on the Particle class level. This is critical in analyses where many events are used. I ran into this trap recently. The Particle class was always using a static array of 25 entries for the properties. Now the memory is dynamically added with an index map which is more memory efficient, even in the dense Oscar2013Extended format.
This is a benchmark run after the current implementation:

Python 3.10.20 (main, Mar  3 2026, 09:24:47) [GCC 13.3.0]
numpy 1.26.0

=======================================================
  JETSCAPE (100k particles)
=======================================================
  Particles created  : 100,000
  Total allocated    : 20.03 MiB  (21,007,883 bytes)
  Per-particle avg   : 210.1 bytes
  Peak usage         : 20.04 MiB

  First particle (old implementation):
    sys.getsizeof(particle)  = 40 bytes
    sys.getsizeof(data_)     = 162 bytes
    shallow sum              = 202 bytes

=======================================================
  Oscar2013Extended (100k particles)
=======================================================
  Particles created  : 100,000
  Total allocated    : 29.95 MiB  (31,401,152 bytes)
  Per-particle avg   : 314.0 bytes
  Peak usage         : 29.95 MiB

  First particle (old implementation):
    sys.getsizeof(particle)  = 40 bytes
    sys.getsizeof(data_)     = 266 bytes
    shallow sum              = 306 bytes

Compared to the issue #396 benchmark run, this is an improvement of ~42% for the JETSCAPE format and 13% for the Oscar2013Extended format.

This closes #396.

I just add all of you to this PR. I don't know who has time to review at the moment.

@Hendrik1704 Hendrik1704 added this to the SPARKX 2.3 milestone May 22, 2026
@Hendrik1704 Hendrik1704 self-assigned this May 22, 2026
@Hendrik1704 Hendrik1704 added the enhancement New feature or request label May 22, 2026
@Hendrik1704

Copy link
Copy Markdown
Collaborator Author

The missing CHANGELOG entry is in PR #398 with a new section for the next version. I didn't add it here to avoid conflicts.

@Hendrik1704
Hendrik1704 merged commit c540e59 into sparkx_devel Jun 12, 2026
3 checks passed
@Hendrik1704
Hendrik1704 deleted the roch/memory_efficiency_particle branch June 12, 2026 09:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants