feat(berlinmod): streaming throughput benchmark on the canonical BerlinMOD corpus (stacks on #30)#31
Open
estebanzimanyi wants to merge 5 commits into
Conversation
This was referenced Jun 11, 2026
Closed
Closed
Closed
Closed
Closed
Closed
Closed
estebanzimanyi
force-pushed
the
consolidate/flink-meos-integration
branch
from
6d06a5f to
db2e51b
Compare
estebanzimanyi
force-pushed
the
consolidate/flink-benchmark
branch
from
60737a5 to
a803ba7
Compare
… surface Rebuild the bundled JMEOS jar from the regenerated JMEOS surface: the hand-rolled functions.functions facade is gone, GeneratedFunctions carries the sret-bound by-value struct returns (the *Split family) and the collation-init fix, and the org.mobilitydb.meos MeosOps* facade is regenerated against it. Migrate the aisdata processors off the removed functions.functions facade onto GeneratedFunctions (meos_initialize / meos_initialize_timezone / meos_finalize); the berlinmod package already used GeneratedFunctions + the MeosOps facade. Verified: flink-processor compiles and its test suite is green (10 tests, 0 failures) against the refreshed jar.
…inMOD corpus Adds the BerlinMOD streaming benchmark harness (BerlinMODBenchmark, per Q-form throughput over the DataStream topology), the benchmark-results document, and the Kafka producer script for feeding the corpus. The benchmark runs on the canonical BerlinMOD corpus only (--csv berlinmod_instants.csv); the invented synthetic fallback is gone.
estebanzimanyi
force-pushed
the
consolidate/flink-benchmark
branch
from
a803ba7 to
c80f17b
Compare
Rebuild the union JMEOS.jar against pin 14l: functions.GeneratedFunctions from JMEOS-14l (4466 methods, 2-arg count accessors) + the org.mobilitydb.meos.MeosOps* streaming facades regenerated from the 14l catalog/signatures. Refresh the bundled libmeos.so to 14l. The jar is now PURE bindings (JMEOS + generated facades). MeosSetSetJoin — the NxN cross-stream set-set join operator (edwithin/adisjoint/tdwithin_tgeoarr_tgeoarr + spanset_as_hexwkb) — was previously bundled as a source-less .class; it is application logic the facade codegen deliberately leaves to the caller, so it now lives as tracked source under org/mobilitydb/meos/ and is compiled from source against the jar.
estebanzimanyi
force-pushed
the
consolidate/flink-benchmark
branch
from
5051555 to
7de1706
Compare
Rebuild the union jar via streaming-codegen/build-union-jar.sh against the JMEOS regen off pin 15c (4466->4469 methods: + the eintersects_tpcpoint_geo / nad_tpcpoint_geo pointcloud spatial rels) + the 58 generated MeosOps* facades. Refresh bundled libmeos to 15c.
…pin-2026-06-22a
Add tools/codegen_facades.py — the per-engine streaming-facade generator (the
Flink analogue of JMEOS' FunctionsGenerator and MobilitySpark's UDF generator)
— and the streaming-relevance baseline it classifies against. It regenerates
the org.mobilitydb.meos.MeosOps* facades as 1:1 forwarders to the JMEOS
functions.GeneratedFunctions surface, so the facades are a pure projection of
the pin and never drift from the bundled jar:
tools/codegen_facades.py --jar flink-processor/jar/JMEOS.jar \
--out flink-processor --engine flink
Bump flink-processor/jar/JMEOS.jar to the raw-FFI JMEOS regenerated at
ecosystem-pin-2026-06-22a (the facades no longer ship in the jar) and
flink-processor/lib/libmeos.so to the matching all-families native library, and
commit the 57 regenerated facade classes. The streaming suite (10 tests incl.
BerlinMODSetSetJoinTest and the cbuffer/pose/npoint/facade smokes) passes.
estebanzimanyi
force-pushed
the
consolidate/flink-benchmark
branch
from
03f2755 to
5697d6d
Compare
estebanzimanyi
added a commit
to estebanzimanyi/MobilityFlink
that referenced
this pull request
Jun 25, 2026
Add tools/pin/compose-order.txt (the canonical fold manifest; MobilityDB#30 merged, MobilityDB#31 carries the facade generator + benchmark) and GENERATION.md (the per-binding generator policy: MobilityFlink binds the JMEOS jar and generates the org.mobilitydb.meos facades via tools/codegen_facades.py).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Path-B consolidation — Flink benchmark (scaffold merged · MEOS integration #30 · benchmark).
Adds the BerlinMOD streaming benchmark harness on top of #30:
BerlinMODBenchmark(per Q-form throughput over the DataStream topology),docs/benchmark-results.md, and the Kafka producer script.Canonical / regularity
The benchmark runs on the canonical BerlinMOD corpus only (
--csv berlinmod_instants.csv); the invented synthetic fallback is removed.Verification
mvn test→ 10 tests, 0 failures (the benchmark is amainentry point, compiled but not run in CI).