BerlinMOD benchmark: run the single canonical query suite, drop per-tool copies#202
Open
estebanzimanyi wants to merge 1 commit into
Open
BerlinMOD benchmark: run the single canonical query suite, drop per-tool copies#202estebanzimanyi wants to merge 1 commit into
estebanzimanyi wants to merge 1 commit into
Conversation
The benchmark now runs the one canonical BerlinMOD suite shared unchanged by all three DB tools — vendored as the berlinmod/suite git submodule (estebanzimanyi/berlinmod-portability: q01..q17 + qrt, pure portable SQL). run_queries.py reads the canonical SQL and applies the DuckDB-specific output handling (.mode csv / .output, EXPLAIN ANALYZE) in the harness, in Python — there is NO per-tool adaptation in the query files. The 17 local query_N.sql + 17 explain copies (which embedded .mode/.output) are removed. Follow-up (separate): single-source the schema/load onto the canonical schema.sql, and the benchmark run needs MEOS pin >= 2026-06-14d for the th3 prefilter naming the canonical queries use.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The BerlinMOD benchmark now runs the one canonical query suite shared, unchanged, by all three DB tools (MobilityDB / MobilityDuck / MobilitySpark) — vendored as the
benchmark/BerlinMOD/suitegit submodule (berlinmod-portability:q01..q17+qrt, pure portable SQL).No per-tool adaptation
The query files are pure canonical SQL. The DuckDB-specific output handling (
.mode csv/.output,EXPLAIN ANALYZE) is applied byrun_queries.pyin Python, not embedded in the SQL. The 17 localquery_N.sql+ 17explain/copies (which embedded.mode/.output— the adaptation) are removed, so there are no stale local copies to drift.Follow-ups (separate, noted not done here)
schema.sql(thesql/load/scripts still create a local schema).bump/meos-pin-2026-06-14cre-points to 14d).After clone:
git submodule update --init benchmark/BerlinMOD/suite.