PR #438 established a pattern for nmdc-submissions-to-mongo: the per-environment .env file holds all per-env config (MONGO_URI, BASE_URL, OUTPUT_FILE, plus the refresh token), and the Makefile target chooses which file to load. Variable names are consistent across prod and dev; user overrides on the command line work uniformly; recipes never echo credentialed values.
Several other Makefiles in the repo carry the same kind of per-env divergence at smaller scale and would benefit from the same pattern:
Makefiles/ncbi_to_duckdb.Makefile — MONGO_HOST/MONGO_PORT/MONGO_DB triple, would benefit from a unified env-file approachMakefiles/test_workflow.Makefile — SOURCE_* / TARGET_* variables (intentional source-vs-target pattern, not prod-vs-dev, so possibly out of scope but worth a look)Makefiles/env_triads.Makefile and others where MONGO_URI is the only knob
Proposed direction
Pick one Makefile at a time, migrate to:
- A single
*_ENV variable per Makefile, defaulting to local/.env.<pipeline>
- Per-env config (URI, paths, base URLs) inside the env file
- Single recipe shape per logical operation; target-specific defaults for dev/test variants
This is mostly a documentation/refactoring task once the canonical pattern is settled. See sibling issues for the broader connection-config story.
Related
PR #438 established a pattern for
nmdc-submissions-to-mongo: the per-environment.envfile holds all per-env config (MONGO_URI,BASE_URL,OUTPUT_FILE, plus the refresh token), and the Makefile target chooses which file to load. Variable names are consistent across prod and dev; user overrides on the command line work uniformly; recipes never echo credentialed values.Several other Makefiles in the repo carry the same kind of per-env divergence at smaller scale and would benefit from the same pattern:
Makefiles/ncbi_to_duckdb.Makefile—MONGO_HOST/MONGO_PORT/MONGO_DBtriple, would benefit from a unified env-file approachMakefiles/test_workflow.Makefile—SOURCE_*/TARGET_*variables (intentional source-vs-target pattern, not prod-vs-dev, so possibly out of scope but worth a look)Makefiles/env_triads.Makefileand others whereMONGO_URIis the only knobProposed direction
Pick one Makefile at a time, migrate to:
*_ENVvariable per Makefile, defaulting tolocal/.env.<pipeline>This is mostly a documentation/refactoring task once the canonical pattern is settled. See sibling issues for the broader connection-config story.
Related