feat: Support excluding recent builds using timestamp (`--exclude-newer`) by jezdez · Pull Request #4228 · mamba-org/mamba

jezdez · 2026-04-10T12:27:26Z

Description

Adds an optional exclude_newer_timestamp field to Database::Settings, allowing callers to exclude packages with a timestamp newer than the given Unix epoch cutoff during repodata loading.

This enables conda's --exclude-newer feature to work with the libmamba solver backend, matching what rattler already supports natively. See conda/conda#15759 for the full tracking and conda/ceps#154 for the CEP proposing indexed_timestamp in repodata.

This PR adds only the low-level global libmamba API: a single cutoff timestamp applied when loading a repo. It intentionally does not implement conda's richer channel- or package-scoped exclude_newer policy; conda-libmamba-solver applies scoped policy in Python-side repodata filtering when needed and uses this native cutoff for the simple global-policy case.

Fix #4254.

How it works

The cutoff is applied at load time in both ingestion paths:

JSON path (mamba_read_json -> set_repo_solvables_impl): after a solvable is parsed, its policy timestamp is compared to the cutoff. If newer, the solvable is removed via repo.remove_solvable(id, true) -- the same pattern used for parse failures. For repodata JSON, indexed_timestamp is preferred over timestamp when present (matching conda semantics).
PackageInfo path (add_repo_from_packages_impl_loop): the package timestamp (normalized from ms to seconds) is checked before adding a solvable. If newer, the package is skipped entirely.

The .solv cache path is not filtered. Callers (e.g. conda-libmamba-solver) are expected to invalidate the cache when exclude_newer_timestamp changes, the same way they handle repodata_filters.

Other changes

Moved MAX_CONDA_TIMESTAMP from the anonymous namespace in helpers.cpp to helpers.hpp and added normalize_conda_timestamp for shared timestamp normalization.
Python bindings: Database.__init__() accepts an optional exclude_newer_timestamp keyword argument.

Context

--exclude-newer is a supply chain security feature that excludes recently published packages, giving security vendors time to flag malicious uploads. It's already supported by uv, pip, pixi, and rattler.
CEP draft proposing indexed_timestamp: conda/ceps#154
Andrew Nesbitt's overview: https://nesbitt.io/2026/03/04/package-managers-need-to-cool-down.html
conda implementation: conda/conda#15761
rattler support: conda/rattler#654
micromamba CLI tracking: mamba-org/mamba#4254

Test plan

C++ tests for add_repo_from_packages, millisecond normalization, repodata JSON filtering, unfiltered vs filtered package_count, and indexed_timestamp precedence
Python binding tests for packages API, repodata JSON, and None default

Convert the exclude_newer duration/timestamp from conda's config into a Unix timestamp cutoff and pass it to libmambapy's Database constructor as exclude_newer_timestamp. This enables native --exclude-newer filtering in the libmamba solver backend. Depends on: - conda/conda#15761 (exclude_newer config + parse_duration_to_seconds) - mamba-org/mamba#4228 (exclude_newer_timestamp in Database::Settings)

codecov · 2026-04-10T13:31:15Z

Codecov Report

❌ Patch coverage is 88.00000% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 55.10%. Comparing base (6bf836f) to head (96d2ca7).

Files with missing lines	Patch %	Lines
libmambapy/bindings/solver_libsolv.cpp	0.00%	3 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4228      +/-   ##
==========================================
+ Coverage   55.08%   55.10%   +0.02%     
==========================================
  Files         240      241       +1     
  Lines       30329    30346      +17     
  Branches     3241     3246       +5     
==========================================
+ Hits        16707    16723      +16     
- Misses      13619    13620       +1     
  Partials        3        3

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

The kwarg doesn't exist in released libmambapy yet (pending mamba-org/mamba#4228), so passing None causes a TypeError. Only include the kwarg when there's an actual timestamp value.

jjerphan · 2026-04-22T06:58:58Z

Thank you for this contribution, @jezdez. 🙂

Please let us know when this can be reviewed.

jjerphan

Thank you, @jezdez.

I just have a few comments. I also will let other maintainers review this PR.

jjerphan · 2026-04-22T09:07:02Z

                    }
-                    if constexpr (std::is_same_v<Action, Solution::Reinstall>
-                                  || std::is_same_v<Action, Solution::Omit>)
+                    if constexpr (std::is_same_v<Action, Solution::Reinstall> || std::is_same_v<Action, Solution::Omit>)


It seems like a spurious change which needs to be reverted.

Fixed in fbf2a8b by restoring the original line wrapping, so this formatting-only change no longer appears in the PR diff.

jjerphan · 2026-04-22T09:12:56Z

-                        on_parsed(filename);
+                        if (exclude_newer_timestamp && pkg_timestamp > *exclude_newer_timestamp)
+                        {
+                            repo.remove_solvable(id, /* reuse_id= */ true);


Side-note: Removing solvable here is the most appropriate to me. I think that we might even be able (in the case of sharded repodata usage) to filter builds on timestamp at the earliest (see #4214); yet I am not sure that there will be massive benefits (more maintenance and complexity for probably not the most reduced overhead).

Agreed. I kept the native libmamba implementation at the solvable insertion/removal point for this PR. The conda-side scoped policy uses Python repodata filtering before libmamba; earlier shard-level filtering can be revisited separately once the shard APIs settle.

SylvainCorlay · 2026-04-22T10:11:43Z

It could be interesting to have a variant of this offering some kind of "as-of" for reproducibility without a lock.

jjerphan · 2026-06-02T08:21:05Z

Hello @jezdez,

Would you authorize us to finish this PR? 🙂

jezdez · 2026-06-08T12:13:35Z

Thanks @jjerphan, I’d appreciate the handover if you’re still willing to help finish this PR.

One thing worth keeping explicit while doing that: this PR currently adds only the low-level/global libmamba API, exclude_newer_timestamp, so it can filter a repo against one cutoff timestamp. That intentionally differs from the current conda implementation, which now has a richer policy with global, channel-specific, and package-specific exclude_newer settings.

For conda, the downstream solver integrations use libmamba’s native global cutoff only for the simple global-policy case. When conda has channel or package overrides, conda-libmamba-solver applies the conda policy in Python-side repodata filtering before handing records to libmamba. So I would not try to squeeze scoped policy into this PR unless standalone mamba/micromamba wants its own matching UX and public API; that likely deserves a separate design/API change.

I also pushed fbf2a8b to address the review comments and align the native filter with conda’s timestamp semantics by preferring indexed_timestamp over timestamp for the cutoff decision.

jjerphan · 2026-06-10T08:48:49Z

Thank you for explaining this.

To me it makes sense to respect the policies which has been adopted by conda and to implement what is necessary to support them (or at least first the global exclude-newer) in mamba and micromamba; in this regards, I would have the support of the configuration elements be supported by this PR.

jjerphan · 2026-06-11T15:17:10Z

Everything should have been implemented in the last commits (it makes sense to me to have everything in a feat PR, especially given that the diff is OK, but I am not against proceeding in several steps). I think we could add many more tests though. What do people think?

Hind-M · 2026-06-16T08:17:32Z

+        exclude_newer.get_cli_config<std::string>(),
+        exclude_newer.description()
+    );
+


Shouldn't we add an option for exclude_newer_package as well?

exclude_newer_package is, as far as I understand this issue and from the current implementation proposal for conda, only part of the configuration.

We could introduce it on the CLI, but I do not really know how could one specify per-package policies.

jjerphan · 2026-06-16T09:24:15Z

I wonder whether we should use elements of std::chrono now since we support C++20.

What do people think?

We need to check if we can parse all the formats, but if so, that would make the code much simpler.

Done in #4344.

Hind-M · 2026-06-17T09:09:57Z

        solver::Request::Flags solver_flags = {};
-        std::string exclude_newer;
-        std::map<std::string, std::string> exclude_newer_package;
+        ExcludeNewerPolicy exclude_newer_policy;


Not sure if we really want to include the whole ExcludeNewerPolicy in the context. Seems ok for now (not too heavy), but maybe others in the team would mind.

I guess this is the best tradeoff to keep up with the ecosystem.

Hind-M · 2026-06-17T09:12:29Z

I'm wondering if we should add a section in the documentation explaining the usage.
It could be done in a follow-up PR and that would be part of a broader security section (what mamba provides in that regard).

jjerphan · 2026-06-17T13:04:28Z

I would wait for it to have at least one other review and preferably two.

JohanMabille · 2026-06-30T12:15:15Z

More generally, using std::chrono::parse and the standard date and duration primitive from the C++ standard library would make the code more high level and easier to maintain.

When set, packages with a timestamp newer than the cutoff are excluded during repodata loading. This covers both ingestion paths: - JSON (mamba_read_json -> set_repo_solvables_impl): after parsing a solvable, check its timestamp and remove it if newer than the cutoff - PackageInfo (add_repo_from_packages_impl_loop): skip the package before adding a solvable The .solv cache path is not filtered; callers (e.g. conda-libmamba-solver) are expected to invalidate the cache when exclude_newer changes. Also moves MAX_CONDA_TIMESTAMP to helpers.hpp so both helpers.cpp and database.cpp can use it without duplication. Python bindings expose the new parameter as an optional exclude_newer_timestamp keyword argument on Database.__init__(). This enables conda's --exclude-newer feature to work with the libmamba solver backend. See conda/conda#15759 for full tracking.

- C++ tests (test_database.cpp): verify add_repo_from_packages filters packages by timestamp, normalizes millisecond timestamps, and filters packages loaded from repodata JSON - Python tests (test_solver_libsolv.py): verify the libmambapy binding accepts exclude_newer_timestamp, filters packages from both the packages API and repodata JSON, and that None keeps all packages

The repodata JSON filter was reading solv.timestamp() to check against the cutoff, but libsolv attributes require internalize() before they can be read back. Since internalize() runs after all packages are loaded, the timestamp was always 0 at filter time. Fix by passing the parsed timestamp out of set_solvable via an out parameter and using that directly in the filter comparison. Also fix the millisecond normalization test to use a timestamp that actually triggers the normalization path (> MAX_CONDA_TIMESTAMP), and add subdir/depends fields to the Python repodata test fixture to match real repodata structure.

Apply clang-format to ternary expression in database.cpp and a pre-existing line-break issue in helpers.cpp.

jjerphan · 2026-07-01T11:06:51Z

As discussed privately, I opened #4344 to have this present PR a simpler and smaller one to review, focusing on what @jezdez initially intended.

jezdez mentioned this pull request Apr 10, 2026

Add --exclude-newer support across the conda ecosystem conda/conda#15759

Open

7 tasks

jezdez mentioned this pull request Apr 10, 2026

Wire exclude-newer policy into libmambapy Database conda/conda-libmamba-solver#905

Draft

3 tasks

jezdez changed the title ~~Add exclude_newer_timestamp to Database::Settings~~ Add exclude_newer_timestamp to Database::Settings Apr 10, 2026

jezdez marked this pull request as draft April 13, 2026 13:55

jjerphan changed the title ~~Add exclude_newer_timestamp to Database::Settings~~ feat: Support excluding recent builds using timestramp (--exclude-newer) Apr 22, 2026

jjerphan added the release::enhancements For enhancements PRs or implementing features label Apr 22, 2026

jjerphan changed the title ~~feat: Support excluding recent builds using timestramp (--exclude-newer)~~ feat: Support excluding recent builds using timestamp (--exclude-newer) Apr 22, 2026

jjerphan reviewed Apr 22, 2026

View reviewed changes

jjerphan mentioned this pull request May 4, 2026

Add an option to constrain dependency resolution with a relative cooldown period #4254

Open

This was referenced May 6, 2026

Add option to use a relative dependency cooldown period (--exclude-newer) conda/conda-lock#908

Open

RFC/META Harden CI security scikit-learn/scikit-learn#34006

Open

jezdez mentioned this pull request Jun 8, 2026

Add first-party --exclude-newer policy conda/conda#15761

Open

3 tasks

jjerphan marked this pull request as ready for review June 10, 2026 08:33

jjerphan force-pushed the feature/exclude-newer branch 4 times, most recently from 603645e to 1f95c97 Compare June 11, 2026 11:42

Hind-M reviewed Jun 16, 2026

View reviewed changes

jjerphan reviewed Jun 16, 2026

View reviewed changes

jjerphan force-pushed the feature/exclude-newer branch 3 times, most recently from 6a96376 to de43e12 Compare June 17, 2026 09:09

Hind-M reviewed Jun 17, 2026

View reviewed changes

jjerphan force-pushed the feature/exclude-newer branch from b930219 to 4223123 Compare June 17, 2026 09:19

Hind-M mentioned this pull request Jun 17, 2026

Add security section in documentation #4321

Open

jjerphan force-pushed the feature/exclude-newer branch from 4223123 to fb568b6 Compare June 17, 2026 09:26

Hind-M approved these changes Jun 17, 2026

View reviewed changes

jjerphan force-pushed the feature/exclude-newer branch 2 times, most recently from d635b0a to 4aca7a7 Compare June 26, 2026 15:12

JohanMabille requested changes Jun 30, 2026

View reviewed changes

jezdez added 5 commits June 30, 2026 15:37

Fix clang-format violations

891ee17

Apply clang-format to ternary expression in database.cpp and a pre-existing line-break issue in helpers.cpp.

Address exclude-newer review feedback

96d2ca7

jjerphan force-pushed the feature/exclude-newer branch 4 times, most recently from a41f30a to 96d2ca7 Compare July 1, 2026 10:53

jjerphan mentioned this pull request Jul 1, 2026

feat: Complete exclude_newer policy #4344

Draft

11 tasks

Uh oh!

Conversation

jezdez commented Apr 10, 2026 • edited by jjerphan Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

How it works

Other changes

Context

Test plan

Uh oh!

codecov Bot commented Apr 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

jjerphan commented Apr 22, 2026

Uh oh!

jjerphan left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

SylvainCorlay commented Apr 22, 2026

Uh oh!

jjerphan commented Jun 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jezdez commented Jun 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jjerphan commented Jun 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jjerphan commented Jun 11, 2026

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jjerphan Jun 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Hind-M commented Jun 17, 2026

Uh oh!

jjerphan commented Jun 17, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jezdez commented Apr 10, 2026 •

edited by jjerphan

Loading

codecov Bot commented Apr 10, 2026 •

edited

Loading

jjerphan commented Jun 2, 2026 •

edited

Loading

jezdez commented Jun 8, 2026 •

edited

Loading

jjerphan commented Jun 10, 2026 •

edited

Loading

jjerphan Jun 16, 2026 •

edited

Loading

JohanMabille commented Jun 30, 2026 •

edited

Loading