Skip to content

Use registry config for plugin repos and default module fallback#7180

Draft
jorgee wants to merge 3 commits into
masterfrom
multi-plugin-registry
Draft

Use registry config for plugin repos and default module fallback#7180
jorgee wants to merge 3 commits into
masterfrom
multi-plugin-registry

Conversation

@jorgee
Copy link
Copy Markdown
Contributor

@jorgee jorgee commented May 27, 2026

Related to #6771

Problem

  • Plugins: only a single plugin repository could be specified (via
    indexUrl / NXF_PLUGINS_REGISTRY_URL). There was no way to declare
    additional plugin registries.
  • Modules: when a registry config block defined a custom URL, the default
    Nextflow registry was dropped from the resolution list entirely — the
    user-defined list fully replaced it.

These two behaviours were inconsistent. This PR aligns them: the default
Nextflow registry is always queried first as the baseline, and the
registry config scope contributes additional registries that are queried
afterwards, in the order they are listed.

Summary

Wires the registry config scope into plugin resolution and makes module
resolution always query the default registry first.

Plugins (nf-commons)

  • PluginUpdater.addRegistryRepos(RegistryConfig) appends an
    HttpPluginRepository for each URL in the registry scope, after the primary
    repo (derived from indexUrl / NXF_PLUGINS_REGISTRY_URL) and before the
    test repos. URLs already present are skipped so the default isn't queried
    twice; repo ids are generated as registry-N with collision guard.
  • PluginsFacade.load(config) extracts the registry block and calls
    addRegistryRepos on the existing updater (no updater recreation).
  • HttpPluginRepository.refresh() is now null-safe — required because
    UpdateManager.addRepository() calls refresh() immediately, before
    prefetch() has populated the metadata map.

Modules (nextflow)

  • RegistryClientFactory.forConfig() always prepends the default registry
    (deduplicated), so it is queried first and a user-defined registry.url no
    longer excludes the public registry. This matches the plugin ordering
    (default primary, then configured registries). A @TestOnly setDefaultRegistry() hook lets tests point the default away from the real
    registry. Publishing is unaffected — it targets registry.url explicitly.

RegistryConfig itself is intentionally left unchanged — it stays a faithful
representation of the user's config block. The default is injected only at the
module consumer (RegistryClientFactory), so the plugin path keeps honoring
NXF_PLUGINS_REGISTRY_URL opt-out.

Test plan

  • ./gradlew :nf-commons:test — incl. new PluginUpdaterTest cases:
    append from config, dedup against existing repo, offline no-op, null config no-op
  • ./gradlew :nextflow:test — incl. new RegistryClientFactoryTest cases:
    default queried first, no duplicate when already configured
  • Full nextflow + nf-commons suites pass locally
  • Manual test: Include resgistry-dev in config and run with a plugin only available in registry-dev

TODO:

  • Add registry config block in documentation
  • Add apiKey authentication in plugin calls.
  • Deprecate NXF_PLUGINS_REGISTRY_URL and use NXF_DEFAULT_REGISTRY_URL for both pluigns and modules. Allow testing registry changes in different development and staging environments.

Implementation note

We also considered routing the plugin calls through RegistryClient — it takes
a list of URLs and performs the calls until one replies successfully. This works
well for modules, but it doesn't fit pf4j's PluginUpdater and how plugins are
managed, where it expects to manage them separately.

Moreover, there is a chicken-and-egg problem between plugin loading and config resolution:
PluginUpdater is initialized before the config is resolved, because some SCM
and filesystem extensions are themselves provided by plugins and are required at
that stage. At that point only the default plugin registry is available. Later,
at load time — once the config is resolved — the remaining registries from the
registry scope are added, and plugins are resolved using all the defined
registries.

jorgee added 2 commits May 26, 2026 17:19
Signed-off-by: jorgee <jorge.ejarque@seqera.io>
Signed-off-by: jorgee <jorge.ejarque@seqera.io>
@netlify
Copy link
Copy Markdown

netlify Bot commented May 27, 2026

Deploy Preview for nextflow-docs-staging ready!

Name Link
🔨 Latest commit 69aa013
🔍 Latest deploy log https://app.netlify.com/projects/nextflow-docs-staging/deploys/6a16c065f6916100086cda1e
😎 Deploy Preview https://deploy-preview-7180--nextflow-docs-staging.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@jorgee jorgee marked this pull request as draft May 27, 2026 09:03
Align module registry resolution with the plugin behaviour: the default
Nextflow registry is always queried first, then the registries configured
in the `registry` scope. Document the `registry` config scope and update
the module/plugin registry guides accordingly.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: jorgee <jorge.ejarque@seqera.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants