Add OVN network support, OVN as default (bridge as opt-out)

[jochumdev]

Background (why this isn't just --type=ovn)

• OVN networks don't require features.networks=true — they can be
  created directly in default too (confirmed: the single-host OVN setup
  howto does exactly this, no project flag). The feature flag only decides
  whether that project's networks are isolated to it.
• features.networks=true breaks bridge networks in that project, it
  doesn't just change the default. Once the flag is on, explicitly
  requesting type=bridge there is hard-rejected —
  netTypeInfo.Projects is false for bridge, so any non-default project
  hits Network type does not support non-default projects. If type is
  omitted, default project -> bridge, feature-enabled project -> ovn
  (cmd/incusd/networks.go:423-428, "Only OVN networks are allowed inside
  network enabled projects") — but that's the default, not a fallback;
  bridge is categorically unavailable in a feature-enabled project either
  way. This same redirect (project.NetworkProject()) applies identically
  to network ACLs, network forwards, peers, etc. — one mechanism, not
  bridge/OVN-specific.

DNS-based service discovery does not carry over to OVN

incus-compose's existing service-name resolution (a service reachable by its
compose name, e.g. db, docker-compose parity) is not a generic Incus
feature — it's built entirely on dnsmasq, and dnsmasq only runs on bridge
networks. OVN networks have no dnsmasq process at all: DHCP and DNS are
both handled natively inside OVN itself
(internal/server/network/driver_ovn.go,
internal/server/network/ovn/ovn_nb_actions.go). This issue's driver choice
has to account for that, or the feature regresses — and not gracefully — the
moment a project ends up on OVN.

What incus-compose does today (bridge only):
client/client_dnswatcher.go's RegisterDNSWatcher, together with
client/resource_network.go's updateDNSAliases, hooks instance
create/start/stop, reads the network's raw.dnsmasq config key, rewrites
address=/<service>/<ip> and cname= lines for the services this run owns
(preserving everything else, including user-supplied raw.dnsmasq
content), and writes it back. Setting raw.dnsmasq is what makes Incus
(re)start dnsmasq with that content appended to its config — that is the
entire mechanism, there's no separate "DNS API."

Why this doesn't just quietly stop working on OVN — it hard-fails.
raw.dnsmasq isn't a recognized OVN network config key at all (OVN's
config-key namespaces are bridge/dns/ipv4/ipv6/security/user —
raw.dnsmasq is bridge-only). Incus's per-driver config validation rejects
unrecognized keys outright ("Invalid option for network %q option %q",
internal/server/network/driver_common.go:161) instead of silently
ignoring them. So RegisterDNSWatcher's hook, unmodified, errors on every
instance create/start/stop against an OVN-backed network — this is a
guaranteed up/start failure the moment a project's networks are OVN, not
a silent feature gap. Whatever driver-detection lands in this issue must
gate RegisterDNSWatcher off raw.dnsmasq entirely for OVN targets before
OVN can ship as a live default.

What OVN has instead, and why it isn't a drop-in replacement. OVN
networks get their own native DNS mechanism (dns.mode, default managed;
dns.domain, default incus) — no dnsmasq, no daemon per network at all.
Each instance's OVN switch port carries a DNS record
(Logical_Switch.dns_records in OVN's northbound DB) that ovn-controller
answers directly on the wire — DNS "snooping" at the flow level, not a
listening resolver at a routable IP, which is why it works regardless of
which nameserver the instance thinks it's asking. Two differences from
dnsmasq matter here:

1. The registered name is hardcoded to the Incus instance name
   (DNSName: d.inst.Name(), internal/server/device/nic_ovn.go:989,1205),
   resolved as <instance-name>.<dns.domain>. There is no equivalent of
   raw.dnsmasq's address=/cname= escape hatch to register an
   arbitrary extra name — incus-compose can't point a db alias at
   whichever instance currently backs that service the way it does today.
2. No multi-replica aggregation. dnsmasq's address= records let
   incus-compose point one service name at every replica's IP (crude
   round-robin via multiple A records for the same name). OVN's per-port
   record is 1:1 with the instance; there's no built-in "one name, many
   replicas" concept to hook into.

Network Zones (doc/howto/network_zones.md) don't fill the gap either:
they're an AXFR zone-transfer source, not a resolver — "It cannot be
directly queried for DNS records" — actual resolution needs an external
authoritative server (bind9/nsd) that transfers from Incus and answers
instances itself. That's real added infrastructure incus-compose up can't
wire up transparently, and the auto-generated records are still 1:1 with
instance name; only manually-created records
(incus network zone record create) could carry a service-style alias, and
that means driving a second, heavier API alongside the OVN network, not a
drop-in swap for the current raw.dnsmasq code path.

Consequence for this issue: RegisterDNSWatcher must skip (not attempt,
not error) any network whose live Type is ovn — symmetric with the
CIDR/selector branch the ACL issue's Resolution section already has to make
for the same reason. It cannot just stop there, though: silently losing
service-name resolution the moment a project lands on OVN is the kind of
regression that doesn't announce itself until something can't find db
anymore, so this issue also has to ship the replacement (below), not just
document the gap.

Fix: Knot as a DNS sidecar

incus-compose ships a small, per-project Knot DNS instance as a sidecar
whenever a project would otherwise lose service-name resolution (in
practice: whenever its networks are OVN). Implementation is its own
follow-up work — not sketched here — but the reasoning for why Knot is
worth pinning down now, since it's the direct swap-in for the mechanism
that doesn't exist on OVN, not a new concept:

• raw.dnsmasq has no OVN equivalent (see above), and OVN's own native DNS
  registers exactly one name per instance — the real Incus instance name,
  not a compose service alias, and no multi-replica aggregation.
• Network Zones can't be queried directly by instances (Incus's built-in
  DNS server only serves AXFR zone transfers), so they need an external
  authoritative server regardless — the "do nothing extra" option was never
  on the table.
• Of the two realistic authoritative servers, only one can actually do the
  job: NSD's control interface (nsd-control) has no per-record command at
  all — every subcommand operates on whole zones, TSIG keys, or server
  state — and NSD doesn't implement RFC 2136 dynamic updates either, so the
  only way to change a record is rewriting and reloading the entire zone
  file. Knot's knotc transaction workflow (zone-begin/zone-set/
  zone-unset/zone-commit) adds and removes individual records directly,
  the same shape of operation updateDNSAliases already performs against
  raw.dnsmasq today. That's the deciding factor, not resource cost — at
  incus-compose's scale (a handful of records per project) either daemon's
  footprint is a non-issue.

Opt-out

x-incus-compose:
  dns-sidecar: auto # auto (default) | none

• auto — spin up the Knot sidecar only when a project actually needs it
  (OVN in play, service-name resolution otherwise unavailable).
• none — never spin one up, even on OVN networks. For anyone already
  running their own DNS automation against Incus directly (e.g.
  dnsweaver-style tools watching Incus and pushing records to their own
  provider), a second writer managing the same names would fight the first
  one rather than help.

Schema

Project-level, default auto, also --network-driver /
INCUS_COMPOSE_NETWORK_DRIVER (CLI wins, same precedence as every other
flag):

x-incus-compose:
  network-driver: auto # auto (default) | ovn | bridge

• bridge — skip detection entirely, always today's behavior. Zero risk for
  anyone who sets it.
• ovn — require OVN; fail loudly if unavailable, don't silently downgrade.
  For anything relying on the isolation guarantee features.networks
  provides, a silent fallback to bridge would be the wrong failure mode.
• auto — run detection (below), log the outcome plainly either way so it's
  never a silent, undiscoverable fact about the project
  (project "myapp": OVN networking enabled (server supports OVN) /
  ... bridge networking (OVN not available on this server)).

Per-network, only meaningful when the project itself isn't already
feature-forced to ovn (i.e. mode bridge, or auto where detection came
back negative) — opts a single network into OVN type while the project stays
in the shared default namespace, without taking on project-wide isolation:

networks:
  backend:
    driver: ovn
    x-incus-compose:
      parent: incusbr0 # optional, see below

x-incus-compose.parent maps to OVN's network config key (uplink). Keep
it optional — incus-compose can't know the server's uplink topology at
compose-authoring time, and Incus already resolves this correctly on its
own: exactly one eligible uplink -> auto-picked; two or more, omitted ->
clear Option "network" is required error. Don't pre-validate or guess,
just let that surface.

Opting a whole project into features.networks=true needs no new code at
all — already works today via the existing project-level x-incus escape
hatch (x-incus: {"features.networks": "true"}), documented as accepting
"any Project option."

Decision mechanism (the auto algorithm)

Decided once, at project creation, never revisited. EnsureProject
already forks create vs. get; the decision only needs to happen on the
create path. Once made, features.networks lives on the Incus project
object forever — that's the persistence layer, no incus-compose-side cache
needed. Every later up/down just reads what's already there via the
existing get path, which never touches feature flags.

Backward compatible by construction. Any project that already exists
before this ships is untouched — the auto default only ever applies to a
project's first-ever creation. No migration, no retroactive change to a
running deployment.

Ordering constraint — the flag must be right before the project's first
network is created, not inferred from it. Creating the project without
features.networks=true and then trying a real type=ovn network to "see
if it works" doesn't work: NetworkProjectFromRecord silently redirects
that network into default (since the flag is still false at request time),
so success proves nothing about isolated operation, and there's no clean way
to fix it after the fact — you'd have an orphaned, wrongly-scoped network in
default plus still need to create the real one. The probe has to be
independent of the project being created.

Detection, in order:

1. Cheap check first: GetNetworksAllProjects()
   scan for any Type == "ovn" && Status == "Created" anywhere on the server.
   Read-only, no side effects, no OVN backend touched. If found, OVN is proven available, skip the probe entirely. Only shortcuts the positive case — an empty result doesn't prove unavailability, just that nothing's been created yet.
2. Fallback probe: create a throwaway type=ovn, network=none network
   directly in default (isolated, no uplink needed, sidesteps the uplink
   question entirely for the probe itself), check success, delete it either
   way. Runs at most once per server in practice — after the first project
   (or a pre-seeded network, see Testing below), every subsequent project
   anywhere on that server hits the cheap path instead.

Both extension-gated and requires an unrestricted credential to see across
projects — same "incus-compose is never restricted today" assumption
already baked into the rest of the client.

Known sharp edges

1. Uplink candidates are polluted by every bridge network ever created,
   including incus-compose's own. allowedUplinkNetworks() scans every
   bridge/physical network in default — since bridge networks are
   pinned there permanently, every incus-compose project's bridge network
   (from any project, not just this one) is a candidate. parent becomes
   required the moment a server has more than one. incus-compose can't
   shrink or pre-validate this list; only Incus's own error at
   network-creation time surfaces it. Switching new projects to
   auto/ovn stops the pool from growing further (OVN networks are
   excluded from the bridge/physical filter) but doesn't retroactively fix
   a server that already has several bridge networks in default.
2. Treat features.networks as effectively permanent for a project's
   lifetime. Not independently verified whether Incus technically allows
   flipping it after networks exist, but given bridge is categorically
   rejected the moment the flag is on (see Background), there's no
   in-place middle ground to design around — moving a project between
   modes means tearing down and recreating its networks either way. See
   Migration below.
3. No universal cheap capability signal exists, full stop. No API
   extension gates OVN, no NetworkDrivers-style field exists in server
   environment (only storage gets that treatment), and the server config key
   that looked promising (network.ovn.northbound_connection) is
   meaningless when unset — it silently defaults to a local Unix socket
   (unix:/run/ovn/ovnnb_db.sock), so a server can have zero explicit OVN
   config and still be fully functional (confirmed live against a real
   server). The two-tier list-check-then-probe approach above is the
   practical ceiling, not a stopgap for something better later.
4. Service-name DNS resolution stops working on OVN networks, and does so
   by hard-failing rather than degrading. See "DNS-based service
   discovery does not carry over to OVN" above — raw.dnsmasq isn't a
   valid OVN config key, so RegisterDNSWatcher must be made driver-aware
   before OVN ships, or every instance create/start/stop against an
   OVN-backed network errors out. The Knot sidecar (see "Fix: Knot as a DNS
   sidecar") is what actually replaces the lost resolution — gating off
   raw.dnsmasq alone only stops the crash, it doesn't restore the feature.

Testing

E2E test servers pre-seed a permanent, otherwise-unused
type=ovn, network=none network at provisioning time, so every test run
gets the cheap detection path unconditionally regardless of execution order
or concurrency — no dependency on some earlier test having already created
one. It's type: ovn, so it never shows up as an uplink candidate either
(allowedUplinkNetworks() only counts bridge/physical) — zero
interaction with sharp edge #1.

Need both server flavors to cover both branches of auto: an OVN-enabled
server (seeded as above) and a plain bridge-only server (no OVN packages
installed at all) so detection is exercised failing and succeeding.

Migration: bridge -> OVN

Not automatic — incus-compose only decides features.networks on a
project's first creation (see Decision mechanism above), so an existing
project needs a manual, one-time push through:

incus-compose down
incus --project=<incus-project-name> project set features.networks=true
incus-compose up

For that last up to work with no compose file change, network creation
needs to check the project's actual features.networks value, not just
default Type client-side the way resource_network.go's newNetwork()
does today (if config.Type == "" { config.Type = "bridge" }, unconditional,
never looks at the project). That's a real implementation requirement of
this issue, not an optional nicety — without it, the migration above
hard-fails on the last step (Network type does not support non-default projects), and the project's features.networks value stops being the
single source of truth it's supposed to be.

incus-compose up afterward recreates everything fresh as OVN in the now
feature-enabled project — new addressing, since nothing carries over from
the old bridge subnet. Service-name DNS aliasing switches mechanism at the
same time: raw.dnsmasq records don't carry over (the mechanism doesn't
exist on the other side, see "DNS-based service discovery does not carry
over to OVN"), and the Knot sidecar (see "Fix: Knot as a DNS sidecar")
takes over instead, unless dns-sidecar: none was set.

Out of scope for this issue

• Network peers / cross-project OVN connectivity — separate Incus object,
  referenced by the ACL issue's OVN appendix but not built here.
• Implementation details of the Knot sidecar itself (deployment shape,
  record-sync mechanics, lifecycle wiring) — see "Fix: Knot as a DNS
  sidecar"; that it's needed and why Knot specifically is in scope here,
  the how is its own follow-up work.
• OVN ACL selectors/groups themselves — that's the ACL issue; this issue
  only gets real OVN networks created with a sane default.