Commit 4c19e6b
committed
feat: RDMA multinode GPU support (chain SDK pieces)
Implements the akash-chain-sdk slice of the RDMA spec at
provider/_docs/infiniband-implementation-spec.md, covering Linear tickets
AKT-401..AKT-403, AKT-405, AKT-406 (CS-1, CS-2, CS-3, CS-5, CS-6). The
shared-storage track (CS-4) is intentionally dropped: per spec decision #3
every service gets its own RWO PVC and the provider does not produce
ReadWriteMany volumes.
Everything in this commit is off-chain — no on-chain proto messages change,
no validator upgrade required.
CS-1 — Inventory v1
(proto/provider/akash/inventory/v1/{node,resources}.proto +
go/inventory/v1/{node,resources}.pb.go regenerated via `make proto-gen-go`):
* Add `ResourcePair rdma = 7` to `NodeResources`.
* Add `rdma_resource_name`, `rdma_fabric`, `nccl_hca_prefix` strings to
`NodeCapabilities` with gogoproto.customname annotations
(`RDMAResourceName`, `RDMAFabric`, `NCCLHCAPrefix`).
* Extend `Dup()` helpers in node.go and resources.go.
* Tests: `node_test.go`, `resources_test.go` round-trip the new fields
through Dup().
CS-2 — SDL parser: `gpu.attributes.rdma: true` (go/sdl/gpu.go):
* Accept `rdma: true|false` under `gpu.attributes`. When true, emit a
flat on-chain GPU attribute `rdma=true` so providers that advertise
`capabilities/gpu/rdma=true` match.
* Tests: `rdma_gpu_test.go::TestV2ResourceGPU_RDMAFlag` plus the existing
`TestV2ResourceGPU` regression continues to pass.
CS-3 — SDL parser: `gpu.attributes.rdma_group: <name>` →
off-chain manifest field
(go/sdl/gpu.go,
proto/provider/akash/manifest/v2beta3/service.proto regenerated to
go/manifest/v2beta3/service.pb.go,
go/sdl/groupBuilder_v2{,_1}.go):
* Parser captures rdma_group via an internal sentinel attribute key
(`__rdma_group__`) inside `v2GPUAttributes.UnmarshalYAML`. The parent
`v2ResourceGPU.UnmarshalYAML` strips the sentinel before the slice ever
reaches on-chain `Resources.GPU.attributes` and surfaces the value on
a dedicated `v2ResourceGPU.RDMAGroup` field. `Validate()` is deferred
to the parent so the sentinel can be removed before the
attribute-key regex runs.
* Manifest `Service.proto` gains `rdma_group = 11` with
`gogoproto.customname = "RDMAGroup"`. Bindings regenerated via
`make proto-gen-go`.
* Both v2 / v2.1 group builders now read `compute.Resources.GPU.RDMAGroup`
and propagate it onto `manifest.Service.RDMAGroup`.
* Tests: `TestV2ResourceGPU_RDMAGroupRoutedOffChain` (asserts the
sentinel never escapes) and `TestV2ResourceGPU_RDMAGroupOmitted`.
CS-5 — Parser cross-field validations
(go/sdl/v2.go, validate() → new validateRDMA()):
1. Any compute profile with `gpu.attributes.rdma: true` requires its
placement attributes to include `capabilities/rdma=true`.
2. Any compute profile with `gpu.attributes.rdma_group` set must also
declare `gpu.attributes.rdma: true` on the same profile.
3. Within one placement, no implicit-default-plus-explicit mixing: if
any profile sets rdma_group, every RDMA-using profile must.
Helpers `gpuAttributesHaveRDMA` and `placementRequiresRDMA` are kept
package-local so the SDL parser owns the policy.
* Tests: `rdma_validation_test.go` (6 positive + negative fixtures).
CS-6 — Reservation commit path audit
(go/node/deployment/v1beta4/rdma_commit_audit_test.go):
Table-driven regression test pinning down that `GroupSpec.Dup()` and the
four concrete `ResourceGroup`-shaped values the provider's reservation
path can hold (`*Group`, `Group`, `*GroupSpec`, `GroupSpec`) all preserve
`Requirements.Attributes` (carrying `capabilities/rdma=true`) AND each
resource's `GPU.Attributes` (carrying `rdma=true`). A future change that
silently drops either slice — the exact failure mode the spec calls
out — fails this test loudly.
All `.pb.go` files were regenerated via `make proto-gen-go` (buf v1.47.2,
protoc v29.1, gogoproto v1.7.2). Running `make proto-gen-go` against this
tree should be a no-op.
Tests (go test ./...):
- pkg.akt.dev/go/inventory/v1 PASS
- pkg.akt.dev/go/manifest/v2beta3 PASS
- pkg.akt.dev/go/node/deployment/v1beta4 PASS (+ new CS-6 audit)
- pkg.akt.dev/go/sdl PASS (+ new CS-2/CS-3/CS-5)
Follow-ups for reviewers:
- TypeScript bindings (`ts/`) are not touched here and need a separate
`make proto-gen-ts` pass before the TS SDK consumes the new fields.
Linear: AKT-401, AKT-402, AKT-403, AKT-405, AKT-406
(AKT-404 / CS-4 cancelled — see spec decision #3)
Fix make/setup-cache.mk: defer GOLANGCI_LINT_MAJOR computation to
recipe-execute time and depend on $(SEMVER), so the install isn't
given an empty major and the broken module path
.../golangci-lint/v/cmd/golangci-lint. This bug had `lint/go` red on
main for several commits prior.1 parent bf983bc commit 4c19e6b
34 files changed
Lines changed: 1598 additions & 228 deletions
File tree
- go
- inventory/v1
- manifest/v2beta3
- node/deployment/v1beta4
- sdl
- make
- proto/provider/akash
- inventory/v1
- manifest/v2beta3
- testdata/sdl/output-fixtures
- v2.0
- gpu-basic
- http-options
- ip-endpoint
- multiple-services
- persistent-storage
- placement
- port-ranges
- pricing
- simple
- storage-classes
- v2.1
- credentials
- ip-endpoint
- ts/src/sdl/manifest/__snapshots__
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2 | 2 | | |
3 | 3 | | |
4 | 4 | | |
5 | | - | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
6 | 9 | | |
7 | 10 | | |
8 | 11 | | |
| |||
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
47 | 47 | | |
48 | 48 | | |
49 | 49 | | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
50 | 61 | | |
51 | | - | |
52 | | - | |
53 | | - | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
54 | 69 | | |
55 | | - | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
56 | 82 | | |
57 | 83 | | |
58 | 84 | | |
59 | 85 | | |
60 | 86 | | |
61 | | - | |
62 | | - | |
63 | 87 | | |
64 | 88 | | |
65 | 89 | | |
| |||
0 commit comments