Skip to content

[RIP-2] Add ProxyAdminService proto definitions for online client query#111

Open
messere1 wants to merge 2 commits into
apache:mainfrom
messere1:contest/rip2-proxy-admin-api
Open

[RIP-2] Add ProxyAdminService proto definitions for online client query#111
messere1 wants to merge 2 commits into
apache:mainfrom
messere1:contest/rip2-proxy-admin-api

Conversation

@messere1

@messere1 messere1 commented Jul 8, 2026

Copy link
Copy Markdown

Motivation

As part of RIP-2: Proxy Admin API, the Proxy needs admin RPCs to query online client connections. Currently there is no gRPC API to list connected clients or inspect their details — operators must rely on CLI tools or logs.

This PR adds the proto definitions for two new admin RPCs:

  • ListClients — List online client connections on the responding Proxy node with optional filtering and cursor-based pagination.
  • DescribeClient — Get detailed information for a specific client by client_id, including negotiated Settings, active subscriptions, and recent heartbeat records.

Design

Follows Option B from the DISCUSS email: introduce a dedicated ProxyAdminService rather than extending the existing Admin service.

Key decisions (updated per community feedback)

Decision Choice Rationale
D1 — Service placement Dedicated ProxyAdminService (Option B) Per community feedback on issue #10601: the Proxy admin API surface will grow (connection management, diagnostics, etc.); a dedicated service avoids bloating the messaging Admin service
D2 — Scope Per-node local view Each Proxy reports only its own clients; callers aggregate across nodes for cluster-wide view
D3 — Cross-proxy correlation proxy_group_id + proxy_endpoint + epoch Consumers can correlate local views across proxies without external configuration (suggested by bot evaluation)
D4 — Pagination Cursor-based (next_token) Client connections are dynamic; offset-based pagination would be unreliable

New messages

ClientFilter          — 6 filter fields (group, topic, id_prefix, language, role, time)
ListClientsRequest    — filter + page_size + next_token
ClientInstance        — 10 fields (id, language, version, endpoint, times, role, groups, auth, protocol)
ListClientsResponse   — status + clients + next_token + proxy_endpoint + epoch + proxy_group_id
DescribeClientRequest — client_id
HeartbeatRecord       — time + groups
ClientDetail          — instance + settings + subscriptions + heartbeats + auth_status

ClientInstance.Protocol enum

PROTOCOL_UNSPECIFIED = 0
GRPC = 1
REMOTING = 2

RocketMQ Proxy accepts both gRPC (v5 SDK) and Remoting (v4 SDK) client connections. The protocol field allows operators to distinguish them.

Service layout

// Existing — unchanged
service Admin {
  rpc ChangeLogLevel(ChangeLogLevelRequest) returns (ChangeLogLevelResponse) {}
}

// New — dedicated Proxy admin service
service ProxyAdminService {
  rpc ListClients(ListClientsRequest) returns (ListClientsResponse) {}
  rpc DescribeClient(DescribeClientRequest) returns (ClientDetail) {}
}

Example Usage

# List all Java consumers subscribed to "order_topic" on a Proxy
request = ListClientsRequest(
    filter=ClientFilter(topic="order_topic", language=LANGUAGE_JAVA, role=PUSH_CONSUMER),
    page_size=100
)
response = proxy_admin.ListClients(request)
for client in response.clients:
    print(f"{client.client_id} @ {client.access_point} (group: {response.proxy_group_id})")

Compatibility

  • Binary compatible: Existing Admin service and ChangeLogLevel RPC are unchanged. All new types are additive.
  • Source compatible: New service/RPCs/messages are additive. Clients that don't use the new RPCs are unaffected.

Changes in this PR

  • Commit 1 (83f3ce0): Initial proto draft with Option A (extend Admin)
  • Commit 2 (d8f7286): Switch to Option B — dedicated ProxyAdminService + add proxy_group_id

Follow-up

  • Server-side implementation in apache/rocketmq (Proxy module) — tracked separately
  • rocketmq-dashboard integration — consume ListClients for client management UI
  • Multi-proxy aggregation strategy — a CLI tool or dashboard-level fan-out

Related

Extend the Admin service with two new RPCs to support online client
query on the Proxy:

- ListClients: list online client connections on the responding Proxy
  node with optional filtering (group, topic, client_id prefix, language,
  role, connect time) and cursor-based pagination. Each Proxy returns
  only its local view; callers aggregate across nodes.

- DescribeClient: get detailed information for a specific client by
  client_id, including negotiated Settings, active subscriptions, and
  recent heartbeat records.

New messages: ClientFilter, ListClientsRequest, ClientInstance,
ListClientsResponse, DescribeClientRequest, HeartbeatRecord, ClientDetail
(with nested AuthStatus).

Reuses existing types from definition.proto: Language, ClientType, Status,
Settings, SubscriptionEntry.

Design follows Option A from the RIP-2 DISCUSS email (extend existing
Admin service rather than introducing a separate ProxyAdminService).

See: apache/rocketmq#10601
Copilot AI review requested due to automatic review settings July 8, 2026 13:02
@fanlunyong

fanlunyong commented Jul 8, 2026 via email

Copy link
Copy Markdown

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends the RocketMQ v2 Admin gRPC surface with new proto definitions to support Proxy-side online client introspection (listing connected clients and describing a specific client in detail), as proposed in RIP-2.

Changes:

  • Added ListClients RPC with ClientFilter and cursor-based pagination (next_token).
  • Added client-visibility messages (ClientInstance, ClientDetail, HeartbeatRecord) to represent connected client runtime state.
  • Extended Admin service with ListClients and DescribeClient RPC declarations.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread apache/rocketmq/v2/admin.proto
Comment thread apache/rocketmq/v2/admin.proto
Comment thread apache/rocketmq/v2/admin.proto
Comment thread apache/rocketmq/v2/admin.proto
Comment thread apache/rocketmq/v2/admin.proto
Based on community feedback on RIP issue #10601 (D1), switch from
extending the existing Admin service to a dedicated ProxyAdminService
for Proxy admin operations.

Changes:
- Restore Admin service to its original scope (ChangeLogLevel only)
- Add new ProxyAdminService with ListClients + DescribeClient RPCs
- Add proxy_group_id field to ListClientsResponse (D3 feedback from
  github-manager-bot evaluation: allows consumers to correlate local
  views across proxies without external configuration)
@messere1 messere1 changed the title [RIP-2] Add Proxy Admin API proto definitions for online client query [RIP-2] Add ProxyAdminService proto definitions for online client query Jul 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants