dashboards: add initial agentgateway dashboard#3
Conversation
Signed-off-by: John Howard <john.howard@solo.io>
|
cc @dhaifley |
There was a problem hiding this comment.
I ran into some issues trying to test this. The agentgateway_build_info metric is somehow getting turned into agentgateway_build when it hits prometheus in my test setup. Inspecting the output of /metrics HELP / TYPE:
# HELP agentgateway_build Agentgateway build information.
# TYPE agentgateway_build info
agentgateway_build_info{tag="0.10.5"} 1
And, the _info part is stripped resulting in queries not working for me.
| "x": 0, | ||
| "y": 53 | ||
| }, | ||
| "id": 0, |
There was a problem hiding this comment.
Panel id's are all 0 and query refId's are all "", but that did not cause any errors for me, so just a nit.
| "repeatDirection": "h", | ||
| "targets": [ | ||
| { | ||
| "expr": "sum(label_replace(sum by (pod,namespace) (container_memory_working_set_bytes{image=\"\",namespace=~\"$namespace\"}), \"nsnameexists\", \"yes\", \"namespace\", \".*\") / on(pod, namespace, nsnameexists) label_replace(label_join(agentgateway_build_info, \"nsname\", \"/\", \"namespace\", \"gateway_networking_k8s_io_gateway_name\"), \"nsnameexists\", \"yes\", \"nsname\", \"$gateway\")) without (nsnameexists)", |
There was a problem hiding this comment.
The agentgateway_build_info metric does not seem to exist for me. I see just agentgateway_build. This is causing issues with several expressions and queries for me.
There was a problem hiding this comment.
fixed in agentgateway/agentgateway#689, its a bug in agentgateway making prometheus fail to parse the metrics
| "label": "Gateway", | ||
| "multi": true, | ||
| "name": "gateway", | ||
| "query": "query_result(label_join(agentgateway_build_info{namespace=~\"$namespace\"}, \"text\", \"/\", \"namespace\", \"gateway_networking_k8s_io_gateway_name\"))", |
There was a problem hiding this comment.
These variables don't have a default datasource and the query for gateway does not work in my test setup.
There was a problem hiding this comment.
ack, it worked for me sinec my install had a "default" data source set. I made it explicit
I think this is related to an issue in the openmetrics parsing in the kgateway otel stack setup. I have been meaning to debug this orthogonally to this dashboard. will report back |
|
hmm, there is one issue. The dashboard expects the pod labels, namely |
Signed-off-by: John Howard <john.howard@solo.io>
|
@timflannagan can you get this merged? |
3 participants
Description
add initial agentgateway dashboard
Change Type
/kind feature
Changelog
Additional Notes