higress 如何通过控制并发数保护上游服务？ || higress How to protect upstream services by controlling the number of concurrency?

[haozaiing]

环境：higress 2.1.8 + K8s
官网插件只有基于key的限流，本质是基于请求（或者说是基于用户）的限流。
但我希望通过控制并发数来保护上游服务，该用什么插件？ （上游服务是大模型的流式接口）

---

Environment: higress 2.1.8 + K8s
The official website plug-in only has key-based current limiting, which is essentially request-based (or user-based) current limiting.
But I want to protect the upstream service by controlling the number of concurrencies. What plug-in should I use? (The upstream service is the streaming interface of the large model)

[haozaiing]

补充提问：higress 是否有类似 apisix 提供的 请求速率限制插件，根据令牌桶或者时间窗口限制请求上游的速率？

---

Additional question: Does higress have a request rate limiting plug-in similar to that provided by apisix, which limits the rate of requests to the upstream based on the token bucket or time window?

[CH3CHO]

可以看一下集群限流插件

…

---- 回复的原邮件 ---- | 发件人 | ***@***.***> | | 发送日期 | 2025年10月24日 11:44 | | 收件人 | alibaba/higress ***@***.***> | | 抄送人 | Subscribed ***@***.***> | | 主题 | Re: [alibaba/higress] higress 如何通过控制并发数保护上游服务？ || higress How to protect upstream services by controlling the number of concurrency? (Discussion #3046) | 补充提问：higress 是否有类似 apisix 提供的 请求速率限制插件，根据令牌桶或者时间窗口限制请求上游的速率？ Additional question: Does higress have a request rate limiting plug-in similar to that provided by apisix, which limits the rate of requests to the upstream based on the token bucket or time window? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[allenz92]

集群限流插件目前是按qps/qpm级别的，有没有限制当前的Connections的方法啊

[haozaiing]

这个问题我已经解决了，我通过给请求染色的方式，给所有请求加上固定请求头，如：request_limited: true，
然后基于这个请求头做流控。
虽然有些粗糙，但这也是比较简单的方式来实现对上游大模型限流保护了。