【发布时间】：2022-01-12 11:19:49
【问题描述】：
我们正面临 varnish Max Threads hit & backend 和 session 连接峰值的问题。我们不确定原因，但我们观察到，当源服务器响应时间较长并最终返回不可缓存 (502) 响应时会发生这种情况。
清漆使用：
我们已经在 nginx 代理后面配置了 varnish，所以传入的请求首先到达 nginx，然后始终平衡到 n varnish。 Varnish，万一错过，调用源nginx主机，这里是example.com。
在我们的例子中，我们只缓存 HTTP GET 请求，所有这些请求都有 JSON 负载作为响应，大小从 0.001 MB 到 2 MB 不等。
示例请求：
HTTP GET : http://test.com/test/abc?arg1=val1&arg2=val2
预期的 xkey : test/abc
响应：Json 有效负载
大约 QPS：60-80 个 HTTP GET 请求
平均 obj ttl : 2d
平均 obj 优雅：1d
附加 vcl 文件、统计信息和 varnish 运行命令以进行调试。
监控统计：
清漆和 VCL 配置：
清漆版本：Linux,5.4.0,x86_64,varnish-6.5.1
varnishd -F -j unix,user=nobody -a :6081 -T localhost:6082 -f /etc/varnish/default.vcl -s file,/opt/varnishdata/cache,750G
vcl 4.0;
import xkey;
import std;
acl purgers {
"localhost";
}
backend default {
.host = "example.com";
.port = "80";
}
sub vcl_recv {
unset req.http.Cookie;
if (req.method == "PURGE") {
if (client.ip !~ purgers) {
return (synth(403, "Forbidden"));
}
if (req.http.xkey) {
set req.http.n-gone = xkey.softpurge(req.http.xkey);
return (synth(200, "Invalidated "+req.http.n-gone+" objects"));
}
else {
return (purge);
}
}
# remove request id from request
set req.url = regsuball(req.url, "reqid=[-_A-z0-9+()%.]+&?", "");
# remove trailing ? or &
set req.url = regsub(req.url, "[?|&]+$", "");
# set hostname for backend request
set req.http.host = "example.com";
}
sub vcl_backend_response {
# Sets default TTL in case the baackend does not send a Caching related header
set beresp.ttl = std.duration(beresp.http.X-Cache-ttl, 2d);
# Grace period to keep serving stale entries
set beresp.grace = std.duration(beresp.http.X-Cache-grace, 1d);
# extract xkey
if (bereq.url ~ "/some-string/") {
set beresp.http.xkey = regsub (bereq.url,".*/some-string/([^?]+).*","\1");
}
# This block will make sure that if the upstream return a 5xx, but we have the response in the cache (even if it's expired),
# we fall back to the cached value (until the grace period is over).
if ( beresp.status != 200 && beresp.status != 422 ){
# This check is important. If is_bgfetch is true, it means that we've found and returned the cached object to the client,
# and triggered an asynchronous background update. In that case, if it was a 5xx, we have to abandon, otherwise the previously cached object
# would be erased from the cache (even if we set uncacheable to true).
if (bereq.is_bgfetch)
{
return (abandon);
}
# We should never cache a 5xx response.
set beresp.uncacheable = true;
}
}
sub vcl_deliver {
unset resp.http.X-Varnish;
unset resp.http.Via;
set resp.http.X-Cached = req.http.X-Cached;
}
sub vcl_hit {
if (obj.ttl >= 0s) {
set req.http.X-Cached = "HIT";
return (deliver);
}
if (obj.ttl + obj.grace > 0s) {
set req.http.X-Cached = "STALE";
return (deliver);
}
set req.http.X-Cached = "MISS";
}
sub vcl_miss {
set req.http.X-Cached = "MISS";
}
如果有任何改进当前配置的建议或调试配置所需的任何其他内容，请告诉我们。
谢谢
阿布舍克调查
【问题讨论】：
标签： varnish varnish-vcl