【问题标题】:Varnish - purge.soft does not change TTL or anything清漆 - purge.soft 不会改变 TTL 或任何东西
【发布时间】:2022-09-26 00:26:40
【问题描述】:

我正在尝试仅对某些 req.url 值进行软清除,所有其他失效都通过禁令进行管理。 当禁令生效时,purge.soft(0s,30s) 不会修改缓存中的任何内容,TTL 保持标准(7200 秒)并且缓存保持活动状态。 我究竟做错了什么?

完整的 VCL 代码:

https://pastebin.com/QLmBh0hw

这里是 PURGE 期间的 varnishlog

curl -XPURGE http://<varnish-host>/prodotti/catene/catene-di-luci/

    *   << Request  >> 1310905
-   Begin          req 1310904 rxreq
-   Timestamp      Start: 1664123093.111004 0.000000 0.000000
-   Timestamp      Req: 1664123093.111004 0.000000 0.000000
-   ReqStart       172.16.2.136 35172 a0
-   ReqMethod      PURGE
-   ReqURL         /prodotti/catene/catene-di-luci/
-   ReqProtocol    HTTP/1.1
-   ReqHeader      Host: 172.16.3.37
-   ReqHeader      User-Agent: curl/7.68.0
-   ReqHeader      Accept: */*
-   ReqHeader      X-Forwarded-For: 172.16.2.136
-   VCL_call       RECV
-   ReqURL         /prodotti/catene/catene-di-luci/
-   ReqURL         /prodotti/catene/catene-di-luci/
-   VCL_acl        MATCH purgers \"ecommerce-node1-prod\"
-   VCL_return     hash
-   VCL_call       HASH
-   VCL_return     lookup
-   VCL_call       MISS
-   ReqHeader      purged: 0
-   VCL_return     synth
-   Timestamp      Process: 1664123093.111061 0.000057 0.000057
-   RespProtocol   HTTP/1.1
-   RespStatus     404
-   RespReason     Not Found
-   RespReason     Not Found
-   RespHeader     Date: Sun, 25 Sep 2022 16:24:53 GMT
-   RespHeader     Server: Varnish
-   RespHeader     X-Varnish: 1310905
-   VCL_call       SYNTH
-   RespHeader     purged: 0
-   VCL_return     deliver
-   RespHeader     Content-Length: 0
-   Storage        malloc Transient
-   RespHeader     Connection: keep-alive
-   Timestamp      Resp: 1664123093.111089 0.000085 0.000027
-   ReqAcct        108 0 108 154 0 154
-   End

    标签: varnish varnish-vcl


    【解决方案1】:

    我自己测试过,没有遇到任何问题。

    VCL 代码

    这是我从https://varnish-cache.org/docs/6.0/reference/vmod_generated.html#example 获取并改编的标准vmod_purge 示例VCL 代码:

    • 我删除了 ACL 检查
    • 我将purge.hard() 转换为purge.soft(0s,30s)
    sub vcl_recv {
        if (req.method == "PURGE") {
            return (hash);
        }
    }
    
    sub my_purge {
        set req.http.purged = purge.soft(0s,30s);
        if (req.http.purged == "0") {
            return (synth(404));
        }
        else {
            return (synth(200));
        }
    }
    
    sub vcl_hit {
        if (req.method == "PURGE") {
            call my_purge;
        }
    }
    
    sub vcl_miss {
        if (req.method == "PURGE") {
            call my_purge;
        }
    }
    
    sub vcl_synth {
        if (req.method == "PURGE") {
            if (req.http.purged) {
                set resp.http.purged = req.http.purged;
            }
            return (deliver);
        }
    }
    

    软清洗有什么作用

    您配置软清除的方式将确保对象被视为陈旧,但是由于您将宽限时间设置为 30 秒,异步重新验证可能仍会发生,直到对象生命周期结束后 30 秒。

    示例 HTTP 调用

    以下是一些示例调用,说明了软清除发生时会发生什么。

    第 1 步:调用端点并接收一个新对象

    ➜  ~ curl -I localhost
    HTTP/1.1 200 OK
    Content-Type: application/json; charset=utf-8
    Content-Length: 595
    ETag: W/"253-BCiA67uc9pD4vCc07ppQaevMbj0"
    Date: Thu, 22 Sep 2022 07:51:22 GMT
    X-Varnish: 65546 12
    Age: 18
    Via: 1.1 varnish (Varnish/6.6)
    Accept-Ranges: bytes
    Connection: keep-alive
    

    如您所见,我们正在调用http://localhost 端点并获得带有Age: 18 标头的结果。这意味着对象已被缓存 18 秒。

    第 2 步:清除

    ➜  ~ curl -XPURGE -I localhost
    HTTP/1.1 200 OK
    Date: Thu, 22 Sep 2022 07:51:42 GMT
    Server: Varnish
    X-Varnish: 32784
    purged: 1
    Content-Length: 0
    Accept-Ranges: bytes
    Connection: keep-alive
    

    在第 2 步中,我们正在执行软清除。 purged: 1 标头表示已清除 1 个对象。

    第 3 步:启动宽限模式

    ➜  ~ curl -I localhost
    HTTP/1.1 200 OK
    Content-Type: application/json; charset=utf-8
    Content-Length: 595
    ETag: W/"253-BCiA67uc9pD4vCc07ppQaevMbj0"
    Date: Thu, 22 Sep 2022 07:51:22 GMT
    X-Varnish: 65552 12
    Age: 26
    Via: 1.1 varnish (Varnish/6.6)
    Accept-Ranges: bytes
    Connection: keep-alive
    

    清除后,我们仍然看到缓存的内容被提供,因为 Age: 26 标头暗示对象已被缓存 26 秒。

    但是varnishlog -g request 的输出显示,在对新内容进行异步获取的同时提供过时的内容。这是调用purge.soft(0s, 30s) 的直接结果:

    *   << Request  >> 65552
    -   Begin          req 65551 rxreq
    -   Timestamp      Start: 1663833108.524685 0.000000 0.000000
    -   Timestamp      Req: 1663833108.524685 0.000000 0.000000
    -   VCL_use        boot
    -   ReqStart       172.24.0.1 58300 http
    -   ReqMethod      HEAD
    -   ReqURL         /
    -   ReqProtocol    HTTP/1.1
    -   ReqHeader      Host: localhost
    -   ReqHeader      User-Agent: curl/7.79.1
    -   ReqHeader      Accept: */*
    -   ReqHeader      X-Forwarded-For: 172.24.0.1
    -   VCL_call       RECV
    -   VCL_return     hash
    -   VCL_call       HASH
    -   VCL_return     lookup
    -   Hit            12 -1.375188 30.000000 0.000000
    -   VCL_call       HIT
    -   VCL_return     deliver
    -   Link           bereq 65553 bgfetch
    -   Timestamp      Fetch: 1663833108.524864 0.000179 0.000179
    -   RespProtocol   HTTP/1.1
    -   RespStatus     200
    -   RespReason     OK
    -   RespHeader     Content-Type: application/json; charset=utf-8
    -   RespHeader     Content-Length: 595
    -   RespHeader     ETag: W/"253-BCiA67uc9pD4vCc07ppQaevMbj0"
    -   RespHeader     Date: Thu, 22 Sep 2022 07:51:22 GMT
    -   RespHeader     X-Varnish: 65552 12
    -   RespHeader     Age: 26
    -   RespHeader     Via: 1.1 varnish (Varnish/6.6)
    -   VCL_call       DELIVER
    -   VCL_return     deliver
    -   Timestamp      Process: 1663833108.524876 0.000191 0.000011
    -   Filters
    -   RespHeader     Accept-Ranges: bytes
    -   RespHeader     Connection: keep-alive
    -   Timestamp      Resp: 1663833108.524937 0.000252 0.000061
    -   ReqAcct        74 0 74 275 0 275
    **  << BeReq    >> 65553
    --  Begin          bereq 65552 bgfetch
    --  VCL_use        boot
    --  Timestamp      Start: 1663833108.524815 0.000000 0.000000
    --  BereqMethod    HEAD
    --  BereqURL       /
    --  BereqProtocol  HTTP/1.1
    --  BereqHeader    Host: localhost
    --  BereqHeader    User-Agent: curl/7.79.1
    --  BereqHeader    Accept: */*
    --  BereqHeader    X-Forwarded-For: 172.24.0.1
    --  BereqMethod    GET
    --  BereqHeader    Accept-Encoding: gzip
    --  BereqHeader    If-None-Match: W/"253-BCiA67uc9pD4vCc07ppQaevMbj0"
    --  BereqHeader    X-Varnish: 65553
    --  VCL_call       BACKEND_FETCH
    --  VCL_return     fetch
    --  Timestamp      Fetch: 1663833108.524843 0.000027 0.000027
    --  Timestamp      Connected: 1663833108.525006 0.000191 0.000163
    --  BackendOpen    31 default 172.24.2.11 80 172.24.2.14 52026 connect
    --  Timestamp      Bereq: 1663833108.525047 0.000231 0.000040
    --  Timestamp      Beresp: 1663833108.530071 0.005256 0.005024
    --  BerespProtocol HTTP/1.1
    --  BerespStatus   200
    --  BerespReason   OK
    --  BerespHeader   Content-Type: application/json; charset=utf-8
    --  BerespHeader   Content-Length: 598
    --  BerespHeader   ETag: W/"256-1rfBjX0LanGZOnmzdwpQfzIjU30"
    --  BerespHeader   Date: Thu, 22 Sep 2022 07:51:48 GMT
    --  BerespHeader   Connection: keep-alive
    --  BerespHeader   Keep-Alive: timeout=5
    --  TTL            RFC 120 10 0 1663833109 1663833109 1663833108 0 0 cacheable
    --  VCL_call       BACKEND_RESPONSE
    --  VCL_return     deliver
    --  Timestamp      Process: 1663833108.530118 0.005302 0.000046
    --  Filters
    --  Storage        malloc s0
    --  Fetch_Body     3 length stream
    --  BackendClose   31 default recycle
    --  Timestamp      BerespBody: 1663833108.530294 0.005479 0.000176
    --  Length         598
    --  BereqAcct      195 0 195 214 598 812
    --  End
    

    日志中的Hit 12 -1.375188 30.000000 0.000000 行显示该对象的剩余生命周期为-1.375188 秒,仍然超过宽限期-30

    Link bereq 65553 bgfetch 日志行证明对后端进行了后台提取以存储最新版本的内容。 Link 标记中的事务 ID 与日志中的 BeReq 事务匹配。

    因此,在事务65553 中获取新对象时,返回的响应仍然是带有Age: 26 的旧对象来证明这一点。

    第 4 步:接下来的下一个请求获取新鲜内容

    因为如果还有宽限时间,宽限模式将触发异步提取,因此当前用户看不到该提取的影响。但是,下一个用户将获得新鲜的内容。以下cURL 输出说明了这一点:

    ➜  ~ curl -I localhost
    HTTP/1.1 200 OK
    Content-Type: application/json; charset=utf-8
    Content-Length: 598
    ETag: W/"256-1rfBjX0LanGZOnmzdwpQfzIjU30"
    Date: Thu, 22 Sep 2022 07:51:48 GMT
    X-Varnish: 32790 65553
    Age: 0
    Via: 1.1 varnish (Varnish/6.6)
    Accept-Ranges: bytes
    Connection: keep-alive
    

    因为设置了Age: 0,所以这是一个全新的对象。但是随着时间的推移,年龄计数器会增加。

    结论

    具有非零宽限值的软清除不会立即从缓存中删除对象。相反,它会将对象标记为已过期并为其提供宽限时间。

    这确保了软清除后的第一个访问者不必等待后端提取完成。

    这是在立即提供新鲜内容和让用户受益于 Varnish 的性能之间的权衡,后者将异步获取内容并在发生这种情况时提供陈旧的对象。

    【讨论】:

    • 在我的情况下, my_purge 被调用,但在 varnishlog 中我看到 purged: 0 并且命中和年龄不断增长。我注意到在清除呼叫期间有一个 '''VCL_call MISS'''
    • 您能否将您的完整 VCL 代码添加到问题中,并添加相关的varnishlog 输出,以便我可以检查VCL_call MISS 的影响。
    猜你喜欢
    • 2018-08-20
    • 2020-11-15
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 2021-03-26
    相关资源
    最近更新 更多