http://aos-ostree.rhev-ci-vms.eng.rdu2.redhat.com/rhcos/images/cloud/latest/rhcos-qemu.qcow2.gz" rel="nofollow">http://aos-ostree.rhev-ci-vms.eng.rdu2.redhat.com/rhcos/images/cloud/latest/rhcos-qemu.qcow2.gz $ curl -I "${URL}" HTTP/1.1 200 OK Server: nginx/1.8.0 Date: Wed, 19 Sep 2018 04:32:19 GMT Content-Type: application/octet-stream Content-Length: 684934062 Last-Modified: Tue, 18 Sep 2018 20:05:24 GMT Connection: keep-alive ETag: "5ba15a84-28d343ae" Accept-Ranges: bytes $ curl -sIH 'If-None-Match: "5ba15a84-28d343ae"' "${URL}" | head -n1 HTTP/1.1 304 Not Modified $ curl -sIH 'If-Modified-Since: Tue, 18 Sep 2018 20:05:24 GMT' "${URL}" | head -n1 HTTP/1.1 304 Not Modified $ curl -sIH 'If-Modified-Since: Tue, 18 Sep 2015 20:05:24 GMT' "${URL}" | head -n1 HTTP/1.1 200 OK $ curl -sIH 'If-Modified-Since: Tue, 18 Sep 2018 20:05:25 GMT' "${URL}" | grep 'HTTP\|Last-Modified' HTTP/1.1 200 OK Last-Modified: Tue, 18 Sep 2018 20:05:24 GMT That last entry should have 304ed, although the spec has [4]: When used for cache updates, a cache will typically use the value of the cached message's Last-Modified field to generate the field value of If-Modified-Since. This behavior is most interoperable for cases where clocks are poorly synchronized or when the server has chosen to only honor exact timestamp matches (due to a problem with Last-Modified dates that appear to go "back in time" when the origin server's clock is corrected or a representation is restored from an archived backup). However, caches occasionally generate the field value based on other data, such as the Date header field of the cached message or the local clock time that the message was received, particularly when the cached message does not contain a Last-Modified field. ... When used for cache updates, a cache will typically use the value of the cached message's Last-Modified field to generate the field value of If-Modified-Since. This behavior is most interoperable for cases where clocks are poorly synchronized or when the server has chosen to only honor exact timestamp matches (due to a problem with Last-Modified dates that appear to go "back in time" when the origin server's clock is corrected or a representation is restored from an archived backup). However, caches occasionally generate the field value based on other data, such as the Date header field of the cached message or the local clock time that the message was received, particularly when the cached message does not contain a Last-Modified field. So the server is violating the SHOULD by not 304ing dates greater than Last-Modified, but it's not violating a MUST-level requirement. The server requirements around If-None-Match are MUST-level [2], so using it should be more portable. The RFC also seems to prefer clients use If(-None)-Match [4,5]. I'm using gregjones/httpcache for the caching, since that implemenation seems reasonably popular and the repo's been around for a few years. That library uses the belt-and-suspenders approach of setting both If-None-Match (to the cached ETag) and If-Modified-Since (to the cached Last-Modified) [6], so we should be fine. UserCacheDir requires Go 1.11 [7,8,9]. [1]: https://tools.ietf.org/html/rfc7232#section-2.3 [2]: https://tools.ietf.org/html/rfc7232#section-3.2 [3]: https://tools.ietf.org/html/rfc7232#section-2.2 [4]: https://tools.ietf.org/html/rfc7232#section-3.3 [5]: https://tools.ietf.org/html/rfc7232#section-2.4 [6]: https://github.com/gregjones/httpcache/blob/9cad4c3443a7200dd6400aef47183728de563a38/httpcache.go#L169-L181 [7]: golang/go#22536 [8]: golang/go@816154b [9]: golang/go@50bd1c4

Commits

Commits on Sep 22, 2018

config/libvirt/cache: Caching for libvirt pulls

config/libvirt/cache: Stub in $XDG_CACHE_HOME

config/libvirt/cache: Decompress when URI has ".gz" suffix

*: Update BUILD.bazel