[Bug 326208] New: libcurl forces "pragma: no-cache" which makes libzypp not play well with squid
https://bugzilla.novell.com/show_bug.cgi?id=326208 Summary: libcurl forces "pragma: no-cache" which makes libzypp not play well with squid Product: openSUSE 10.3 Version: Beta 3 Platform: Other OS/Version: Other Status: NEW Severity: Normal Priority: P5 - None Component: libzypp AssignedTo: kkaempf@novell.com ReportedBy: jmichael@novell.com QAContact: kkaempf@novell.com Found By: --- By default, libcurl adds "Pragma: no-cache" to every web request which forces squid to re-fetch everything no matter if it's changed or not. We specifically set up a squid proxy with large cache sizes to lessen the pain of upgrading multiple machines in the office that are all tracking factory, but the forced no-caching prevents that from helping and results in far more network usage than it should. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=326208#c1 --- Comment #1 from Michal Marek <mmarek@novell.com> 2007-09-19 07:14:32 MST --- struct curl_slist *_headers = NULL; .. _headers = curl_slist_append(_headers, "Pragma:"); curl_easy_setopt(_curl, CURLOPT_HTTPHEADER, _headers); .. curl_easy_cleanup(_curl); curl_slist_free_all(_headers); should do the trick (not tested). -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=326208 Klaus Kämpf <kkaempf@novell.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |poeml@novell.com, aschnell@novell.com AssignedTo|kkaempf@novell.com |dmacvicar@novell.com QAContact|kkaempf@novell.com |visnov@novell.com -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=326208#c2 --- Comment #2 from Peter Poeml <poeml@novell.com> 2007-09-20 01:39:23 MST --- I initially thought this would be an anachronism of zypp, but surprising enough, it is indeed the default behaviour of curl. (Listed in the FAQ even...) Cacheability is obviously important to miminize load on origin servers and local networks. It is in our (download.opensuse.org) interest to be well cacheable. We attempt to set appropriate HTTP cache control headers that play well with caches _and_ with the client. Rather aggressively, mtadata of the Factory trees are specifically served with must-revalidate, because they change frequently. so proxy caches will revalidated it before serving it. zypp should not use this header under normal circumstances. If there is any scenario, where zypp is required to do end-to-end reload or revalidation, it should do a If-Modified-Since request, rather than using Pragma: no-cache. repomd.xml is special. It is small enough to simply fetch it. And since it contains hashes over all other (rpm-md) metadata files, it saves revalidation on the other metadata files. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=326208 Duncan Mac-Vicar <dmacvicar@novell.com> changed: What |Removed |Added ---------------------------------------------------------------------------- AssignedTo|dmacvicar@novell.com |jkupec@novell.com -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=326208 Ján Kupec <jkupec@novell.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |ASSIGNED -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=326208 User mmarek@novell.com added comment https://bugzilla.novell.com/show_bug.cgi?id=326208#c3 Michal Marek <mmarek@novell.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Component|libzypp |libzypp Product|openSUSE 10.3 |openSUSE 11.0 Version|Beta 3 |Factory --- Comment #3 from Michal Marek <mmarek@novell.com> 2008-04-16 07:15:09 MST --- Still valid for 11.0. Please have a look at comment #1, it has an example how to remove the Pragma: header. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=326208 Duncan Mac-Vicar <dmacvicar@novell.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dmacvicar@novell.com Priority|P5 - None |P4 - Low -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=326208 User crrodriguez@novell.com added comment https://bugzilla.novell.com/show_bug.cgi?id=326208#c4 --- Comment #4 from Cristian Rodriguez <crrodriguez@novell.com> 2008-07-15 10:52:29 MDT --- (In reply to comment #2 from Peter Poeml)
I initially thought this would be an anachronism of zypp, but surprising enough, it is indeed the default behaviour of curl. (Listed in the FAQ even...)
Do you know what is the rationale on it ? sounds quite strange to me..
repomd.xml is special. It is small enough to simply fetch it. And since it contains hashes over all other (rpm-md) metadata files, it saves revalidation on the other metadata files.
yes, but IMHO it should be retrived the same way, if HTTP code 304 is the response then it simple uses the repomd.xml already stored on disk. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=326208 User jnelson-suse@jamponi.net added comment https://bugzilla.novell.com/show_bug.cgi?id=326208#c5 Jon Nelson <jnelson-suse@jamponi.net> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jnelson-suse@jamponi.net --- Comment #5 from Jon Nelson <jnelson-suse@jamponi.net> 2008-07-22 19:21:19 MDT --- One way to temporarily work around this problem (which I also have!) is to add (between ===) the following to /root/.curlrc: === header "Pragma:" === -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=326208 User jnelson-suse@jamponi.net added comment https://bugzilla.novell.com/show_bug.cgi?id=326208#c6 --- Comment #6 from Jon Nelson <jnelson-suse@jamponi.net> 2008-10-28 08:35:25 MDT --- I'm not sure when it stopped working, but the suggestion in Comment 5 no longer works!
From the strace (grepping for Pragma)
6291 read(5, "header \"Pragma:\"\n", 8191) = 17 6291 sendto(6, "GET http://download.opensuse.org/repositories/openSUSE:/Tools/openSUSE_11.0/repo... HTTP/1.1\r\nProxy-Authorization: Basic Og==\r\nUser-Agent: ZYpp 4.27.3 (curl 7.18.1)\r\nHost: download.opensuse.org\r\nPragma: no-cache\r\nAccept: */*\r\nProxy-Connection: Keep-Alive\r\nIf-Modified-Since: Thu, 01 Jan 1970 00:00:00 GMT\r\n\r\n", 320, MSG_NOSIGNAL, NULL, 0) = 320 Furthermore, why it is it sending Proxy-Authorization at all? The proxy does not require auth! The string Og== decodes to a ':' (sans quotes) which means no username and no password. Some proxies would get confused at the presence of a Proxy-Authorization header if one has not been requested. Clearly the request is using If-Modified-Since (with the WRONG DATE). What is going on here? Using the squid option (NOT recommended) "reload_into_ims on" I was able to correct the bad curl (and zypp) usage but it's a total kluge. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=326208 User jnelson-suse@jamponi.net added comment https://bugzilla.novell.com/show_bug.cgi?id=326208#c7 --- Comment #7 from Jon Nelson <jnelson-suse@jamponi.net> 2008-10-28 08:43:15 MDT --- Why is libzypp making HEAD requests *and* GET requests when using If-Modified-Since? If it was using If-Modified-Since (or etags) correctly, it would not need the HEAD request at all. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=326208 User crrodriguez@novell.com added comment https://bugzilla.novell.com/show_bug.cgi?id=326208#c8 --- Comment #8 from Cristian Rodríguez <crrodriguez@novell.com> 2008-10-28 08:46:25 MDT --- (In reply to comment #6 from Jon Nelson)
Furthermore, why it is it sending Proxy-Authorization at all? The proxy does not require auth!
new bug report please.
Clearly the request is using If-Modified-Since (with the WRONG DATE).
huh!.. it is using the Unix Epoch, other new bug report...
What is going on here?
We have bugs :-) -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=326208 User crrodriguez@novell.com added comment https://bugzilla.novell.com/show_bug.cgi?id=326208#c9 --- Comment #9 from Cristian Rodríguez <crrodriguez@novell.com> 2008-10-28 08:49:21 MDT --- (In reply to comment #7 from Jon Nelson)
If it was using If-Modified-Since (or etags) correctly, it would not need the HEAD request at all.
Right, but that is, as far as I know a design flaw :-( -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=326208 Ján Kupec <jkupec@novell.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Priority|P4 - Low |P3 - Medium -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=326208 User dmacvicar@novell.com added comment https://bugzilla.novell.com/show_bug.cgi?id=326208#c10 Duncan Mac-Vicar <dmacvicar@novell.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |NEEDINFO Info Provider| |jkupec@novell.com --- Comment #10 from Duncan Mac-Vicar <dmacvicar@novell.com> 2008-11-03 07:38:24 MDT --- Commited patches from Jon Nelson that fix this. They will be available in libzypp 5.20.0 Please close this bug when you see it in factory. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=326208 User jkupec@novell.com added comment https://bugzilla.novell.com/show_bug.cgi?id=326208#c11 Ján Kupec <jkupec@novell.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEEDINFO |ASSIGNED Info Provider|jkupec@novell.com | Status Whiteboard| |in-svn --- Comment #11 from Ján Kupec <jkupec@novell.com> 2008-11-04 06:16:23 MST --- Thanx a lot! -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=326208 User jkupec@novell.com added comment https://bugzilla.novell.com/show_bug.cgi?id=326208#c12 Ján Kupec <jkupec@novell.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |RESOLVED Resolution| |FIXED Status Whiteboard|in-svn | --- Comment #12 from Ján Kupec <jkupec@novell.com> 2008-11-07 09:01:35 MST --- submitted -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
participants (1)
-
bugzilla_noreply＠novell.com