On 2016-06-09 16:31, Per Jessen wrote:
Dave Howorth wrote:
On 2016-06-09 15:32, Per Jessen wrote:
Well, I can at least report some progress -
Congratulations.
a) I have a test system up and running.
b) I have xdebug working, I find the documentation is a little lacking, but I'm getting there.
c) In utter desperation, I actually went and googled "php many lstat calls":
http://grokbase.com/t/php/php-internals/087f1t68mm/lstat-call-on-each-direct...
(precisely what I am seeing).
Looks like you struck gold.
Yes, very much so.
If it is doing five lstats for every file and there are 4500 files then hitting each file once with no cacheing would give you the bulk of your lstats but why would it be hitting every file? Making a sitemap?
I think it happens with include()/require() too, and processwire is quite modularised.
I am using "open_basedir", of course. Can't get through to bugs.php.net at the moment. I'm still reading the bugreport.
I can access http://bugs.php.net/bug.php?id=52312 but I'm not sure what you're having trouble with if you're already reading the bug report.
When I started writing the above, bugs.php.net was hanging ... forgot to delete the middle sentence before I posted.
Ah, OK.
On my test-system, a plain office PC, but directly attached disk (vs. iSCSI), the 15000 lstat calls are done in about 500ms.
The interesting question to me there is why is that system faster than your server? Or rather, why is your server slow?
The test system has a local disk, whereas the server uses iSCSI to the SAN. It's not even about bandwidth available, I'm guessing it's simply a longer path for each call. When I googled, people were reporting similar issues with NFS.
I'll try to summarise the problem -
the stat() calls are made by realpath() which resolves a path with relative and symbolic links to an absolute name. These calls are normally cached, except when open_basedir is active. (which it typically would be in a shared environment).
It is a pity that there is no official patch for this yet, it's been going on for a few years. I found something called "turbo_realpath" which seems to works, at least on the test system. It enables the realpath() cache and disables function for creating sym/links.
The other possibility is to disable open_basedir. As we're using mpm-itk, every request is run with user permissions anyway, so open_basedir might not actually be necessary.
The bug doesn't seem to have had any updates from Rasmus in recent years. The suggestion to only do the realpath test when actually opening the file seemed sensible. I wonder if the discussion has moved somewhere else? Maybe post to the PHP-INTERNALS list and ask? Disabling open_basedir while also disabling symlinks in PHP and ideally in Apache if possible might be a fix. I haven't looked closely enough to know. I'm nervous about deliberately opening security holes for good reason since the bad guys will be testing for those. Adding a local disk is almost certainly a fix, if possible. Making a RAM-based filesystem to cache that part of your disks might be a possibility, if you can separate out anything thta changes state whilst the service is active. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org