On Mon, 14 Mar 2016 16:02, Michael Matz wrote:
On Mon, 14 Mar 2016, Yamaban wrote:
contain timestamps of their source file and will not be used unless it matches exactly. Why do we (and redhat) even include them? In Debian packages have only .py files and the precompiled .pyc files get added upon package installation.
I wonder if .pyc or .pyo bring any advantages at all at this time?
On slow HW, esp with rotating rust, YES.
Any numbers to convince me? :)
For a good storage compare try this: HW: raspi2 -- for the slow / puny / frail cpu Test 1: slow / older USB2 hdd w/ 5400rpm, little cache (e.g. 20MByte/s ~ 180Mbit/s Test 2: very fast USB3 - SSD / Flash (yes, the interface is 480Mbit/s only) This 2.5x difference in through-put and the much bigger difference in 'seek'ing time becomes significant during the search for the dependencies. put the python files on that, without the .pyc and .pyo, load a bigger (using many libs / py files) app on both setups, see the difference wrt the storage, and why it makes the difference rust vs. solid On my old Atom330 (Dual Core, HT, 1600Mhz, 4GB Ram) with openSUSE 12.3, I saw the difference between HDD and SSD, too, but also the difference between available compiled python and not. E.g. loading calibre was a pain in waiting without doing a precompile beforehand (My notes speak of 75 sec on rust w/o precompile vs 15 sec on SSD with precompile). May that box rest in peace, the fans got loud at the end. Now I'm working on a Haswell i3-4330T Passive Cooling Box w/ Leap 42.1 (3Ghz, 16GB Ram, Samsung Evo 250GB mSata, 2.5k Display) On the new Box I can not tell the difference between with precompile and not -- the Storage is FAST, the Ram is BIG, the cpu has POWER.
But "compile on install" ala debian - best controlled by config setting - seems the the way to go forward fast in this aspect.
Actually I would do such compilation in a deferred scheme, not directly at install time (or well, maybe that's acceptable as well if not taking too much time).
That is what I meant with "config setting": 1: at install, put into /usr/lib..., 2: deferered cache /var/cache/python/... (per user would be the most easy), 3: no precompile at all, fresh every time, as "noarch" as possible opt 1 for physically limited machines, pay the penalty once, opt 2 for those that, start python apps all the time, but have the machine power, opt 3 for those that use phython very seldom, or are heavy into python dev, or simply have the fast storage and cpu power that they do not care. IMHO, bet would be a cli-tool, lets call it "python-precomile" with a few options: --remove :: either with additional lib/program name, or implied all --install :: either with additional lib/program name, or implied all --clear-cache :: either with system, user, or implied "all" --prewarm-cache :: (it this is wanted or needed) --respect-config :: read config, remove or install as configured That called as root during postinstall with "python-precomile --respect-config <package>" should reduce the hassle for the packager as much as possible, and allow the admin to change his/her mind later on.
This will become more imporatant, as other script languages will follow with precompiled files, esp. for system-libs, say hello to the debates on perl6 doing such a thing, and ruby devs also looks into that, never mind what php7 tries.
Yeah, true. I actually loath "script" languages that have a compiled form, that seems a work-around for some fundamental mis-design of them ("whohooo, our parser is so slow" - "yeah, well, it's a script language, right?" - "but ... but, I want it to be fast" - "and I want a rosa pony, go dying").
*Grin* Yep, wanna bake the cake and eat the batter beforhand. I know when to use "script" and when "compile", but then, my training started around 1985, during school, the teacher was a 50yo 'retired programming sysadmin', she knew her stuff from the pike up. (had fascinating, and hair-rising stories, too) - Yamaban -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org