[opensuse-packaging] How to deal with Python .pyc and .pyo files?
Hello, what is the recommended way for packaging Python stuff? Either have byte-compiled Python .pyc and .pyo files packaged into the RPM or let this be done by the currently actually installed Python system during run-time on the end-user's computer and not by whatever (perhaps different) Python system during package build-time? Kind Regards Johannes Meixner -- SUSE LINUX Products GmbH, Maxfeldstrasse 5, 90409 Nuernberg, Germany AG Nuernberg, HRB 16746, GF: Markus Rex --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-packaging+help@opensuse.org
On Wed, 2007-11-14 at 15:08 +0100, Johannes Meixner wrote:
Hello,
what is the recommended way for packaging Python stuff?
Either have byte-compiled Python .pyc and .pyo files packaged into the RPM or let this be done by the currently actually installed Python system during run-time on the end-user's computer and not by whatever (perhaps different) Python system during package build-time?
This is kind of what I was trying to figure out myself for
python-gtkglext. The python automake macros seem to create both at
install time, but the files are identical.
-JP
--
JP Rosevear
On 2007-11-14 09:33:03 -0500, JP Rosevear wrote:
On Wed, 2007-11-14 at 15:08 +0100, Johannes Meixner wrote:
Hello,
what is the recommended way for packaging Python stuff?
Either have byte-compiled Python .pyc and .pyo files packaged into the RPM or let this be done by the currently actually installed Python system during run-time on the end-user's computer and not by whatever (perhaps different) Python system during package build-time?
This is kind of what I was trying to figure out myself for python-gtkglext. The python automake macros seem to create both at install time, but the files are identical.
as we build with the same python version as the user will run it later. you can safely package the .pyc files. i would even recommend packaging them so they get removed if you uninstall the package. hope this helps darix -- openSUSE - SUSE Linux is my linux openSUSE is good for you www.opensuse.org --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-packaging+help@opensuse.org
Hello, On Nov 14 16:00 Marcus Rueckert wrote (shortened):
On Wed, 2007-11-14 at 15:08 +0100, Johannes Meixner wrote:
Hello,
what is the recommended way for packaging Python stuff?
Either have byte-compiled Python .pyc and .pyo files packaged into the RPM or let this be done by the currently actually installed Python system during run-time on the end-user's computer and not by whatever (perhaps different) Python system during package build-time?
as we build with the same python version as the user will run it later. you can safely package the .pyc files.
I understand that it is the same Python version directly after a new system installation. But what happens when the user later change/update his Pyhon? Think about the "worst case" when the user replaces our Pyhon with whatever self-compiled Pyhon. Are byte-compiled Python .pyc and .pyo files the same for any Python and/or is any Python sufficiently smart to know when .pyc and/or .pyo files are outdated (even if the matching .py files are unchanged)?
i would even recommend packaging them so they get removed if you uninstall the package.
Removal when uninstalling could also be done by whatever other magic (e.g. via %ghost or a %preun scriptlet in RPM). I wonder why in this case small RPMs seem not to count. For example the installed .pyc and .pyo files in python on a openSUSE 10.3 i386 system are more than 7MB and still about 1.6MB after "bzip2" so that those files should make the python RPM about 1.6MB bigger. At least according to http://www.python.org/doc/1.5.1p1/tut/node43.html I wonder if there is a noticeable advantage to provide .pyc and .pyo files in the RPM because it reads: "A program doesn't run any faster when it is read from a ".pyc" or ".pyo" file than when it is read from a ".py" file; the only thing that's faster about ".pyc" or ".pyo" files is the speed with which they are loaded." I am no Python expert at all and I would be happy if a Python expert could provide some background information. Kind Regards Johannes Meixner -- SUSE LINUX Products GmbH, Maxfeldstrasse 5, 90409 Nuernberg, Germany AG Nuernberg, HRB 16746, GF: Markus Rex --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-packaging+help@opensuse.org
On Thursday 15 November 2007, Johannes Meixner wrote:
Think about the "worst case" when the user replaces our Pyhon with whatever self-compiled Pyhon.
Are byte-compiled Python .pyc and .pyo files the same for any Python and/or is any Python sufficiently smart to know when .pyc and/or .pyo files are outdated (even if the matching .py files are unchanged)?
they`re installed in a versioned directory. python does not do any checking other than timestamp comparison, so it will never read an outdated bytecompiled version. Thats about it.
I wonder why in this case small RPMs seem not to count.
So far printing with more than 100MB of data is the bigger factor compared to a couple of mb we could save by not packaging pyc files.
I am no Python expert at all and I would be happy if a Python expert could provide some background information.
if you %fdupe the pyc/pyo files, the overhead is lower. and the parsing overhead is significant for smaller short lived python scripts. Greetings, Dirk -- RPMLINT information under http://en.opensuse.org/Packaging/RpmLint --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-packaging+help@opensuse.org
Hello, On Nov 15 14:25 Dirk Mueller wrote (shortened):
On Thursday 15 November 2007, Johannes Meixner wrote:
Think about the "worst case" when the user replaces our Pyhon with whatever self-compiled Pyhon.
Are byte-compiled Python .pyc and .pyo files the same for any Python and/or is any Python sufficiently smart to know when .pyc and/or .pyo files are outdated (even if the matching .py files are unchanged)?
they`re installed in a versioned directory.
Does this mean that .pyc and .pyo files can be different for different Python versions? Python .pyc and .pyo files are not necessarily installed in a versioned directory like /usr/lib/python2.5/. They are installed where the matching .py file is. On a openSUSE 10.3 i386 default system I find .py files in /usr/share/emacs/22.1/etc/ /usr/share/hplip/* /usr/share/texmf/doc/generic/enctex/ /usr/lib/gimp/2.0/* /usr/lib/ooo-2.0/* /usr/bin/ What happens if Python version N creates /somewhere/file.pyc and later there is an update to Python version M? Is Python version M sufficiently smart to know that /somewhere/file.pyc is probably outdated?
I wonder why in this case small RPMs seem not to count.
So far printing with more than 100MB of data is the bigger factor compared to a couple of mb we could save by not packaging pyc files.
I do not understand this logic. I would understand if you said that packaging pyc files doesn't matter regarding RPM package size but I wonder why you suddenly start talking about printing? What do you want to tell reagrding printing, Python, and RPM package size? Kind Regards Johannes Meixner -- SUSE LINUX Products GmbH, Maxfeldstrasse 5, 90409 Nuernberg, Germany AG Nuernberg, HRB 16746, GF: Markus Rex --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-packaging+help@opensuse.org
Am Mittwoch 14 November 2007 schrieb JP Rosevear:
On Wed, 2007-11-14 at 15:08 +0100, Johannes Meixner wrote:
Hello,
what is the recommended way for packaging Python stuff?
Either have byte-compiled Python .pyc and .pyo files packaged into the RPM or let this be done by the currently actually installed Python system during run-time on the end-user's computer and not by whatever (perhaps different) Python system during package build-time?
This is kind of what I was trying to figure out myself for python-gtkglext. The python automake macros seem to create both at install time, but the files are identical.
The problem is: they don't have to be identical. And python -O <myscript> will look for .pyo and python <myscript> looks for .pyc. So you need to package both, but can hardlink them if they are the same (fdupes is your friend) Greetings, Stephan --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-packaging+help@opensuse.org
On Wednesday 14 November 2007, Stephan Kulow wrote:
will look for .pyo and python <myscript> looks for .pyc. So you need to package both, but can hardlink them if they are the same (fdupes is your friend)
Oh, we did test meanwhile that this works? ;) judging from the strace it looks like python correctly unlinks the file before trying to re-write it, so it should be okay. -- RPMLINT information under http://en.opensuse.org/Packaging/RpmLint --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-packaging+help@opensuse.org
participants (5)
-
Dirk Mueller -
Johannes Meixner -
JP Rosevear -
Marcus Rueckert -
Stephan Kulow