Am 6. August 2013 00:23:46 schrieb Robert Schweikert <rjschwei@suse.com>:

Hi,

I need to package gcutil and gsutil, starting with gcutil (https://code.google.com/p/google-compute-engine-tools/ , https://developers.google.com/compute/docs/gcutil/#install)

For better or worse the guys at Google decided not to create a setup.py file and assume that everything they need is installed in the location where the main "executable" is installed.

That's not very nice of course from a packaging perspective as I think their modules should end up in site-packages and the script should live in /usr/bin.

Do you know about any Google code that does things the standard way? ;-)

I've created the following setup.py file:

import setuptools

setuptools.setup( name = 'gcutil', version = '1.8.2', description = 'Utilities to control Google Compute Engine resources', author = 'Robert Schweikert', author_email = 'rjschwei@suse.com', url = 'https://code.google.com/p/google-compute-engine-tools/', packages = setuptools.find_packages('lib'), package_dir = {'': 'lib'}, )

With this I ended up with /usr/lib/python2.7/site-packages/gcutil-1.8.2-py2.7.egg/EGG-INFO/scripts/gcutil where "gcutil" is the "executable". None of the code in the "lib" directory ended up in site-packages.

What's the correct approach here?

Normally, you would add "scripts=["gcutil"]," to the setup() call to have the script wrapper installed into %_bindir. But that wouldn't work since the top-level wrapper script (from the tarball) looks at a hardcoded path and does an exec :-( FWIW, this code was written by absolute Python weenies. But let's cd into lib/google_compute_engine/gcutil. There's the real "main" binary and a Makefile :-) This one installs everything into /usr/share/gcutil/ and symlinks the main executable to /usr/bin/gcutil. Also, lib/google_compute_engine/gcutil/gcutil imports is a funky module called "path_initializer". This is what it does:

...

...

import path_initializer import sys p1 = sys.path path_initializer.InitializeSysPath() p2 = sys.path p1 ['', '/usr/bin', '/usr/lib/python27.zip', '/usr/lib64/python2.7', '/usr/lib64/python2.7/plat-linux2', '/usr/lib64/python2.7/lib-tk', '/usr/lib64/python2.7/lib-old', '/usr/lib64/python2.7/lib-dynload', '/usr/lib64/python2.7/site-packages', '/usr/local/lib64/python2.7/site -packages', '/usr/local/lib/python2.7/site-packages', '/usr/lib64/python2.7/site-packages/gtk-2.0', '/usr/lib/python2.7/site-packages'] p2 ['/root/Projects/suse/packaging/Cloud:Tools/gcutils/gcutil-1.8.2/lib/google_apputils', '/root/Projects/suse/packaging/Cloud:Tools/gcutils/gcutil-1.8.2/lib/ipaddr', '/root/Projects/suse/packaging/Cloud:Tools/gcutils/gcutil-1.8.2/lib/python_gflags', '/root/Projects/suse/pa ckaging/Cloud:Tools/gcutils/gcutil-1.8.2/lib/google_compute_engine', '/root/Projects/suse/packaging/Cloud:Tools/gcutils/gcutil-1.8.2/lib/google_api_python_client', '/root/Projects/suse/packaging/Cloud:Tools/gcutils/gcutil-1.8.2/lib/httplib2', '/root/Projects/suse/packagi ng/Cloud:Tools/gcutils/gcutil-1.8.2/lib/iso8601', '', '/usr/bin', '/usr/lib/python27.zip', '/usr/lib64/python2.7', '/usr/lib64/python2.7/plat-linux2', '/usr/lib64/python2.7/lib-tk', '/usr/lib64/python2.7/lib-old', '/usr/lib64/python2.7/lib-dynload', '/usr/lib64/python2.7 /site-packages', '/usr/local/lib64/python2.7/site-packages', '/usr/local/lib/python2.7/site-packages', '/usr/lib64/python2.7/site-packages/gtk-2.0', '/usr/lib/python2.7/site-packages']

So first of all you would need to remove the local copies of various Python libs since it's adding all those local copies of system modules to the PYTHONPATH. Instead, you would need requires on python-ipaddr, python-gflags, python-httplib2 (NASTY/EVIL) and python-iso8601.

I could of course just simply copy the whole kit and caboodle to the site-packages directory and create a link from /usr/bin, but somehow that does not feel very satisfying and appears like taking the easy way out, but maybe that is acceptable?

So, 1) Add "scripts" to your setup.py using the real main executable to have this one installed into /usr/bin 2) Patch out path_initializer to not mess with PYTHONPATH 3) Only install lib/google_* (means not using setuptools.find_packages but listing them individually. technically distutils should be sufficient if you don't use that function anymore) 3.1) lib/google_app_utils/google goes into %python_sitelib/google 3.2) google_compute_engine/gcutil goes into %python_sitelib/gcutil 4) require stuff mentioned above. Err, or not: % grep -re "from.*google.*import" ... httplib2/httplib2/__init__.py: from google.appengine.api import apiproxy_stub_map I can see real genius at work, they patched the upstream module to use their custom cr*p o_O. It's used in two places: % grep -re "import.*httplib" google_api_python_client/oauth2client/appengine.py:import httplib2 ... google_compute_engine/gcutil/version_checker.py:import httplib2 ... So you could install a local copy of lib/httplib2 into both %python_sitelib/google and %python_sitelib/gcutil. Since "." is the fist entry on PYTHONPATH, both modules installed in %python_sitelib would prefer their local copy (well, depends on absolute_import). But maybe they modified the other libs too... Long story short, it's the same story as with any other code written by Google engineers. They just copy the stuff they need in-tree and modify it however they see fit. Their idea of upstream contribution is what I would call f*ckstream. This is not worth messing with, just install the whole nonsense into /opt and be done.

Eventually I'd like to have this in Factory.

No way.

Help is much appreciated. As I do not know the correct direction to take I have o spec file or package to share in OBS yet.