zc.buildout: KeyError during "buildout init" with distribute 0.6.25

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

zc.buildout: KeyError during "buildout init" with distribute 0.6.25

Felix Schwarz-5
Hey,

I use Python 2.7 (Fedora 16). However zc.buildout (1.5.2) does not work for me:
$ buildout init
Creating '/home/fs/temp/buildout.cfg'.
Creating directory '/home/fs/temp/bin'.
Creating directory '/home/fs/temp/parts'.
Creating directory '/home/fs/temp/eggs'.
Creating directory '/home/fs/temp/develop-eggs'.
Getting distribution for 'distribute'.
Traceback (most recent call last):
...
    add_activation_listener(lambda dist: dist.activate())
  File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 2230, in activate
    map(declare_namespace, self._get_metadata('namespace_packages.txt'))
  File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 1815, in
declare_namespace
    path = sys.modules[parent].__path__
KeyError: 'peak'
An error occurred when trying to install distribute 0.6.25. Look above this
message for any errors that were output by easy_install.
While:
  Bootstrapping.
  Getting distribution for 'distribute'.
Error: Couldn't install: distribute 0.6.25


The code which causes the exception is:
def declare_namespace(packageName):
...
        if '.' in packageName:
            parent = '.'.join(packageName.split('.')[:-1])
            declare_namespace(parent)
            if parent not in _namespace_packages:
                __import__(parent)
            try:
                path = sys.modules[parent].__path__
            except AttributeError:
                raise TypeError("Not a package:", parent)

The package which triggers the bug is 'decoratortools' (python package
'peak.utils'). If I install other namespace packages (e.g. zope.interface)
everything works fine.

The error happens because 'peak' is not in sys.modules when buildout calls
it's easy_install implementation. If I add some code to ignore the keyerror,
everything works as expected though I'm not sure that this is the right fix.

Any idea how I can fix this? Pointers welcome.
fs
_______________________________________________
Distutils-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/distutils-sig
Reply | Threaded
Open this post in threaded view
|

Re: zc.buildout: KeyError during "buildout init" with distribute 0.6.25

Felix Schwarz-5
Also I should mention that the problem is somehow connected to the way how
Fedora installs the decoratortools.

If I install 'decoratortools' manually through easy_install (system-wide), the
problem does not appear.

When I traced the code I found that in the latter case
zc.buildout/easy_install does not check for peak.util after it prints
   Getting distribution for 'distribute'
while it checks for peak.util when decoratortools is installed via Fedora's
packages.


I don't see an obvious difference: Fedora does not do anything special to the
source and builds/installs it like that:
%{__python} setup.py build
%{__python} setup.py install --skip-build --root %{buildroot}

Other Fedora namespace packages (e.g. paste, logilab, zope) don't cause issues.

What's so special about peak?

The only obvious difference I see is that the peak packages install individual
Python files (decorators.py, symbols.py) in a common python package
"peak.utils" while other libraries seem to prefer separate packages (e.g.
zope.interface, zope.proxy).

I'm not sure why this seems to make a difference but the complexity
Python's/setuptools' site activation mechanism is currently over my head.

fs

_______________________________________________
Distutils-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/distutils-sig
Reply | Threaded
Open this post in threaded view
|

Re: zc.buildout: KeyError during "buildout init" with distribute 0.6.25

Felix Schwarz-5

Am 06.04.2012 12:45, schrieb Felix Schwarz:
> Also I should mention that the problem is somehow connected to the way how
> Fedora installs the decoratortools.
>
> If I install 'decoratortools' manually through easy_install (system-wide), the
> problem does not appear.

Actually I understood why that's happening:
pkg_resources will activate all namespace packages on sys.path. And Fedora
puts the decoratortools in 'site-packages/peak/...' so it is found while
easy_install stores the code in 'site-packages/DecoratorTools-1.8-py2.7.egg/'.


I think I also know now why peak.utils triggers the problem. Here's the code
from pkg_resources.py

def declare_namespace(packageName):
...
path, parent = sys.path, None
if '.' in packageName:
    parent = '.'.join(packageName.split('.')[:-1])
    declare_namespace(parent)
    if parent not in _namespace_packages:
        __import__(parent)
    try:
        path = sys.modules[parent].__path__
    except AttributeError:
        raise TypeError("Not a package:", parent)

All other namespace packages on my system (not many) just used a simple
namespace package like 'logilab' or 'paste'. Therefore the extra
  if '.' in packageName:
code path is not executed.

And that path expects that the parent is actually loaded in sys.modules which
is not valid in zc.buildout (as it manually resets 'sys.path' after loading
site.py).

Suggestions how the problem can be fixed? I think buildout is currently broken
for nested namespace packages directly installed in site-packages.
fs

_______________________________________________
Distutils-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/distutils-sig
Reply | Threaded
Open this post in threaded view
|

Re: zc.buildout: KeyError during "buildout init" with distribute 0.6.25

Felix Schwarz-5

Had an idea how to fix this in zc.buildout (see attached patch). Of course it
leads to some activated namespaces which should be inactive but as this is
mostly relevant for "buildout init" I hope it's anyway acceptable.

Comments welcome.

fs

_______________________________________________
Distutils-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/distutils-sig

python-zc-buildout-namespace-packages.patch (1K) Download Attachment