Running `distribute` without installing it globally

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Running `distribute` without installing it globally

Tim Cuthbertson
Hi all,

I'm attempting to create a [zero install](http://0install.net) feed /
package for distribute, for the purposes of being able to build
software (using distribute) without having to first manually install
distribute. Specifically nosetests, but I imagine it will be useful
for lots of other software.

This means I'll only really want the part of distribute that *builds*
software - running 2to3, compiling C extensions if present, etc. If
there's a specific subset of distribute I should be using that, it'd
be good to know - but for now I'm building the the whole thing and
just not invoking the other features.

Anyway, the main requirements of a zero install feed are that the
program must be relocatable, and must be runnable "as-is", without
installing anything to well-known locations. This is similar to
wanting to run a built version from within a checked-out workspace, in
that the location of the built files is not some special system
location.




In attempting to do this (specifically for python 3), I've encountered
some issues. The first is that `distutils.require("distribute")`
doesn't work if distribute is not installed globally. This failed on
program startup, because setuptools/package_index.py contains:

user_agent = "Python-urllib/%s distribute/%s" % (
   sys.version[:3], require('distribute')[0].version
 )

I've changed this to use setuptools.__version__, which is not the
exact same thing but is probably close enough for a user-agent?
Otherwise, perhaps the user-agent could be lazy-initted since you'd
have little reason to use it if you're just building packages.





The second issue is much more surprising: Once I got it running, any
invocation would give me the error:

$ python3 lib/easy_install.py --help
usage: easy_install.py [options] requirement_or_url ...
   or: easy_install.py --help

error: invalid command 'easy_install'


I've taken the following line out of setuptools/command/easy_install.py:
    script_args = ['-q','easy_install', '-v']+argv,
and replaced it with just:
    script_args = argv,

I feel surely there was a good reason for this being here in the first
place, so I'm curious about why it was broken for me.

I've pushed my changes to bitbucket:
https://bitbucket.org/gfxmonk/distribute. After these changes, it
seems to at least run from an arbitrary directory without being
installed globally.




After all this, trying to build nosetests (using `python3 setup.py
build`) still fails though, unfortunately - it seems to be using the
system version of the setup() function, because I get the following
warnings:

/usr/lib/python3.2/distutils/dist.py:257: UserWarning: Unknown
distribution option: 'test_suite'
  warnings.warn(msg)
/usr/lib/python3.2/distutils/dist.py:257: UserWarning: Unknown
distribution option: 'use_2to3'
  warnings.warn(msg)
/usr/lib/python3.2/distutils/dist.py:257: UserWarning: Unknown
distribution option: 'zip_safe'
  warnings.warn(msg)
/usr/lib/python3.2/distutils/dist.py:257: UserWarning: Unknown
distribution option: 'entry_points'
  warnings.warn(msg)


..and the built library gets syntax on errors when run under py3,
presumably because py3 didn't actually run. Any idea how to get some
more information on what path is being taken during the build process
and why it's not invoking the distribute hotness?

Cheers,
 - Tim.
_______________________________________________
Distutils-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/distutils-sig
Reply | Threaded
Open this post in threaded view
|

Re: Running `distribute` without installing it globally

Tim Cuthbertson
On Sat, Mar 17, 2012 at 12:04 PM, Tim Cuthbertson <[hidden email]> wrote:

> Hi all,
>
> I'm attempting to create a [zero install](http://0install.net) feed /
> package for distribute, for the purposes of being able to build
> software (using distribute) without having to first manually install
> distribute. Specifically nosetests, but I imagine it will be useful
> for lots of other software.
>
> This means I'll only really want the part of distribute that *builds*
> software - running 2to3, compiling C extensions if present, etc. If
> there's a specific subset of distribute I should be using that, it'd
> be good to know - but for now I'm building the the whole thing and
> just not invoking the other features.
>
> Anyway, the main requirements of a zero install feed are that the
> program must be relocatable, and must be runnable "as-is", without
> installing anything to well-known locations. This is similar to
> wanting to run a built version from within a checked-out workspace, in
> that the location of the built files is not some special system
> location.
>
>
>
>
> In attempting to do this (specifically for python 3), I've encountered
> some issues. The first is that `distutils.require("distribute")`
> doesn't work if distribute is not installed globally. This failed on
> program startup, because setuptools/package_index.py contains:
>
> user_agent = "Python-urllib/%s distribute/%s" % (
>   sys.version[:3], require('distribute')[0].version
>  )
>
> I've changed this to use setuptools.__version__, which is not the
> exact same thing but is probably close enough for a user-agent?
> Otherwise, perhaps the user-agent could be lazy-initted since you'd
> have little reason to use it if you're just building packages.
>
>
>
>
>
> The second issue is much more surprising: Once I got it running, any
> invocation would give me the error:
>
> $ python3 lib/easy_install.py --help
> usage: easy_install.py [options] requirement_or_url ...
>   or: easy_install.py --help
>
> error: invalid command 'easy_install'
>
>
> I've taken the following line out of setuptools/command/easy_install.py:
>    script_args = ['-q','easy_install', '-v']+argv,
> and replaced it with just:
>    script_args = argv,
>
> I feel surely there was a good reason for this being here in the first
> place, so I'm curious about why it was broken for me.
>
> I've pushed my changes to bitbucket:
> https://bitbucket.org/gfxmonk/distribute. After these changes, it
> seems to at least run from an arbitrary directory without being
> installed globally.
>
>
>
>
> After all this, trying to build nosetests (using `python3 setup.py
> build`) still fails though, unfortunately - it seems to be using the
> system version of the setup() function, because I get the following
> warnings:
>
> /usr/lib/python3.2/distutils/dist.py:257: UserWarning: Unknown
> distribution option: 'test_suite'
>  warnings.warn(msg)
> /usr/lib/python3.2/distutils/dist.py:257: UserWarning: Unknown
> distribution option: 'use_2to3'
>  warnings.warn(msg)
> /usr/lib/python3.2/distutils/dist.py:257: UserWarning: Unknown
> distribution option: 'zip_safe'
>  warnings.warn(msg)
> /usr/lib/python3.2/distutils/dist.py:257: UserWarning: Unknown
> distribution option: 'entry_points'
>  warnings.warn(msg)
>
>
> ..and the built library gets syntax on errors when run under py3,
> presumably because py3 didn't actually run. Any idea how to get some
> more information on what path is being taken during the build process
> and why it's not invoking the distribute hotness?
>
> Cheers,
>  - Tim.

So I'm not entirely sure *how* this happened, but it seems that after
switching to use the "install" command instead of "build", none of
these problems occur. I'm pretty confused as to why, but certainly not
complaining. If anybody is interested in zero install, I've now got a
working feed for distribute here:
http://gfxmonk.net/dist/0install/python-distribute.xml

If you're curious as tot he point of all this, it means that one can
depend upon distribute to build source packages into python2 and
python3 "binary" implementations (not really binary since they're just
relocatable python code) without having to manually install distribute
for either python 2 or 3. Which I've made use of for the `nosetests`
feed: http://gfxmonk.net/dist/0install/nose.xml

Cheers,
 - Tim.
_______________________________________________
Distutils-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/distutils-sig