Installing subpackages under common superpackage

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Installing subpackages under common superpackage

Otto Maddox
Hi,

I have a Python project which follows a client-server architecture. It
is natural to develop it as two different projects and in two different
source distributions, myproject-server.tar.gz and
myproject-client.tar.gz.

Neither package depends on the other, but because they are part of a
larger, overarching project, I want install each of them so that they
appear as subpackages (myproject.server and myproject.client) under a
common superpackage (myproject). I am looking for the cleanest and most
correct way to do this using distutils and setup.py, so that I can end
up with a structure like this:

site-packages/myproject
site-packages/myproject/__init__.py

site-packages/myproject/server/__init__.py
site-packages/myproject/server/file1.py
site-packages/myproject/server/file2.py

site-packages/myproject/client/__init__.py
site-packages/myproject/client/file3.py
site-packages/myproject/client/file4.py

and so that any of the following commands (and their standard
variations) work:

import myproject
import myproject.server
import myproject.client

Is there a way to write distutils/setup.py to do this?

I have tried something like packages = ["myproject", "myproject.client"]
in the client's setup.py and packages = ["myproject",
"myproject.server"] in the server's setup.py. This kind of works, but it
feels wrong because the second package which gets installed will
overwrite files from the first -- for example, there is no clean way to
install myproject/__init__.py.

I can't imagine that I'm the first person to want to address this use
case. What would be the best way of doing this?

Finally, does anybody have any examples of other projects which do a
similar thing? Thanks.

--
http://www.fastmail.fm - Email service worth paying for. Try it for free

_______________________________________________
Distutils-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/distutils-sig
Reply | Threaded
Open this post in threaded view
|

Re: Installing subpackages under common superpackage

Rob Healey
Dear Otto:

The only way that I could tell you about a project that kind of does what you are looking for is already done somewhat in Distutils2-1.0a4 tarball...

You may get the tarball from here, and look at its setup.py and setup.cfg files...

http://pypi.python.org/pypi/Distutils2

Sincerely yours,
Rob G. Healey




On Tue, Apr 3, 2012 at 8:35 AM, Otto Maddox <[hidden email]> wrote:
Hi,

I have a Python project which follows a client-server architecture. It
is natural to develop it as two different projects and in two different
source distributions, myproject-server.tar.gz and
myproject-client.tar.gz.

Neither package depends on the other, but because they are part of a
larger, overarching project, I want install each of them so that they
appear as subpackages (myproject.server and myproject.client) under a
common superpackage (myproject). I am looking for the cleanest and most
correct way to do this using distutils and setup.py, so that I can end
up with a structure like this:

site-packages/myproject
site-packages/myproject/__init__.py

site-packages/myproject/server/__init__.py
site-packages/myproject/server/file1.py
site-packages/myproject/server/file2.py

site-packages/myproject/client/__init__.py
site-packages/myproject/client/file3.py
site-packages/myproject/client/file4.py

and so that any of the following commands (and their standard
variations) work:

import myproject
import myproject.server
import myproject.client

Is there a way to write distutils/setup.py to do this?

I have tried something like packages = ["myproject", "myproject.client"]
in the client's setup.py and packages = ["myproject",
"myproject.server"] in the server's setup.py. This kind of works, but it
feels wrong because the second package which gets installed will
overwrite files from the first -- for example, there is no clean way to
install myproject/__init__.py.

I can't imagine that I'm the first person to want to address this use
case. What would be the best way of doing this?

Finally, does anybody have any examples of other projects which do a
similar thing? Thanks.

--
http://www.fastmail.fm - Email service worth paying for. Try it for free

_______________________________________________
Distutils-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/distutils-sig



--
Sincerely yours,
Rob G. Healey



_______________________________________________
Distutils-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/distutils-sig
Reply | Threaded
Open this post in threaded view
|

Re: Installing subpackages under common superpackage

Jonathan Ballet
In reply to this post by Otto Maddox
On Tue, Apr 03, 2012 at 04:35:17PM +0100, Otto Maddox wrote:

> Hi,
>
> I have a Python project which follows a client-server architecture. It
> is natural to develop it as two different projects and in two different
> source distributions, myproject-server.tar.gz and
> myproject-client.tar.gz.
>
> Neither package depends on the other, but because they are part of a
> larger, overarching project, I want install each of them so that they
> appear as subpackages (myproject.server and myproject.client) under a
> common superpackage (myproject). I am looking for the cleanest and most
> correct way to do this using distutils and setup.py, so that I can end
> up with a structure like this:
>
> site-packages/myproject
> site-packages/myproject/__init__.py
>
> site-packages/myproject/server/__init__.py
> site-packages/myproject/server/file1.py
> site-packages/myproject/server/file2.py
>
> site-packages/myproject/client/__init__.py
> site-packages/myproject/client/file3.py
> site-packages/myproject/client/file4.py
>
> and so that any of the following commands (and their standard
> variations) work:
>
> import myproject
> import myproject.server
> import myproject.client
>
> Is there a way to write distutils/setup.py to do this?

You would probably interested about setuptools' namespace support for
doing this, you can have a look at
http://packages.python.org/distribute/setuptools.html#namespace-packages

Additionnaly, if you are looking for examples, the Zope community is
well known for the use of namespaced packages - have a look at the
zope.,  zc., z3c., collective., plone., etc. namespaces.

 Jonathan

_______________________________________________
Distutils-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/distutils-sig
Reply | Threaded
Open this post in threaded view
|

Re: Installing subpackages under common superpackage

Otto Maddox
In reply to this post by Rob Healey
Hi Rob,
 
Thank you for your answer. Could you please point me to the specific functionality/documentation which I need to look at?
 
Andrew
 
 
On Tue, Apr 3, 2012, at 08:53 AM, Rob Healey wrote:
Dear Otto:

The only way that I could tell you about a project that kind of does what you are looking for is already done somewhat in Distutils2-1.0a4 tarball...

You may get the tarball from here, and look at its setup.py and setup.cfg files...

http://pypi.python.org/pypi/Distutils2

Sincerely yours,
Rob G. Healey




On Tue, Apr 3, 2012 at 8:35 AM, Otto Maddox <[hidden email]> wrote:
Hi,

I have a Python project which follows a client-server architecture. It
is natural to develop it as two different projects and in two different
source distributions, myproject-server.tar.gz and
myproject-client.tar.gz.

Neither package depends on the other, but because they are part of a
larger, overarching project, I want install each of them so that they
appear as subpackages (myproject.server and myproject.client) under a
common superpackage (myproject). I am looking for the cleanest and most
correct way to do this using distutils and setup.py, so that I can end
up with a structure like this:

site-packages/myproject
site-packages/myproject/__init__.py

site-packages/myproject/server/__init__.py
site-packages/myproject/server/file1.py
site-packages/myproject/server/file2.py

site-packages/myproject/client/__init__.py
site-packages/myproject/client/file3.py
site-packages/myproject/client/file4.py

and so that any of the following commands (and their standard
variations) work:

import myproject
import myproject.server
import myproject.client

Is there a way to write distutils/setup.py to do this?

I have tried something like packages = ["myproject", "myproject.client"]
in the client's setup.py and packages = ["myproject",
"myproject.server"] in the server's setup.py. This kind of works, but it
feels wrong because the second package which gets installed will
overwrite files from the first -- for example, there is no clean way to
install myproject/__init__.py.

I can't imagine that I'm the first person to want to address this use
case. What would be the best way of doing this?

Finally, does anybody have any examples of other projects which do a
similar thing? Thanks.

--
http://www.fastmail.fm - Email service worth paying for. Try it for free

_______________________________________________
Distutils-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/distutils-sig



--
Sincerely yours,
Rob G. Healey

 
 
-- 
http://www.fastmail.fm - Does exactly what it says on the tin

_______________________________________________
Distutils-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/distutils-sig
Reply | Threaded
Open this post in threaded view
|

Re: Installing subpackages under common superpackage

Otto Maddox
In reply to this post by Jonathan Ballet
On Tue, Apr 3, 2012, at 11:41 PM, Jonathan Ballet wrote:

> On Tue, Apr 03, 2012 at 04:35:17PM +0100, Otto Maddox wrote:
> > I have a Python project which follows a client-server architecture. It
> > is natural to develop it as two different projects and in two different
> > source distributions, myproject-server.tar.gz and
> > myproject-client.tar.gz.
> >
> > Neither package depends on the other, but because they are part of a
> > larger, overarching project, I want install each of them so that they
> > appear as subpackages (myproject.server and myproject.client) under a
> > common superpackage (myproject). I am looking for the cleanest and most
> > correct way to do this using distutils and setup.py, so that I can end
> > up with a structure like this:

[snip]

> You would probably interested about setuptools' namespace support for
> doing this, you can have a look at
> http://packages.python.org/distribute/setuptools.html#namespace-packages

That looks like it might be do the trick. I will evaluate this sort of
solution, and compare it with Rob's suggestion to use distutils2.

> Additionnaly, if you are looking for examples, the Zope community is
> well known for the use of namespaced packages - have a look at the
> zope.,  zc., z3c., collective., plone., etc. namespaces.

Thank you.

--
http://www.fastmail.fm - Faster than the air-speed velocity of an
                          unladen european swallow

_______________________________________________
Distutils-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/distutils-sig
Reply | Threaded
Open this post in threaded view
|

Re: Installing subpackages under common superpackage

Éric Araujo
In reply to this post by Rob Healey
Hi,

Le 03/04/2012 11:53, Rob Healey a écrit :
> The only way that I could tell you about a project that kind of does what
> you are looking for is already done somewhat in Distutils2-1.0a4 tarball...
This is incorrect.  What Otto asked for is called namespace packages and
its status is a bit complicated.

- If you use only the standard library, then
http://docs.python.org/library/pkgutil#pkgutil.extend_path can be used.
  On the packaging side, you can use distutils to package your two
projects and the second installed will overwrite __init__.py, which is
okay given that the contents will be the same.

- Most people use the flavor of namespace packages provided by
setuptools.  (setuptools is a set of extensions to distutils which
people tend to love or hate.)  I don’t remember how it differs from the
pkgutil flavor, but the docs for setuptools and distribute (a setuptools
fork) should explain it.

- There is an ongoing discussion to add better namespace packages in
Python 3.3.  When it’s done, packaging/distutils2 will add support for it.

Cheers
_______________________________________________
Distutils-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/distutils-sig