Can't count on -S to get a clean Python (at least on mac)

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Can't count on -S to get a clean Python (at least on mac)

Jim Fulton
On my mac (Snow Leopard):

$ which python
/usr/bin/python

$ python -S
Python 2.6.1 (r261:67515, Jun 24 2010, 21:47:49)
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
>>> import setuptools
>>> setuptools.__file__
'/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/setuptools/__init__.pyc'
>>> import os
>>> os.listdir('/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python')
['altgraph', 'altgraph-0.6.8.dev-py2.6.egg-info', 'bdist_mpkg',
'bdist_mpkg-0.4.3.dev-py2.6.egg-info', 'bonjour',
'bonjour_py-0.3-py2.6.egg-info', 'CoreGraphics', 'dateutil',
'easy_install.py', 'easy_install.pyc', 'ez_setup', 'fetchmailconf.py',
'fetchmailconf.pyc', 'fetchmailconf.pyo', 'libsvn', 'macholib',
'macholib-1.2.1.dev-py2.6.egg-info', 'modulegraph',
'modulegraph-0.7.2.dev-py2.6.egg-info', 'numpy',
'numpy-1.2.1-py2.6.egg-info', 'OpenSSL', 'pkg_resources.py',
'pkg_resources.pyc', 'py2app', 'py2app-0.4.2-py2.6.egg-info',
'PyObjC', 'PyObjC.pth', 'pyOpenSSL-0.7-py2.6.egg-info',
'PyRSS2Gen-1.0.0-py2.6.egg-info', 'PyRSS2Gen.py', 'PyRSS2Gen.pyc',
'python_dateutil-1.2-py2.6.egg-info', 'setuptools',
'setuptools-0.6c9-py2.6.egg-info', 'site.py', 'site.pyc', 'site.pyo',
'svn', 'twisted', 'Twisted-8.2.0-py2.6.egg-info',
'wx-2.8-mac-unicode', 'wx.pth', 'wxaddons',
'wxaddons-2.8.8.1-py2.6.egg-info',
'wxPython_common-2.8.8.1-py2.6.egg-info', 'wxversion.py',
'wxversion.pyc', 'xattr', 'xattr-0.5-py2.6.egg-info', 'zope',
'zope.interface-3.5.1-py2.6.egg-info']

whimper

I get something similar on a Lion machine.

I haven't been able to reproduce this lamosity on any linux systems.

Jim

--
Jim Fulton
http://www.linkedin.com/in/jimfulton
Jerky is better than bacon! http://www.dublinstore.com/
_______________________________________________
Distutils-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/distutils-sig
Reply | Threaded
Open this post in threaded view
|

Re: Can't count on -S to get a clean Python (at least on mac)

Ned Deily
In article
<CAPDm-FiEKjVoU=x7gMY8WDLsK0pHw=DD7rE8o7kvW=[hidden email]>,
 Jim Fulton <[hidden email]> wrote:

> On my mac (Snow Leopard):
>
> $ which python
> /usr/bin/python
>
> $ python -S
> Python 2.6.1 (r261:67515, Jun 24 2010, 21:47:49)
> [GCC 4.2.1 (Apple Inc. build 5646)] on darwin
> >>> import setuptools
> >>> setuptools.__file__
> '/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/se
> tuptools/__init__.pyc'
> >>> import os
> >>> os.listdir('/System/Library/Frameworks/Python.framework/Versions/2.6/Extra
> >>> s/lib/python')
> ['altgraph', 'altgraph-0.6.8.dev-py2.6.egg-info', 'bdist_mpkg',
> 'bdist_mpkg-0.4.3.dev-py2.6.egg-info', 'bonjour',
> 'bonjour_py-0.3-py2.6.egg-info', 'CoreGraphics', 'dateutil',
> 'easy_install.py', 'easy_install.pyc', 'ez_setup', 'fetchmailconf.py',
> 'fetchmailconf.pyc', 'fetchmailconf.pyo', 'libsvn', 'macholib',
> 'macholib-1.2.1.dev-py2.6.egg-info', 'modulegraph',
> 'modulegraph-0.7.2.dev-py2.6.egg-info', 'numpy',
> 'numpy-1.2.1-py2.6.egg-info', 'OpenSSL', 'pkg_resources.py',
> 'pkg_resources.pyc', 'py2app', 'py2app-0.4.2-py2.6.egg-info',
> 'PyObjC', 'PyObjC.pth', 'pyOpenSSL-0.7-py2.6.egg-info',
> 'PyRSS2Gen-1.0.0-py2.6.egg-info', 'PyRSS2Gen.py', 'PyRSS2Gen.pyc',
> 'python_dateutil-1.2-py2.6.egg-info', 'setuptools',
> 'setuptools-0.6c9-py2.6.egg-info', 'site.py', 'site.pyc', 'site.pyo',
> 'svn', 'twisted', 'Twisted-8.2.0-py2.6.egg-info',
> 'wx-2.8-mac-unicode', 'wx.pth', 'wxaddons',
> 'wxaddons-2.8.8.1-py2.6.egg-info',
> 'wxPython_common-2.8.8.1-py2.6.egg-info', 'wxversion.py',
> 'wxversion.pyc', 'xattr', 'xattr-0.5-py2.6.egg-info', 'zope',
> 'zope.interface-3.5.1-py2.6.egg-info']
>
> whimper
>
> I get something similar on a Lion machine.
>
> I haven't been able to reproduce this lamosity on any linux systems.

Blame it on Apple's modifications to the system Pythons and the
setuptools they supply in OS X.  The third-party packages they supply
with the system Python are not installed in the same directory where
user-installed site-packages go, for instance.

You won't see this behavior with the Pythons for OS X installed from
python.org:

$ /usr/local/bin/python2.7
Python 2.7.2 (v2.7.2:8527427914a2, Jun 11 2011, 15:22:34)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import setuptools
>>> setuptools.__file__
'/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-pac
kages/distribute-0.6.24-py2.7.egg/setuptools/__init__.pyc'
>>> ^D
$ /usr/local/bin/python2.7 -S
Python 2.7.2 (v2.7.2:8527427914a2, Jun 11 2011, 15:22:34)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
>>> import setuptools
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named setuptools
>>> ^D

--
 Ned Deily,
 [hidden email]

_______________________________________________
Distutils-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/distutils-sig
Reply | Threaded
Open this post in threaded view
|

Re: Can't count on -S to get a clean Python (at least on mac)

Zvezdan Petkovic
In reply to this post by Jim Fulton
I know what you mean when you say clean Python.

However, for vendors, clean Python is the distribution that supports all their features and does not break their promises.  They do not promise Python with no additional extra packages.  For example, Apple has the support for the scripting bridge towards Objective-C libraries, py2app builder, and probably some other features that are obviously important to them.

On Apr 7, 2012, at 4:24 PM, Jim Fulton wrote:

> On my mac (Snow Leopard):
>
> $ which python
> /usr/bin/python
>
> $ python -S
> Python 2.6.1 (r261:67515, Jun 24 2010, 21:47:49)
> [GCC 4.2.1 (Apple Inc. build 5646)] on darwin
>>>> import setuptools
>>>> setuptools.__file__
> '/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/setuptools/__init__.pyc'
>>>


The only thing this shows is that setuptools are in the Extras and that Extras are obviously always in sys.path.

This doesn't mean that site.py was loaded.

Do notice that site-packages are not included with python -S:

$ python
Python 2.6.1 (r261:67515, Jun 24 2010, 21:47:49)
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> '/Library/Python/2.6/site-packages' in sys.path
True
>>> '/Users/zvezdan/Library/Python/2.6/site-packages' in sys.path
True
>>> '/Users/zvezdan/.local/lib/python2.6/site-packages' in sys.path
True
>>> ^D

$ python -S
Python 2.6.1 (r261:67515, Jun 24 2010, 21:47:49)
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
>>> import sys
>>> '/Library/Python/2.6/site-packages' in sys.path
False
>>> '/Users/zvezdan//Library/Python/2.6/site-packages' in sys.path
False
>>> '/Users/zvezdan/.local/lib/python2.6/site-packages' in sys.path
False
>>> ^D

The python2.6 man page says this:

-S Disable the import of the module  site  and  the site-dependent
        manipulations of sys.path that it entails.

According to the tests above that promise is fulfilled.
Nothing site dependent was loaded.

> whimper

Your expectations of the -S option might be too high. :-)

> I haven't been able to reproduce this lamosity on any linux systems.


Probably because they are not trying to support a scripting bridge or an app builder.  Perhaps they do, but don't care if it breaks when -S is used. :-)  I'm pretty sure I've seen python installations on Linux that had their own warts.

I know you are trying to do something with this (probably for buildout) and that not being able to rely on -S is frustrating, to put it mildly.
I'm not arguing that, I just tried to point out what I thought was behind this behavior.

Regards,

        Zvezdan

_______________________________________________
Distutils-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/distutils-sig
Reply | Threaded
Open this post in threaded view
|

Re: Can't count on -S to get a clean Python (at least on mac)

Jim Fulton
On Sat, Apr 7, 2012 at 6:42 PM, Zvezdan Petkovic <[hidden email]> wrote:
> I know what you mean when you say clean Python.
>
> However, for vendors, clean Python is the distribution that supports
  all their features and does not break their promises.  They do not
  promise Python with no additional extra packages.  For example,
  Apple has the support for the scripting bridge towards Objective-C
  libraries, py2app builder, and probably some other features that are
  obviously important to them.

Yup.  In some ways, a vendor-supplied (aka "system") Python is like an
application or application platform with lots of specific components
that the OS may depend on.

>
> On Apr 7, 2012, at 4:24 PM, Jim Fulton wrote:
>
>> On my mac (Snow Leopard):
>>
>> $ which python
>> /usr/bin/python
>>
>> $ python -S
>> Python 2.6.1 (r261:67515, Jun 24 2010, 21:47:49)
>> [GCC 4.2.1 (Apple Inc. build 5646)] on darwin
>>>>> import setuptools
>>>>> setuptools.__file__
>> '/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/setuptools/__init__.pyc'
>>>>
>
>
> The only thing this shows is that setuptools are in the Extras and
  that Extras are obviously always in sys.path.

Yup.

>
> This doesn't mean that site.py was loaded.

I never said it was.

> Do notice that site-packages are not included with python -S:

But, as you and I pointed out, Extras is and contains lots of
non-standard stuff that you can't avoid using a platform-independent
mechanism.

...

>
> The python2.6 man page says this:
>
> -S   Disable the import of the module site and the site-dependent
>    manipulations of sys.path that it entails.
>
> According to the tests above that promise is fulfilled.
> Nothing site dependent was loaded.

My point was that using -S didn't yield a (even a relatively) clean
Python. (Check the subject of this thread.)

For years, some people have been confused and annoyed at my assertion
that applications shouldn't be built on system Pythons because the
results are too unpredictable.

At the Python language summit in Atlanta last year, I suggested that
it would be useful to define a standard for a baseline Python that
applications could depend on. System packagers could provide this
alongside their loaded Python installations. At the time, it was
suggested that all that was needed was to to use -S to avoid
system-supplied custom packages. There was even a change made in 3.3
to make the -S option more reliable by allowing useful utilities
previously available in site.py available without importing it.
(In most versions of Python, even if -S is used, site.py may get
imported by something trying to get at one of these utilities.)

Of course, there were other issues like unicode sizes (seemingly
addresses by recent unicode enhancements) and other build options, but
it was thought that extra libraries were the biggest issue.

>> whimper
>
> Your expectations of the -S option might be too high. :-)

My expectations weren't too high, but I suspect other's are.

>> I haven't been able to reproduce this lamosity on any linux systems.
>
>
> Probably because they are not trying to support a scripting bridge
  or an app builder. Perhaps they do, but don't care if it breaks when
  -S is used. :-) I'm pretty sure I've seen python installations on
  Linux that had their own warts.

I think it's more likely because they used the correct mechanism and
add extras to the path in site.py.  I would call the Apple packaging a
bug.

> I know you are trying to do something with this (probably for
> buildout) and that not being able to rely on -S is frustrating, to
> put it mildly.  I'm not arguing that, I just tried to point out what
> I thought was behind this behavior.

Buildout 1.5 tries hard to leverage -S to provide isolation.  The new
buildout will as well, although it won't be very effective.

The position of the new (simpler and more maintainable buildout) is
that buildout will provide facilities for working with system Pythons,
but it will make few if any promises and it won't work nearly as hard
as zc.buildout 1.5 does.  The position will be that if you're
building non-trivial applications, you'd better use a clean Python.

I still think it would be useful for the Python community/PSF to
define a "clean" or baseline Python for applications and work out how
such a thing could live alongside a loaded/extra-batteries-included
installation.

Jim

--
Jim Fulton
http://www.linkedin.com/in/jimfulton
Jerky is better than bacon! http://www.dublinstore.com/
_______________________________________________
Distutils-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/distutils-sig