shrink app bundle size (removing unused stuff)?

classic Classic list List threaded Threaded
18 messages Options
Reply | Threaded
Open this post in threaded view
|

shrink app bundle size (removing unused stuff)?

guano
Hi there,

I've been reasonably successful in building an app bundle with py2app.
I'm running OS X Lion, python 2.7.2 (from python.org), wxpython
2.9beta, and my app
uses matplotlib and numpy.

Using py2exe, the final folder has 14MB, or 11MB compressed.

Using py2app, the .app has 250MB (!) or 100MB compressed with Disk utility

Examining the Contents of the .app, I can see that there's A LOT of
stuff I don't need,
like Editra (11MB), Locales, or Matplotlib "tests" folder (38MB)

So, how could I remove these itens from my app?
I tried the 'excludes' option in py2app setup.py, but without any luck.

Also, what are those "codecs" in lib-dynload? I have no idea what they are.

my setup.py follows. (BTW, I tried to run it without any reference to
'wx' but then
the app crashes)

many thanks

Carlos





from setuptools import setup
from glob import glob

import shutil
shutil.rmtree("build", ignore_errors=True)
shutil.rmtree("dist", ignore_errors=True)

APPNAME = 'OpenStereo'
APP = ['resources/OpenStereo.py']
VERSION = '0.1.2g' # must be in X.X.X format

DATA_FILES = ['license/gpl-3.0.txt'] #[glob(r'resources/*')]

OPTIONS = {
            'plist':dict(
            CFBundleName                = 'OpenStereo',
            CFBundleShortVersionString  = VERSION,
            CFBundleVersion             = VERSION,
            CFBundleGetInfoString       = 'OpenStereo, Open-Source
Structural Geology Analysis',
            CFBundleExecutable          = 'OpenStereo',
            CFBundleIdentifier          = "br.usp.igc",
            NSHumanReadableCopyright    = 'Carlos H. Grohmann'),

            'argv_emulation': True,
            'optimize': 1,
            'compressed': True,
            'strip': True,
            'semi_standalone': False,
            'packages': ['wx'],
            'dylib_excludes': ['Tcl.framework','Tk.framework'],
            'resources': ['icons/openstereo_icon_noname_256x256x32.png'],
            'iconfile': 'icons/openstereo_icon_noname.icns',
            }

setup(
    app=APP,
    data_files=DATA_FILES,
    options={'py2app': OPTIONS},
    setup_requires=['py2app'],
)






--
Prof. Carlos Henrique Grohmann - Geologist D.Sc.
Institute of Geosciences - Univ. of São Paulo, Brazil
---
http://www.igc.usp.br/pessoais/guano
http://digitalelevation.wordpress.com/
http://lattes.cnpq.br/5846052449613692 (CV)
---
Twitter: @CarlosGrohmann
http://carlosgrohmann.tumblr.com/
________________
Can’t stop the signal.
_______________________________________________
Pythonmac-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: http://mail.python.org/mailman/options/Pythonmac-SIG
Reply | Threaded
Open this post in threaded view
|

Re: shrink app bundle size (removing unused stuff)?

Kevin Walzer-5
On 3/7/12 6:09 PM, Carlos Grohmann wrote:

> Hi there,
>
> I've been reasonably successful in building an app bundle with py2app.
> I'm running OS X Lion, python 2.7.2 (from python.org), wxpython
> 2.9beta, and my app
> uses matplotlib and numpy.
>
> Using py2exe, the final folder has 14MB, or 11MB compressed.
>
> Using py2app, the .app has 250MB (!) or 100MB compressed with Disk utility
>
> Examining the Contents of the .app, I can see that there's A LOT of
> stuff I don't need,
> like Editra (11MB), Locales, or Matplotlib "tests" folder (38MB)
>
> So, how could I remove these itens from my app?
> I tried the 'excludes' option in py2app setup.py, but without any luck.
>
> Also, what are those "codecs" in lib-dynload? I have no idea what they are.
>
> my setup.py follows. (BTW, I tried to run it without any reference to
> 'wx' but then
> the app crashes)
>
> many thanks
>
> Carlos
>

I hope this isn't stating the obvious, but how about simply removing the
unwanted bits after running setup.py? I run my py2app builds from a
shell script because I have to do some additional manipulation of the
app bundle. Here's an excerpt from my build script:

#!/bin/sh

echo "Starting dist process..."


if [ -a cbk/Phynchronicity.dmg ]
then
rm cbk/Phynchronicity.dmg
fi

if [ -a cbk/Phynchronicity.app ]
then
rm -rf cbk/Phynchronicity.app
fi

python2.7 setup.py py2app

mkdir dist/Phynchronicity.app/Contents/lib

#copy the supporting libraries
for lib in libs/*
do
   echo "Copying $lib library..."
   cp -R -f $lib dist/Phynchronicity.app/Contents/lib
done


cp -rf dist/Phynchronicity.app cbk/Phynchronicity.app
ditto --rsrc  --arch i386 --arch x86_64 cbk/Phynchronicity.app
cbk/Phynchronicity_intel.app
rm -rf cbk/Phynchronicity.app
mv cbk/Phynchronicity_intel.app cbk/Phynchronicity.app


...and so on.

Hope this helps,
Kevin

--
Kevin Walzer
Code by Kevin
http://www.codebykevin.com
_______________________________________________
Pythonmac-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: http://mail.python.org/mailman/options/Pythonmac-SIG
Reply | Threaded
Open this post in threaded view
|

Re: shrink app bundle size (removing unused stuff)?

Ronald Oussoren
In reply to this post by guano

On 7 Mar, 2012, at 15:09, Carlos Grohmann wrote:

> Hi there,
>
> I've been reasonably successful in building an app bundle with py2app.
> I'm running OS X Lion, python 2.7.2 (from python.org), wxpython
> 2.9beta, and my app
> uses matplotlib and numpy.
>
> Using py2exe, the final folder has 14MB, or 11MB compressed.
>
> Using py2app, the .app has 250MB (!) or 100MB compressed with Disk utility
>
> Examining the Contents of the .app, I can see that there's A LOT of
> stuff I don't need,
> like Editra (11MB), Locales, or Matplotlib "tests" folder (38MB)
matplatlib is included as is because it won't work when you include only the bits that are referenced in the "import" graph, I'll check how hard it will be to exclude the tests in this copy.

The only editra I could find was <http://editra.org/> which is an editor, I don't know why that gets copied into your application. How did you install Editra, did you use the binary DMG on the website, or did you install from source?

I don't know what "Locales" is at all. Is there a python package on  your system that is named "Locales"?

>
> So, how could I remove these itens from my app?
> I tried the 'excludes' option in py2app setup.py, but without any luck.

That this didn't work for matplotlib is expected, a py2app recipe for matplotlib overrides your exclude option.
>
> Also, what are those "codecs" in lib-dynload? I have no idea what they are.

The codecs are the stdlib support for translating between bytestrings and unicode in various encodings.  These are implicitly imported when you use the codecs module (either directory or through somestring.decoe("encoding").  Py2app cannot detect if you do this and therefore conservatively includes all encodings just in case they are needed.  Adding a way to specify which encodings should be included would be nice, I'm not sure when I'll get around to implementing this.

>
> my setup.py follows. (BTW, I tried to run it without any reference to
> 'wx' but then
> the app crashes)

Can you reproduce this using a smaller example?  

BTW. Are you trying to package this OpenStereo: http://www.igc.usp.br/index.php?id=openstereo. If so, I can try to reproduce the issue on my machine.  

Ronald

>
> many thanks
>
> Carlos
>
>
>
>
>
> from setuptools import setup
> from glob import glob
>
> import shutil
> shutil.rmtree("build", ignore_errors=True)
> shutil.rmtree("dist", ignore_errors=True)
>
> APPNAME = 'OpenStereo'
> APP = ['resources/OpenStereo.py']
> VERSION = '0.1.2g' # must be in X.X.X format
>
> DATA_FILES = ['license/gpl-3.0.txt'] #[glob(r'resources/*')]
>
> OPTIONS = {
>            'plist':dict(
>            CFBundleName                = 'OpenStereo',
>            CFBundleShortVersionString  = VERSION,
>            CFBundleVersion             = VERSION,
>            CFBundleGetInfoString       = 'OpenStereo, Open-Source
> Structural Geology Analysis',
>            CFBundleExecutable          = 'OpenStereo',
>            CFBundleIdentifier          = "br.usp.igc",
>            NSHumanReadableCopyright    = 'Carlos H. Grohmann'),
>
>            'argv_emulation': True,
>            'optimize': 1,
>            'compressed': True,
>            'strip': True,
>            'semi_standalone': False,
>            'packages': ['wx'],
>            'dylib_excludes': ['Tcl.framework','Tk.framework'],
>            'resources': ['icons/openstereo_icon_noname_256x256x32.png'],
>            'iconfile': 'icons/openstereo_icon_noname.icns',
>            }
>
> setup(
>    app=APP,
>    data_files=DATA_FILES,
>    options={'py2app': OPTIONS},
>    setup_requires=['py2app'],
> )
>
>
>
>
>
>
> --
> Prof. Carlos Henrique Grohmann - Geologist D.Sc.
> Institute of Geosciences - Univ. of São Paulo, Brazil
> ---
> http://www.igc.usp.br/pessoais/guano
> http://digitalelevation.wordpress.com/
> http://lattes.cnpq.br/5846052449613692 (CV)
> ---
> Twitter: @CarlosGrohmann
> http://carlosgrohmann.tumblr.com/
> ________________
> Can’t stop the signal.
> _______________________________________________
> Pythonmac-SIG maillist  -  [hidden email]
> http://mail.python.org/mailman/listinfo/pythonmac-sig
> unsubscribe: http://mail.python.org/mailman/options/Pythonmac-SIG

_______________________________________________
Pythonmac-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: http://mail.python.org/mailman/options/Pythonmac-SIG

smime.p7s (6K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: shrink app bundle size (removing unused stuff)?

Chris Barker
> The only editra I could find was <http://editra.org/> which is an editor, I don't know why that gets copied into your application. How did you install Editra, did you use the binary DMG on the website, or did you install from source?

Editra is shipped with wxPython -- so it may have snuck in that way.

But I agree with Kevin, while the more we can clean this up with
py2app, it's an imperfect system -- I"ve generally ended with some
custom code to clean things out a bit after a build -- I write mine in
Python, and put it at the end of the setup.py, but the idea is the
same.

-Chris

> I don't know what "Locales" is at all. Is there a python package on  your system that is named "Locales"?
>
>>
>> So, how could I remove these itens from my app?
>> I tried the 'excludes' option in py2app setup.py, but without any luck.
>
> That this didn't work for matplotlib is expected, a py2app recipe for matplotlib overrides your exclude option.
>>
>> Also, what are those "codecs" in lib-dynload? I have no idea what they are.
>
> The codecs are the stdlib support for translating between bytestrings and unicode in various encodings.  These are implicitly imported when you use the codecs module (either directory or through somestring.decoe("encoding").  Py2app cannot detect if you do this and therefore conservatively includes all encodings just in case they are needed.  Adding a way to specify which encodings should be included would be nice, I'm not sure when I'll get around to implementing this.
>
>>
>> my setup.py follows. (BTW, I tried to run it without any reference to
>> 'wx' but then
>> the app crashes)
>
> Can you reproduce this using a smaller example?
>
> BTW. Are you trying to package this OpenStereo: http://www.igc.usp.br/index.php?id=openstereo. If so, I can try to reproduce the issue on my machine.
>
> Ronald
>
>>
>> many thanks
>>
>> Carlos
>>
>>
>>
>>
>>
>> from setuptools import setup
>> from glob import glob
>>
>> import shutil
>> shutil.rmtree("build", ignore_errors=True)
>> shutil.rmtree("dist", ignore_errors=True)
>>
>> APPNAME = 'OpenStereo'
>> APP = ['resources/OpenStereo.py']
>> VERSION = '0.1.2g' # must be in X.X.X format
>>
>> DATA_FILES = ['license/gpl-3.0.txt'] #[glob(r'resources/*')]
>>
>> OPTIONS = {
>>            'plist':dict(
>>            CFBundleName                = 'OpenStereo',
>>            CFBundleShortVersionString  = VERSION,
>>            CFBundleVersion             = VERSION,
>>            CFBundleGetInfoString       = 'OpenStereo, Open-Source
>> Structural Geology Analysis',
>>            CFBundleExecutable          = 'OpenStereo',
>>            CFBundleIdentifier          = "br.usp.igc",
>>            NSHumanReadableCopyright    = 'Carlos H. Grohmann'),
>>
>>            'argv_emulation': True,
>>            'optimize': 1,
>>            'compressed': True,
>>            'strip': True,
>>            'semi_standalone': False,
>>            'packages': ['wx'],
>>            'dylib_excludes': ['Tcl.framework','Tk.framework'],
>>            'resources': ['icons/openstereo_icon_noname_256x256x32.png'],
>>            'iconfile': 'icons/openstereo_icon_noname.icns',
>>            }
>>
>> setup(
>>    app=APP,
>>    data_files=DATA_FILES,
>>    options={'py2app': OPTIONS},
>>    setup_requires=['py2app'],
>> )
>>
>>
>>
>>
>>
>>
>> --
>> Prof. Carlos Henrique Grohmann - Geologist D.Sc.
>> Institute of Geosciences - Univ. of São Paulo, Brazil
>> ---
>> http://www.igc.usp.br/pessoais/guano
>> http://digitalelevation.wordpress.com/
>> http://lattes.cnpq.br/5846052449613692 (CV)
>> ---
>> Twitter: @CarlosGrohmann
>> http://carlosgrohmann.tumblr.com/
>> ________________
>> Can’t stop the signal.
>> _______________________________________________
>> Pythonmac-SIG maillist  -  [hidden email]
>> http://mail.python.org/mailman/listinfo/pythonmac-sig
>> unsubscribe: http://mail.python.org/mailman/options/Pythonmac-SIG
>
>
> _______________________________________________
> Pythonmac-SIG maillist  -  [hidden email]
> http://mail.python.org/mailman/listinfo/pythonmac-sig
> unsubscribe: http://mail.python.org/mailman/options/Pythonmac-SIG
>



--

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

[hidden email]
_______________________________________________
Pythonmac-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: http://mail.python.org/mailman/options/Pythonmac-SIG
Reply | Threaded
Open this post in threaded view
|

Re: shrink app bundle size (removing unused stuff)?

Ronald Oussoren

On 8 Mar, 2012, at 22:51, Chris Barker wrote:

>> The only editra I could find was <http://editra.org/> which is an editor, I don't know why that gets copied into your application. How did you install Editra, did you use the binary DMG on the website, or did you install from source?
>
> Editra is shipped with wxPython -- so it may have snuck in that way.
>
> But I agree with Kevin, while the more we can clean this up with
> py2app, it's an imperfect system -- I"ve generally ended with some
> custom code to clean things out a bit after a build -- I write mine in
> Python, and put it at the end of the setup.py, but the idea is the
> same.

Py2app will never be able to do everything, but I'd prefer to get closer than we're currently. Are there things you have to do every time that could (easily) be expressed as configuration for py2app?  It seems that python is slowly moving towards a more declarative way of specifying builds using a setup.cfg file instead of setup.py though the distutils2/packaging project and py2app should do the same when that happens (without dropping support for setup.py files of course).

Ronald


_______________________________________________
Pythonmac-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: http://mail.python.org/mailman/options/Pythonmac-SIG

smime.p7s (6K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: shrink app bundle size (removing unused stuff)?

guano
I think that if the 'excludes' weren't overwritten by the
wx/numpy/matplotlib/etc recipes, it would be easier to fine-tune our
options (maybe run the recipes first and aftrer run the excludes?)

best

Carlos



On Fri, Mar 9, 2012 at 12:22, Ronald Oussoren <[hidden email]> wrote:

>
> On 8 Mar, 2012, at 22:51, Chris Barker wrote:
>
>>> The only editra I could find was <http://editra.org/> which is an editor, I don't know why that gets copied into your application. How did you install Editra, did you use the binary DMG on the website, or did you install from source?
>>
>> Editra is shipped with wxPython -- so it may have snuck in that way.
>>
>> But I agree with Kevin, while the more we can clean this up with
>> py2app, it's an imperfect system -- I"ve generally ended with some
>> custom code to clean things out a bit after a build -- I write mine in
>> Python, and put it at the end of the setup.py, but the idea is the
>> same.
>
> Py2app will never be able to do everything, but I'd prefer to get closer than we're currently. Are there things you have to do every time that could (easily) be expressed as configuration for py2app?  It seems that python is slowly moving towards a more declarative way of specifying builds using a setup.cfg file instead of setup.py though the distutils2/packaging project and py2app should do the same when that happens (without dropping support for setup.py files of course).
>
> Ronald
>



--
Prof. Carlos Henrique Grohmann - Geologist D.Sc.
Institute of Geosciences - Univ. of São Paulo, Brazil
---
http://www.igc.usp.br/pessoais/guano
http://digitalelevation.wordpress.com/
http://lattes.cnpq.br/5846052449613692 (CV)
---
Twitter: @CarlosGrohmann
http://carlosgrohmann.tumblr.com/
________________
Can’t stop the signal.
_______________________________________________
Pythonmac-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: http://mail.python.org/mailman/options/Pythonmac-SIG
Reply | Threaded
Open this post in threaded view
|

Re: shrink app bundle size (removing unused stuff)?

Ronald Oussoren

On 9 Mar, 2012, at 10:55, Carlos Grohmann wrote:

> I think that if the 'excludes' weren't overwritten by the
> wx/numpy/matplotlib/etc recipes, it would be easier to fine-tune our
> options (maybe run the recipes first and aftrer run the excludes?)

The matplotlib issue seems to be a bug that's not related to recipes: py2app copies package data into the application bundle, and this code accidently also copied all subpackages like the matplotlib.test package.   I have a patch, but need to test if this actually fixes the bug before I commit.

Ronald


_______________________________________________
Pythonmac-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: http://mail.python.org/mailman/options/Pythonmac-SIG

smime.p7s (6K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: shrink app bundle size (removing unused stuff)?

Ronald Oussoren

On 9 Mar, 2012, at 16:33, Ronald Oussoren wrote:

>
> On 9 Mar, 2012, at 10:55, Carlos Grohmann wrote:
>
>> I think that if the 'excludes' weren't overwritten by the
>> wx/numpy/matplotlib/etc recipes, it would be easier to fine-tune our
>> options (maybe run the recipes first and aftrer run the excludes?)
>
> The matplotlib issue seems to be a bug that's not related to recipes: py2app copies package data into the application bundle, and this code accidently also copied all subpackages like the matplotlib.test package.   I have a patch, but need to test if this actually fixes the bug before I commit.

It didn't, but I did make some progress:

1) I added a py2app recipe that does the right thing with the pubsub library in wxPython. That library uses __path__ hacks and that confuses modulegraph (and hence py2app). Because of this you no longer have to explicitly include 'wx' in the application bundle

2) I tweaked the matplotlib recipe in py2app, it no longer copies the entire package but adds a hook to tell matplotlib where its data is. This solution is not fully complete yet, I'm currently including mpl-data twice (once in Resources, once in site-packages.zip) and the later copy is not needed.

With this the application bundle already shrinks a little (curently 123 MByte, before I started it was about 160 MByte). About 70 Mbyte of this is wx itself (the wxWidgets library and wxPython extensions). Another 21 MByte is because all of numpy is included, not just the bits that are actually needed. That's because of another py2app recipe which can hopefully be improved.

I just pushed updates to modulegraph and py2app to my bitbucket repositories.

Ronald

>
> Ronald
>
> _______________________________________________
> Pythonmac-SIG maillist  -  [hidden email]
> http://mail.python.org/mailman/listinfo/pythonmac-sig
> unsubscribe: http://mail.python.org/mailman/options/Pythonmac-SIG


_______________________________________________
Pythonmac-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: http://mail.python.org/mailman/options/Pythonmac-SIG

smime.p7s (6K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: shrink app bundle size (removing unused stuff)?

Ronald Oussoren

On 10 Mar, 2012, at 8:47, Ronald Oussoren wrote:

> With this the application bundle already shrinks a little (curently 123 MByte, before I started it was about 160 MByte). About 70 Mbyte of this is wx itself (the wxWidgets library and wxPython extensions). Another 21 MByte is because all of numpy is included, not just the bits that are actually needed. That's because of another py2app recipe which can hopefully be improved.

BTW. The size of the binaries is at least partially due to the support for multiple architectures, the python installer contains support for two architectures and wx even includes support for 3 different architectures.

Ronald


_______________________________________________
Pythonmac-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: http://mail.python.org/mailman/options/Pythonmac-SIG

smime.p7s (6K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: shrink app bundle size (removing unused stuff)?

Chris Barker
In reply to this post by Ronald Oussoren
On Fri, Mar 9, 2012 at 7:22 AM, Ronald Oussoren
> Py2app will never be able to do everything, but I'd prefer to get closer than we're currently.

A laudable goal, for sure -- and we much appreciate your work on py2app.

> Are there things you have to do every time that could
> (easily) be expressed as configuration for py2app?

As suggested, 'include', 'excludes', and 'packages' could perhaps be
more robust.

I haven't checked recently -- have you fixed the issue where you had
to explicitly include the parent package when you want to includ ea
sub-package?

ie. you want to be able to :

import libA.libB

(but nothing else in libA)

IN the past, I've had to do:

packages = ['libA', 'libA.libB']

ANother major thought:

I've thought that maybe we need a way to build a module graph from
run-time information, rather than code inspection -- that would
address both:

importing packages you don't need
and
fancy dynamic imports.

The trick is that you'd need to have a way to run your code that
guaranteed that eveything ws imported that might be needed (i.e.
dynamci import), but it seems that you'd want test code that did that
anyway.

I may be missing something, but it seems all you'd need to do is look
at sys.modules, and write out a modulegraph (or would it even need to
be  graph? a simple list would do...)

Then Py2app would need a way to load the modulegraph (or list) from a file.

Even without a runtime inspector -- this could be handy as you could:

1) only re-run the code inspection part when you knew you'd added a
module, rather than on every build

2) you could tweak the module graph by hand and re-use it, rather than
tweaking the resulting package.

Anyway, maybe too much architectural work for now...

> 1) I added a py2app recipe that does the right thing
> with the pubsub library in wxPython. That library uses
> __path__ hacks and that confuses modulegraph (and
> hence py2app). Because of this you no longer have to
> explicitly include 'wx' in the application bundle

Very nice! I happen to need that for a project right now.

> 2) I tweaked the matplotlib recipe in py2app, it no
> longer copies the entire package but adds a hook to
> tell matplotlib where its data is. This solution is not fully
> complete yet, I'm currently including mpl-data twice
> (once in Resources, once in site-packages.zip) and
> the later copy is not needed.

also nice -- did you use:

matplotlib.get_py2exe_datafiles()

That is handy for py2exe.

Thanks for you work...

-Chris













  It seems that python is slowly moving towards a more declarative way
of specifying builds using a setup.cfg file instead of setup.py though
the distutils2/packaging project and py2app should do the same when
that happens (without dropping support for setup.py files of course).
>
> Ronald
>



--

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

[hidden email]
_______________________________________________
Pythonmac-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: http://mail.python.org/mailman/options/Pythonmac-SIG
Reply | Threaded
Open this post in threaded view
|

Re: shrink app bundle size (removing unused stuff)?

Ronald Oussoren

On 10 Mar, 2012, at 15:02, Chris Barker wrote:

>> 2) I tweaked the matplotlib recipe in py2app, it no
>> longer copies the entire package but adds a hook to
>> tell matplotlib where its data is. This solution is not fully
>> complete yet, I'm currently including mpl-data twice
>> (once in Resources, once in site-packages.zip) and
>> the later copy is not needed.
>
> also nice -- did you use:
>
> matplotlib.get_py2exe_datafiles()
That seemed to have assumptions about how py2exe bundles applications. I currently set and environment variable in the bootstrap module and that seems to work fine.

Ronald
_______________________________________________
Pythonmac-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: http://mail.python.org/mailman/options/Pythonmac-SIG

smime.p7s (6K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: shrink app bundle size (removing unused stuff)?

guano
In reply to this post by Ronald Oussoren
Can I build for only one (i386) architecture?

cheers

Carlos



On Sat, Mar 10, 2012 at 14:03, Ronald Oussoren <[hidden email]> wrote:

On 10 Mar, 2012, at 8:47, Ronald Oussoren wrote:

> With this the application bundle already shrinks a little (curently 123 MByte, before I started it was about 160 MByte). About 70 Mbyte of this is wx itself (the wxWidgets library and wxPython extensions). Another 21 MByte is because all of numpy is included, not just the bits that are actually needed. That's because of another py2app recipe which can hopefully be improved.

BTW. The size of the binaries is at least partially due to the support for multiple architectures, the python installer contains support for two architectures and wx even includes support for 3 different architectures.

Ronald




--
Prof. Carlos Henrique Grohmann - Geologist D.Sc.
Institute of Geosciences - Univ. of São Paulo, Brazil
---
http://www.igc.usp.br/pessoais/guano
http://digitalelevation.wordpress.com/
http://lattes.cnpq.br/5846052449613692 (CV)
---
Twitter: @CarlosGrohmann
http://carlosgrohmann.tumblr.com/
________________
Can’t stop the signal.

_______________________________________________
Pythonmac-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: http://mail.python.org/mailman/options/Pythonmac-SIG
Reply | Threaded
Open this post in threaded view
|

Re: shrink app bundle size (removing unused stuff)?

João Vale-3
I run this command in my setup.py after the .app is generated, and it's been serving me well:

ditto --rsrc --arch i386 file.app new_file.app

Cheers,
João Vale



On Mon, Mar 12, 2012 at 3:59 PM, Carlos Grohmann <[hidden email]> wrote:
Can I build for only one (i386) architecture?

cheers

Carlos



On Sat, Mar 10, 2012 at 14:03, Ronald Oussoren <[hidden email]> wrote:

On 10 Mar, 2012, at 8:47, Ronald Oussoren wrote:

> With this the application bundle already shrinks a little (curently 123 MByte, before I started it was about 160 MByte). About 70 Mbyte of this is wx itself (the wxWidgets library and wxPython extensions). Another 21 MByte is because all of numpy is included, not just the bits that are actually needed. That's because of another py2app recipe which can hopefully be improved.

BTW. The size of the binaries is at least partially due to the support for multiple architectures, the python installer contains support for two architectures and wx even includes support for 3 different architectures.

Ronald




--
Prof. Carlos Henrique Grohmann - Geologist D.Sc.
Institute of Geosciences - Univ. of São Paulo, Brazil
---
http://www.igc.usp.br/pessoais/guano
http://digitalelevation.wordpress.com/
http://lattes.cnpq.br/5846052449613692 (CV)
---
Twitter: @CarlosGrohmann
http://carlosgrohmann.tumblr.com/
________________
Can’t stop the signal.

_______________________________________________
Pythonmac-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: http://mail.python.org/mailman/options/Pythonmac-SIG



_______________________________________________
Pythonmac-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: http://mail.python.org/mailman/options/Pythonmac-SIG
Reply | Threaded
Open this post in threaded view
|

Re: shrink app bundle size (removing unused stuff)?

Ronald Oussoren

On 12 Mar, 2012, at 11:34, João Vale wrote:

> I run this command in my setup.py after the .app is generated, and it's been serving me well:
>
> ditto --rsrc --arch i386 file.app new_file.app

Cool trick, and it doesn't even rely on the developer tools.

I've filed an issue in the macholib tracker to remind me that this should be functionality in that library (which can then be used by py2app). Macholib already understands a  lot about the structure of mac binaries, with some luck it will be easy to remove architectures from binaries.

Ronald


_______________________________________________
Pythonmac-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: http://mail.python.org/mailman/options/Pythonmac-SIG

smime.p7s (6K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: shrink app bundle size (removing unused stuff)?

guano
In reply to this post by Ronald Oussoren
Hi Ronald,

I downloaded your new version from bitbucket and installed it with

sudo python setup.py install

but when I run py2app I got this:



GuanoMac:build_Mac guano$ python setup.py py2app
running py2app
creating /Users/guano/Documents/Arbeit/OpenStereo/development/build_Mac/build
creating /Users/guano/Documents/Arbeit/OpenStereo/development/build_Mac/build/bdist.macosx-10.6-intel
creating /Users/guano/Documents/Arbeit/OpenStereo/development/build_Mac/build/bdist.macosx-10.6-intel/python2.7-standalone
creating /Users/guano/Documents/Arbeit/OpenStereo/development/build_Mac/build/bdist.macosx-10.6-intel/python2.7-standalone/app
creating /Users/guano/Documents/Arbeit/OpenStereo/development/build_Mac/build/bdist.macosx-10.6-intel/python2.7-standalone/app/collect
creating /Users/guano/Documents/Arbeit/OpenStereo/development/build_Mac/build/bdist.macosx-10.6-intel/python2.7-standalone/app/temp
creating /Users/guano/Documents/Arbeit/OpenStereo/development/build_Mac/dist
creating build/bdist.macosx-10.6-intel/python2.7-standalone/app/lib-dynload
creating build/bdist.macosx-10.6-intel/python2.7-standalone/app/Frameworks
*** using recipe: virtualenv ***
*** using recipe: sip ***
*** using recipe: matplotlib ***
*** using recipe: numpy ***
*** using recipe: wx ***
Traceback (most recent call last):
  File "setup.py", line 75, in <module>
    setup_requires=['py2app'],
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/core.py", line 152, in setup
    dist.run_commands()
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/py2app-0.6.5-py2.7.egg/py2app/build_app.py", line 480, in run
    self._run()
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/py2app-0.6.5-py2.7.egg/py2app/build_app.py", line 643, in _run
    self.run_normal()
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/py2app-0.6.5-py2.7.egg/py2app/build_app.py", line 716, in run_normal
    self.process_recipes(mf, filters, flatpackages, loader_files)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/py2app-0.6.5-py2.7.egg/py2app/build_app.py", line 624, in process_recipes
    find_needed_modules(mf, includes=rval['includes'])
  File "build/bdist.macosx-10.6-intel/egg/modulegraph/find_modules.py", line 182, in find_needed_modules
  File "build/bdist.macosx-10.6-intel/egg/modulegraph/modulegraph.py", line 643, in import_hook
  File "build/bdist.macosx-10.6-intel/egg/modulegraph/modulegraph.py", line 734, in load_tail
ImportError: No module named wx.lib.pubsub.pubsub2
GuanoMac:build_Mac guano$ 



Still some issues with pubsub..

tks

Carlos



On Sat, Mar 10, 2012 at 13:47, Ronald Oussoren <[hidden email]> wrote:

On 9 Mar, 2012, at 16:33, Ronald Oussoren wrote:

>
> On 9 Mar, 2012, at 10:55, Carlos Grohmann wrote:
>
>> I think that if the 'excludes' weren't overwritten by the
>> wx/numpy/matplotlib/etc recipes, it would be easier to fine-tune our
>> options (maybe run the recipes first and aftrer run the excludes?)
>
> The matplotlib issue seems to be a bug that's not related to recipes: py2app copies package data into the application bundle, and this code accidently also copied all subpackages like the matplotlib.test package.   I have a patch, but need to test if this actually fixes the bug before I commit.

It didn't, but I did make some progress:

1) I added a py2app recipe that does the right thing with the pubsub library in wxPython. That library uses __path__ hacks and that confuses modulegraph (and hence py2app). Because of this you no longer have to explicitly include 'wx' in the application bundle

2) I tweaked the matplotlib recipe in py2app, it no longer copies the entire package but adds a hook to tell matplotlib where its data is. This solution is not fully complete yet, I'm currently including mpl-data twice (once in Resources, once in site-packages.zip) and the later copy is not needed.

With this the application bundle already shrinks a little (curently 123 MByte, before I started it was about 160 MByte). About 70 Mbyte of this is wx itself (the wxWidgets library and wxPython extensions). Another 21 MByte is because all of numpy is included, not just the bits that are actually needed. That's because of another py2app recipe which can hopefully be improved.

I just pushed updates to modulegraph and py2app to my bitbucket repositories.

Ronald

>
> Ronald
>
> _______________________________________________
> Pythonmac-SIG maillist  -  [hidden email]
> http://mail.python.org/mailman/listinfo/pythonmac-sig
> unsubscribe: http://mail.python.org/mailman/options/Pythonmac-SIG




--
Prof. Carlos Henrique Grohmann - Geologist D.Sc.
Institute of Geosciences - Univ. of São Paulo, Brazil
---
http://www.igc.usp.br/pessoais/guano
http://www.igc.usp.br/openstereo
http://lattes.cnpq.br/5846052449613692 (CV)
---
Twitter: @CarlosGrohmann
http://carlosgrohmann.tumblr.com/
________________
Can’t stop the signal.

_______________________________________________
Pythonmac-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: http://mail.python.org/mailman/options/Pythonmac-SIG
Reply | Threaded
Open this post in threaded view
|

Re: shrink app bundle size (removing unused stuff)?

Chris Barker
Nothing to do with your issue (I don't think) but:


On Tue, Mar 13, 2012 at 5:17 PM, Carlos Grohmann > *** using recipe:
virtualenv ***
> *** using recipe: sip ***
> *** using recipe: matplotlib ***
> *** using recipe: numpy ***
> *** using recipe: wx ***

recipe sip is usually used for PyQT (as sip is used to build pyQT) It
seems odd that you'd have both SIP and wx in one app.

Or is sip used fro something else that you're using?

-Chris

--

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

[hidden email]
_______________________________________________
Pythonmac-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: http://mail.python.org/mailman/options/Pythonmac-SIG
Reply | Threaded
Open this post in threaded view
|

Re: shrink app bundle size (removing unused stuff)?

Chris Barker
On Wed, Mar 14, 2012 at 2:24 PM, Carlos Grohmann
<[hidden email]> wrote:
> I don't use sip at all. don't know why it's been used by py2app.

one guess -- there is an optional QT back-end to matplotlib -- is
py2app picking that up (It may drag in TK and even GTK, too). MIght be
worth taking a look.

-Chris





> Carlos
>
> On Wed, Mar 14, 2012 at 17:46, Chris Barker <[hidden email]> wrote:
>>
>> Nothing to do with your issue (I don't think) but:
>>
>>
>> On Tue, Mar 13, 2012 at 5:17 PM, Carlos Grohmann > *** using recipe:
>> virtualenv ***
>> > *** using recipe: sip ***
>> > *** using recipe: matplotlib ***
>> > *** using recipe: numpy ***
>> > *** using recipe: wx ***
>>
>> recipe sip is usually used for PyQT (as sip is used to build pyQT) It
>> seems odd that you'd have both SIP and wx in one app.
>>
>> Or is sip used fro something else that you're using?
>>
>> -Chris
>>
>> --
>>
>> Christopher Barker, Ph.D.
>> Oceanographer
>>
>> Emergency Response Division
>> NOAA/NOS/OR&R            (206) 526-6959   voice
>> 7600 Sand Point Way NE   (206) 526-6329   fax
>> Seattle, WA  98115       (206) 526-6317   main reception
>>
>> [hidden email]
>> _______________________________________________
>> Pythonmac-SIG maillist  -  [hidden email]
>> http://mail.python.org/mailman/listinfo/pythonmac-sig
>> unsubscribe: http://mail.python.org/mailman/options/Pythonmac-SIG
>
>
>
>
> --
> Prof. Carlos Henrique Grohmann - Geologist D.Sc.
> Institute of Geosciences - Univ. of São Paulo, Brazil
> ---
> http://www.igc.usp.br/pessoais/guano
> http://www.igc.usp.br/openstereo
> http://lattes.cnpq.br/5846052449613692 (CV)
> ---
> Twitter: @CarlosGrohmann
> http://carlosgrohmann.tumblr.com/
> ________________
> Can’t stop the signal.



--

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

[hidden email]
_______________________________________________
Pythonmac-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: http://mail.python.org/mailman/options/Pythonmac-SIG
Reply | Threaded
Open this post in threaded view
|

Re: shrink app bundle size (removing unused stuff)?

Ronald Oussoren
Py2app does indeed see all backends and therefore copies basicly every GUI library it can find into your bundle.

Note that the recipe lines in the output don't necessarily mean anything, py2app prints something when it calls a recipe.

Ronald

Sent from my iPhone

On 14 mrt. 2012, at 14:55, Chris Barker <[hidden email]> wrote:

> On Wed, Mar 14, 2012 at 2:24 PM, Carlos Grohmann
> <[hidden email]> wrote:
>> I don't use sip at all. don't know why it's been used by py2app.
>
> one guess -- there is an optional QT back-end to matplotlib -- is
> py2app picking that up (It may drag in TK and even GTK, too). MIght be
> worth taking a look.
>
> -Chris
>
>
>
>
>
>> Carlos
>>
>> On Wed, Mar 14, 2012 at 17:46, Chris Barker <[hidden email]> wrote:
>>>
>>> Nothing to do with your issue (I don't think) but:
>>>
>>>
>>> On Tue, Mar 13, 2012 at 5:17 PM, Carlos Grohmann > *** using recipe:
>>> virtualenv ***
>>>> *** using recipe: sip ***
>>>> *** using recipe: matplotlib ***
>>>> *** using recipe: numpy ***
>>>> *** using recipe: wx ***
>>>
>>> recipe sip is usually used for PyQT (as sip is used to build pyQT) It
>>> seems odd that you'd have both SIP and wx in one app.
>>>
>>> Or is sip used fro something else that you're using?
>>>
>>> -Chris
>>>
>>> --
>>>
>>> Christopher Barker, Ph.D.
>>> Oceanographer
>>>
>>> Emergency Response Division
>>> NOAA/NOS/OR&R            (206) 526-6959   voice
>>> 7600 Sand Point Way NE   (206) 526-6329   fax
>>> Seattle, WA  98115       (206) 526-6317   main reception
>>>
>>> [hidden email]
>>> _______________________________________________
>>> Pythonmac-SIG maillist  -  [hidden email]
>>> http://mail.python.org/mailman/listinfo/pythonmac-sig
>>> unsubscribe: http://mail.python.org/mailman/options/Pythonmac-SIG
>>
>>
>>
>>
>> --
>> Prof. Carlos Henrique Grohmann - Geologist D.Sc.
>> Institute of Geosciences - Univ. of São Paulo, Brazil
>> ---
>> http://www.igc.usp.br/pessoais/guano
>> http://www.igc.usp.br/openstereo
>> http://lattes.cnpq.br/5846052449613692 (CV)
>> ---
>> Twitter: @CarlosGrohmann
>> http://carlosgrohmann.tumblr.com/
>> ________________
>> Can’t stop the signal.
>
>
>
> --
>
> Christopher Barker, Ph.D.
> Oceanographer
>
> Emergency Response Division
> NOAA/NOS/OR&R            (206) 526-6959   voice
> 7600 Sand Point Way NE   (206) 526-6329   fax
> Seattle, WA  98115       (206) 526-6317   main reception
>
> [hidden email]
> _______________________________________________
> Pythonmac-SIG maillist  -  [hidden email]
> http://mail.python.org/mailman/listinfo/pythonmac-sig
> unsubscribe: http://mail.python.org/mailman/options/Pythonmac-SIG

_______________________________________________
Pythonmac-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: http://mail.python.org/mailman/options/Pythonmac-SIG

smime.p7s (6K) Download Attachment