Quantcast

[IPython-User] Neat trick: Quick memory usage benchmarking in IPython

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[IPython-User] Neat trick: Quick memory usage benchmarking in IPython

Fernando Perez
using the new magics api:

http://blog.vene.ro/2012/06/30/quick-memory-usage-benchmarking-in-ipython/

let's test this out, and if it works well we may just ship it.
_______________________________________________
IPython-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/ipython-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Neat trick: Quick memory usage benchmarking in IPython

klo uo
On Sat, Jun 30, 2012 at 6:20 PM, Fernando Perez <[hidden email]> wrote:
> using the new magics api:
>
> http://blog.vene.ro/2012/06/30/quick-memory-usage-benchmarking-in-ipython/
>
> let's test this out, and if it works well we may just ship it.

It looks like it's handy function simplifying memory_profiler usage,
but I won't vote for shipping if my vote counts.

It requires additional module (perhaps popular one and tiny, but
nonetheless), which requires yet additional module - psutil (optional
on Linux but recommended and necessary on Windows).
Introducing two new modules as a new dependency for single handy
function may be seen as too much to otherwise sensitive application as
IPython is toward as minimal requirement as possible, while offering
so much. This is IMHO clear line between IPython and various similar
interfaces

Also script needs to be corrected for Python3 compatibility.
I understand that I should just copy the script in IPython interpreter
and can test it? If that's correct here is what I got on Ubuntu with
IPython 0.13:

  In [3]: %memit np.ones(1e6)
  best of 3: 0.117188 MB per loop
  Out[3]: 0.1171875

which is different from result author got for some reason, while
profiling it with memory_profiler I get expected result.

On Windows I get this traceback: http://pastebin.com/Mmc7mHQh

Anyhow I'd vote to put this script in IPython wiki, where other
interesting magic functions can be hosted perhaps


Cheers
_______________________________________________
IPython-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/ipython-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Neat trick: Quick memory usage benchmarking in IPython

klo uo
Huh, while looking in documentation for newly featured magic
functions, so that I could learn how custom ones can be employed, I
see that some magics have external dependencies depending of the
function, so my previous post about requirements doesn't stand.

Maybe it just needs more testing and considering Python3 compatibility
_______________________________________________
IPython-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/ipython-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Neat trick: Quick memory usage benchmarking in IPython

Fernando Perez
In reply to this post by klo uo
On Sun, Jul 1, 2012 at 10:24 AM, klo uo <[hidden email]> wrote:
> Anyhow I'd vote to put this script in IPython wiki, where other
> interesting magic functions can be hosted perhaps

Go for it!  And thanks for the more detailed look, indeed it seems
it's best to just reference but not bother with shipping it for now.

Best

f
_______________________________________________
IPython-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/ipython-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Neat trick: Quick memory usage benchmarking in IPython

klo uo
On Sun, Jul 1, 2012 at 9:27 PM, Fernando Perez wrote:
> On Sun, Jul 1, 2012 at 10:24 AM, klo uo wrote:
>> Anyhow I'd vote to put this script in IPython wiki, where other
>> interesting magic functions can be hosted perhaps
>
> Go for it!  And thanks for the more detailed look, indeed it seems
> it's best to just reference but not bother with shipping it for now.

Done: http://wiki.ipython.org/Cookbook/Custom_magic_examples

If anyone knows of other examples to this new IPython feature, I could
append created wiki page
_______________________________________________
IPython-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/ipython-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Neat trick: Quick memory usage benchmarking in IPython

Fernando Perez
On Mon, Jul 2, 2012 at 10:32 AM, klo uo <[hidden email]> wrote:
> Done: http://wiki.ipython.org/Cookbook/Custom_magic_examples

Great, thanks!
_______________________________________________
IPython-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/ipython-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Neat trick: Quick memory usage benchmarking in IPython

vene
Hello, author of the gist here.

I have meanwhile updated the %memit magic along with writing an %mprun clone of the famous line_profiler magic, for inclusion with memory_profiler. I believe the home of these magics to be in that package, not in IPython, as much as I'd like to have some code in there :) I've also patched vbench to use these tricks. More info on my blog [1] and github [2]

Alas, I had to drop back to the old-style syntax, but right now, when I saw that you linked to my gist [3], I promptly edited it so it's up to date with the memory_profiler version and more robust.

This work is part of my GSoC benchmarking project for scikit-learn.

Thank you for linking, and thanks Fernando for noticing my post.

[1] http://blog.vene.ro/
[2] https://github.com/vene/
[3] https://gist.github.com/3022718/
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Neat trick: Quick memory usage benchmarking in IPython

Fernando Perez
On Fri, Jul 6, 2012 at 4:01 PM, vene <[hidden email]> wrote:
>
> Thank you for linking, and thanks Fernando for noticing my post.

Our pleasure.  As these tools mature, keep the ipython lists posted;
once this is completely solid it might be something we'd like to ship
as part of the project, since it's a perfect compation to our other
generic timing/profiling utilities.

Best,

f
_______________________________________________
IPython-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/ipython-user
Loading...