Changing functions in a running program?

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Changing functions in a running program?

Andreas Yankopolus
Is it possible to run a program to a breakpoint in IPython and then change functions within that program? This would be for interactively experimenting with program logic without having to restart the program from scratch. I’m running IPython3 from Emacs using Elpy for interacting with the IPython3 process.

Consider foo.py as follows:

#!/usr/bin/env ipython3

import sys
from IPython.core.debugger import Pdb


def print_name():
    print ("Alice")

def name():
    print_name()

def main(argv):
    print ("In main.")
    Pdb().set_trace()
    name()

if __name__ == "__main__":
    main(sys.argv[1:])

Running it gives me:

ayank@snorri:~/Documents$ ./foo.py 
In main.
> /home/ayank/Documents/foo.py(16)main()
     14     print ("In main.")
     15     Pdb().set_trace()
---> 16     name()
     17 
     18 if __name__ == "__main__":


ipdb> 

I now want to change the definition of print_name() to print “Bob” instead of “Alice”. I can make this change in Emacs, send the new function to IPython with C-c C-y f, but when I then type name(), I get “Alice”. The reference to print_name() in name() is not getting updated to point to the new definition of print_name(). Likely I’m going about this process incorrectly, as I can make these kind of changes at an IPython3 prompt but not at an ipdb one.

Thanks,

Andreas

_______________________________________________
IPython-dev mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/ipython-dev
Reply | Threaded
Open this post in threaded view
|

Re: Changing functions in a running program?

Wes Turner

On Sat, Jan 12, 2019 at 6:11 PM Andreas Yankopolus <[hidden email]> wrote:
Is it possible to run a program to a breakpoint in IPython and then change functions within that program? This would be for interactively experimenting with program logic without having to restart the program from scratch. I’m running IPython3 from Emacs using Elpy for interacting with the IPython3 process.

Consider foo.py as follows:

#!/usr/bin/env ipython3

import sys
from IPython.core.debugger import Pdb


def print_name():
    print ("Alice")

def name():
    print_name()

def main(argv):
    print ("In main.")
    Pdb().set_trace()
    name()

if __name__ == "__main__":
    main(sys.argv[1:])

Running it gives me:

ayank@snorri:~/Documents$ ./foo.py 
In main.
> /home/ayank/Documents/foo.py(16)main()
     14     print ("In main.")
     15     Pdb().set_trace()
---> 16     name()
     17 
     18 if __name__ == "__main__":


ipdb> 

I now want to change the definition of print_name() to print “Bob” instead of “Alice”. I can make this change in Emacs, send the new function to IPython with C-c C-y f, but when I then type name(), I get “Alice”. The reference to print_name() in name() is not getting updated to point to the new definition of print_name(). Likely I’m going about this process incorrectly, as I can make these kind of changes at an IPython3 prompt but not at an ipdb one.

Thanks,

Andreas
_______________________________________________
IPython-dev mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/ipython-dev

_______________________________________________
IPython-dev mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/ipython-dev
Reply | Threaded
Open this post in threaded view
|

Re: Changing functions in a running program?

Wes Turner
In searching for a solution, I found your question here:

It's funny, I've been coding in Python for very many years and I've never encountered this need.
With TDD, you basically never do this: you write a different function and test that;
if it needs to be parametrized ("dependency injection"),
you just pass a function reference as an argument.
(If the code is designed to be changed at runtime (polymorphic),
that should be part of the callable parameter spec.)

But that doesn't help with this simple example
(or probably the code you're actually working with).

IIUC, this is ocurring because
print_name in the main() function is already bound to the previous instance.

in order to change that reference,
we need to update __self__/__module__.print_name.
Maybe coincidentally I also have no idea how to do this without importing inspect?
If it was a class method, a simple assignment would accomplish your objective.

This works:

ipdb> !import inspect; inspect.currentframe().f_globals['print_name'] = lambda: print("Bob")

But there may be a simpler solution?


On Sat, Jan 12, 2019 at 8:57 PM Andreas Yankopolus <[hidden email]> wrote:
Wes,

https://stackoverflow.com/questions/48112226/update-a-function-during-debugging-pdb-or-ipdb
- Does this work from just a plain ipdb shell (when not sending the new function definition from emacs)?

This is getting close, but updating the function as proposed doesn’t work. Using the example code from my previous post:

ayank@snorri:~/Documents$ ./foo.py 
In main.
/home/ayank/Documents/foo.py(16)main()
     14     print ("In main.")
     15     Pdb().set_trace()
---> 16     name()
     17 
     18 if __name__ == "__main__":

ipdb> print_name()
Alice
ipdb> name()
Alice
ipdb> !def print_name(): print ("Bob")
ipdb> print_name()
Bob
ipdb> name()
Alice

We can see that the function is updated, but functions that call it do not get updated to use the new definition. Any idea what’s going on here?

This kind of thing is no problem in Lisp, but its macro facility means that support for changing code at runtime is baked deep into the language. Perhaps there’s something missed in the IPython REPL?

Thanks,

Andreas



On Sat, Jan 12, 2019 at 8:37 PM Wes Turner <[hidden email]> wrote:

On Sat, Jan 12, 2019 at 6:11 PM Andreas Yankopolus <[hidden email]> wrote:
Is it possible to run a program to a breakpoint in IPython and then change functions within that program? This would be for interactively experimenting with program logic without having to restart the program from scratch. I’m running IPython3 from Emacs using Elpy for interacting with the IPython3 process.

Consider foo.py as follows:

#!/usr/bin/env ipython3

import sys
from IPython.core.debugger import Pdb


def print_name():
    print ("Alice")

def name():
    print_name()

def main(argv):
    print ("In main.")
    Pdb().set_trace()
    name()

if __name__ == "__main__":
    main(sys.argv[1:])

Running it gives me:

ayank@snorri:~/Documents$ ./foo.py 
In main.
> /home/ayank/Documents/foo.py(16)main()
     14     print ("In main.")
     15     Pdb().set_trace()
---> 16     name()
     17 
     18 if __name__ == "__main__":


ipdb> 

I now want to change the definition of print_name() to print “Bob” instead of “Alice”. I can make this change in Emacs, send the new function to IPython with C-c C-y f, but when I then type name(), I get “Alice”. The reference to print_name() in name() is not getting updated to point to the new definition of print_name(). Likely I’m going about this process incorrectly, as I can make these kind of changes at an IPython3 prompt but not at an ipdb one.

Thanks,

Andreas
_______________________________________________
IPython-dev mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/ipython-dev

_______________________________________________
IPython-dev mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/ipython-dev
Reply | Threaded
Open this post in threaded view
|

Re: Changing functions in a running program?

Thomas Kluyver-2
In reply to this post by Andreas Yankopolus
Hi Andreas,

If you define a function or variable at the breakpoint, it's probably making it a local variable inside main(), so it's not in scope for name().

You may be able to get round this by defining the new function and then explicitly making it a global variable, something like this:

globals()['print_name'] = print_name

Not exactly elegant, but hopefully it works.
Thomas

On Sun, 13 Jan 2019 at 00:10, Andreas Yankopolus <[hidden email]> wrote:
Is it possible to run a program to a breakpoint in IPython and then change functions within that program? This would be for interactively experimenting with program logic without having to restart the program from scratch. I’m running IPython3 from Emacs using Elpy for interacting with the IPython3 process.

Consider foo.py as follows:

#!/usr/bin/env ipython3

import sys
from IPython.core.debugger import Pdb


def print_name():
    print ("Alice")

def name():
    print_name()

def main(argv):
    print ("In main.")
    Pdb().set_trace()
    name()

if __name__ == "__main__":
    main(sys.argv[1:])

Running it gives me:

ayank@snorri:~/Documents$ ./foo.py 
In main.
> /home/ayank/Documents/foo.py(16)main()
     14     print ("In main.")
     15     Pdb().set_trace()
---> 16     name()
     17 
     18 if __name__ == "__main__":


ipdb> 

I now want to change the definition of print_name() to print “Bob” instead of “Alice”. I can make this change in Emacs, send the new function to IPython with C-c C-y f, but when I then type name(), I get “Alice”. The reference to print_name() in name() is not getting updated to point to the new definition of print_name(). Likely I’m going about this process incorrectly, as I can make these kind of changes at an IPython3 prompt but not at an ipdb one.

Thanks,

Andreas
_______________________________________________
IPython-dev mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/ipython-dev

_______________________________________________
IPython-dev mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/ipython-dev
Reply | Threaded
Open this post in threaded view
|

Re: Changing functions in a running program?

Andreas Yankopolus
In reply to this post by Andreas Yankopolus
Thomas,

If you define a function or variable at the breakpoint, it's probably making it a local variable inside main(), so it's not in scope for name().

You may be able to get round this by defining the new function and then explicitly making it a global variable, something like this:

globals()['print_name'] = print_name

Not exactly elegant, but hopefully it works.

Yes—makes sense and does the trick! I’ll have to figure out an Emacs hook to automatically update globals in that manner. My output:

In main.
> /Users/ayank/Documents/programming/python/bar.py(13)main()
     12     import ipdb; ipdb.set_trace()
---> 13     name()
     14 

ipdb> name()
Alice
ipdb> !def print_name(): print ("Bob")
ipdb> name()
Alice
ipdb> globals()['print_name'] = print_name
ipdb> name()
Bob
ipdb> 

To follow up on Wes’s comment regarding TDD—I’m writing signal processing code and using Python + SciPy like Matlab. There are some calculations when the code starts that take a few minutes. I have a breakpoint there and am experimenting with the next steps of the processing chain.

_______________________________________________
IPython-dev mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/ipython-dev
Reply | Threaded
Open this post in threaded view
|

Re: Changing functions in a running program?

Wes Turner
There are likely more convenient patterns for a functionally composed signal processing pipeline; though ipdb may be good enough.
https://pypi.org/project/pdbpp/ supports {tab-completion,}

Decomposing to functions that accept state with a standard interface like {data: [], kwargs: {}} may have advantages.
You can assign the output of one step of the pipeline to a global; or, more ideally, a key of a dict (or an attribute of a class instance); so you don't need to modify globals() from a different scope.

The Jupyter notebook way to do this would be to put the first processing stage in one cell, and then work with it in the next. e g. Spyder and VSCode support markers in Python sources so that you can execute a top-level block of code at a time.

I'm not too familiar with signal processing. Someone has likely already distilled the workflow into some standard interfaces like sklearn.pipeline.Pipeline.steps?

/q="scikit signal"
- scipy.signal
- scikit-signal

/q="python signal processing"



# This logs IPython input and output to a file;
# but not ipdb i/o AFAIU
%logstart -o example.py
%logstart -h

To avoid logging code that modifies globals(),
it's probably better and more convenient to pass print_name as an argument to name():

def name(print_name=print_name):
    print_name()

With a config dict/object:

def name(conf):
    print_name = conf['print_name']
    print_name(conf['data'])


On Monday, January 14, 2019, Andreas Yankopolus <[hidden email]> wrote:
Thomas,

If you define a function or variable at the breakpoint, it's probably making it a local variable inside main(), so it's not in scope for name().

You may be able to get round this by defining the new function and then explicitly making it a global variable, something like this:

globals()['print_name'] = print_name

Not exactly elegant, but hopefully it works.

Yes—makes sense and does the trick! I’ll have to figure out an Emacs hook to automatically update globals in that manner. My output:

In main.
> /Users/ayank/Documents/programming/python/bar.py(13)main()
     12     import ipdb; ipdb.set_trace()
---> 13     name()
     14 

ipdb> name()
Alice
ipdb> !def print_name(): print ("Bob")
ipdb> name()
Alice
ipdb> globals()['print_name'] = print_name
ipdb> name()
Bob
ipdb> 

To follow up on Wes’s comment regarding TDD—I’m writing signal processing code and using Python + SciPy like Matlab. There are some calculations when the code starts that take a few minutes. I have a breakpoint there and am experimenting with the next steps of the processing chain.

_______________________________________________
IPython-dev mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/ipython-dev