Is there a utility to tally function calls from other files?

classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

Is there a utility to tally function calls from other files?

C.D. Reimer
Greetings,

I'm converting 101 BASIC games from an old book
(http://www.atariarchives.org/basicgames/) into Python scripts. After a
dozen conversions, I've created a helper file for common functions
(i.e., pick_card(), replay_game() and roll_dice()). The functions for
card and dice games are obvious candidates for the helper file, but
other functions that might be one offs that shouldn't be included.

Is there utility that will take the function names from the helper file
and scan the other files to tally up how many times a particular
function got called?

Or do I need to write a script?

Thanks,

Chris R.

Reply | Threaded
Open this post in threaded view
|

Is there a utility to tally function calls from other files?

Chris Angelico
On Sun, Jun 14, 2015 at 5:10 AM, C.D. Reimer <chris at cdreimer.com> wrote:

> I'm converting 101 BASIC games from an old book
> (http://www.atariarchives.org/basicgames/) into Python scripts. After a
> dozen conversions, I've created a helper file for common functions (i.e.,
> pick_card(), replay_game() and roll_dice()). The functions for card and dice
> games are obvious candidates for the helper file, but other functions that
> might be one offs that shouldn't be included.
>
> Is there utility that will take the function names from the helper file and
> scan the other files to tally up how many times a particular function got
> called?

Can you guarantee that your function names are absolutely unique text
strings? If so, grep can do what you want. I don't have a huge Python
project to demo this on, but grep is language-agnostic.

rosuav at sikorsky:~/Gypsum$ grep gtksignal *.pike plugins/*.pike
globals.pike://Usage:
gtksignal(some_object,"some_signal",handler,arg,arg,arg) --> save that
object.
globals.pike:class gtksignal(object obj)
globals.pike: mi->signals=({gtksignal(mi->menuitem,"activate",menu_clicked)});
globals.pike: mi->signals=({gtksignal(mi->menuitem,"activate",menu_clicked)});
globals.pike: //a gtksignal object or the integer 0, though as of
20150103 nothing depends on this.
globals.pike: gtksignal(win->mainwindow,"delete_event",closewindow),
globals.pike: win->stock_close &&
gtksignal(win->stock_close,"clicked",closewindow),
globals.pike: win->signals+=({gtksignal(obj,parts[i+1..]*"_",this[key],arg,UNDEFINED,b4)});
globals.pike: if (actionbtn)
win->signals+=({gtksignal(win->pb_action,"clicked",action_callback)});
globals.pike: statustxt->signals=({gtksignal(statustxt->evbox,"button_press_event",mousedown)});
window.pike: //an additional argument to signal_connect/gtksignal
would provide that value here)
plugins/hpgraph.pike:
win->signals+=({gtksignal(win["color"+j+i],"clicked",update_color,i)});
plugins/hpgraph.pike:
info->signals=({gtksignal(info->evbox,"button_press_event",mousedown)});
plugins/rmimport.pike:
gtksignal(win->pb_selectall,"clicked",pb_select_click,1), //Same
handler for these, just an arg
plugins/rmimport.pike:
gtksignal(win->pb_selectnone,"clicked",pb_select_click,0),
//Technically, since the default is 0, I could call this
sig_pb_selectnone_clicked, but this way is clearer.
plugins/search.pike:
statustxt->signals=({gtksignal(statustxt->ef,"key_press_event",keypress,0,UNDEFINED,1)});
plugins/zoneinfo.pike:
win->signals+=({gtksignal(win[pfx+"_"+sfx],"changed",pfx=="Thresh"?convert_th:convert_rl,pfx)});

Each of those lines is one usage of the gtksignal() class/function.
One of them is actually a comment (in window.pike), and the first two
at the top of globals.pike are the class definition, with an
associated usage comment, but all the rest are actual uses. As long as
you can depend on names like roll_dice not occurring anywhere else,
this is a great way to find out what you need to know.

If you _can't_ rely on that, it's a bit harder. You could possibly
append an open parenthesis to it, to require classic function-call
syntax, but that may not necessarily be enough (depends how strict
your style guide is). You can do lexical analysis of a Python program
to see where the name is used in a function call, which might be
better. Or you could instrument the function to find out where it's
being called from, and then run every program and see how many of them
call it - something like this:

def pick_card():
    import __main__
    print("Called from %s"%__main__.__file__)
    # rest of pick_card's definition

Log that to a file instead of the screen, and you have yourself a
guaranteed log that shows which programs use that function. Of course,
that depends on actually running the program far enough to get to the
call, which might require interaction.

Depending on your requirements, it could be anywhere from easy to
hard. Good luck :)

ChrisA

Reply | Threaded
Open this post in threaded view
|

Is there a utility to tally function calls from other files?

C.D. Reimer
On 6/13/2015 12:31 PM, Chris Angelico wrote:
> Depending on your requirements, it could be anywhere from easy to
> hard. Good luck:)

I don't have  grep on my Windows machine. Writing a script might be easier.

Each file has a import statement for the helper file:

     from bg_helper import replay_game, roll_dice

I just need to grab the function names from the helper file, walk the
directory, find the import line from each file, and parse each line for
the function names to build the tally.

Thanks,

Chris R.


Reply | Threaded
Open this post in threaded view
|

Is there a utility to tally function calls from other files?

Chris Angelico
On Sun, Jun 14, 2015 at 5:53 AM, C.D. Reimer <chris at cdreimer.com> wrote:

> On 6/13/2015 12:31 PM, Chris Angelico wrote:
>>
>> Depending on your requirements, it could be anywhere from easy to
>> hard. Good luck:)
>
>
> I don't have  grep on my Windows machine. Writing a script might be easier.
>
> Each file has a import statement for the helper file:
>
>     from bg_helper import replay_game, roll_dice
>
> I just need to grab the function names from the helper file, walk the
> directory, find the import line from each file, and parse each line for the
> function names to build the tally.

Hmm, I think the Windows 'find' command can do the same sort of job.
Though it's not hard to grab a Windows port of grep and use that.
Should be easier than writing your own script.

ChrisA

Reply | Threaded
Open this post in threaded view
|

Is there a utility to tally function calls from other files?

Terry Reedy
In reply to this post by C.D. Reimer
On 6/13/2015 3:53 PM, C.D. Reimer wrote:
> On 6/13/2015 12:31 PM, Chris Angelico wrote:
>> Depending on your requirements, it could be anywhere from easy to
>> hard. Good luck:)
>
> I don't have  grep on my Windows machine. Writing a script might be easier.

If you have a normal Python installation, you do.  Idle's Find in Files
is a grep function with a gui interface.

--
Terry Jan Reedy


Reply | Threaded
Open this post in threaded view
|

Is there a utility to tally function calls from other files?

Laura Creighton-2
In reply to this post by C.D. Reimer
In a message of Sat, 13 Jun 2015 12:53:43 -0700, "C.D. Reimer" writes:
>On 6/13/2015 12:31 PM, Chris Angelico wrote:
>> Depending on your requirements, it could be anywhere from easy to
>> hard. Good luck:)
>
>I don't have  grep on my Windows machine. Writing a script might be easier.

Does Idle run?

Because Idle's 'Find in Files' is grep with a gui around the outside.

(And if it doesn't, getting a python from python.org should fix that
particular problem.)

Laura

Reply | Threaded
Open this post in threaded view
|

Is there a utility to tally function calls from other files?

C.D. Reimer
In reply to this post by Chris Angelico
On 6/13/2015 12:58 PM, Chris Angelico wrote:
> Hmm, I think the Windows 'find' command can do the same sort of job.
> Though it's not hard to grab a Windows port of grep and use that.
> Should be easier than writing your own script.

Writing the script would be easier for me since I'll be cobbling
together several scripts I've used before.

Thanks,

Chris R.

Reply | Threaded
Open this post in threaded view
|

Is there a utility to tally function calls from other files?

C.D. Reimer
In reply to this post by Terry Reedy
On 6/13/2015 1:02 PM, Terry Reedy wrote:
> If you have a normal Python installation, you do.  Idle's Find in
> Files is a grep function with a gui interface.

I have Notepad++ on Windows. The find function for that program does the
same thing. If I have a working Mac, Text Wrangler does the same thing
as well.

Thanks,

Chris

Reply | Threaded
Open this post in threaded view
|

Is there a utility to tally function calls from other files?

Laura Creighton-2
In a message of Sat, 13 Jun 2015 13:45:10 -0700, "C.D. Reimer" writes:

>On 6/13/2015 1:02 PM, Terry Reedy wrote:
>> If you have a normal Python installation, you do.  Idle's Find in
>> Files is a grep function with a gui interface.
>
>I have Notepad++ on Windows. The find function for that program does the
>same thing. If I have a working Mac, Text Wrangler does the same thing
>as well.
>
>Thanks,
>
>Chris

Idle is written in pure python.  Steal from:
https://hg.python.org/cpython/file/74d182cf0187/Lib/idlelib/GrepDialog.py
if you do not have a copy locally.

Laura



Reply | Threaded
Open this post in threaded view
|

Is there a utility to tally function calls from other files?

C.D. Reimer
On 6/13/2015 1:59 PM, Laura Creighton wrote:
> Idle is written in pure python.  Steal from:
> https://hg.python.org/cpython/file/74d182cf0187/Lib/idlelib/GrepDialog.py
> if you do not have a copy locally.

I've been using Notepad++ (Windows) and TextWrangler (Mac) for over a
decade now. Idle may be pure Python, but that's not enough for me to
switch. :)

Thanks,

Chris R.

Reply | Threaded
Open this post in threaded view
|

Is there a utility to tally function calls from other files?

Mark Lawrence
In reply to this post by C.D. Reimer
On 13/06/2015 20:53, C.D. Reimer wrote:
> On 6/13/2015 12:31 PM, Chris Angelico wrote:
>> Depending on your requirements, it could be anywhere from easy to
>> hard. Good luck:)
>
> I don't have  grep on my Windows machine. Writing a script might be easier.
>

http://getgnuwin32.sourceforge.net/ might come in handy for you one day.

--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence


Reply | Threaded
Open this post in threaded view
|

Is there a utility to tally function calls from other files?

Steven D'Aprano-11
In reply to this post by Laura Creighton-2
On Sat, 13 Jun 2015 14:09:48 -0700, C.D. Reimer wrote:

> On 6/13/2015 1:59 PM, Laura Creighton wrote:
>> Idle is written in pure python.  Steal from:
>> https://hg.python.org/cpython/file/74d182cf0187/Lib/idlelib/
GrepDialog.py
>> if you do not have a copy locally.
>
> I've been using Notepad++ (Windows) and TextWrangler (Mac) for over a
> decade now. Idle may be pure Python, but that's not enough for me to
> switch. :)


You don't have to switch for good. You can fire up IDLE to do this one
job, then never use it again.



--
Steven D'Aprano

Reply | Threaded
Open this post in threaded view
|

Is there a utility to tally function calls from other files?

Laura Creighton-2
In a message of 14 Jun 2015 01:59:10 +0000, "Steven D'Aprano" writes:

>On Sat, 13 Jun 2015 14:09:48 -0700, C.D. Reimer wrote:
>
>> On 6/13/2015 1:59 PM, Laura Creighton wrote:
>>> Idle is written in pure python.  Steal from:
>>> https://hg.python.org/cpython/file/74d182cf0187/Lib/idlelib/
>GrepDialog.py
>>> if you do not have a copy locally.
>>
>> I've been using Notepad++ (Windows) and TextWrangler (Mac) for over a
>> decade now. Idle may be pure Python, but that's not enough for me to
>> switch. :)
>
>
>You don't have to switch for good. You can fire up IDLE to do this one
>job, then never use it again.
>

Or my idea was, if his plan was to write a script to grep through his
files, he could just go out and steal the code that comes with idle
where it is already written for him.

Of course, by now he may have already written one ...

Laura


Reply | Threaded
Open this post in threaded view
|

Is there a utility to tally function calls from other files?

C.D. Reimer
Here's my code and output for the curious. I'm sure someone will come up
with a five-line version. :)

import os, time

def main():

     def_str = 'def '

     line_limit = 5

     call_tally = []

     import_str = 'from bg_helper import '

     path = 'scripts'

     startTime = time.time()

     

     # pull call names from helper file

     with open(os.path.join(path, 'bg_helper.py')) as f:

         for line in f:

         

             search = line.find(def_str)

             

             if search != -1:

             

                 call_tally.append([(line[len(def_str):].split('('))[0], 0])

     

     call_tally.sort()

     

     # walk through the scripts directory

     fileList = next(os.walk(path))[2]

     fileList.remove('bg_helper.py')

     fileList.remove('bg_helper.pyc')

     

     # check files for import statements

     for filename in fileList:

     

         i = 0

         

         with open(os.path.join(path, filename)) as f:

         

             for line in f:

             

                 if i < line_limit:

                 

                     search = line.find(import_str)

                 

                     if search != -1:

           

                         found = line[len(import_str):].strip('\n').split(', ')

                     

                         found.sort()

 

                         for found in enumerate(found):

                     

                             for tally in enumerate(call_tally):

                                 

                                 if tally[1][0] == found[1]:

                                 

                                    tally[1][1] += 1

                         

                     i += 1

                     

                 else:

                 

                     break

     # print the results

     print 'FUNCTION CALLS\t\tTALLY\tNOTES\n'

                 

     for tally in enumerate(call_tally):

     

         print tally[1][0],'\t\t', tally[1][1],

         

         if tally[1][1] == 0:

         

             print '\t** NO CALLS **'

             

         else:

         

             print ''

         

     print '\nEvaluated',len(fileList),'files in',

     

     print '{0:.1g}'.format(time.time() - startTime),'seconds\n'

         

                 

if __name__ == "__main__":

     main()



FUNCTION CALLS          TALLY   NOTES

card_faces              1

pick_card               1

replay_game             3

roll_dice               2

slot_faces              0       ** NO CALLS **

Evaluated 10 files in 0.08 seconds


Reply | Threaded
Open this post in threaded view
|

Is there a utility to tally function calls from other files?

Steven D'Aprano-11
In reply to this post by Steven D'Aprano-11
On Sun, 14 Jun 2015 04:11:59 +0200, Laura Creighton wrote:

> In a message of 14 Jun 2015 01:59:10 +0000, "Steven D'Aprano" writes:
>>On Sat, 13 Jun 2015 14:09:48 -0700, C.D. Reimer wrote:
>>
>>> On 6/13/2015 1:59 PM, Laura Creighton wrote:
>>>> Idle is written in pure python.  Steal from:
>>>> https://hg.python.org/cpython/file/74d182cf0187/Lib/idlelib/
>>GrepDialog.py
>>>> if you do not have a copy locally.
>>>
>>> I've been using Notepad++ (Windows) and TextWrangler (Mac) for over a
>>> decade now. Idle may be pure Python, but that's not enough for me to
>>> switch. :)
>>
>>
>>You don't have to switch for good. You can fire up IDLE to do this one
>>job, then never use it again.
>>
>>
> Or my idea was, if his plan was to write a script to grep through his
> files, he could just go out and steal the code that comes with idle
> where it is already written for him.

Well, yes, but since IDLE already contains that script, why not just use
IDLE?


> Of course, by now he may have already written one ...

Indeed.



--
Steven D'Aprano

Reply | Threaded
Open this post in threaded view
|

Is there a utility to tally function calls from other files?

Laura Creighton-2
In a message of 14 Jun 2015 02:53:15 +0000, "Steven D'Aprano" writes:
>Well, yes, but since IDLE already contains that script, why not just use
>IDLE?

I don't know any way to tell Idle to run in the background, generate this
result, and hand it to me as a list, or a csv or whatever.  Have
I been missing something?  The nice thing about grep is that it doesn't
demand that you have an interactive experience with it in order to get
your results out.

Laura