How does Python do Pointers?

classic Classic list List threaded Threaded
60 messages Options
123
Reply | Threaded
Open this post in threaded view
|

How does Python do Pointers?

David MacQuigg
This was the question from a student at my recent lecture to a class of engineering students studying C.  My answer was brief: It doesn't - arguments are passed by value.  Then I thought, this is really misleading for students of C, where pass-by-value means making a copy of the passed object.  So I'm looking for a simple answer that is complete, but doesn't baffle freshmen whose only experience so far is an introductory course in C.  

How about this:

'''
Python doesn't use pointers, but they really aren't needed in Python, because the way objects in memory are accessed is fundamentally different than C.  Objects in Python are "bound" to variable names.  These names are associated with pointers, but that is at a lower level not seen by the Python programmer.  If you need to get down to that level, you should be using C, not Python.
~ '''

That's a little better than my first answer, but I could elaborate with further discussion about argument passing.

'''
You know that in C there are two ways to pass a value to a function: call-by-value and call-by-reference.  Call-by-value puts a copy of the value in the function's memory area.  Call-by-reference passes to the function a pointer to the original value in the caller's memory.  In Python, passing a value to a function is done by "binding" the name of the parameter in the function to the value in the caller's memory.  This is like call-by-reference in C, but the Python programmer never sees the pointer.
~ '''

Note that Martelli says "all argument passing in Python is by value" (Python in a Nutshell, p.74), but I think this may be an error.

I would be interested in seeing how others would handle this question.

-- Dave

P.S. The lecture went very well.  The students were enthusiastic, and the Mandelbrot images were stunning!  http://ece.arizona.edu/~edatools/ece175/Lecture  The purpose of the lecture was to introduce students to higher-level programming, and show how C can still play a role when performance is critical.  I'm hoping to expand this to a 3-week sequence of lectures.


_______________________________________________
Edu-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/edu-sig
Reply | Threaded
Open this post in threaded view
|

How does Python do Pointers?

Michael H. Goldwasser

Hi Dave,

  For students familiar with C, the most straightforward analogy is
  that ALL of Python's names are akin to C's pointers.  This is
  certainly the closest match of the three C/C++ models (value,
  pointer, reference).

  Even for new programmers with no previous knowledge, we make sure to
  portray names distinctly as pointers to the underlying object in
  memory, rather than to cloud the role of the name versus the value.

  Advantages of this analogy are the consistency of the following
  semantics:

  * Assignment "a = b" creates an alias, whereby a and b both
    reference the same underlying object.

  * Parameter passing is equivalent to passing a pointer in C.
    With a signature,  def foo(bar):, the calling syntax
    foo(myList) simply assigns the formal parameter bar to point to
    the same underlying object as the actual parameter myList. In
    fact, it is exactly the semantics of the standard assignment
    "bar = myList" other than the distinction in scope between the
    local variable and the caller's.

  * None in Python is analogous to NULL in C.

    As an aside, this is why I prefer to compare Python's model to
    pointers in C/C++ rather than to references.  When passing a
    reference variable in C++, the formal parameter must be bound to
    an actual object (you cannot pass a null reference).


  Having used this analogy, there are a few things that should be
  explained to those used to C.

  * You cannot directly manipulate memory addresses in Python as you
    can in C.  Internally, Python's interpretter maintains those
    memory address.  In fact, the value of id(x) is typically the
    underlying memory address.   But you cannot use that address to
    access anything, nor can you do pointer arithmetic as in C.

  * There is an independent issue in that Python's primitive types
    (int/str/tuple) are immutable.  So when passing an integer as a
    parameter, the mechanism is still pass-by-reference however this
    does not give the callee any power to alter the original value.
    I think this is why there are some who errantly characterize
    Python as using pass-by-value .

  As a final aside, if student have any familiarity with Java, the
  Python model is quite simple to explain.  It is precisely that which
  Java uses for all object types.

With regard,
Michael


On Sunday May 4, 2008, David MacQuigg wrote:

>    This was the question from a student at my recent lecture to a class of engineering students studying C.  My answer was brief: It doesn't - arguments are passed by value.  Then I thought, this is really misleading for students of C, where pass-by-value means making a copy of the passed object.  So I'm looking for a simple answer that is complete, but doesn't baffle freshmen whose only experience so far is an introductory course in C.  
>    
>    How about this:
>    
>    '''
>    Python doesn't use pointers, but they really aren't needed in Python, because the way objects in memory are accessed is fundamentally different than C.  Objects in Python are "bound" to variable names.  These names are associated with pointers, but that is at a lower level not seen by the Python programmer.  If you need to get down to that level, you should be using C, not Python.
>    ~ '''
>    
>    That's a little better than my first answer, but I could elaborate with further discussion about argument passing.
>    
>    '''
>    You know that in C there are two ways to pass a value to a function: call-by-value and call-by-reference.  Call-by-value puts a copy of the value in the function's memory area.  Call-by-reference passes to the function a pointer to the original value in the caller's memory.  In Python, passing a value to a function is done by "binding" the name of the parameter in the function to the value in the caller's memory.  This is like call-by-reference in C, but the Python programmer never sees the pointer.
>    ~ '''
>    
>    Note that Martelli says "all argument passing in Python is by value" (Python in a Nutshell, p.74), but I think this may be an error.
>    
>    I would be interested in seeing how others would handle this question.
>    
>    -- Dave
>    
>    P.S. The lecture went very well.  The students were enthusiastic, and the Mandelbrot images were stunning!  http://ece.arizona.edu/~edatools/ece175/Lecture  The purpose of the lecture was to introduce students to higher-level programming, and show how C can still play a role when performance is critical.  I'm hoping to expand this to a 3-week sequence of lectures.

       +-----------------------------------------------
       | Michael Goldwasser
       | Associate Professor
       | Dept. Mathematics and Computer Science
       | Saint Louis University
       | 220 North Grand Blvd.
       | St. Louis, MO 63103-2007

_______________________________________________
Edu-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/edu-sig
Reply | Threaded
Open this post in threaded view
|

Fwd: How does Python do Pointers?

Anna Ravenscroft
In reply to this post by David MacQuigg
Martelli's response. ;-)


---------- Forwarded message ----------
From: Alex Martelli <[hidden email]>
Date: Sun, May 4, 2008 at 2:36 PM
Subject: Re: [Edu-sig] How does Python do Pointers?
To: Anna Ravenscroft <[hidden email]>
Cc: [hidden email]


With thanks to Anna for forwarding this...


 >  From: David MacQuigg <[hidden email]>
   ...

>   Note that Martelli says "all argument passing in Python is by value"
 >  (Python in a Nutshell, p.74), but I think this may be an error.

 Taken completely out of context, it might well be, because of the
 connotations often carried by the "by value" tag; if you however read
 the whole paragraph (just six short lines) you'll also see "Python
 passes the object itself, not a copy" which should abundantly suffice
 to dispel the mistaken belief that "by value means a copy".

 Within six short lines, I can't cover all the permutations and nuances
 that might best teach this issue to students coming from many
 different languages.  Somebody coming from Java (perhaps the most
 popular first-language these days) has the least need for
 clarification, as I might basically say "it's just like in Java"
 (perhaps adding "without the complications and exceptions whereby e.g.
 int values appear to be treated differently";-). See for example
 http://javadude.com/articles/passbyvalue.htm -- "Java is
 Pass-by-Value, Dammit!" by Scott Stanchfield -- I do agree with the
 "Dammit!" part at least;-). A similar position is expounded at
 http://www.javaranch.com/campfire/StoryPassBy.jsp .

 Somebody coming from Fortran or Basic (still surprisingly popular
 first-languages, even though dated) needs most hand-holding, because
 they're used to think in terms of "passing a variable" rather than
 "passing a value".

 C of course is thoroughly by-value BUT with the added twist of an
 implied copy at each argument-passing (or assignment -- identical
 semantics to argument passing of course). (Scott would disagree as he
 claims that by-value MEANS copy -- but he also says Java has pointers,
 "Dog d;" declares a POINTER to a Dog, etc...).  To somebody knowing
 only C, I might explain that "everything in Python is a pointer" (on
 which you however cannot perform pointer-arithmetic), much like the
 above Java articles explain things (but without the need to mention
 'int' as somehow distinguished); or I might say a bit more precisely
 that every NAME is a pointer, and also ITEMS in containers are
 pointers. So argument passing (and assignment) most definitely ARE by
 value -- but of course, value of the POINTERS (_those_ pointers get
 copied around -- but the copy of a pointer points to the same object
 as the original pointer!), because that's what names always are in
 Python.


 Alex



--
cordially,
Anna
--
Walking through the water. Trying to get across.
Just like everybody else.
_______________________________________________
Edu-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/edu-sig
Reply | Threaded
Open this post in threaded view
|

Re: How does Python do Pointers?

John Zelle
In reply to this post by Michael H. Goldwasser
Hello all,

This is all good explanation, but could use a bit of tweaking. Where the
pointer analogy with C breaks down is that C pointers require explicit
dereferencing. In this regard, the Python model is actually closer to C
++ reference variables, which are automatically dereferenced. Of course
it also differs from references in that in Python the referent for a
variable can change (just reassign it). Still the intuition should be
just about right for C++ programmers.

I myself like to think of Python variables as "sticky notes" that are
attached to values so we can look them up later. This thinking is a
clean break from pointers or references. Variables are not locations to
have values placed in them. They are names that can be attached to
objects.

The discussion of NULL below is not quite accurate. NULL is a value
stored into a variable in C/C++, that is, it is a value to say the the
pointer/reference does not point to anything. However, None in Python is
actually just the name (i.e. a variable) denoting a particular object.
When you assign None, you are just setting the variable to point to that
object. In C++, if you created a None object, then you could certainly
pass any variable containing None as the actual argument for a reference
parameter.

In the final analysis Martelli is absolutely right. All parameters are
passed by _value_ in Python. It's just that all variables are references
to objects, so that means when you get a copy of the variable that copy
is just another reference to the same object (i.e. you get a copy of the
reference). Again, for C programmers, it is similar to getting a copy of
a pointer.

What I particularly like from a teaching standpoint is that parameter
passing in Python works exactly like assignment. Just think: "assign the
actual parameters of the caller to the formal parameters in the callee
and execute the function body." There's nothing else to it.

--John



On Sun, 2008-05-04 at 15:57 -0500, Michael H. Goldwasser wrote:

> Hi Dave,
>
>   For students familiar with C, the most straightforward analogy is
>   that ALL of Python's names are akin to C's pointers.  This is
>   certainly the closest match of the three C/C++ models (value,
>   pointer, reference).
>
>   Even for new programmers with no previous knowledge, we make sure to
>   portray names distinctly as pointers to the underlying object in
>   memory, rather than to cloud the role of the name versus the value.
>
>   Advantages of this analogy are the consistency of the following
>   semantics:
>
>   * Assignment "a = b" creates an alias, whereby a and b both
>     reference the same underlying object.
>
>   * Parameter passing is equivalent to passing a pointer in C.
>     With a signature,  def foo(bar):, the calling syntax
>     foo(myList) simply assigns the formal parameter bar to point to
>     the same underlying object as the actual parameter myList. In
>     fact, it is exactly the semantics of the standard assignment
>     "bar = myList" other than the distinction in scope between the
>     local variable and the caller's.
>
>   * None in Python is analogous to NULL in C.
>
>     As an aside, this is why I prefer to compare Python's model to
>     pointers in C/C++ rather than to references.  When passing a
>     reference variable in C++, the formal parameter must be bound to
>     an actual object (you cannot pass a null reference).
>
>
>   Having used this analogy, there are a few things that should be
>   explained to those used to C.
>
>   * You cannot directly manipulate memory addresses in Python as you
>     can in C.  Internally, Python's interpretter maintains those
>     memory address.  In fact, the value of id(x) is typically the
>     underlying memory address.   But you cannot use that address to
>     access anything, nor can you do pointer arithmetic as in C.
>
>   * There is an independent issue in that Python's primitive types
>     (int/str/tuple) are immutable.  So when passing an integer as a
>     parameter, the mechanism is still pass-by-reference however this
>     does not give the callee any power to alter the original value.
>     I think this is why there are some who errantly characterize
>     Python as using pass-by-value .
>
>   As a final aside, if student have any familiarity with Java, the
>   Python model is quite simple to explain.  It is precisely that which
>   Java uses for all object types.
>
> With regard,
> Michael
>
>
> On Sunday May 4, 2008, David MacQuigg wrote:
>
> >    This was the question from a student at my recent lecture to a class of engineering students studying C.  My answer was brief: It doesn't - arguments are passed by value.  Then I thought, this is really misleading for students of C, where pass-by-value means making a copy of the passed object.  So I'm looking for a simple answer that is complete, but doesn't baffle freshmen whose only experience so far is an introductory course in C.  
> >    
> >    How about this:
> >    
> >    '''
> >    Python doesn't use pointers, but they really aren't needed in Python, because the way objects in memory are accessed is fundamentally different than C.  Objects in Python are "bound" to variable names.  These names are associated with pointers, but that is at a lower level not seen by the Python programmer.  If you need to get down to that level, you should be using C, not Python.
> >    ~ '''
> >    
> >    That's a little better than my first answer, but I could elaborate with further discussion about argument passing.
> >    
> >    '''
> >    You know that in C there are two ways to pass a value to a function: call-by-value and call-by-reference.  Call-by-value puts a copy of the value in the function's memory area.  Call-by-reference passes to the function a pointer to the original value in the caller's memory.  In Python, passing a value to a function is done by "binding" the name of the parameter in the function to the value in the caller's memory.  This is like call-by-reference in C, but the Python programmer never sees the pointer.
> >    ~ '''
> >    
> >    Note that Martelli says "all argument passing in Python is by value" (Python in a Nutshell, p.74), but I think this may be an error.
> >    
> >    I would be interested in seeing how others would handle this question.
> >    
> >    -- Dave
> >    
> >    P.S. The lecture went very well.  The students were enthusiastic, and the Mandelbrot images were stunning!  http://ece.arizona.edu/~edatools/ece175/Lecture  The purpose of the lecture was to introduce students to higher-level programming, and show how C can still play a role when performance is critical.  I'm hoping to expand this to a 3-week sequence of lectures.
>
>        +-----------------------------------------------
>        | Michael Goldwasser
>        | Associate Professor
>        | Dept. Mathematics and Computer Science
>        | Saint Louis University
>        | 220 North Grand Blvd.
>        | St. Louis, MO 63103-2007
>
> _______________________________________________
> Edu-sig mailing list
> [hidden email]
> http://mail.python.org/mailman/listinfo/edu-sig
>
--
John M. Zelle, Ph.D.             Wartburg College
Professor of Computer Science    Waverly, IA    
[hidden email]          (319) 352-8360  

_______________________________________________
Edu-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/edu-sig
Reply | Threaded
Open this post in threaded view
|

Re: How does Python do Pointers?

David MacQuigg
In reply to this post by David MacQuigg
Many thanks to Michael, Anna and John for the very thorough answers to this question.  I especially like John's sticky-note analogy.  I have been using a similar analogy with "labels", but that gets confused with the other more common uses of the word "label".  Sticky Note is unique and more memorable.

I'm disturbed that there doesn't seem to be any agreement on simple definitions for "call by reference" and "call by value".  A Google search for [python "call by value"] shows there is widespread confusion in the Python world.  I'm not a computer scientist, but somewhere I did learn these concepts as being simply copying the value of a variable vs passing just a reference to the original.  I see that my simple understanding agrees with Kernighan and Ritchie and with http://en.wikipedia.org/wiki/Call_by_value. (Note that "call-by-reference", as defined in this article, clearly includes Python's calling method.)

Luckily, we can answer the question without wading into this swamp.  Here is my latest revision:

'''
Python doesn't need pointers because of the simple relationship between variables and the objects they refer to.  A variable has a name and a pointer to an object.  An object has a type, a value, and an address in memory.  Unlike C, the type is with the object, not the variable.  A variable is "bound" to an object much like a sticky note on a box.  Notes can be moved from one box to another, and a box can have more than one note, or no notes at all.

Python's pointers are "under the hood", not something the programmer sees or ever needs to worry about.  This part of what makes Python a higher level language than C.  You gain a lot in simplicity and readability, while losing the ability to work directly with memory addresses.  You also lose a little memory, because objects are more complex than raw data values.

The sticky-note analogy has a flaw.  You can't stick one note on top of another.  When you say x = y = z, all three variables now point to the object originally pointed to by z.  Then when you say y = 8, y now points to an integer object 8, but x doesn't move with y.  Python's "sticky notes" have a special glue that sticks only to an object, not to another variable.

See http://effbot.org/zone/python-objects.htm for more on Python objects and variables.

Note:  You can get the address of an object x with the function id(x), but this is used only to provide a unique identity for the object, not to access it.  If id(x) == id(y), you know that variables x and y both point to the same object.
'''

-- Dave


_______________________________________________
Edu-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/edu-sig
Reply | Threaded
Open this post in threaded view
|

Re: How does Python do Pointers?

kirby urner-4
Useful discussion for me too, although my teenager students usually
aren't that familiar with C, so mapping that shoptalk to Python is
less of the issue.

On the other hand, understanding that the same objects might have many
names remains critical, also the idea of immutability, with reference
to strings, tuples and such (a good counter to having so many names,
each potentially a source of changes, as the name also "anchors" the
"api" to its object i.e. any name might change objects' values as seen
by others, via the various syntax triggers such as dot notation).

I like that we call them "names" in Python and not variables so much,
per the cited http://effbot.org/zone/python-objects.htm

People new to programming know about objects, and that objects have
names (maybe multiple, aliases and such), whereas the term "variable"
is maybe not so accessible, sounds more like old algebra, plus its not
the names that vary, but what they stick to.  To say "the identity of
x has changed" is to undo the work we just did, to distinguish
x-the-name from what it points to (its object, not its identity).

We also like calling them names because its namespaces they live in
(have currency and defined meaning within) -- which is why a given
name might reference different objects because defined in different
namespaces (still a possibility for name collisions if not careful
with importing -- why from "module import * " is often not a good
idea).  A key verb:  "to disambiguate" (a purpose of namespaces).

And having no names at all, in the sense of no references, no sticky
notes, pointers from anywhere:  that's what makes an object a
candidate for garbage collection (reference count zero **), something
else C doesn't provide out of the box either (but then CPython is C
after all, made to do trix, behave like a snake (very agile)).

Kirby


On Mon, May 5, 2008 at 3:06 PM, David MacQuigg <[hidden email]> wrote:

> Many thanks to Michael, Anna and John for the very thorough answers to this question.  I especially like John's sticky-note analogy.  I have been using a similar analogy with "labels", but that gets confused with the other more common uses of the word "label".  Sticky Note is unique and more memorable.
>
>  I'm disturbed that there doesn't seem to be any agreement on simple definitions for "call by reference" and "call by value".  A Google search for [python "call by value"] shows there is widespread confusion in the Python world.  I'm not a computer scientist, but somewhere I did learn these concepts as being simply copying the value of a variable vs passing just a reference to the original.  I see that my simple understanding agrees with Kernighan and Ritchie and with http://en.wikipedia.org/wiki/Call_by_value. (Note that "call-by-reference", as defined in this article, clearly includes Python's calling method.)
>
>  Luckily, we can answer the question without wading into this swamp.  Here is my latest revision:
>
>  '''
>  Python doesn't need pointers because of the simple relationship between variables and the objects they refer to.  A variable has a name and a pointer to an object.  An object has a type, a value, and an address in memory.  Unlike C, the type is with the object, not the variable.  A variable is "bound" to an object much like a sticky note on a box.  Notes can be moved from one box to another, and a box can have more than one note, or no notes at all.
>
>  Python's pointers are "under the hood", not something the programmer sees or ever needs to worry about.  This part of what makes Python a higher level language than C.  You gain a lot in simplicity and readability, while losing the ability to work directly with memory addresses.  You also lose a little memory, because objects are more complex than raw data values.
>
>  The sticky-note analogy has a flaw.  You can't stick one note on top of another.  When you say x = y = z, all three variables now point to the object originally pointed to by z.  Then when you say y = 8, y now points to an integer object 8, but x doesn't move with y.  Python's "sticky notes" have a special glue that sticks only to an object, not to another variable.
>
>  See http://effbot.org/zone/python-objects.htm for more on Python objects and variables.
>
>  Note:  You can get the address of an object x with the function id(x), but this is used only to provide a unique identity for the object, not to access it.  If id(x) == id(y), you know that variables x and y both point to the same object.
>  '''
>
>  -- Dave
>
>
>
>
>  _______________________________________________
>  Edu-sig mailing list
>  [hidden email]
>  http://mail.python.org/mailman/listinfo/edu-sig
>
_______________________________________________
Edu-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/edu-sig
Reply | Threaded
Open this post in threaded view
|

Re: How does Python do Pointers?

Brad Miller-6
In reply to this post by David MacQuigg
Just to muddy the waters even further, David Ranum and I have adopted the terminology call by assignment for our upcoming CS1 book.  we found this terminology a few Python references and liked it.  Here's an excerpt:

There are many different ways to pass parameters and different 
programming languages have chosen to use a variety of them. In Python, however, all parameters 
are passed using a single mechanism known as call by assignment parameter passing. 

Call by assignment parameter passing uses a simple two step process to pass data when the 
function is called, also known as invocation. The first thing that happens is that the actual parame- 
ters are evaluated. This evaluation results in an object reference to the result. In the first case from 
Session 6.11, evaluating a literal number simply returns a reference to the number itself. In the 
second example, evaluating a variable name returns the object reference named by that variable. 

Once the evaluation of the actual parameters is complete, the object references are passed to 
and received by the formal parameters in the function. The formal parameter becomes a new 
name for the reference that is passed. In a sense it is as if we executed the assignment statement 
formal parameter = actual parameter. 

Here is Session 6.11:   You can fill in the blanks for yourself on the definition of the hypotenuse function.

>>> hypotenuse(3,4) 
5.0 
>>> 
>>> side1 = 3 
>>> side2 = 4 
>>> hypotenuse(side1,side2) 
5.0 
>>> 
>>> hypotenuse(side1*2, side2*2) 
10.0 

Brad


On May 5, 2008, at 5:06 PM, David MacQuigg wrote:

Many thanks to Michael, Anna and John for the very thorough answers to this question.  I especially like John's sticky-note analogy.  I have been using a similar analogy with "labels", but that gets confused with the other more common uses of the word "label".  Sticky Note is unique and more memorable.

I'm disturbed that there doesn't seem to be any agreement on simple definitions for "call by reference" and "call by value".  A Google search for [python "call by value"] shows there is widespread confusion in the Python world.  I'm not a computer scientist, but somewhere I did learn these concepts as being simply copying the value of a variable vs passing just a reference to the original.  I see that my simple understanding agrees with Kernighan and Ritchie and with http://en.wikipedia.org/wiki/Call_by_value. (Note that "call-by-reference", as defined in this article, clearly includes Python's calling method.)

Luckily, we can answer the question without wading into this swamp.  Here is my latest revision:

'''
Python doesn't need pointers because of the simple relationship between variables and the objects they refer to.  A variable has a name and a pointer to an object.  An object has a type, a value, and an address in memory.  Unlike C, the type is with the object, not the variable.  A variable is "bound" to an object much like a sticky note on a box.  Notes can be moved from one box to another, and a box can have more than one note, or no notes at all.

Python's pointers are "under the hood", not something the programmer sees or ever needs to worry about.  This part of what makes Python a higher level language than C.  You gain a lot in simplicity and readability, while losing the ability to work directly with memory addresses.  You also lose a little memory, because objects are more complex than raw data values.

The sticky-note analogy has a flaw.  You can't stick one note on top of another.  When you say x = y = z, all three variables now point to the object originally pointed to by z.  Then when you say y = 8, y now points to an integer object 8, but x doesn't move with y.  Python's "sticky notes" have a special glue that sticks only to an object, not to another variable.

See http://effbot.org/zone/python-objects.htm for more on Python objects and variables.

Note:  You can get the address of an object x with the function id(x), but this is used only to provide a unique identity for the object, not to access it.  If id(x) == id(y), you know that variables x and y both point to the same object.
'''

-- Dave


_______________________________________________
Edu-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/edu-sig


_______________________________________________
Edu-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/edu-sig
Reply | Threaded
Open this post in threaded view
|

Re: How does Python do Pointers?

Michael H. Goldwasser
In reply to this post by kirby urner-4

This is a great discussion.   While on the topic, I'd like to share a
few more thoughts.

* The original discussion in comparing Python's model to close analogs
  from C was specifically in the context of Dave's original posting.
  I wholeheartedly agree with John/Kirby in that there is no reason to
  dig that deep with new (e.g., untainted) programmers.

* For newbies, the sticky note analogy is wonderful.  I carry around a
  pad of yellow post-it notes for the entire semester so that I can
  literally stick them to the objects on the classroom board.  It
  allows us to instill a distinction between names and underlying
  objects and to portray the assignment semantics.

* The analogy also carries over to portray the repeated assignment of
  a for loop variable to the container's elements.  If you don't mind,
  take a quick peek at the figures on pages 127-132 in Ch. 4 of our book
  (http://esminfo.prenhall.com/computing/goldwasser/Goldwasser_ch04.pdf).
  You'll notice the portrayal of the names "guests" and  "person" in
  the context of a "for person in guests:" loop.

* As an aside, when picking the style for those images, we
  intentionally emphasize that the type-awareness is encapsulated
  within the state of each object (as opposed to with the label).

* The analogy of sticky notes also cleanly portrays the parameter
  passing model.  As John and I mentioned in earlier posts, it is
  precisely the same semantics as assignment.  The formal parameter is
  assigned to the same object identified by the actual parameter.

  In the classroom we take this a step further and start using a
  second color post-it note to start laying the framework for the
  concept of namespaces.   If we had been using yellow notes for the
  names in the caller's context, we start using green notes for names
  in the local namespace of the function.

  The same mechanism is used for the return value.  BTW, as for the
  "pass-by-value", "pass-by-reference" terminology, part of the
  problem is that in the programming language community, those terms
  aren't always used consistently.  In truth, Python is more akin to
  the notion of "pass-by-name" as in Algol (however the pass-by-name
  terminology has not really caught on in general).

* There is one way in which the sticky note analogy breaks down. It
  does not accurately portray the internals of a container that
  references other objects.  For example if you look back at our
  figure on page 127, it gives a false impression that the contents
  of the list are contained "within" the list.  For beginners, this
  analogy suffices.  Also, in this particular case, the elements of
  the list are immutable and so the distinction is less significant.

  But in the long run, it is important for students to understand that
  the list is a list of references (not a list of objects).  The
  sticky note analogy does not work here because because there is no
  name per se to give an individual element (guests[i] is
  syntactically useful, but that is not a true name and the particular
  index is not persistent over time).

  To portray the underlying objects separate from the container's
  state, it becomes necessary to draw a picture more akin to the
  classic "pointer" diagrams.   Yet by this point, it is helpful to do
  so and awareness of the underlying distinction is important.

* As a final note, if students are subsequently introduced to the use
  of dictionaries in their own programming, you can go under the hood
  to give a true explanation of Python's model.  Each namespace is
  implemented as a dictionary and the names are simply strings that
  are keys in the dictionary, mapped to the associated value.  From
  within a function body, locals() command returns access to the
  dictionary representing the local namespace, while globals() returns
  the global namespace.  Further lessons of name resolution can be
  explored as vars(x) gives the instance-level dictionary for an
  object x (other than for built-in primitives which do not rely on
  dictionaries for their internals).  vars(Foo) gives the class-level
  dictionary for a class Foo.


       +-----------------------------------------------
       | Michael Goldwasser
       | Associate Professor
       | Dept. Mathematics and Computer Science
       | Saint Louis University
       | 220 North Grand Blvd.
       | St. Louis, MO 63103-2007

_______________________________________________
Edu-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/edu-sig
Reply | Threaded
Open this post in threaded view
|

Re: How does Python do Pointers?

Michael H. Goldwasser
In reply to this post by Brad Miller-6

Hi Brad,

  The problem I see with the "call-by-assignment" terminology is that
  it is ambiguous.  In fact, all parameter passing in Python, Java,
  and C++ would satisfy your call-by-assignment description.  The key
  is that the assignment semantics is different in different
  circumstances.

  To illustrate, consider a Java method with the following signature.

      public void doSomething(int x, Widget w) {  ... }

  If a caller invokes this function using a syntax such as

      foo.doSomething(myAge, myToaster)

  The information passing is precisely the same as assigning formal
  parameters to actual parameters as in

      x = myAge;
      w = myToaster;

  The difference in parameter passing models really boils down to the
  distinction between assignment semantics for the int type versus an
  Object type.

  In C++, the different forms of parameter passing also are all based
  on the underlying assignment semantics.  Calling a function with
  signature

      void doSomething(int x, Widget w) {  ... }

  is essentially the same as if the first action within the body were
  the assignments

      int x = myAge;
      Widget w = myToaster;

  Notice that both of these are value variables and so we get a new
  int and a new Widget (techincally using the copy constructor rather
  than the assignment operator, but close enough).

  If we declare a signature such as

      void doSomething(int& x, Widget& w) {  ... }

  it would be akin to the declaration and assignments

      int& x = myAge;
      Widget& w = myToaster;

  Finally, if pointers are passed with a signature

      void doSomething(int* x, Widget* w) {  ... }

  and of course now the caller uses syntax doSomething(&myAge, &myToaster)
  to send pointers, we again get precisely the same semantics as the
  implicit assignments

      int* x = &myAge;
      Widget* w = &myToaster;


  So in all of these cases, as with Python, the key seems to be having
  a correct understanding of the assignment semantics.  I wish there
  were a better commonly accepted name for the parameter passing.
  Python's model is quite clean, it just doesn't match up perfectly
  with the standard call-by-??? terminology.


With regard,
Michael


On Monday May 5, 2008, Brad Miller wrote:

>   Just to muddy the waters even further, David Ranum and I have adopted  
>   the terminology call by assignment for our upcoming CS1 book.  we  
>   found this terminology a few Python references and liked it.  Here's  
>   an excerpt:
>  
>   There are many di������erent ways to pass parameters and di������erent
>   programming languages have chosen to use a variety of them. In Python,  
>   however, all parameters
>   are passed using a single mechanism known as call by assignment  
>   parameter passing.
>  
>   Call by assignment parameter passing uses a simple two step process to  
>   pass data when the
>   function is called, also known as invocation. The ������rst thing that  
>   happens is that the actual parame-
>   ters are evaluated. This evaluation results in an object reference to  
>   the result. In the ������rst case from
>   Session 6.11, evaluating a literal number simply returns a reference  
>   to the number itself. In the
>   second example, evaluating a variable name returns the object  
>   reference named by that variable.
>  
>   Once the evaluation of the actual parameters is complete, the object  
>   references are passed to
>   and received by the formal parameters in the function. The formal  
>   parameter becomes a new
>   name for the reference that is passed. In a sense it is as if we  
>   executed the assignment statement
>   formal parameter = actual parameter.
       +-----------------------------------------------
       | Michael Goldwasser
       | Associate Professor
       | Dept. Mathematics and Computer Science
       | Saint Louis University
       | 220 North Grand Blvd.
       | St. Louis, MO 63103-2007



_______________________________________________
Edu-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/edu-sig
Reply | Threaded
Open this post in threaded view
|

Re: How does Python do Pointers?

David MacQuigg
In reply to this post by Brad Miller-6
I agree, there is no reason to dig into call-by-??? terminology with new "untainted" programmers.  The sticky-note analogy is all we need for these students.  The figures in Michael's book are excellent.

However, having been tainted by C, I am finding the discussion interesting.  I just don't understand why there is so much confusion with these call-by terms.  Does the function get a copy of the object or a reference to the original object?  It's got to be one or the other.

At 10:11 PM 5/5/2008 -0500, Michael H.Goldwasser wrote:

>  Python's model is quite clean, it just doesn't match up perfectly
>  with the standard call-by-??? terminology.

In call-by-reference evaluation, a function receives an implicit reference to the argument, rather than a copy of its value.
-- http://en.wikipedia.org/wiki/Call_by_value

By this definition, Python's model is call-by-reference, even though the reference (pointer) is not seen by the user.  C's model is call-by-value, even though that value can be an explicitly-evaluated address (pointer).

-- Dave


_______________________________________________
Edu-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/edu-sig
Reply | Threaded
Open this post in threaded view
|

Re: How does Python do Pointers?

kirby urner-4
On Tue, May 6, 2008 at 7:45 AM, David MacQuigg <[hidden email]> wrote:
> I agree, there is no reason to dig into call-by-??? terminology with new "untainted" programmers.  The sticky-note analogy is all we need for these students.  The figures in Michael's book are excellent.
>
>  However, having been tainted by C, I am finding the discussion interesting.  I just don't understand why there is so much confusion with these call-by terms.  Does the function get a copy of the object or a reference to the original object?  It's got to be one or the other.

A succinct summary?:

A function gets its own referent for the duration of its scope, but
its a referent to the original object, per names passed in the
"function mouth" (OK to not call it that, but eat( ) has that emoticon
mouth look).

The name is what passes by value, in handing off its value to a local
in-function name, a name behaving sort of like a C pointer, but more
like a generic pointer i.e. like a finger or stick pointing.

IDLE 1.2.1
>>> class Foo:
        pass

>>> o = Foo()
>>> id(o)
15107256

>>> def f(x):  print id(x)

>>> f(o)
15107256

Also (continuing in same namespace)...

>>> def g(x):  x.myname = 'dude'  # add gratuitous attribute to Foo object

>>> g(o)
>>> o.myname  # even though x is out of scope, the original object has changed
'dude'

Thanks to default arguments, we can pass arguments in any order to
some functions:

>>> f()
1 2
>>> f(y=100, x='ardvarrk') # mentioning y first, types irrelevant
ardvarrk 100

I think this is an important feature to dwell on, as many APIs give a
lot of parameters, VPython a great example, but they're all given
defaults, ergo ball(pos=(1,0,0)) and ball(color=(1,0,0)) are both
meaningful.  As the programmer using these APIs, you don't have to
care about order of the names, just the names, and even then you can
skip some.

Kirby

>
>
>  At 10:11 PM 5/5/2008 -0500, Michael H.Goldwasser wrote:
>
>  >  Python's model is quite clean, it just doesn't match up perfectly
>  >  with the standard call-by-??? terminology.
>
>  In call-by-reference evaluation, a function receives an implicit reference to the argument, rather than a copy of its value.
>  -- http://en.wikipedia.org/wiki/Call_by_value
>
>  By this definition, Python's model is call-by-reference, even though the reference (pointer) is not seen by the user.  C's model is call-by-value, even though that value can be an explicitly-evaluated address (pointer).
>
>
>
>  -- Dave
>
>
>  _______________________________________________
>  Edu-sig mailing list
>  [hidden email]
>  http://mail.python.org/mailman/listinfo/edu-sig
>
_______________________________________________
Edu-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/edu-sig
Reply | Threaded
Open this post in threaded view
|

Re: How does Python do Pointers?

kirby urner-4
>  Thanks to default arguments, we can pass arguments in any order to
>  some functions:
>

Oops, I missed my redefinition of f:

>>> def f(x=1, y=2):  print x, y

>  >>> f()
>  1 2
>  >>> f(y=100, x='ardvarrk') # mentioning y first, types irrelevant
>  ardvarrk 100
>

Plus misspelled 'aardvark' (sigh).

Kirby
_______________________________________________
Edu-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/edu-sig
Reply | Threaded
Open this post in threaded view
|

Re: How does Python do Pointers?

John Zelle
In reply to this post by David MacQuigg
David,

Actually, I don't think there is that much confusion among the folks who
understand/design/study programming languages. Python uses call by
value, period, as that term is technically used. But your note below
shows the continued confusion (among programmers) about what parameter
passing terminology. In classifying calling mechanisms, the question is
what happens with a variable being passed and the value it (contains in
the traditional usage of those words); it has nothing to do with
objects. When modeling Python semantics, a variable actually holds a
reference (pointer, address) of an object. When the variable is passed
as a parameter, its value (the reference/pointer) is copied into the
formal parameter variable. That's it. It's call by value.

The confusion and fuzzy thinking comes because the automatic
dereferencing of Python variables when they are used in expressions
leads people to think about objects being "stored in" variables. Then
they talk about whether objects being copied or not copied in function
call parameters. But an object is _never_ stored into a Python
variable.

To see the difference between passing a mutable object in Python vs. a
reference parameter in other languages, you just have to ask if you can
change the value of the calling code's variable by assigning a value to
the corresponding formal parameter in a function. In C++ or Pascal,
setting the value of a reference parameter actually changes the contents
of the variable in the calling program.

def func(formal_param):
     # change the value of formal_param to be a reference to foo_object
     formal_param = foo_object

x = bar_object
func(x)
# for pass by reference, x would now contain a foo_object
# In Python (pass by value) x is still bar_object.

In Python, you cannot change the actual parameter variable (x above). No
matter what you do in the function, the actual parameter still contains
the same thing, a reference to the same object it was always pointing to
at the time of call. That's because the function operates on it own copy
of that reference, not the _actual_variable_ of the caller.

Summary: Python's parameter passing is just call by value. But Python
variable values are always references and Python semantics dictates that
those values are dereferenced when variables are used in expressions.
That leads some programmers to (imprecisely) talk about parameter
passing in terms of variables having objects as values. I often do this
loosely myself, because it's easy to think of a Python name as a
variable that stores and object.

--John



On Tue, 2008-05-06 at 07:45 -0700, David MacQuigg wrote:

> I agree, there is no reason to dig into call-by-??? terminology with new "untainted" programmers.  The sticky-note analogy is all we need for these students.  The figures in Michael's book are excellent.
>
> However, having been tainted by C, I am finding the discussion interesting.  I just don't understand why there is so much confusion with these call-by terms.  Does the function get a copy of the object or a reference to the original object?  It's got to be one or the other.
>
> At 10:11 PM 5/5/2008 -0500, Michael H.Goldwasser wrote:
>
> >  Python's model is quite clean, it just doesn't match up perfectly
> >  with the standard call-by-??? terminology.
>
> In call-by-reference evaluation, a function receives an implicit reference to the argument, rather than a copy of its value.
> -- http://en.wikipedia.org/wiki/Call_by_value
>
> By this definition, Python's model is call-by-reference, even though the reference (pointer) is not seen by the user.  C's model is call-by-value, even though that value can be an explicitly-evaluated address (pointer).
>
> -- Dave
>
>
> _______________________________________________
> Edu-sig mailing list
> [hidden email]
> http://mail.python.org/mailman/listinfo/edu-sig
>

_______________________________________________
Edu-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/edu-sig
Reply | Threaded
Open this post in threaded view
|

Re: How does Python do Pointers?

Richard Enbody
In reply to this post by David MacQuigg
This discussion has been interesting: we could really drive this
discussion into the obscure by bringing in  'passing by name' where
actual parameters remain unevaluated until the point of use.  :-)

I also agree that Michael's 'post-it-note' analogy is a good one for
novices.

An interesting issue to consider is how to best set up students to
understand pointers in subsequent languages.  The 'post-it-note' analogy
seems to be a reasonable starting point.  If one understands naming from
that standpoint, transitioning to pointers could be reasonable.  I
believe we found that to be the case in our transition to a C++ based
CS2 course, but I haven't examined the data carefully yet.

-rich
[hidden email]

David MacQuigg wrote:

> I agree, there is no reason to dig into call-by-??? terminology with new "untainted" programmers.  The sticky-note analogy is all we need for these students.  The figures in Michael's book are excellent.
>
> However, having been tainted by C, I am finding the discussion interesting.  I just don't understand why there is so much confusion with these call-by terms.  Does the function get a copy of the object or a reference to the original object?  It's got to be one or the other.
>
> At 10:11 PM 5/5/2008 -0500, Michael H.Goldwasser wrote:
>
>  
>>  Python's model is quite clean, it just doesn't match up perfectly
>>  with the standard call-by-??? terminology.
>>    
>
> In call-by-reference evaluation, a function receives an implicit reference to the argument, rather than a copy of its value.
> -- http://en.wikipedia.org/wiki/Call_by_value
>
> By this definition, Python's model is call-by-reference, even though the reference (pointer) is not seen by the user.  C's model is call-by-value, even though that value can be an explicitly-evaluated address (pointer).
>
> -- Dave
>
>
> _______________________________________________
> Edu-sig mailing list
> [hidden email]
> http://mail.python.org/mailman/listinfo/edu-sig
>  
_______________________________________________
Edu-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/edu-sig
Reply | Threaded
Open this post in threaded view
|

Re: How does Python do Pointers?

David MacQuigg
In reply to this post by John Zelle
John,

This is the best explanation I've heard so far for why Python is "call by value", but it still leaves me dissatisfied that what I thought was simple (call-by-value) is really complicated and not very useful to me, and what is truly simple (passing a copy of the object) is a useful concept with no name.  I also wonder if I'm completely misunderstanding the Wikipedia article (and other texts like K&R - Kernighan and Ritchie p. 27), or if there aren't two schools of thought with conflicting terminology.

As usual, the confusion in the discussion arises when there are differing underlying assumptions, unstated and unrecognized.  I was assuming the word "value" referred to a part of the object in the simple scheme - variable:(name, pointer) --> object:(type, value, address).  I see now that "value" can mean the pointer, and that making a copy of the pointer can be passing a "value".  Perhaps we should refer to the object as having (type, data, address) and save the word "value" for the meaning you assume.

At 10:55 AM 5/6/2008 -0500, John Zelle wrote:

>David,
>
>Actually, I don't think there is that much confusion among the folks who
>understand/design/study programming languages. Python uses call by
>value, period, as that term is technically used. But your note below
>shows the continued confusion (among programmers) about what parameter
>passing terminology. In classifying calling mechanisms, the question is
>what happens with a variable being passed and the value it (contains in
>the traditional usage of those words); it has nothing to do with
>objects. When modeling Python semantics, a variable actually holds a
>reference (pointer, address) of an object. When the variable is passed
>as a parameter, its value (the reference/pointer) is copied into the
>formal parameter variable. That's it. It's call by value.

Understood now.  The value is the pointer, not the data.

>The confusion and fuzzy thinking comes because the automatic
>dereferencing of Python variables when they are used in expressions
>leads people to think about objects being "stored in" variables. Then
>they talk about whether objects being copied or not copied in function
>call parameters. But an object is _never_ stored into a Python
>variable.

Never any misunderstanding for me.

>To see the difference between passing a mutable object in Python vs. a
>reference parameter in other languages, you just have to ask if you can
>change the value of the calling code's variable by assigning a value to
>the corresponding formal parameter in a function.

With your definition of "value", it never changes.  The assignment creates a new local variable in the function's namespace.

> In C++ or Pascal,
>setting the value of a reference parameter actually changes the contents
>of the variable in the calling program.

Understood, but now we are using the previous definition of "value".  If we use the definition above, changing the "value" just points the parameter name in the function to some other address.

In C if I have a parameter (int *x) then x is the address of an int.  The actual value passed is an address, and an assignment statement in the function can change the data at that address.  But it is still "call-by-value" according to K&R, since the value in this case is an explicit address, with 32 bits of data looking just like an unsigned integer.

>def func(formal_param):
>     # change the value of formal_param to be a reference to foo_object
>     formal_param = foo_object
>
>x = bar_object
>func(x)
># for pass by reference, x would now contain a foo_object
># In Python (pass by value) x is still bar_object.
>
>In Python, you cannot change the actual parameter variable (x above). No
>matter what you do in the function, the actual parameter still contains
>the same thing, a reference to the same object it was always pointing to
>at the time of call. That's because the function operates on it own copy
>of that reference, not the _actual_variable_ of the caller.
>
>Summary: Python's parameter passing is just call by value. But Python
>variable values are always references and Python semantics dictates that
>those values are dereferenced when variables are used in expressions.
>That leads some programmers to (imprecisely) talk about parameter
>passing in terms of variables having objects as values. I often do this
>loosely myself, because it's easy to think of a Python name as a
>variable that stores and object.

When I hear "variable x has value y" I think of x pointing to object y, not storing y, not even loosely.  My confusion is not due to misunderstanding what Python actually does.

It seems like there might be a simple alternative to the traditional definition of "call-by-value" in terms of result rather than mechanism.  The traditional definition (K&R) is simple if you think of mechanism.  The new definition is simple if you think of result (argument in calling program not changeable by the called function).

Should we add a note to that Wikipedia page?

-- Dave


>On Tue, 2008-05-06 at 07:45 -0700, David MacQuigg wrote:
>> I agree, there is no reason to dig into call-by-??? terminology with new "untainted" programmers.  The sticky-note analogy is all we need for these students.  The figures in Michael's book are excellent.
>>
>> However, having been tainted by C, I am finding the discussion interesting.  I just don't understand why there is so much confusion with these call-by terms.  Does the function get a copy of the object or a reference to the original object?  It's got to be one or the other.
>>
>> At 10:11 PM 5/5/2008 -0500, Michael H.Goldwasser wrote:
>>
>> >  Python's model is quite clean, it just doesn't match up perfectly
>> >  with the standard call-by-??? terminology.
>>
>> In call-by-reference evaluation, a function receives an implicit reference to the argument, rather than a copy of its value.
>> -- http://en.wikipedia.org/wiki/Call_by_value
>>
>> By this definition, Python's model is call-by-reference, even though the reference (pointer) is not seen by the user.  C's model is call-by-value, even though that value can be an explicitly-evaluated address (pointer).
>>
>> -- Dave
>>
>>
>> _______________________________________________
>> Edu-sig mailing list
>> [hidden email]
>> http://mail.python.org/mailman/listinfo/edu-sig
>>


_______________________________________________
Edu-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/edu-sig
Reply | Threaded
Open this post in threaded view
|

Re: How does Python do Pointers?

David MacQuigg
Fredrik Lundh has yet another point-of-view:

http://effbot.org/zone/call-by-object.htm
'''
Python’s model is neither "call by value" nor "call by reference" (because any attempt to use those terms for Python requires you to use non-standard definitions of the words "value" and "reference"). The most accurate description is CLU’s "call by object" or "call by sharing". Or, if you prefer, "call by object reference".
...
IT IS NOT CALL BY VALUE because mutations of arguments performed by the called routine will be visible to the caller. And IT IS NOT CALL BY REFERENCE because access is not given to the variables of the caller, but merely to certain objects.
~ '''

I guess what we need is a taxonomy of definitions: strict/loose, mechanism/result.  I prefer a loose definition based on mechanism, like Kernighan and Ritchie.  So for me it's call-by-reference, but I don't mind softening that a bit with "like call-by-reference".  For Lundh, its strict definitions only, and Python fits neither.

Anyway, it looks like the waters are so muddy now, that the simple concepts of the early days have lost their value.  None of this should pollute our classrooms.  Let's all stick with sticky notes. :>)

-- Dave


At 11:47 AM 5/6/2008 -0700, David MacQuigg wrote:

>John,
>
>This is the best explanation I've heard so far for why Python is "call by value", but it still leaves me dissatisfied that what I thought was simple (call-by-value) is really complicated and not very useful to me, and what is truly simple (passing a copy of the object) is a useful concept with no name.  I also wonder if I'm completely misunderstanding the Wikipedia article (and other texts like K&R - Kernighan and Ritchie p. 27), or if there aren't two schools of thought with conflicting terminology.
>
>As usual, the confusion in the discussion arises when there are differing underlying assumptions, unstated and unrecognized.  I was assuming the word "value" referred to a part of the object in the simple scheme - variable:(name, pointer) --> object:(type, value, address).  I see now that "value" can mean the pointer, and that making a copy of the pointer can be passing a "value".  Perhaps we should refer to the object as having (type, data, address) and save the word "value" for the meaning you assume.
>
>At 10:55 AM 5/6/2008 -0500, John Zelle wrote:
>
>>David,
>>
>>Actually, I don't think there is that much confusion among the folks who
>>understand/design/study programming languages. Python uses call by
>>value, period, as that term is technically used. But your note below
>>shows the continued confusion (among programmers) about what parameter
>>passing terminology. In classifying calling mechanisms, the question is
>>what happens with a variable being passed and the value it (contains in
>>the traditional usage of those words); it has nothing to do with
>>objects. When modeling Python semantics, a variable actually holds a
>>reference (pointer, address) of an object. When the variable is passed
>>as a parameter, its value (the reference/pointer) is copied into the
>>formal parameter variable. That's it. It's call by value.
>
>Understood now.  The value is the pointer, not the data.
>
>>The confusion and fuzzy thinking comes because the automatic
>>dereferencing of Python variables when they are used in expressions
>>leads people to think about objects being "stored in" variables. Then
>>they talk about whether objects being copied or not copied in function
>>call parameters. But an object is _never_ stored into a Python
>>variable.
>
>Never any misunderstanding for me.
>
>>To see the difference between passing a mutable object in Python vs. a
>>reference parameter in other languages, you just have to ask if you can
>>change the value of the calling code's variable by assigning a value to
>>the corresponding formal parameter in a function.
>
>With your definition of "value", it never changes.  The assignment creates a new local variable in the function's namespace.
>
>> In C++ or Pascal,
>>setting the value of a reference parameter actually changes the contents
>>of the variable in the calling program.
>
>Understood, but now we are using the previous definition of "value".  If we use the definition above, changing the "value" just points the parameter name in the function to some other address.
>
>In C if I have a parameter (int *x) then x is the address of an int.  The actual value passed is an address, and an assignment statement in the function can change the data at that address.  But it is still "call-by-value" according to K&R, since the value in this case is an explicit address, with 32 bits of data looking just like an unsigned integer.
>
>>def func(formal_param):
>>     # change the value of formal_param to be a reference to foo_object
>>     formal_param = foo_object
>>
>>x = bar_object
>>func(x)
>># for pass by reference, x would now contain a foo_object
>># In Python (pass by value) x is still bar_object.
>>
>>In Python, you cannot change the actual parameter variable (x above). No
>>matter what you do in the function, the actual parameter still contains
>>the same thing, a reference to the same object it was always pointing to
>>at the time of call. That's because the function operates on it own copy
>>of that reference, not the _actual_variable_ of the caller.
>>
>>Summary: Python's parameter passing is just call by value. But Python
>>variable values are always references and Python semantics dictates that
>>those values are dereferenced when variables are used in expressions.
>>That leads some programmers to (imprecisely) talk about parameter
>>passing in terms of variables having objects as values. I often do this
>>loosely myself, because it's easy to think of a Python name as a
>>variable that stores and object.
>
>When I hear "variable x has value y" I think of x pointing to object y, not storing y, not even loosely.  My confusion is not due to misunderstanding what Python actually does.
>
>It seems like there might be a simple alternative to the traditional definition of "call-by-value" in terms of result rather than mechanism.  The traditional definition (K&R) is simple if you think of mechanism.  The new definition is simple if you think of result (argument in calling program not changeable by the called function).
>
>Should we add a note to that Wikipedia page?
>
>-- Dave
>
>
>>On Tue, 2008-05-06 at 07:45 -0700, David MacQuigg wrote:
>>> I agree, there is no reason to dig into call-by-??? terminology with new "untainted" programmers.  The sticky-note analogy is all we need for these students.  The figures in Michael's book are excellent.
>>>
>>> However, having been tainted by C, I am finding the discussion interesting.  I just don't understand why there is so much confusion with these call-by terms.  Does the function get a copy of the object or a reference to the original object?  It's got to be one or the other.
>>>
>>> At 10:11 PM 5/5/2008 -0500, Michael H.Goldwasser wrote:
>>>
>>> >  Python's model is quite clean, it just doesn't match up perfectly
>>> >  with the standard call-by-??? terminology.
>>>
>>> In call-by-reference evaluation, a function receives an implicit reference to the argument, rather than a copy of its value.
>>> -- http://en.wikipedia.org/wiki/Call_by_value
>>>
>>> By this definition, Python's model is call-by-reference, even though the reference (pointer) is not seen by the user.  C's model is call-by-value, even though that value can be an explicitly-evaluated address (pointer).
>>>
>>> -- Dave
>>>
>>>
>>> _______________________________________________
>>> Edu-sig mailing list
>>> [hidden email]
>>> http://mail.python.org/mailman/listinfo/edu-sig
>>>
>
>
>_______________________________________________
>Edu-sig mailing list
>[hidden email]
>http://mail.python.org/mailman/listinfo/edu-sig


_______________________________________________
Edu-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/edu-sig
Reply | Threaded
Open this post in threaded view
|

Re: How does Python do Pointers?

kirby urner-4
Although I like the post-its analogy, good advertising for 3M, there's
an implied thesis about proximity (post-its "stick to" something)
whereas I'd like students to thing of objects as potentially very
heavy and far removed, connected more by wire or rope.

This matches very early conversations about edu-sig about Python's
bias against copying, i.e. you need to import from a separate module,
and even then, deepcopy is something special, considered usually
unneeded, i.e. Python the design, like any design, encourages and
discourages ways of thinking, and Guido is pretty clear about what
he's trying to cut down on (copying).

Because once your names really do point to objects, potentially
associated with huge amounts of data (as objects may be), then you
really *don't* want some x = y assignment to involve truly copying
anything, as that implies potentially major consequences in memory
sometimes.

So I'd say Python's emphasis on names sharing an object, is consistent
with Python's strong bias against copying.

I like the point made earlier that parameter passing at the function
gateway is all about assignment, and assignment is all about cloning a
pointer or reference, for use within the local scope, and these names
are by definition something tiny, whereas the objects themselves --
who knows how big these might be.

Kirby

2008/5/6 David MacQuigg <[hidden email]>:

> Fredrik Lundh has yet another point-of-view:
>
>  http://effbot.org/zone/call-by-object.htm
>  '''
>  Python's model is neither "call by value" nor "call by reference" (because any attempt to use those terms for Python requires you to use non-standard definitions of the words "value" and "reference"). The most accurate description is CLU's "call by object" or "call by sharing". Or, if you prefer, "call by object reference".
>  ...
>  IT IS NOT CALL BY VALUE because mutations of arguments performed by the called routine will be visible to the caller. And IT IS NOT CALL BY REFERENCE because access is not given to the variables of the caller, but merely to certain objects.
>  ~ '''
>
>  I guess what we need is a taxonomy of definitions: strict/loose, mechanism/result.  I prefer a loose definition based on mechanism, like Kernighan and Ritchie.  So for me it's call-by-reference, but I don't mind softening that a bit with "like call-by-reference".  For Lundh, its strict definitions only, and Python fits neither.
>
>  Anyway, it looks like the waters are so muddy now, that the simple concepts of the early days have lost their value.  None of this should pollute our classrooms.  Let's all stick with sticky notes. :>)
>
>  -- Dave
>
>
>
>
>  At 11:47 AM 5/6/2008 -0700, David MacQuigg wrote:
>
>  >John,
>  >
>  >This is the best explanation I've heard so far for why Python is "call by value", but it still leaves me dissatisfied that what I thought was simple (call-by-value) is really complicated and not very useful to me, and what is truly simple (passing a copy of the object) is a useful concept with no name.  I also wonder if I'm completely misunderstanding the Wikipedia article (and other texts like K&R - Kernighan and Ritchie p. 27), or if there aren't two schools of thought with conflicting terminology.
>  >
>  >As usual, the confusion in the discussion arises when there are differing underlying assumptions, unstated and unrecognized.  I was assuming the word "value" referred to a part of the object in the simple scheme - variable:(name, pointer) --> object:(type, value, address).  I see now that "value" can mean the pointer, and that making a copy of the pointer can be passing a "value".  Perhaps we should refer to the object as having (type, data, address) and save the word "value" for the meaning you assume.
>  >
>  >At 10:55 AM 5/6/2008 -0500, John Zelle wrote:
>  >
>  >>David,
>  >>
>  >>Actually, I don't think there is that much confusion among the folks who
>  >>understand/design/study programming languages. Python uses call by
>  >>value, period, as that term is technically used. But your note below
>  >>shows the continued confusion (among programmers) about what parameter
>  >>passing terminology. In classifying calling mechanisms, the question is
>  >>what happens with a variable being passed and the value it (contains in
>  >>the traditional usage of those words); it has nothing to do with
>  >>objects. When modeling Python semantics, a variable actually holds a
>  >>reference (pointer, address) of an object. When the variable is passed
>  >>as a parameter, its value (the reference/pointer) is copied into the
>  >>formal parameter variable. That's it. It's call by value.
>  >
>  >Understood now.  The value is the pointer, not the data.
>  >
>  >>The confusion and fuzzy thinking comes because the automatic
>  >>dereferencing of Python variables when they are used in expressions
>  >>leads people to think about objects being "stored in" variables. Then
>  >>they talk about whether objects being copied or not copied in function
>  >>call parameters. But an object is _never_ stored into a Python
>  >>variable.
>  >
>  >Never any misunderstanding for me.
>  >
>  >>To see the difference between passing a mutable object in Python vs. a
>  >>reference parameter in other languages, you just have to ask if you can
>  >>change the value of the calling code's variable by assigning a value to
>  >>the corresponding formal parameter in a function.
>  >
>  >With your definition of "value", it never changes.  The assignment creates a new local variable in the function's namespace.
>  >
>  >> In C++ or Pascal,
>  >>setting the value of a reference parameter actually changes the contents
>  >>of the variable in the calling program.
>  >
>  >Understood, but now we are using the previous definition of "value".  If we use the definition above, changing the "value" just points the parameter name in the function to some other address.
>  >
>  >In C if I have a parameter (int *x) then x is the address of an int.  The actual value passed is an address, and an assignment statement in the function can change the data at that address.  But it is still "call-by-value" according to K&R, since the value in this case is an explicit address, with 32 bits of data looking just like an unsigned integer.
>  >
>  >>def func(formal_param):
>  >>     # change the value of formal_param to be a reference to foo_object
>  >>     formal_param = foo_object
>  >>
>  >>x = bar_object
>  >>func(x)
>  >># for pass by reference, x would now contain a foo_object
>  >># In Python (pass by value) x is still bar_object.
>  >>
>  >>In Python, you cannot change the actual parameter variable (x above). No
>  >>matter what you do in the function, the actual parameter still contains
>  >>the same thing, a reference to the same object it was always pointing to
>  >>at the time of call. That's because the function operates on it own copy
>  >>of that reference, not the _actual_variable_ of the caller.
>  >>
>  >>Summary: Python's parameter passing is just call by value. But Python
>  >>variable values are always references and Python semantics dictates that
>  >>those values are dereferenced when variables are used in expressions.
>  >>That leads some programmers to (imprecisely) talk about parameter
>  >>passing in terms of variables having objects as values. I often do this
>  >>loosely myself, because it's easy to think of a Python name as a
>  >>variable that stores and object.
>  >
>  >When I hear "variable x has value y" I think of x pointing to object y, not storing y, not even loosely.  My confusion is not due to misunderstanding what Python actually does.
>  >
>  >It seems like there might be a simple alternative to the traditional definition of "call-by-value" in terms of result rather than mechanism.  The traditional definition (K&R) is simple if you think of mechanism.  The new definition is simple if you think of result (argument in calling program not changeable by the called function).
>  >
>  >Should we add a note to that Wikipedia page?
>  >
>  >-- Dave
>  >
>  >
>  >>On Tue, 2008-05-06 at 07:45 -0700, David MacQuigg wrote:
>  >>> I agree, there is no reason to dig into call-by-??? terminology with new "untainted" programmers.  The sticky-note analogy is all we need for these students.  The figures in Michael's book are excellent.
>  >>>
>  >>> However, having been tainted by C, I am finding the discussion interesting.  I just don't understand why there is so much confusion with these call-by terms.  Does the function get a copy of the object or a reference to the original object?  It's got to be one or the other.
>  >>>
>  >>> At 10:11 PM 5/5/2008 -0500, Michael H.Goldwasser wrote:
>  >>>
>  >>> >  Python's model is quite clean, it just doesn't match up perfectly
>  >>> >  with the standard call-by-??? terminology.
>  >>>
>  >>> In call-by-reference evaluation, a function receives an implicit reference to the argument, rather than a copy of its value.
>  >>> -- http://en.wikipedia.org/wiki/Call_by_value
>  >>>
>  >>> By this definition, Python's model is call-by-reference, even though the reference (pointer) is not seen by the user.  C's model is call-by-value, even though that value can be an explicitly-evaluated address (pointer).
>  >>>
>  >>> -- Dave
>  >>>
>  >>>
>  >>> _______________________________________________
>  >>> Edu-sig mailing list
>  >>> [hidden email]
>  >>> http://mail.python.org/mailman/listinfo/edu-sig
>  >>>
>  >
>  >
>  >_______________________________________________
>  >Edu-sig mailing list
>  >[hidden email]
>  >http://mail.python.org/mailman/listinfo/edu-sig
>
>
>
> _______________________________________________
>  Edu-sig mailing list
>  [hidden email]
>  http://mail.python.org/mailman/listinfo/edu-sig
>
>
_______________________________________________
Edu-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/edu-sig
Reply | Threaded
Open this post in threaded view
|

Re: How does Python do Pointers?

kirby urner-4
Just to add a few words, I think a big dog, like an English Mastiff,
tugging really hard on a leash, would be a good place to start, in
some cartoon YouTube about Python.

When I get to some dog washing storefront, I hand another leash to the
dog washer, same dog at the end, washer goes inside, has control of
the dog object, does local scope thing with soap. The dog at the other
end of my leash (was there the whole time) gets a good cleanup (thanks
to this function).

Maybe better than a post-it is a remote controller like for robot, or
console game system?  Lots of little buttons, some strangely shaped.

Your having a name (handle, token) means you can do stuff like
dog.attribute, dog( ) -- called calling, triggers __call__ -- dog[ ]
(__getitem__) other syntax games (maybe use operators as in dogA +
dogB).

The point is your "post-it" (remote) has all these triggering
abilities, makes you the puppet master of the (maybe faraway) object.

So it's a combination of those images:  like a leash, or a rope, a
connection, but also a set of controls, ways to send messages, read
state, trigger methods (however we want to talk about it -- standard
OO ways of doing so).

When we get to a function, we hand over a set of controls (a remote,
an api), which get destroyed when they go out of scope, but in the
meantime give the local scope some control or at least contact with
the underlying controlees (the objects).

These objects usually have no way of knowing the identities of their
potentially many handlers -- you could code that in some higher level
MVC, but Python out of the box keeps objects in the dark about such
matters.

Kirby
4D

Sorry for typos below:

On Tue, May 6, 2008 at 3:43 PM, kirby urner <[hidden email]> wrote:

> Although I like the post-its analogy, good advertising for 3M, there's
>  an implied thesis about proximity (post-its "stick to" something)
>  whereas I'd like students to thing of objects as potentially very
>  heavy and far removed, connected more by wire or rope.
>
>  This matches very early conversations about edu-sig about Python's
>  bias against copying, i.e. you need to import from a separate module,
>  and even then, deepcopy is something special, considered usually
>  unneeded, i.e. Python the design, like any design, encourages and
>  discourages ways of thinking, and Guido is pretty clear about what
>  he's trying to cut down on (copying).
>
>  Because once your names really do point to objects, potentially
>  associated with huge amounts of data (as objects may be), then you
>  really *don't* want some x = y assignment to involve truly copying
>  anything, as that implies potentially major consequences in memory
>  sometimes.
>
>  So I'd say Python's emphasis on names sharing an object, is consistent
>  with Python's strong bias against copying.
>
>  I like the point made earlier that parameter passing at the function
>  gateway is all about assignment, and assignment is all about cloning a
>  pointer or reference, for use within the local scope, and these names
>  are by definition something tiny, whereas the objects themselves --
>  who knows how big these might be.
>
>  Kirby
_______________________________________________
Edu-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/edu-sig
Reply | Threaded
Open this post in threaded view
|

Re: How does Python do Pointers?

kirby urner-4
>  >  I like the point made earlier that parameter passing at the function
>  >  gateway is all about assignment, and assignment is all about cloning a
>  >  pointer or reference, for use within the local scope, and these names
>  >  are by definition something tiny, whereas the objects themselves --
>  >  who knows how big these might be.
>  >
>  >  Kirby
>

Per my blog entry on this topic (mostly just pointing back here

http://mybizmo.blogspot.com/2008/05/memory-managment.html

), you might explicitly do the assignment x = y right in the
function header e.g. def g (x = y): is legal provide y has meaning
in the global scope by the time we reach this definition.

So to prove arguments "pass by assignment" you could go:

>>> class MegaLith:
        pass

>>> y = MegaLith()    # y gets to control one

>>> def f(x = y):
        x.color = "grey"  #  x does too, interimly...

Note you'd get an error if y weren't already defined.  The
more normal think is to have your right side defaults
be hard-coded values, not variables (names).
       
>>> f( )              # don't forget to actually do the work
>>> y.color
'grey'
>>> # tada!
>>>

The picture here would be x and y both tied to the same
MegaLith object, but x only while we're in the scope of
function f.

Kirby
_______________________________________________
Edu-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/edu-sig
Reply | Threaded
Open this post in threaded view
|

Re: How does Python do Pointers?

John Posner-3
In reply to this post by David MacQuigg
> The sticky-note analogy has a flaw.  You can't stick one
> note on top
> of another.  When you say x = y = z, all three variables
> now point to
> the object originally pointed to by z.  Then when you say y = 8,
> y now points to an integer object 8, but x doesn't move with y.  
> Python's "sticky notes" have a special glue that sticks only to an
> object, not to another variable.

IMHO it's not a flaw, but a blessing. Extending the "sticky notes" analogy
by referring to "special glue" is a very intuitive way to explain the
language feature you describe above -- assigning then reassigning "y". You
might not like that language feature, but don't place the blame on the
analogy!


>  In the classroom we take this a step further and start using a
>  second color post-it note to start laying the framework for the
>  concept of namespaces.   If we had been using yellow notes for the
>  names in the caller's context, we start using green notes for names
>  in the local namespace of the function.

Excellent! And you could also use multiple sticky-note colors (or multiple
shapes -- they exist, too) to introduce the concept of typed variables.
Switching for a moment to that other thread ("Introducing Python at our
community college"), I don't think that anyone mentioned using "assert" to
implement typed variables. Example:

    def myfunc(stringparm, listparm, intparm):
        """
        function with typed parameters,
        courtesy of "assert"
        """
        # check parameter types
        assert type(stringparm) is type('abc')
        assert type(listparm) is type(['a', 'b'])
        assert type(intparm) is type(123)

        # do the work
        print "All OK"
        return
    # ------------------------------------------
    >>> f.myfunc('a', range(3), 44)
    All OK

    >>> f.myfunc(range(3), 44, 'a')
    Traceback (most recent call last):
      File "c:\temp\<string>", line 1, in <module>
      File "c:\temp\f.py", line 6, in myfunc
        assert type(stringparm) is type('abc')
    AssertionError:

> * There is one way in which the sticky note analogy breaks down. It
>   does not accurately portray the internals of a container that
>   references other objects.  For example if you look back at our
>   figure on page 127, it gives a false impression that the contents
>   of the list are contained "within" the list.  For beginners, this
>   analogy suffices.  Also, in this particular case, the elements of
>   the list are immutable and so the distinction is less significant.

I'm not sure, but I think the sticky-notes analogy *can* be extended to
container objects (though it seems that John Zelle doesn't do this in his
textbook):

  * dictionary: an unordered set of sticky-notes
  * list: an ordered set of sticky-notes, in which the names on the notes
    can be changed dynamically by the Python interpreter -- e.g. when a new
member
    of the list is inserted.

I would agree, though, that analogies shouldn't be extended past the point
where they help students understand the actual language features.


And just to get a word in on the original question -- how does Python do
pointers -- what about iterators? From a functional viewpoint, isn't the
next() operation just like incrementing a pointer?

-John





_______________________________________________
Edu-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/edu-sig
123