Building CPython

classic Classic list List threaded Threaded
54 messages Options
123
Reply | Threaded
Open this post in threaded view
|

Building CPython

BartC-3
I'm interested in playing with the CPython sources. I need to be able to
build under Windows, but don't want to use make files (which rarely work
properly), nor do a 6GB installation of Visual Studio Express which is
what seems to be needed (I'm hopeless with complicated IDEs anyway).

Is it possible to do this by using mingw-gcc to compile the .c files of
the Python sources one by one, or is it one of those complicated
projects where some of the source is generated as it goes along?

I thought I'd start with the source file containing Py_Main and continue
from there, but some modules compile and some don't, obscure errors that
I don't want to investigate unless it's going to be worthwhile (ie.
eventually ending up with a python.exe that can run simple .py programs).

--
Bartc


Reply | Threaded
Open this post in threaded view
|

Building CPython

Mark Lawrence
On 13/05/2015 20:36, BartC wrote:

> I'm interested in playing with the CPython sources. I need to be able to
> build under Windows, but don't want to use make files (which rarely work
> properly), nor do a 6GB installation of Visual Studio Express which is
> what seems to be needed (I'm hopeless with complicated IDEs anyway).
>
> Is it possible to do this by using mingw-gcc to compile the .c files of
> the Python sources one by one, or is it one of those complicated
> projects where some of the source is generated as it goes along?
>
> I thought I'd start with the source file containing Py_Main and continue
> from there, but some modules compile and some don't, obscure errors that
> I don't want to investigate unless it's going to be worthwhile (ie.
> eventually ending up with a python.exe that can run simple .py programs).
>

Before you spend too much time on the mingw route, check out the
outstanding issues for it on the bug tracker.  Then you'll maybe realise
that using the supported VS setup is far easier.  Everything you need is
in the PCBuild directory, including pcbuild.sln for use with VS and
build.bat for command line use.  Details here
https://docs.python.org/devguide/setup.html#windows

--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



Reply | Threaded
Open this post in threaded view
|

Building CPython

Terry Reedy
In reply to this post by BartC-3
On 5/13/2015 3:36 PM, BartC wrote:
> I'm interested in playing with the CPython sources. I need to be able to
> build under Windows, but don't want to use make files (which rarely work
> properly), nor do a 6GB installation of Visual Studio Express which is
> what seems to be needed (I'm hopeless with complicated IDEs anyway).

Once installed hg or tortoisehg (I use this) and VSE are installed and
repository cloned, are half done. At command prompt, with top directory
of repository as current directory enter
tools\scripts\external.bat
Double-clicking file in Explorer does not work.
Usually only needs to be done once per branch after x.y.0 release as
dependencies are usually not updated for bugfix releases.

Then in same directory enter
pcbuild\python.sln
or double click in Explorer or open VSE and open this file.
Hit F7, wait until get line like
========== Build: 1 succeeded, 0 failed, 24 up-to-date, 1 skipped,
hit F5, pin python_d to taskbar (optional, but handy), and go.

And read devguide.

--
Terry Jan Reedy



Reply | Threaded
Open this post in threaded view
|

Building CPython

BartC-3
In reply to this post by BartC-3
On 13/05/2015 23:34, Terry Reedy wrote:

> On 5/13/2015 3:36 PM, BartC wrote:
>> I'm interested in playing with the CPython sources. I need to be able to
>> build under Windows, but don't want to use make files (which rarely work
>> properly), nor do a 6GB installation of Visual Studio Express which is
>> what seems to be needed (I'm hopeless with complicated IDEs anyway).
>
> Once installed hg or tortoisehg (I use this) and VSE are installed and
> repository cloned, are half done. At command prompt, with top directory
> of repository as current directory enter
> tools\scripts\external.bat
> Double-clicking file in Explorer does not work.
> Usually only needs to be done once per branch after x.y.0 release as
> dependencies are usually not updated for bugfix releases.
>
> Then in same directory enter
> pcbuild\python.sln
> or double click in Explorer or open VSE and open this file.
> Hit F7, wait until get line like
> ========== Build: 1 succeeded, 0 failed, 24 up-to-date, 1 skipped,
> hit F5, pin python_d to taskbar (optional, but handy), and go.

OK, the answer seems to be No then - you can't just trivially compile
the C modules that comprise the sources with the nearest compiler to
hand. So much for C's famous portability!

(Actually, I think you already lost me on your first line.)

That's a shame because I wanted to tinker with the main dispatcher loop
to try and find out what exactly is making it slow. Nothing that seems
obvious at first sight. (The comments even talk about improving branch
prediction on certain architectures, even though the performance is a
couple of magnitudes away from that kind of optimisation being relevant.)

Perhaps I was hoping there were some options turned on by default which,
if disabled, would suddenly double the speed of simple benchmarks. Now I
won't be able to find out...

--
Bartc



Reply | Threaded
Open this post in threaded view
|

Building CPython

Chris Angelico
On Fri, May 15, 2015 at 1:51 AM, BartC <bc at freeuk.com> wrote:

> OK, the answer seems to be No then - you can't just trivially compile the C
> modules that comprise the sources with the nearest compiler to hand. So much
> for C's famous portability!
>
> (Actually, I think you already lost me on your first line.)
>
> That's a shame because I wanted to tinker with the main dispatcher loop to
> try and find out what exactly is making it slow. Nothing that seems obvious
> at first sight. (The comments even talk about improving branch prediction on
> certain architectures, even though the performance is a couple of magnitudes
> away from that kind of optimisation being relevant.)

C's portability isn't really sufficient for building a huge project,
so what you generally end up with is a huge slab of common code that
doesn't change from platform to platform, plus a (relatively) tiny
section of platform-specific code, such as makefiles/project files,
linker definitions, and so on. When you start hacking on CPython, you
don't generally have to consider which platform you're aiming at, as
long as you're building on one of the ones that's well supported;
trying to port Python to a new compiler on a known OS is actually
about as much work as porting it to a known compiler on a new OS. (I
know this, because I've attempted both - using mingw on Windows, and
gcc on OS/2. It's a big job either way.)

If you want to just quickly play around with CPython's sources, I
would strongly recommend getting yourself a Linux box. Either spin up
some actual hardware with actual Linux, or grab a virtualization
engine like VMWare, VirtualBox, etc, etc, and installing into a VM.
With a Debian-based Linux (Debian, Ubuntu, Mint, etc), you should
simply be able to:

sudo apt-get build-dep python3

to get all the build dependencies for Python 3; that, plus the source
code, should be enough to get you a'building. Similar simplicities are
available for other Linux distros, but I'll let someone else recommend
them.

Even when you have all the appropriate build tools, Windows can be at
times a pain for building software on. The general philosophy of
Windows is that you should normally be taking ready-made binaries; the
general philosophy of Linux is that it's perfectly normal to spin up
your own binaries from the distributed source code. It's not
impossible to build on Windows, by any means, but be prepared for
extra hassles and less support from the OS than you might like.

ChrisA


Reply | Threaded
Open this post in threaded view
|

Building CPython

Marko Rauhamaa
In reply to this post by BartC-3
BartC <bc at freeuk.com>:

> That's a shame because I wanted to tinker with the main dispatcher
> loop to try and find out what exactly is making it slow. Nothing that
> seems obvious at first sight.

My guess is the main culprit is attribute lookup in two ways:

 * Each object attribute reference involves a dictionary lookup.

 * Each method call involves *two* dictionary lookups plus an object
   creation.

Tell us what you find out.


Marko


Reply | Threaded
Open this post in threaded view
|

Building CPython

BartC-3
In reply to this post by BartC-3
On 14/05/2015 17:09, Chris Angelico wrote:
> On Fri, May 15, 2015 at 1:51 AM, BartC <bc at freeuk.com> wrote:
>> OK, the answer seems to be No then - you can't just trivially compile the C
>> modules that comprise the sources with the nearest compiler to hand. So much
>> for C's famous portability!
>>
>> (Actually, I think you already lost me on your first line.)

> If you want to just quickly play around with CPython's sources, I
> would strongly recommend getting yourself a Linux box. Either spin up
> some actual hardware with actual Linux, or grab a virtualization
> engine like VMWare, VirtualBox, etc, etc, and installing into a VM.
> With a Debian-based Linux (Debian, Ubuntu, Mint, etc), you should
> simply be able to:
>
> sudo apt-get build-dep python3

Actually I had VirtualBox with Ubuntu, but I don't know my way around
Linux and preferred doing things under Windows (and with all my own tools).

But it's now building under Ubuntu.

(Well, I'm not sure what it's doing exactly; the instructions said type
make, then make test, then make install, and it's still doing make test.

I hope there's a quicker way of re-building an executable after a minor
source file change, otherwise doing any sort of development is going to
be impractical.)

--
Bartc


Reply | Threaded
Open this post in threaded view
|

Building CPython

Dave Angel-4
On 05/14/2015 01:02 PM, BartC wrote:

> On 14/05/2015 17:09, Chris Angelico wrote:
>> On Fri, May 15, 2015 at 1:51 AM, BartC <bc at freeuk.com> wrote:
>>> OK, the answer seems to be No then - you can't just trivially compile
>>> the C
>>> modules that comprise the sources with the nearest compiler to hand.
>>> So much
>>> for C's famous portability!
>>>
>>> (Actually, I think you already lost me on your first line.)
>
>> If you want to just quickly play around with CPython's sources, I
>> would strongly recommend getting yourself a Linux box. Either spin up
>> some actual hardware with actual Linux, or grab a virtualization
>> engine like VMWare, VirtualBox, etc, etc, and installing into a VM.
>> With a Debian-based Linux (Debian, Ubuntu, Mint, etc), you should
>> simply be able to:
>>
>> sudo apt-get build-dep python3
>
> Actually I had VirtualBox with Ubuntu, but I don't know my way around
> Linux and preferred doing things under Windows (and with all my own tools).
>
> But it's now building under Ubuntu.
>
> (Well, I'm not sure what it's doing exactly; the instructions said type
> make, then make test, then make install, and it's still doing make test.
>
> I hope there's a quicker way of re-building an executable after a minor
> source file change, otherwise doing any sort of development is going to
> be impractical.)
>

That's what make is good for.  It compares the datestamps of the source
files against the obj files (etc.) and recompiles only when the source
is newer.  (It's more complex, but that's the idea)

--
DaveA


Reply | Threaded
Open this post in threaded view
|

Building CPython

Chris Angelico
In reply to this post by BartC-3
On Fri, May 15, 2015 at 3:02 AM, BartC <bc at freeuk.com> wrote:

> Actually I had VirtualBox with Ubuntu, but I don't know my way around Linux
> and preferred doing things under Windows (and with all my own tools).
>
> But it's now building under Ubuntu.
>
> (Well, I'm not sure what it's doing exactly; the instructions said type
> make, then make test, then make install, and it's still doing make test.
>
> I hope there's a quicker way of re-building an executable after a minor
> source file change, otherwise doing any sort of development is going to be
> impractical.)

The whole point of 'make' is to rebuild only the parts that need to be
rebuilt (either they've changed, or they depend on something that was
changed). Sometimes practically everything needs to be rebuilt, if you
do some really fundamental change, but generally not.

The three parts to the build process are:

1) make - actually generate an executable. Takes ages the first time,
will be a lot quicker if you haven't changed much.
2) make test - run the entire test suite. Takes just as long every
time, but most of it won't have changed.
3) make install (needs root access, so probably 'sudo make install') -
install this as your primary build of Python.

When you start tinkering, I suggest just running make; rerunning the
test suite isn't necessary till you're all done, and even then it's
only important for making sure that your change hasn't broken anything
anywhere else. Test your actual changes by simply running the
freshly-built Python - most likely that'll be "./python". Working that
way is fairly quick - you can tweak some C code, see the results, and
go back and tweak some more, all without pausing for a sword fight.
But once you go and rerun the full test suite, well... that's when
it's time for some:

https://xkcd.com/303/

ChrisA


Reply | Threaded
Open this post in threaded view
|

Building CPython

BartC-3
In reply to this post by BartC-3
On 14/05/2015 18:11, Chris Angelico wrote:
> On Fri, May 15, 2015 at 3:02 AM, BartC <bc at freeuk.com> wrote:

>> I hope there's a quicker way of re-building an executable after a minor
>> source file change, otherwise doing any sort of development is going to be
>> impractical.)
>
> The whole point of 'make' is to rebuild only the parts that need to be
> rebuilt (either they've changed, or they depend on something that was
> changed). Sometimes practically everything needs to be rebuilt, if you
> do some really fundamental change, but generally not.
>
> The three parts to the build process are:
>
> 1) make - actually generate an executable. Takes ages the first time,
> will be a lot quicker if you haven't changed much.
> 2) make test - run the entire test suite. Takes just as long every
> time, but most of it won't have changed.
> 3) make install (needs root access, so probably 'sudo make install') -
> install this as your primary build of Python.
>
> When you start tinkering, I suggest just running make; rerunning the
> test suite isn't necessary till you're all done, and even then it's
> only important for making sure that your change hasn't broken anything
> anywhere else. Test your actual changes by simply running the
> freshly-built Python - most likely that'll be "./python".

OK, thanks. I didn't even know where the executable was put! Now I don't
need 'make install', while 'make test' I won't bother with any more.

Making a small change and typing 'make' took 5 seconds, which is
reasonable enough (although I had to use the copy of the source in
Windows to find where the main.c file I needed was located).

Now Python 3.4.3 says "Bart's Python".


--
Bartc


Reply | Threaded
Open this post in threaded view
|

Building CPython

Chris Angelico
On Fri, May 15, 2015 at 3:32 AM, BartC <bc at freeuk.com> wrote:
> OK, thanks. I didn't even know where the executable was put! Now I don't
> need 'make install', while 'make test' I won't bother with any more.
>
> Making a small change and typing 'make' took 5 seconds, which is reasonable
> enough (although I had to use the copy of the source in Windows to find
> where the main.c file I needed was located).
>
> Now Python 3.4.3 says "Bart's Python".

Haha. I don't usually bother rebranding my builds of things; though
that's partly because normally I'm making very few changes, and all in
the hope that they'll be accepted upstream anyway.

Incidentally, a quick 'make' can range anywhere from a fraction of a
second to quite a long time, depending mainly on the speed of your
hard drive and the performance of your disk cache. On my Linux, a
"null make" (ie when literally nothing has changed - just rerunning
make) takes about half a second, and that's dealing with a module
that's failing to build. When you rebuild lots of times all at once,
you'll pretty much be working in RAM the whole time, assuming you have
enough of it available.

ChrisA


Reply | Threaded
Open this post in threaded view
|

Building CPython

Terry Reedy
In reply to this post by Chris Angelico
On 5/14/2015 1:11 PM, Chris Angelico wrote:

> 2) make test - run the entire test suite. Takes just as long every
> time, but most of it won't have changed.

The test runner has an option, -jn, to run tests in n processes instead
of just 1.  On my 6 core pentium, -j5 cuts time to almost exactly 1/5th
of otherwise.  -j10 seems faster but have not times it.  I suspect that
'make test' does not use -j option.


--
Terry Jan Reedy



Reply | Threaded
Open this post in threaded view
|

Building CPython

BartC-3
In reply to this post by Marko Rauhamaa
On 14/05/2015 17:29, Marko Rauhamaa wrote:

> BartC <bc at freeuk.com>:
>
>> That's a shame because I wanted to tinker with the main dispatcher
>> loop to try and find out what exactly is making it slow. Nothing that
>> seems obvious at first sight.
>
> My guess is the main culprit is attribute lookup in two ways:
>
>   * Each object attribute reference involves a dictionary lookup.
>
>   * Each method call involves *two* dictionary lookups plus an object
>     creation.
>
> Tell us what you find out.

I'm just starting but I can tell you that it isn't because debug mode
(Py_DEBUG defined) was left on by mistake!

What is interesting however is that on the very simple test I'm doing (a
while loop incrementing a variable up to 100 million), the timings under
Windows are:

Python 2.5       9.2 seconds
Python 3.1      13.1
Python 3.4.3    17.0
Python 3.4.3    14.3 (under Ubuntu on same machine, using the version
                      I built today)

That's quite a big range!

PyPy does it in 0.7 seconds. The same program within my own *simple*
bytecode interpreter (not for Python) has a fastest time of 1.5 seconds
but makes use of ASM. A version 100% in (gcc) C can manage 2 seconds.

So far I haven't find anything that explains the discrepancy (the
languages are different, mine is simpler, but the Python code isn't
doing anything that complicated, only LOAD_FASTs and such, and LOAD_FAST
is apparently just an array access.

But the nearly 2:1 difference between new and old Python versions is
also intriguing.

def whiletest():
        i=0
        while i<=100000000:
                i=i+1

whiletest()

--
Bartc


Reply | Threaded
Open this post in threaded view
|

Building CPython

MRAB-2
On 2015-05-14 22:55, BartC wrote:

> On 14/05/2015 17:29, Marko Rauhamaa wrote:
>> BartC <bc at freeuk.com>:
>>
>>> That's a shame because I wanted to tinker with the main dispatcher
>>> loop to try and find out what exactly is making it slow. Nothing that
>>> seems obvious at first sight.
>>
>> My guess is the main culprit is attribute lookup in two ways:
>>
>>   * Each object attribute reference involves a dictionary lookup.
>>
>>   * Each method call involves *two* dictionary lookups plus an object
>>     creation.
>>
>> Tell us what you find out.
>
> I'm just starting but I can tell you that it isn't because debug mode
> (Py_DEBUG defined) was left on by mistake!
>
> What is interesting however is that on the very simple test I'm doing (a
> while loop incrementing a variable up to 100 million), the timings under
> Windows are:
>
> Python 2.5       9.2 seconds
> Python 3.1      13.1
> Python 3.4.3    17.0
> Python 3.4.3    14.3 (under Ubuntu on same machine, using the version
>                        I built today)
>
> That's quite a big range!
>
> PyPy does it in 0.7 seconds. The same program within my own *simple*
> bytecode interpreter (not for Python) has a fastest time of 1.5 seconds
> but makes use of ASM. A version 100% in (gcc) C can manage 2 seconds.
>
> So far I haven't find anything that explains the discrepancy (the
> languages are different, mine is simpler, but the Python code isn't
> doing anything that complicated, only LOAD_FASTs and such, and LOAD_FAST
> is apparently just an array access.
>
> But the nearly 2:1 difference between new and old Python versions is
> also intriguing.
>
> def whiletest():
> i=0
> while i<=100000000:
> i=i+1
>
> whiletest()
>
Python 2.x has int and long; Python 3 has int, which is the old 'long'.
Try Python 2 with longs.



Reply | Threaded
Open this post in threaded view
|

Building CPython

BartC-3
In reply to this post by BartC-3
On 14/05/2015 22:55, BartC wrote:

 > def whiletest():
 >      i=0
 >      while i<=100000000:
 >          i=i+1
 >
 > whiletest()
 >

> Python 2.5       9.2 seconds
> Python 3.1      13.1
> Python 3.4.3    17.0
> Python 3.4.3    14.3 (under Ubuntu on same machine, using the version
>                       I built today)
>
> That's quite a big range!
>
> PyPy does it in 0.7 seconds. The same program within my own *simple*
> bytecode interpreter (not for Python) has a fastest time of 1.5 seconds
> but makes use of ASM. A version 100% in (gcc) C can manage 2 seconds.

(Actually my ASM-aided code took 0.5 seconds (some crucial section had
been commented out). Faster than PyPy and just using simple brute-force
methods.)

> So far I haven't find anything that explains the discrepancy (the
> languages are different, mine is simpler, but the Python code isn't
> doing anything that complicated, only LOAD_FASTs and such, and LOAD_FAST
> is apparently just an array access.

It turns out the LOAD_FASTs were the simple byte-codes!

I'm just starting to find out just how much of a big complicated mess
this project really is. I wouldn't be surprised if there aren't many
people who actually understand it all, and that would explain why no-one
seems to have had much luck in getting the speed up (if anything, it's
getting slower).

I still have no idea yet exactly what an object comprises. I know that
even an innocuous-looking LOAD_CONST actually loads a tuple from a table
(complete with flag testing and bounds checks, although taking those out
made no discernible difference).

It appears to be those "<=" and "+" operations in the code above where
much of the time is spent. When I trace out the execution paths a bit
more, I'll have a better picture of how many lines of C code are
involved in each iteration.

--
Bartc


Reply | Threaded
Open this post in threaded view
|

Building CPython

Gregory Ewing
BartC wrote:
> It appears to be those "<=" and "+" operations in the code above where
> much of the time is spent. When I trace out the execution paths a bit
> more, I'll have a better picture of how many lines of C code are
> involved in each iteration.

The path from decoding a bytecode to the C code that
implements it can be rather convoluted, but there are
reasons for each of the complications -- mainly to
do with supporting the ability to override operators
with C and/or Python code.

If you removed those abilities, the implemention
would be simpler, and possibly faster. But then the
language wouldn't be Python any more.

--
Greg


Reply | Threaded
Open this post in threaded view
|

Building CPython

Marko Rauhamaa
Gregory Ewing <greg.ewing at canterbury.ac.nz>:

> BartC wrote:
>> It appears to be those "<=" and "+" operations in the code above
>> where much of the time is spent. When I trace out the execution paths
>> a bit more, I'll have a better picture of how many lines of C code
>> are involved in each iteration.
>
> The path from decoding a bytecode to the C code that implements it can
> be rather convoluted, but there are reasons for each of the
> complications -- mainly to do with supporting the ability to override
> operators with C and/or Python code.
>
> If you removed those abilities, the implemention would be simpler, and
> possibly faster. But then the language wouldn't be Python any more.

I agree that Python's raison-d'?tre is its dynamism and expressive
power. It definitely shouldn't be sacrificed for performance.

However, in some respects, Python might be going overboard with its
dynamism; are all those dunder methods really needed? Must "false" be
defined so broadly? Must a method lookup necessarily involve object
creation?


Marko


Reply | Threaded
Open this post in threaded view
|

Building CPython

Marko Rauhamaa
wxjmfauth at gmail.com:

> Implement unicode correctly.

Did they reject your patch?


Marko


Reply | Threaded
Open this post in threaded view
|

Building CPython

BartC-3
In reply to this post by Gregory Ewing
On 15/05/2015 07:05, Gregory Ewing wrote:

> BartC wrote:
>> It appears to be those "<=" and "+" operations in the code above where
>> much of the time is spent. When I trace out the execution paths a bit
>> more, I'll have a better picture of how many lines of C code are
>> involved in each iteration.
>
> The path from decoding a bytecode to the C code that
> implements it can be rather convoluted, but there are
> reasons for each of the complications -- mainly to
> do with supporting the ability to override operators
> with C and/or Python code.
>
> If you removed those abilities, the implemention
> would be simpler, and possibly faster. But then the
> language wouldn't be Python any more.

That's the challenge; programs must still work as they did before. (But
I suppose it can be exasperating for CPython developers if 99% of
programs could be made faster but can't be because of the 1% that rely
on certain features).

Still, I'm just seeing what there is in CPython that limits the
performance, and whether anything can be done /easily/ without going to
solutions such as PyPy which are unsatisfactory IMO (by being even more
fantastically complicated, but they don't always give a speed-up either).

For example, there is a /specific/ byte-code called BINARY_ADD, which
then proceeds to call a /generic/ binary-op handler! This throws away
the advantage of knowing at byte-code generation time exactly which
operation is needed.

(Unless I'm just looking at a bunch of macros or maybe there is some
inlining going on with the compiler reducing all those table-indexing
operations for 'Add', with direct accesses, but it didn't look like it.
I'm just glad it doesn't use C++ which would have made it truly
impossible to figure out what's going on.)

(BTW since I'm having to use Linux to compile this anyway, is there a
tool available that will tell me whether something in the C sources is a
function or macro? (And preferably where the definition might be located.))

--
Bartc


Reply | Threaded
Open this post in threaded view
|

Building CPython

Chris Angelico
In reply to this post by Marko Rauhamaa
On Fri, May 15, 2015 at 6:59 PM, Marko Rauhamaa <marko at pacujo.net> wrote:
> However, in some respects, Python might be going overboard with its
> dynamism; are all those dunder methods really needed?

Yes - at least, most of them. As regards operators, there are three
options: either you have magic methods for all of them (Python style),
or none of them (Java style, no operator overloading), or you
hybridize and permit just a handful of them (and then you have to
decide which). There's really no reason not to have them. The other
dunder methods are a mixed bag; some are to allow you to customize
object creation itself (a class's __new__ method could be considered
equivalent to an instance-specific __call__ method on the type
object), some let you pretend to be different types of number
(__int__, __index__, __complex__), which allows you to duck-type
integerness rather than having to subclass int and rely on magic; and
others let you customize the behaviour of well-known functions, such
as __len__ for len() and __repr__ for repr(). Without dunder methods
for all of these, it would be difficult to make an object "play
nicely" with the overall Python ecosystem. There are others, though,
which are less crucial (__getstate__ and so on for pickle), but you
can ignore them if you're not using those features.

> Must "false" be defined so broadly?

Different languages define true and false differently. REXX says that
"1" is true and "0" is false, and anything else is an error. Pike says
that the integer 0 is false and anything else is true; the philosophy
is that a "thing" is true and the absence of any thing is false.
Python says that an empty "thing" is false and a non-empty "thing" is
true; if next(iter(x)) raises StopIteration, x is probably false, and
vice versa. All three have their merits, all three have their
consequences. One consequence of Python's model is that a __bool__
method is needed on any object that might be empty (unless it defines
__len__, in which case that makes a fine fall-back); it's normal in
Python code to distinguish between "if x:" and "if x is not None:",
where the former sees if x has anything in it, but the latter sees if
there x even exists. (More or less.)

> Must a method lookup necessarily involve object creation?

Actually, no. Conceptually, this method call:

foo.bar(1, 2, 3)

involves looking up 'foo' in the current namespace, looking up
attribute 'bar' on it, treating the result as a function, and calling
it with three integer objects as its arguments. And the language
definition demands that this work even if the "foo.bar" part is broken
out:

def return_function():
    return foo.bar

def call_function():
    return_function()(1, 2, 3)

But a particular Python implementation is most welcome to notice the
extremely common situation of method calls and optimize it. I'm not
sure if PyPy does this, but I do remember reading about at least one
Python that does; CPython has an optimization for the actual memory
allocations involved, though I think it does actually construct some
sort of object for each one; as long as the resulting behaviour is
within spec, objects needn't be created just to be destroyed.

Dynamism doesn't have to be implemented naively, just as long as the
slow path is there if anyone needs it.

ChrisA


123