sys.implementation

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

sys.implementation

Eric Snow-2
The proposal of adding sys.implementation has come up a couple times
over the last few years. [1][2]  While the reaction has been
overwhelmingly positive, nothing has come of it.  I've created a
tracker issue and a patch:

    http://bugs.python.org/issue14673

The patch adds a struct sequence that holds ("name" => "CPython",
"version" => sys.version_info).  If later needs dictate more fields,
we can cross that bridge then.

Are there any objections?  Considering the positive reaction and the
scope of the addition, does this need a PEP?

-eric

[1] http://mail.python.org/pipermail/python-dev/2009-October/092893.html
[2] http://mail.python.org/pipermail/python-ideas/2012-April/014878.html
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: sys.implementation

Barry Warsaw
On Apr 25, 2012, at 11:31 PM, Eric Snow wrote:

>The proposal of adding sys.implementation has come up a couple times
>over the last few years. [1][2]  While the reaction has been
>overwhelmingly positive, nothing has come of it.  I've created a
>tracker issue and a patch:
>
>    http://bugs.python.org/issue14673
>
>The patch adds a struct sequence that holds ("name" => "CPython",
>"version" => sys.version_info).  If later needs dictate more fields,
>we can cross that bridge then.
>
>Are there any objections?  Considering the positive reaction and the
>scope of the addition, does this need a PEP?

It's somewhat of a corner case, but I think a PEP couldn't hurt.  The
rationale section would be useful, at least.

-Barry
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: sys.implementation

Eric Snow-2
On Thu, Apr 26, 2012 at 8:31 AM, Barry Warsaw <[hidden email]> wrote:
> On Apr 25, 2012, at 11:31 PM, Eric Snow wrote:
>>Are there any objections?  Considering the positive reaction and the
>>scope of the addition, does this need a PEP?
>
> It's somewhat of a corner case, but I think a PEP couldn't hurt.  The
> rationale section would be useful, at least.
>
> -Barry

Yeah, I'm finding little bits and pieces that would be nice to have
recorded in one place.  I'll get something up in the next couple days.

-eric
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: sys.implementation

Larry Hastings
In reply to this post by Eric Snow-2
On 04/25/2012 10:31 PM, Eric Snow wrote:
The patch adds a struct sequence that holds ("name" => "CPython",
"version" => sys.version_info).  If later needs dictate more fields,
we can cross that bridge then.

My one bit of bike-shedding: I don't think it's desirable that this object be iterable.  Therefore I suggest you don't use struct sequence.


/arry

_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: sys.implementation

Eric Snow-2
On Thu, Apr 26, 2012 at 9:29 PM, Larry Hastings <[hidden email]> wrote:
> My one bit of bike-shedding: I don't think it's desirable that this object
> be iterable.  Therefore I suggest you don't use struct sequence.

Good point.  Noted.

-eric
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: sys.implementation

Eric Snow-2
In reply to this post by Barry Warsaw
On Thu, Apr 26, 2012 at 8:31 AM, Barry Warsaw <[hidden email]> wrote:
> It's somewhat of a corner case, but I think a PEP couldn't hurt.  The
> rationale section would be useful, at least.

  http://mail.python.org/pipermail/python-ideas/2012-April/014954.html

-eric
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: sys.implementation

Glenn Linderman-3
On 4/27/2012 12:34 AM, Eric Snow wrote:
On Thu, Apr 26, 2012 at 8:31 AM, Barry Warsaw [hidden email] wrote:
It's somewhat of a corner case, but I think a PEP couldn't hurt.  The
rationale section would be useful, at least.
  http://mail.python.org/pipermail/python-ideas/2012-April/014954.html

The idea of having separate versions for CPython and stdlib has been raised recently, although I believe it has been mostly been deferred or discarded.  Should that be resurrected, sys.implementation may be a good repository for the additional version info defining the stdlib version.

However, this PEP raises the following question in my mind: is the sys module part of the stdlib? Before reaching a hasty conclusion, consider the following points:

1) with this proposal, the contents of sys.implementation will vary between implementations.  If stdlib is to be shared among implementations, then it seems sys.implementation should not be part of the stdlib, but rather part of the implementation. Is sys considered part of the implementation or part of the stdlib? I've always perceived it as part of the stdlib, because of the way it is documented.

2) importlib wants to be part of the stdlib, and thus available to other implementations, but it must be built-in or frozen. The goal with importlib is a common implementation in Python, that can be used by all implementations.  I am not clear on whether the accelerated C code is part of the stdlib, or part of an implementation optimization, nor how the structuring of such things is arranged to separate stdlib from implementation (if it is; if it isn't, should it be?)

3) can anything that must be built-in or frozen be part of the stdlib? I don't see why not, if it is common to all implementations, even if it depends on data it obtains from the implementation via some mechanism such as the proposed sys.implementation. However, if it is not common, I don't know how it can be standard/stdlib... which raises issues in my understanding of the various modules available as Python with C accelerators, and I know there are pure C modules that are part of the stdlib. So I think this idea of making the stdlib more sharable between implementations is still a work-in-progress, even a design-in-progress, but maybe part of the solution is to separate, or at least delineate, things that can be common, from things that cannot be common to all implementations.

My conclusion is that sys.implementation clearly should not be part of the stdlib, but rather be part of the language implementation.  Whether it then fits with the rest of what is in sys, or not, I am not qualified to say.  If not, perhaps a new module name is warranted... perhaps "implementation" at the top level of the namespace.

So my thoughts are:

Things that are part of the stdlib should be available in Python source to be shared across implementations.  Things that are not available in Python source cannot be shared across implementations, and therefore should not be part of the stdlib, but rather part of an implementation-specific library, or part of the language specification.

Or maybe stdlib should be an umbrella term, with the following subsets: common (Python implementation available, and not dependent on implementation-specific details except in very standardized ways), implementation-specific (provided by each implementation, either in implementation-specific Python, or some other implementation language), accelerators (a faster version of a common module, provided by an implementation when necessary for performance).

In this situation, sys.implementation, as proposed, should be implementation-specific.


_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: sys.implementation

R. David Murray
On Fri, 27 Apr 2012 10:40:43 -0700, Glenn Linderman <[hidden email]> wrote:

> On 4/27/2012 12:34 AM, Eric Snow wrote:
> > On Thu, Apr 26, 2012 at 8:31 AM, Barry Warsaw<[hidden email]>  wrote:
> >> It's somewhat of a corner case, but I think a PEP couldn't hurt.  The
> >> rationale section would be useful, at least.
> >    http://mail.python.org/pipermail/python-ideas/2012-April/014954.html
>
> My conclusion is that sys.implementation clearly should not be part of
> the stdlib, but rather be part of the language implementation.  Whether
> it then fits with the rest of what is in sys, or not, I am not qualified
> to say.  If not, perhaps a new module name is warranted... perhaps
> "implementation" at the top level of the namespace.

IMO, there are two different things here that you are conflating(*): the
*implementation* of the stdlib, and the stdlib *API*.  sys.implementation
would be a part of the API that any conforming implementation of
python+stdlib would be required to implement.

We also have a goal of making as much of the *implementation* of the
stdlib usable by any python implementation as possible, but as you say
that is a work in progress.

There are, by the way, many things documented in the "library"
documentation that are in fact provided by the language implementation
itself.  All of the fundamental types, for example.

--David

(*) the Oracle lawyers sometimes seem to be trying to get
the judge and jury to make the same mistake.
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: sys.implementation

Glenn Linderman-3
On 4/27/2012 11:49 AM, R. David Murray wrote:
On Fri, 27 Apr 2012 10:40:43 -0700, Glenn Linderman [hidden email] wrote:
On 4/27/2012 12:34 AM, Eric Snow wrote:
On Thu, Apr 26, 2012 at 8:31 AM, Barry Warsaw[hidden email]  wrote:
It's somewhat of a corner case, but I think a PEP couldn't hurt.  The
rationale section would be useful, at least.
   http://mail.python.org/pipermail/python-ideas/2012-April/014954.html
My conclusion is that sys.implementation clearly should not be part of 
the stdlib, but rather be part of the language implementation.  Whether 
it then fits with the rest of what is in sys, or not, I am not qualified 
to say.  If not, perhaps a new module name is warranted... perhaps 
"implementation" at the top level of the namespace.
IMO, there are two different things here that you are conflating(*): the
*implementation* of the stdlib, and the stdlib *API*.  sys.implementation
would be a part of the API that any conforming implementation of
python+stdlib would be required to implement.

Hmm.  OK.

We also have a goal of making as much of the *implementation* of the
stdlib usable by any python implementation as possible, but as you say
that is a work in progress.

OK.
There are, by the way, many things documented in the "library"
documentation that are in fact provided by the language implementation
itself.  All of the fundamental types, for example.

I was aware of this last, but wasn't thinking about it during these musings... although the thoughts of documentation also crossed my mind, I didn't mention them, figuring it could come up later.

So "library" documentation already covers all three categories of stuff that I mentioned, plus one more (restated here for clarity, with better wording):

* language implementation
* implementation dependent modules
* implementation independent modules
* implementation dependent optimizations of implementation independent modules

From the perspective of a user of a single implementation of the language + library, it really doesn't matter how the documentation is organized, or whether the documentation notes which of the above 4 categories an item falls in.

From the perspective of a user of multiple implementations, or perspective of a developer of an implementation other than CPython, knowledge of the category could be useful for both portability and performance planning. Organizing the documentation in some manner to be aware of such categories may help other implementations provide appropriate addenda.  The closer any of them get to tracking the Py3 trunk in real time, the more so.

Here's a ponderable: In the long term, should the documentation be unified for multiple implementations?  Or should it be split into 4 pieces, so that alternate implementations could swap in their own sections for implementation dependent items?


--David

(*) the Oracle lawyers sometimes seem to be trying to get
the judge and jury to make the same mistake.
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/v%2Bpython%40g.nevcal.com




_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: sys.implementation

Nick Coghlan
On Sat, Apr 28, 2012 at 6:11 AM, Glenn Linderman <[hidden email]> wrote:
> Here's a ponderable: In the long term, should the documentation be unified
> for multiple implementations?  Or should it be split into 4 pieces, so that
> alternate implementations could swap in their own sections for
> implementation dependent items?

Probably not, because the boundary between language, standard library
and implementation *is* blurry. The blurriness in the descriptions
reflects the blurriness in reality.

Anything that doesn't have dedicated syntax is, in a formal sense,
part of the standard library rather than the core language definition.
That includes the GC API, the weakref API, the sys module, the
operator module, the builtins module, the types module, etc. The
language specification itself just states that there *is* a builtin
namespace and you *can* do imports. It is then up to the standard
library specification to describe the *contents* of the builtin
namespace, as well as state what other modules and packages can be
imported by default.

However, the various parts of the standard library differ can wildly
in how *closely coupled* they are to a particular implementation.

Some, such as builtins, gc, operator, weakref, types and sys are
*very* tightly coupled with a specific implementation and always will
be. If someone is writing a new implementation of Python, they're
almost certainly going to have to write new version of these modules
from scratch that interoperate correctly with their code generator and
evaluation loop.

Historically, the import machinery was similarly coupled to a specific
implementation. The goal of bootstrapping importlib as the main import
implementation is to change that so that most of the import machinery
is decoupled from the implementation, with the bare minimum remaining
as implementation specific code (specifically, the code needed to
carry out the bootstrapping process, such as supporting frozen and
builtin modules).

Other modules may differ in performance characteristics between
implementations, particular for older C modules in CPython which don't
have a pure Python counterpart.

So, yes, I agree the four categories you list are helpful in
*thinking* about the questions involved, but, no, I don't think it's a
good principle for organising the documentation (precisely because the
categories are related to implementation details that shouldn't matter
all that much to end users).

Cheers,
Nick.

--
Nick Coghlan   |   [hidden email]   |   Brisbane, Australia
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com