Declaring PEP 3333 accepted (was: PEP 444 != WSGI 2.0)

classic Classic list List threaded Threaded
45 messages Options
123
Reply | Threaded
Open this post in threaded view
|

Declaring PEP 3333 accepted (was: PEP 444 != WSGI 2.0)

Jacob Kaplan-Moss-2
On Sun, Jan 2, 2011 at 9:21 AM, Guido van Rossum <[hidden email]> wrote:
> Although [PEP 3333] is still marked as draft, I personally think of it
> as accepted; [...]

What does it take to get PEP 3333 formally marked as accepted? Is
there anything I can do to push that process forward?

The lack of a WSGI answer on Py3 is the main thing that's keeping me,
personally, from feeling excited about the platform. Once that's done
I can feel comfortable coding to it -- and browbeating those who don't
support it.

I understand that PEP 444/Web3/WSGI 2/whatever might be a better
answer, but it's clearly got some way to go. In the meantime, what's
next to get PEP 3333 officially endorsed and accepted?

Jacob
_______________________________________________
Web-SIG mailing list
[hidden email]
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/lists%40nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Declaring PEP 3333 accepted (was: PEP 444 != WSGI 2.0)

Benoit Chesneau-3
On Tue, Jan 4, 2011 at 12:13 AM, Jacob Kaplan-Moss <[hidden email]> wrote:

> On Sun, Jan 2, 2011 at 9:21 AM, Guido van Rossum <[hidden email]> wrote:
>> Although [PEP 3333] is still marked as draft, I personally think of it
>> as accepted; [...]
>
> What does it take to get PEP 3333 formally marked as accepted? Is
> there anything I can do to push that process forward?
>
> The lack of a WSGI answer on Py3 is the main thing that's keeping me,
> personally, from feeling excited about the platform. Once that's done
> I can feel comfortable coding to it -- and browbeating those who don't
> support it.
>
> I understand that PEP 444/Web3/WSGI 2/whatever might be a better
> answer, but it's clearly got some way to go. In the meantime, what's
> next to get PEP 3333 officially endorsed and accepted?
>
> Jacob

I we pep3333 is considered accepted, I will start the port of gunicorn
based on it too. It shouldn't be hard since we are using our own http
parser.

- benoit
_______________________________________________
Web-SIG mailing list
[hidden email]
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/lists%40nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Declaring PEP 3333 accepted (was: PEP 444 != WSGI 2.0)

Guido van Rossum
In reply to this post by Jacob Kaplan-Moss-2
On Mon, Jan 3, 2011 at 3:13 PM, Jacob Kaplan-Moss <[hidden email]> wrote:

> On Sun, Jan 2, 2011 at 9:21 AM, Guido van Rossum <[hidden email]> wrote:
>> Although [PEP 3333] is still marked as draft, I personally think of it
>> as accepted; [...]
>
> What does it take to get PEP 3333 formally marked as accepted? Is
> there anything I can do to push that process forward?
>
> The lack of a WSGI answer on Py3 is the main thing that's keeping me,
> personally, from feeling excited about the platform. Once that's done
> I can feel comfortable coding to it -- and browbeating those who don't
> support it.
>
> I understand that PEP 444/Web3/WSGI 2/whatever might be a better
> answer, but it's clearly got some way to go. In the meantime, what's
> next to get PEP 3333 officially endorsed and accepted?

I haven't heard anyone speak up against it, ever, since it was
submitted. If no-one speaks up in the next 24 hours consider it
accepted (and after that delay, anyone with SVN privileges can mark it
thus).

--
--Guido van Rossum (python.org/~guido)
_______________________________________________
Web-SIG mailing list
[hidden email]
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/lists%40nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Declaring PEP 3333 accepted (was: PEP 444 != WSGI 2.0)

Randy Syring-3
In the server/gateway example, there is a comment in the code that says:

# TODO: this needs to be binary on Py3

The "TODO" part confuses me.  In other areas of the PEP, there are
comments like:

# call must be byte-safe on Py3

which make sense.  But is the TODO meant to be a TODO for the PEP or is
it meant to be a note to the person running the example on Py3.  If the
latter, maybe "TODO" isn't the best prefix.

FWIW, don't consider this an objection, it is just a question I had as I
read through the PEP.

--------------------------------------
Randy Syring
Intelicom
Direct: 502-276-0459
Office: 502-212-9913

For the wages of sin is death, but the
free gift of God is eternal life in
Christ Jesus our Lord (Rom 6:23)


On 01/03/2011 07:43 PM, Guido van Rossum wrote:

> On Mon, Jan 3, 2011 at 3:13 PM, Jacob Kaplan-Moss<[hidden email]>  wrote:
>> On Sun, Jan 2, 2011 at 9:21 AM, Guido van Rossum<[hidden email]>  wrote:
>>> Although [PEP 3333] is still marked as draft, I personally think of it
>>> as accepted; [...]
>> What does it take to get PEP 3333 formally marked as accepted? Is
>> there anything I can do to push that process forward?
>>
>> The lack of a WSGI answer on Py3 is the main thing that's keeping me,
>> personally, from feeling excited about the platform. Once that's done
>> I can feel comfortable coding to it -- and browbeating those who don't
>> support it.
>>
>> I understand that PEP 444/Web3/WSGI 2/whatever might be a better
>> answer, but it's clearly got some way to go. In the meantime, what's
>> next to get PEP 3333 officially endorsed and accepted?
> I haven't heard anyone speak up against it, ever, since it was
> submitted. If no-one speaks up in the next 24 hours consider it
> accepted (and after that delay, anyone with SVN privileges can mark it
> thus).
>
_______________________________________________
Web-SIG mailing list
[hidden email]
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/lists%40nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Declaring PEP 3333 accepted (was: PEP 444 != WSGI 2.0)

PJ Eby
In reply to this post by Guido van Rossum
At 04:43 PM 1/3/2011 -0800, Guido van Rossum wrote:

>On Mon, Jan 3, 2011 at 3:13 PM, Jacob Kaplan-Moss <[hidden email]> wrote:
> > On Sun, Jan 2, 2011 at 9:21 AM, Guido van Rossum <[hidden email]> wrote:
> >> Although [PEP 3333] is still marked as draft, I personally think of it
> >> as accepted; [...]
> >
> > What does it take to get PEP 3333 formally marked as accepted? Is
> > there anything I can do to push that process forward?
> >
> > The lack of a WSGI answer on Py3 is the main thing that's keeping me,
> > personally, from feeling excited about the platform. Once that's done
> > I can feel comfortable coding to it -- and browbeating those who don't
> > support it.
> >
> > I understand that PEP 444/Web3/WSGI 2/whatever might be a better
> > answer, but it's clearly got some way to go. In the meantime, what's
> > next to get PEP 3333 officially endorsed and accepted?
>
>I haven't heard anyone speak up against it, ever, since it was
>submitted. If no-one speaks up in the next 24 hours consider it
>accepted (and after that delay, anyone with SVN privileges can mark it
>thus).

There are a few minor changes to the code samples needed to make them
proper Python 3; I just checked in the hairiest of them, though.

_______________________________________________
Web-SIG mailing list
[hidden email]
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/lists%40nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Declaring PEP 3333 accepted (was: PEP 444 != WSGI 2.0)

PJ Eby
In reply to this post by Randy Syring-3
At 08:04 PM 1/3/2011 -0500, Randy Syring wrote:

>In the server/gateway example, there is a comment in the code that says:
>
># TODO: this needs to be binary on Py3
>
>The "TODO" part confuses me.  In other areas of the PEP, there are
>comments like:
>
># call must be byte-safe on Py3
>
>which make sense.  But is the TODO meant to be a TODO for the PEP or
>is it meant to be a note to the person running the example on
>Py3.  If the latter, maybe "TODO" isn't the best prefix.
>
>FWIW, don't consider this an objection, it is just a question I had
>as I read through the PEP.

Those are my TODO's for the PEP itself, and I've fixed a couple of
them in SVN (probably around the time you were writing the
above).  If somebody can point me to the proper Py3 incantation for
writing bytes to stdout, I'll fix the one remaining TODO marker as well.

_______________________________________________
Web-SIG mailing list
[hidden email]
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/lists%40nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Declaring PEP 3333 accepted (was: PEP 444 != WSGI 2.0)

Guido van Rossum
On Mon, Jan 3, 2011 at 5:29 PM, P.J. Eby <[hidden email]> wrote:

> At 08:04 PM 1/3/2011 -0500, Randy Syring wrote:
>>
>> In the server/gateway example, there is a comment in the code that says:
>>
>> # TODO: this needs to be binary on Py3
>>
>> The "TODO" part confuses me.  In other areas of the PEP, there are
>> comments like:
>>
>> # call must be byte-safe on Py3
>>
>> which make sense.  But is the TODO meant to be a TODO for the PEP or is it
>> meant to be a note to the person running the example on Py3.  If the latter,
>> maybe "TODO" isn't the best prefix.
>>
>> FWIW, don't consider this an objection, it is just a question I had as I
>> read through the PEP.
>
> Those are my TODO's for the PEP itself, and I've fixed a couple of them in
> SVN (probably around the time you were writing the above).  If somebody can
> point me to the proper Py3 incantation for writing bytes to stdout, I'll fix
> the one remaining TODO marker as well.

Would

  sys.stdout.buffer.write(b'abc')

do?

(If you mix this with writing strings to sys.stdout directly, you may
have to call sys.stdout.flush() first.)

--
--Guido van Rossum (python.org/~guido)
_______________________________________________
Web-SIG mailing list
[hidden email]
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/lists%40nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Declaring PEP 3333 accepted (was: PEP 444 != WSGI 2.0)

Graham Dumpleton-2
In reply to this post by Guido van Rossum
On 4 January 2011 11:43, Guido van Rossum <[hidden email]> wrote:

> On Mon, Jan 3, 2011 at 3:13 PM, Jacob Kaplan-Moss <[hidden email]> wrote:
>> On Sun, Jan 2, 2011 at 9:21 AM, Guido van Rossum <[hidden email]> wrote:
>>> Although [PEP 3333] is still marked as draft, I personally think of it
>>> as accepted; [...]
>>
>> What does it take to get PEP 3333 formally marked as accepted? Is
>> there anything I can do to push that process forward?
>>
>> The lack of a WSGI answer on Py3 is the main thing that's keeping me,
>> personally, from feeling excited about the platform. Once that's done
>> I can feel comfortable coding to it -- and browbeating those who don't
>> support it.
>>
>> I understand that PEP 444/Web3/WSGI 2/whatever might be a better
>> answer, but it's clearly got some way to go. In the meantime, what's
>> next to get PEP 3333 officially endorsed and accepted?
>
> I haven't heard anyone speak up against it, ever, since it was
> submitted. If no-one speaks up in the next 24 hours consider it
> accepted (and after that delay, anyone with SVN privileges can mark it
> thus).

I note one issue which I have expressed concern over previously. In
section 'Handling the Content-Length Header; it says:

"""
Under some circumstances, however, the server or gateway may be able
to either generate a Content-Length header, or at least avoid the need
to close the client connection. If the application does not call the
write() callable, and returns an iterable whose len() is 1, then the
server can automatically determine Content-Length by taking the length
of the first bytestring yielded by the iterable.
"""

As documented in:

  http://blog.dscpl.com.au/2009/10/wsgi-issues-with-http-head-requests.html

the automatic addition of a Content-Length response header where
len(iterable) is 1, can cause wrong output for cases where WSGI
application believes that it can itself decide not to return any
actual content for a HEAD response, ignoring the fact that there could
be output filters which rely on headers or content being exactly the
same as for GET.

Do we therefore still want to promote the idea that the optimisation
is a good idea or even allowed?

Next thing is the reference to Jython in section 'Supporting Older
(<2.2) Versions of Python' which are quite out of date with respect to
version of Python it supports. Should that be updated? Should that
whole section be removed now?

Finally, I'd still like to see the CGI gateway example be updated to
properly protect stdin/stdout so that people using print() without
redirecting it to stderr don't stuff themselves up. For example:

    # Keep a reference to the original stdin. We then replace
    # stdin with an empty stream. This is to protect against
    # code from accessing sys.stdin directly and consuming the
    # request content.

    stdin = sys.stdin

    sys.stdin = cStringIO.StringIO('')

    # Keep a reference to the original stdout. We then replace
    # stdout with stderr. This is to protect against code that
    # wants to use 'print' to output debugging. If stdout wasn't
    # protected, then anything output using 'print' would end up
    # being sent as part of the response itself and interfere
    # with the operation of the CGI protocol.

    stdout = sys.stdout

    sys.stdout = sys.stderr

The adapter would then use stdin/stdout local variables and not
sys.stdin/sys.stdout.

The CGI adapter in wsgiref should also really be updated to do
something similar. This would solve the portability problem where code
is written for non CGI hosting environment and they leave 'print'
statements in which breaks on CGI. Even if people who have copied the
CGI gateway from PEP previously don't update their implementations,
have at least set what is best practice for the future.

BTW, I am not across how stdin/stdout works on Windows, but is there
an issue with that CGI example working on Windows because of text
stream vs byte stream issues and CRLF translation?

Graham
_______________________________________________
Web-SIG mailing list
[hidden email]
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/lists%40nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Declaring PEP 3333 accepted (was: PEP 444 != WSGI 2.0)

James Y Knight

On Jan 3, 2011, at 10:39 PM, Graham Dumpleton wrote:

> As documented in:
>
>  http://blog.dscpl.com.au/2009/10/wsgi-issues-with-http-head-requests.html
>
> the automatic addition of a Content-Length response header where
> len(iterable) is 1, can cause wrong output for cases where WSGI
> application believes that it can itself decide not to return any
> actual content for a HEAD response, ignoring the fact that there could
> be output filters which rely on headers or content being exactly the
> same as for GET.
>
> Do we therefore still want to promote the idea that the optimisation
> is a good idea or even allowed?

I think it would be nice if it was allowed -- it makes simple apps easier. Just because some WSGI applications may be broken w.r.t. HEAD, that doesn't make this optimization undesirable.

However, the current description does leave things a bit ambiguous. Why, for example, does it suggest only adding Content-Length if the length of the iterable is 1? Surely "if type(iterable) in (list, tuple)", the server could also set the Content-Length header to "sum(len(s) for s in iterable)". Is that forbidden, or just not explicitly spelled out as allowed?

If your app *wants* to special-case HEAD handling so as to avoid generating the body when it doesn't need to, how can it do that correctly/reliably? If you normally return content with hard-to-determine length, and you want the HEAD processing to thus also omit Content-Length (and not, say, have the server decide it should return Content-Length: 0), what do you have to return to ensure this happens?

James
_______________________________________________
Web-SIG mailing list
[hidden email]
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/lists%40nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Declaring PEP 3333 accepted (was: PEP 444 != WSGI 2.0)

Graham Dumpleton-2
On 4 January 2011 15:43, James Y Knight <[hidden email]> wrote:

>
> On Jan 3, 2011, at 10:39 PM, Graham Dumpleton wrote:
>
>> As documented in:
>>
>>  http://blog.dscpl.com.au/2009/10/wsgi-issues-with-http-head-requests.html
>>
>> the automatic addition of a Content-Length response header where
>> len(iterable) is 1, can cause wrong output for cases where WSGI
>> application believes that it can itself decide not to return any
>> actual content for a HEAD response, ignoring the fact that there could
>> be output filters which rely on headers or content being exactly the
>> same as for GET.
>>
>> Do we therefore still want to promote the idea that the optimisation
>> is a good idea or even allowed?
>
> I think it would be nice if it was allowed -- it makes simple apps easier. Just because some WSGI applications may be broken w.r.t. HEAD, that doesn't make this optimization undesirable.
>
> However, the current description does leave things a bit ambiguous. Why, for example, does it suggest only adding Content-Length if the length of the iterable is 1? Surely "if type(iterable) in (list, tuple)", the server could also set the Content-Length header to "sum(len(s) for s in iterable)". Is that forbidden, or just not explicitly spelled out as allowed?

It just doesn't mention it. From memory it also doesn't mention what
to do about case when len(iterable) is 0 either, which presumably if
such an optimisation was allowed could allow you to set Content-Length
to 0.

> If your app *wants* to special-case HEAD handling so as to avoid generating the body when it doesn't need to, how can it do that correctly/reliably? If you normally return content with hard-to-determine length, and you want the HEAD processing to thus also omit Content-Length (and not, say, have the server decide it should return Content-Length: 0), what do you have to return to ensure this happens?

Which is further support for the WSGI server not making decisions
itself to set Content-Length. But then, if an application isn't going
to generate Content-Length for HEAD, then it can't by rights do it for
GET either for same request else the response headers are different.

Graham
_______________________________________________
Web-SIG mailing list
[hidden email]
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/lists%40nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Declaring PEP 3333 accepted (was: PEP 444 != WSGI 2.0)

Guido van Rossum
In reply to this post by Graham Dumpleton-2
On Mon, Jan 3, 2011 at 7:39 PM, Graham Dumpleton
<[hidden email]> wrote:

> I note one issue which I have expressed concern over previously. In
> section 'Handling the Content-Length Header; it says:
>
> """
> Under some circumstances, however, the server or gateway may be able
> to either generate a Content-Length header, or at least avoid the need
> to close the client connection. If the application does not call the
> write() callable, and returns an iterable whose len() is 1, then the
> server can automatically determine Content-Length by taking the length
> of the first bytestring yielded by the iterable.
> """

That is copied exactly from PEP 333, i.e. WSGI 1.0. I didn't mean to
solicit objections to parts of PEP 3333 that are the same as PEP 333;
PEP 3333 is intended only to specify how WSGI 1.0 compliance is
supposed to work in Python 3. Some clarifications to the original WSGI
1.0 wordings were actually added to PEP 333 around the same time that
PEP 3333 was spun off; AFAIK the changes to PEP 333 were
noncontroversial and merely clarifications of how WSGI already works.
I don't think you can change the above bit of specification (no matter
how bad it is) and still call the resulting spec WSGI 1.0(.x) -- we
don't want to rule out WSGI 1.0 compliance of apps or frameworks that
would be considered compliant under the original 1.0 spec.

--
--Guido van Rossum (python.org/~guido)
_______________________________________________
Web-SIG mailing list
[hidden email]
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/lists%40nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Declaring PEP 3333 accepted (was: PEP 444 != WSGI 2.0)

Graham Dumpleton-2
On 4 January 2011 16:39, Guido van Rossum <[hidden email]> wrote:

> On Mon, Jan 3, 2011 at 7:39 PM, Graham Dumpleton
> <[hidden email]> wrote:
>> I note one issue which I have expressed concern over previously. In
>> section 'Handling the Content-Length Header; it says:
>>
>> """
>> Under some circumstances, however, the server or gateway may be able
>> to either generate a Content-Length header, or at least avoid the need
>> to close the client connection. If the application does not call the
>> write() callable, and returns an iterable whose len() is 1, then the
>> server can automatically determine Content-Length by taking the length
>> of the first bytestring yielded by the iterable.
>> """
>
> That is copied exactly from PEP 333, i.e. WSGI 1.0. I didn't mean to
> solicit objections to parts of PEP 3333 that are the same as PEP 333;
> PEP 3333 is intended only to specify how WSGI 1.0 compliance is
> supposed to work in Python 3. Some clarifications to the original WSGI
> 1.0 wordings were actually added to PEP 333 around the same time that
> PEP 3333 was spun off; AFAIK the changes to PEP 333 were
> noncontroversial and merely clarifications of how WSGI already works.
> I don't think you can change the above bit of specification (no matter
> how bad it is) and still call the resulting spec WSGI 1.0(.x) -- we
> don't want to rule out WSGI 1.0 compliance of apps or frameworks that
> would be considered compliant under the original 1.0 spec.

I don't believe this really causes a compliance issue as it is a
requirement on the WSGI server, not the WSGI application and doesn't
cause any existing WSGI applications to break.

It also says 'can' and not 'must' so technically WSGI servers are not
currently obligated to do it as I read it and certainly mod_wsgi
doesn't do it any more because it was causing problems for people.

But then, since it does say 'can' and not 'must' any WSGI server
implementers who know better can just ignore it anyway if it if left
in, and it can be dealt with in any new major revision.

Graham
_______________________________________________
Web-SIG mailing list
[hidden email]
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/lists%40nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Declaring PEP 3333 accepted (was: PEP 444 != WSGI 2.0)

Graham Dumpleton-2
BTW, to what extent are the examples in the PEP meant to be able to
work on both Python 2.X and Python 3.X as is.

Does it need to be clarified where examples will only work on Python
3.X, in particular the CGI gateway.

Graham

On 4 January 2011 16:49, Graham Dumpleton <[hidden email]> wrote:

> On 4 January 2011 16:39, Guido van Rossum <[hidden email]> wrote:
>> On Mon, Jan 3, 2011 at 7:39 PM, Graham Dumpleton
>> <[hidden email]> wrote:
>>> I note one issue which I have expressed concern over previously. In
>>> section 'Handling the Content-Length Header; it says:
>>>
>>> """
>>> Under some circumstances, however, the server or gateway may be able
>>> to either generate a Content-Length header, or at least avoid the need
>>> to close the client connection. If the application does not call the
>>> write() callable, and returns an iterable whose len() is 1, then the
>>> server can automatically determine Content-Length by taking the length
>>> of the first bytestring yielded by the iterable.
>>> """
>>
>> That is copied exactly from PEP 333, i.e. WSGI 1.0. I didn't mean to
>> solicit objections to parts of PEP 3333 that are the same as PEP 333;
>> PEP 3333 is intended only to specify how WSGI 1.0 compliance is
>> supposed to work in Python 3. Some clarifications to the original WSGI
>> 1.0 wordings were actually added to PEP 333 around the same time that
>> PEP 3333 was spun off; AFAIK the changes to PEP 333 were
>> noncontroversial and merely clarifications of how WSGI already works.
>> I don't think you can change the above bit of specification (no matter
>> how bad it is) and still call the resulting spec WSGI 1.0(.x) -- we
>> don't want to rule out WSGI 1.0 compliance of apps or frameworks that
>> would be considered compliant under the original 1.0 spec.
>
> I don't believe this really causes a compliance issue as it is a
> requirement on the WSGI server, not the WSGI application and doesn't
> cause any existing WSGI applications to break.
>
> It also says 'can' and not 'must' so technically WSGI servers are not
> currently obligated to do it as I read it and certainly mod_wsgi
> doesn't do it any more because it was causing problems for people.
>
> But then, since it does say 'can' and not 'must' any WSGI server
> implementers who know better can just ignore it anyway if it if left
> in, and it can be dealt with in any new major revision.
>
> Graham
>
_______________________________________________
Web-SIG mailing list
[hidden email]
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/lists%40nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Declaring PEP 3333 accepted (was: PEP 444 != WSGI 2.0)

Graham Dumpleton-2
Add another point. FWIW, these are coming up because of questions
being asked on python-dev IRC channel about PEP 3333.

The issue as it came down to was that the PEP may not be clear enough
in explaining that where str() is unicode and as such something like
PATH_INFO, although unicode, is actually bytes decoded as ISO-8859-1,
needed to be re encoded/decoded to get it back to Unicode in the
charset required before use.

They were thinking that because it was unicode already they could use
it as is and not need to do anything. Ie., didn't realise that need to
do:

  path_info = environ.get('PATH_INFO', '')
  path_info = path_info.encode('ISO-8859-1').decode('UTF-8')

for example to get it interpreted as UTF-8 first. They were simply
looking at concatenating new URL bits to the ISO-8859-1 variant from
other unicode strings that weren't bytes represented as ISO-8859-1.

In Python 2.X it was obvious that since it wasn't unicode that you had
to decode it, but confusion may arise for Python 3.X if this
requirement is not explicitly spelled out with a code example like
above.

We all may see it as obvious and yes perhaps it could be covered in
separate articles or commentaries be people, but given this person was
new to it, maybe it is deserving of more explanation in the PEP itself
if they were confused.

It could also be that the PEP covers it adequately already. I am too
tired to read through it again right now.

Graham

On 4 January 2011 20:53, Graham Dumpleton <[hidden email]> wrote:

> BTW, to what extent are the examples in the PEP meant to be able to
> work on both Python 2.X and Python 3.X as is.
>
> Does it need to be clarified where examples will only work on Python
> 3.X, in particular the CGI gateway.
>
> Graham
>
> On 4 January 2011 16:49, Graham Dumpleton <[hidden email]> wrote:
>> On 4 January 2011 16:39, Guido van Rossum <[hidden email]> wrote:
>>> On Mon, Jan 3, 2011 at 7:39 PM, Graham Dumpleton
>>> <[hidden email]> wrote:
>>>> I note one issue which I have expressed concern over previously. In
>>>> section 'Handling the Content-Length Header; it says:
>>>>
>>>> """
>>>> Under some circumstances, however, the server or gateway may be able
>>>> to either generate a Content-Length header, or at least avoid the need
>>>> to close the client connection. If the application does not call the
>>>> write() callable, and returns an iterable whose len() is 1, then the
>>>> server can automatically determine Content-Length by taking the length
>>>> of the first bytestring yielded by the iterable.
>>>> """
>>>
>>> That is copied exactly from PEP 333, i.e. WSGI 1.0. I didn't mean to
>>> solicit objections to parts of PEP 3333 that are the same as PEP 333;
>>> PEP 3333 is intended only to specify how WSGI 1.0 compliance is
>>> supposed to work in Python 3. Some clarifications to the original WSGI
>>> 1.0 wordings were actually added to PEP 333 around the same time that
>>> PEP 3333 was spun off; AFAIK the changes to PEP 333 were
>>> noncontroversial and merely clarifications of how WSGI already works.
>>> I don't think you can change the above bit of specification (no matter
>>> how bad it is) and still call the resulting spec WSGI 1.0(.x) -- we
>>> don't want to rule out WSGI 1.0 compliance of apps or frameworks that
>>> would be considered compliant under the original 1.0 spec.
>>
>> I don't believe this really causes a compliance issue as it is a
>> requirement on the WSGI server, not the WSGI application and doesn't
>> cause any existing WSGI applications to break.
>>
>> It also says 'can' and not 'must' so technically WSGI servers are not
>> currently obligated to do it as I read it and certainly mod_wsgi
>> doesn't do it any more because it was causing problems for people.
>>
>> But then, since it does say 'can' and not 'must' any WSGI server
>> implementers who know better can just ignore it anyway if it if left
>> in, and it can be dealt with in any new major revision.
>>
>> Graham
>>
>
_______________________________________________
Web-SIG mailing list
[hidden email]
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/lists%40nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Declaring PEP 3333 accepted (was: PEP 444 != WSGI 2.0)

PJ Eby
In reply to this post by Guido van Rossum
At 06:30 PM 1/3/2011 -0800, Guido van Rossum wrote:
>Would
>
>   sys.stdout.buffer.write(b'abc')
>
>do?
>
>(If you mix this with writing strings to sys.stdout directly, you may
>have to call sys.stdout.flush() first.)

The current code is:

             sys.stdout.write(data)  # TODO: this needs to be binary on Py3
             sys.stdout.flush()

Should I be using sys.stdout.buffer for both, or just the write?

For the CGI example in the PEP, I don't want to bother trying to make
it fully production-usable; that's what we have wsgiref in the stdlib
for.  So I won't worry about mixing strings and regular output in the
example, even if perhaps wsgiref should add the StringIO's proposed by Graham.

_______________________________________________
Web-SIG mailing list
[hidden email]
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/lists%40nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Declaring PEP 3333 accepted (was: PEP 444 != WSGI 2.0)

PJ Eby
In reply to this post by Graham Dumpleton-2
At 08:53 PM 1/4/2011 +1100, Graham Dumpleton wrote:
>BTW, to what extent are the examples in the PEP meant to be able to
>work on both Python 2.X and Python 3.X as is.
>
>Does it need to be clarified where examples will only work on Python
>3.X, in particular the CGI gateway.

The intention is that PEP 3333 will have examples specific to Python
3 in future.

_______________________________________________
Web-SIG mailing list
[hidden email]
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/lists%40nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Declaring PEP 3333 accepted (was: PEP 444 != WSGI 2.0)

PJ Eby
In reply to this post by Graham Dumpleton-2
At 09:51 PM 1/4/2011 +1100, Graham Dumpleton wrote:

>Add another point. FWIW, these are coming up because of questions
>being asked on python-dev IRC channel about PEP 3333.
>
>The issue as it came down to was that the PEP may not be clear enough
>in explaining that where str() is unicode and as such something like
>PATH_INFO, although unicode, is actually bytes decoded as ISO-8859-1,
>needed to be re encoded/decoded to get it back to Unicode in the
>charset required before use.
>
>They were thinking that because it was unicode already they could use
>it as is and not need to do anything. Ie., didn't realise that need to
>do:
>
>   path_info = environ.get('PATH_INFO', '')
>   path_info = path_info.encode('ISO-8859-1').decode('UTF-8')
>
>for example to get it interpreted as UTF-8 first. They were simply
>looking at concatenating new URL bits to the ISO-8859-1 variant from
>other unicode strings that weren't bytes represented as ISO-8859-1.
>
>In Python 2.X it was obvious that since it wasn't unicode that you had
>to decode it, but confusion may arise for Python 3.X if this
>requirement is not explicitly spelled out with a code example like
>above.
>
>We all may see it as obvious and yes perhaps it could be covered in
>separate articles or commentaries be people, but given this person was
>new to it, maybe it is deserving of more explanation in the PEP itself
>if they were confused.

It would be really awesome if somebody would write separate
Application Authors' Guide and Middleware Authors' Guides to
WSGI.  They don't need to know absolutely everything in the PEP,
unlike server authors.


>It could also be that the PEP covers it adequately already. I am too
>tired to read through it again right now.

It's pretty prominently stated early on that NO strings in the spec
are really unicode, they're just bytes packed into unicode objects.

Obviously, no matter how prominently this is stated, some people will
still make this mistake, but if desired, we could always put some
additional info near the environ part of the spec for clarification.

(It occurs to me in retrospect that I should probably have updated
wsgiref in the stdlib to check the bytesy-ness of strings used to
create Header objects.  Too late for 3.2, though.)

_______________________________________________
Web-SIG mailing list
[hidden email]
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/lists%40nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Declaring PEP 3333 accepted (was: PEP 444 != WSGI 2.0)

Guido van Rossum
In reply to this post by PJ Eby
On Tue, Jan 4, 2011 at 7:48 AM, P.J. Eby <[hidden email]> wrote:

> At 06:30 PM 1/3/2011 -0800, Guido van Rossum wrote:
>>
>> Would
>>
>>  sys.stdout.buffer.write(b'abc')
>>
>> do?
>>
>> (If you mix this with writing strings to sys.stdout directly, you may
>> have to call sys.stdout.flush() first.)
>
> The current code is:
>
>            sys.stdout.write(data)  # TODO: this needs to be binary on Py3
>            sys.stdout.flush()
>
> Should I be using sys.stdout.buffer for both, or just the write?

For both.

But the flush() I was referring to is actually *before* either of
these, suggesting

sys.stdout.flush()
sys.stdout.buffer.write(data)
sys.stdout.buffer.flush()

However the first flush() is only necessary is there's a possibility
that previously something had been written to sys.stdout (not to
sys.stdout.buffer).

> For the CGI example in the PEP, I don't want to bother trying to make it
> fully production-usable; that's what we have wsgiref in the stdlib for.  So
> I won't worry about mixing strings and regular output in the example, even
> if perhaps wsgiref should add the StringIO's proposed by Graham.

Not sure what you mean. Surely copying and pasting the examples should
work? What are the details you'd like to leave out?

--
--Guido van Rossum (python.org/~guido)
_______________________________________________
Web-SIG mailing list
[hidden email]
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/lists%40nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Declaring PEP 3333 accepted (was: PEP 444 != WSGI 2.0)

Graham Dumpleton-2
Can we not let the PEP 444 discussion side track getting PEP 3333
sorted out? This is exactly what has happened numerous times before
when we have been trying to sort out core issues of WSGI on Python 3.
And people wander why I get grumpy now every time this happens. :-(

So, where are we at? It seems the only real issue that needs to be
resolved is the correctness or otherwise of the CGI/WSGI bridge
example. Everything else could be left as is, even if dubious.

For the CGI/WSGI bridge I take it the issue is that for bytes need to
use the internal buffer directly. I don't see a need to flush the top
level stdout because nothing should be writing to stdout besides the
WSGI bridge and so it just needs to ensure it is consistent.

Is that the last thing or do I need to go spend some time and write my
own CGI/WSGI bridge for Python 3 based on my own Python 2 one I have
lying around and just do some final validation checks with a parallel
implementation as a sanity check to make sure we got everything? This
might be a good idea anyway.

Graham

On 5 January 2011 05:33, Guido van Rossum <[hidden email]> wrote:

> On Tue, Jan 4, 2011 at 7:48 AM, P.J. Eby <[hidden email]> wrote:
>> At 06:30 PM 1/3/2011 -0800, Guido van Rossum wrote:
>>>
>>> Would
>>>
>>>  sys.stdout.buffer.write(b'abc')
>>>
>>> do?
>>>
>>> (If you mix this with writing strings to sys.stdout directly, you may
>>> have to call sys.stdout.flush() first.)
>>
>> The current code is:
>>
>>            sys.stdout.write(data)  # TODO: this needs to be binary on Py3
>>            sys.stdout.flush()
>>
>> Should I be using sys.stdout.buffer for both, or just the write?
>
> For both.
>
> But the flush() I was referring to is actually *before* either of
> these, suggesting
>
> sys.stdout.flush()
> sys.stdout.buffer.write(data)
> sys.stdout.buffer.flush()
>
> However the first flush() is only necessary is there's a possibility
> that previously something had been written to sys.stdout (not to
> sys.stdout.buffer).
>
>> For the CGI example in the PEP, I don't want to bother trying to make it
>> fully production-usable; that's what we have wsgiref in the stdlib for.  So
>> I won't worry about mixing strings and regular output in the example, even
>> if perhaps wsgiref should add the StringIO's proposed by Graham.
>
> Not sure what you mean. Surely copying and pasting the examples should
> work? What are the details you'd like to leave out?
>
> --
> --Guido van Rossum (python.org/~guido)
> _______________________________________________
> Web-SIG mailing list
> [hidden email]
> Web SIG: http://www.python.org/sigs/web-sig
> Unsubscribe: http://mail.python.org/mailman/options/web-sig/graham.dumpleton%40gmail.com
>
_______________________________________________
Web-SIG mailing list
[hidden email]
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/lists%40nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Declaring PEP 3333 accepted (was: PEP 444 != WSGI 2.0)

PJ Eby
In reply to this post by Guido van Rossum
At 10:33 AM 1/4/2011 -0800, Guido van Rossum wrote:

>But the flush() I was referring to is actually *before* either of
>these, suggesting
>
>sys.stdout.flush()
>sys.stdout.buffer.write(data)
>sys.stdout.buffer.flush()
>
>However the first flush() is only necessary is there's a possibility
>that previously something had been written to sys.stdout (not to
>sys.stdout.buffer).

Yeah, that sort of error checking seems out of scope for a PEP
example.  If something was written to sys.stdout at that point, your
CGI was already broken.  ;-)


> > For the CGI example in the PEP, I don't want to bother trying to make it
> > fully production-usable; that's what we have wsgiref in the stdlib for.  So
> > I won't worry about mixing strings and regular output in the example, even
> > if perhaps wsgiref should add the StringIO's proposed by Graham.
>
>Not sure what you mean. Surely copying and pasting the examples should
>work? What are the details you'd like to leave out?

I meant that a production CGI gateway should handle various
boundary/error conditions.  Graham was saying that a CGI gateway
should replace sys.stdout to avoid stray print()s causing problems,
and I consider that similar to saying, "what if somebody wrote text
to sys.stdout?" -- i.e., an error handling case that would be a good
idea in a production gateway, but which would obscure the common case
that the example is meant to illustrate.

_______________________________________________
Web-SIG mailing list
[hidden email]
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/lists%40nabble.com
123