Quantcast

PEP 418 is too divisive and confusing and should be postponed

classic Classic list List threaded Threaded
53 messages Options
123
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: PEP 418 is too divisive and confusing and should be postponed

Ethan Furman-2
Cameron Simpson wrote:

> On 04Apr2012 22:23, PJ Eby <[hidden email]> wrote:
> | On Apr 4, 2012 7:28 PM, "Victor Stinner" <[hidden email]> wrote:
> | > More details why it's hard to define such function and why I dropped
> | > it from the PEP.
> | >
> | > If someone wants to propose again such function ("monotonic or
> | > fallback to system" clock), two issues should be solved:
> | >
> | >  - name of the function
> | >  - description of the function
> |
> | Maybe I missed it, but did anyone ever give a reason why the fallback
> | couldn't be to Steven D'Aprano's monotonic wrapper algorithm over the
> | system clock?  (Given a suitable minimum delta.)  That function appeared to
> | me to provide a sufficiently monotonic clock for timeout purposes, if
> | nothing else.
>
> It was pointed out (by Nick Coglan I think?) that if the system clock
> stepped backwards then a timeout would be extended by at least that
> long. For example, code that waited (by polling the synthetic clock)
> for 1s could easily wait an hour if the system clock stepped back that
> far. Probaby undesirable.

Steven D'Aprano's synthetic clock is able to partially avoid that
situation -- worst case is a timeout of double what you asked for -- so
10 seconds instead of 5 (which is much better than 3600!).

~Ethan~
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: PEP 418 is too divisive and confusing and should be postponed

Antoine Pitrou
On Thu, 05 Apr 2012 08:32:22 -0700
Ethan Furman <[hidden email]> wrote:
>
> Steven D'Aprano's synthetic clock is able to partially avoid that
> situation -- worst case is a timeout of double what you asked for -- so
> 10 seconds instead of 5 (which is much better than 3600!).

The remaining issue is that the clock is not system-wide, it's
interpreter-specific.

Regards

Antoine.


_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: PEP 418 is too divisive and confusing and should be postponed

PJ Eby
In reply to this post by Cameron Simpson


On Wed, Apr 4, 2012 at 11:41 PM, Cameron Simpson <[hidden email]> wrote:
On 04Apr2012 22:23, PJ Eby <[hidden email]> wrote:
| On Apr 4, 2012 7:28 PM, "Victor Stinner" <[hidden email]> wrote:
| > More details why it's hard to define such function and why I dropped
| > it from the PEP.
| >
| > If someone wants to propose again such function ("monotonic or
| > fallback to system" clock), two issues should be solved:
| >
| >  - name of the function
| >  - description of the function
|
| Maybe I missed it, but did anyone ever give a reason why the fallback
| couldn't be to Steven D'Aprano's monotonic wrapper algorithm over the
| system clock?  (Given a suitable minimum delta.)  That function appeared to
| me to provide a sufficiently monotonic clock for timeout purposes, if
| nothing else.

It was pointed out (by Nick Coglan I think?) that if the system clock
stepped backwards then a timeout would be extended by at least that
long. For example, code that waited (by polling the synthetic clock)
for 1s could easily wait an hour if the system clock stepped back that
far. Probaby undesirable.

Steven D'Aprano's algorithm doesn't do that.  If the system clock steps backwards, it still stepped forward by a specified minimum delta.  The amount of time that a timeout was extended would be a function of the polling frequency, not the presence of absence of backward steps in the underlying clock.

_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: PEP 418 is too divisive and confusing and should be postponed

PJ Eby
In reply to this post by Victor Stinner-3


On Thu, Apr 5, 2012 at 6:34 AM, Victor Stinner <[hidden email]> wrote:
2012/4/5 PJ Eby <[hidden email]>:
>> More details why it's hard to define such function and why I dropped
>> it from the PEP.
>>
>> If someone wants to propose again such function ("monotonic or
>> fallback to system" clock), two issues should be solved:
>>
>>  - name of the function
>>  - description of the function
>
> Maybe I missed it, but did anyone ever give a reason why the fallback
> couldn't be to Steven D'Aprano's monotonic wrapper algorithm over the system
> clock?  (Given a suitable minimum delta.)  That function appeared to me to
> provide a sufficiently monotonic clock for timeout purposes, if nothing
> else.


Did you read the following section of the PEP?
http://www.python.org/dev/peps/pep-0418/#working-around-operating-system-bugs

Did I miss something? If yes, could you write a patch for the PEP please?

What's missing is that if you're using a monotonic clock for timeouts, then a monotonically-adjusted system clock can do that, subject to the polling frequency -- it does not break just because the system clock is set backwards; it simply loses time proportional to the frequency with which it is polled.

For timeout purposes in a single process, such a clock is useful.  It just isn't suitable for benchmarks, or for interprocess coordination.


_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: PEP 418 is too divisive and confusing and should be postponed

Guido van Rossum
On Thu, Apr 5, 2012 at 9:48 AM, PJ Eby <[hidden email]> wrote:

>
>
> On Thu, Apr 5, 2012 at 6:34 AM, Victor Stinner <[hidden email]>
> wrote:
>>
>> 2012/4/5 PJ Eby <[hidden email]>:
>> >> More details why it's hard to define such function and why I dropped
>> >> it from the PEP.
>> >>
>> >> If someone wants to propose again such function ("monotonic or
>> >> fallback to system" clock), two issues should be solved:
>> >>
>> >>  - name of the function
>> >>  - description of the function
>> >
>> > Maybe I missed it, but did anyone ever give a reason why the fallback
>> > couldn't be to Steven D'Aprano's monotonic wrapper algorithm over the
>> > system
>> > clock?  (Given a suitable minimum delta.)  That function appeared to me
>> > to
>> > provide a sufficiently monotonic clock for timeout purposes, if nothing
>> > else.
>>
>>
>> Did you read the following section of the PEP?
>>
>> http://www.python.org/dev/peps/pep-0418/#working-around-operating-system-bugs
>>
>> Did I miss something? If yes, could you write a patch for the PEP please?
>
>
> What's missing is that if you're using a monotonic clock for timeouts, then
> a monotonically-adjusted system clock can do that, subject to the polling
> frequency -- it does not break just because the system clock is set
> backwards; it simply loses time proportional to the frequency with which it
> is polled.

Depending on the polling frequency sounds like a bad idea, since you
can't know that you're the only user of the clock. Also depending on
the use case, too short a timeout may be worse than too long a
timeout. E.g. imagine hitting a website that usually takes 2 seconds
to respond, and setting a timeout to e.g. 4 seconds to bail. If the
timeout somehow gets reduced to 1 second it will appear as if the
website died, whereas if the timeout was increased to 1 hour, nothing
bad would happen unless the website *actually* started having truly
bad response times.

> For timeout purposes in a single process, such a clock is useful.  It just
> isn't suitable for benchmarks, or for interprocess coordination.

I think it would be better if the proposed algorithm (or whatever
algorithm to "fix" timeouts) was implemented by the
application/library code using the timeout (or provided as a separate
library function), rather than by the clock, since the clock can't
know what fallback behavior the app/lib needs.

--
--Guido van Rossum (python.org/~guido)
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: PEP 418 is too divisive and confusing and should be postponed

Antoine Pitrou
On Thu, 5 Apr 2012 09:56:19 -0700
Guido van Rossum <[hidden email]> wrote:
>
> > For timeout purposes in a single process, such a clock is useful.  It just
> > isn't suitable for benchmarks, or for interprocess coordination.
>
> I think it would be better if the proposed algorithm (or whatever
> algorithm to "fix" timeouts) was implemented by the
> application/library code using the timeout (or provided as a separate
> library function), rather than by the clock, since the clock can't
> know what fallback behavior the app/lib needs.

Agreed with providing it as a separate library function.

Regards

Antoine.


_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: PEP 418 is too divisive and confusing and should be postponed

Victor Stinner-3
>> > For timeout purposes in a single process, such a clock is useful.  It just
>> > isn't suitable for benchmarks, or for interprocess coordination.
>>
>> I think it would be better if the proposed algorithm (or whatever
>> algorithm to "fix" timeouts) was implemented by the
>> application/library code using the timeout (or provided as a separate
>> library function), rather than by the clock, since the clock can't
>> know what fallback behavior the app/lib needs.
>
> Agreed with providing it as a separate library function.

I changed time.monotonic() to not fallback to the system clock exactly
for this reason: Python cannot guess what the developer expects, or
how the developer will use the clock.

Instead of implementing your own clock in your application, it's maybe
easier to patch your OS? I suppose that you are running on GNU/Hurd,
because I didn't find yet other OS not providing a monotonic clock :-)

If you are using an OS that doesn't provide a monotonic clock, do you
really need to implement your own in your application?

Victor
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: PEP 418 is too divisive and confusing and should be postponed

Ethan Furman-2
Victor Stinner wrote:
> I changed time.monotonic() to not fallback to the system clock exactly
> for this reason: Python cannot guess what the developer expects, or
> how the developer will use the clock.

Which is exactly why I like Cameron Simpson's approach to selecting a
clock -- let the developer/user decide what kind of clock they need, and
ask for one that matches their criteria.

~Ethan~
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: PEP 418 is too divisive and confusing and should be postponed

PJ Eby
In reply to this post by Guido van Rossum
On Thu, Apr 5, 2012 at 12:56 PM, Guido van Rossum <[hidden email]> wrote:
Depending on the polling frequency sounds like a bad idea, since you
can't know that you're the only user of the clock. Also depending on
the use case, too short a timeout may be worse than too long a
timeout.

Given a small enough delta, the timeout won't be too short. (Steven's original code sample I believed used either 0 or 1 as a delta, but it could be as small a fraction as will add correctly in the datatype used.)   And the worst case polling frequency is the length of the timeout, meaning you can't end up with more than double your intended timeout.

In the opposite scenario, where the time is polled in a tight loop, then as long as Python doesn't sample the raw clock so often that the summed fractional deltas exceeds the real clock speed, the timeout won't be shortened by any appreciable amount.  In fact, this can be guaranteed by measuring time as a (raw, increment) tuple, where the increment can be an arbitrarily-large integer.  Each new time value is greater than the one before, yet the real component remains untouched.  With this approach, the timeout can only be delayed for however long the system clock *stops*, and the timeout can only be shortened by the system clock skipping ahead.

Okay, having thought that out, I now agree that there are too many fine points to make this cover enough of the use cases without needing parameters.

Or more to the point, "If the implementation is hard to explain, it's a bad idea."  ;-)
 

_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: PEP 418 is too divisive and confusing and should be postponed

zooko
In reply to this post by Victor Stinner-3
Folks:

Good job, Victor Stinner on baking the accumulated knowledge of this
thread into PEP 418. Even though I'm very interested in the topic, I
haven't been able to digest the whole thread(s) on the list and
understand what the current collective understanding is. The detailed
PEP document helps a lot.

I think there are still some mistakes, either in our collective
understanding as reflected by the PEP, or in my own head.

For starters, I still don't understand the first, most basic thing:
what do people mean when they say "monotonic clock"? I don't
understand the current text of PEP 418 with regard to the definition
of that word.

Allow me to resort to an analogy. There is an infinitely long,
perfectly straight and flat racetrack. There is a flag that gets
dragged along it at a constant rate, with the label "REAL TIME" on the
flag. There are some runners, each with a different label on their
chest:

Runner A: a helicopter hovers over Runner A. Occasionally it picks him
up and plops him down right next to the flag. Also, he wears a headset
and listens to instructions from his coach to run a little faster or
slower, as necessary, to remain abreast of the flag.

Runner B: a helicopter hovers over Runner B. If he is behind the flag,
it will pick him up and plop him down right next to the flag. However,
if he is ahead of the flag it will not pick him up.

Runner C: no helicopter ever picks up Runner C, but he does wear a
headset and listens to instructions from his coach to run a little
faster or a little slower. His coach tells him to run a little faster
if he is behind the flag or run a little slower if he is in front of
the flag, with the goal of eventually having him right next to the
flag.

Runner D: like Runner C, he never gets picked up, but he listens to
instructions to run a little faster or a little slower. However,
instead of telling him to run faster in order to catch up to the flag,
or to run slower in order to "catch down" to the flag, his coach
instead tells him to run a little faster if he is moving slower than
the flag is moving, and to run a little slower if he is moving faster
than the flag is moving. Note that this is very different from Runner
C, in that it is not intended to cause him to eventually be right next
to the flag, and indeed if it is done right it guarantees that he will
*never* be right next to the flag, although he will be moving just as
fast as the flag is moving.

Runner E: no helicopter, no headset. He just proceeds at his own pace,
blissfully unaware of the exhortations of others.

Now: which ones of these five runners do you call "monotonic"? Which
ones do you call "steady"?

Regards,

Zooko
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: PEP 418 is too divisive and confusing and should be postponed

Cameron Simpson
In reply to this post by Guido van Rossum
On 05Apr2012 09:56, Guido van Rossum <[hidden email]> wrote:
| On Thu, Apr 5, 2012 at 9:48 AM, PJ Eby <[hidden email]> wrote:
| > What's missing is that if you're using a monotonic clock for timeouts, then
| > a monotonically-adjusted system clock can do that, subject to the polling
| > frequency -- it does not break just because the system clock is set
| > backwards; it simply loses time proportional to the frequency with which it
| > is polled.
|
| Depending on the polling frequency sounds like a bad idea, since you
| can't know that you're the only user of the clock.

You can if you're handed a shiny new "clock" object in some way, with a
not-a-singleton guarrentee. Of course, such a clock is immediately
_less_ reliable to synchornisation with other clock users:-)

| Also depending on
| the use case, too short a timeout may be worse than too long a
| timeout. [...]
|
| > For timeout purposes in a single process, such a clock is useful.  It just
| > isn't suitable for benchmarks, or for interprocess coordination.
|
| I think it would be better if the proposed algorithm (or whatever
| algorithm to "fix" timeouts) was implemented by the
| application/library code using the timeout (or provided as a separate
| library function), rather than by the clock, since the clock can't
| know what fallback behavior the app/lib needs.

Absolutely. I for one would be happy with a clocktools module or
something offering a bunch of synthetic clocks. Especially if they were
compatible in API with whatever clock objects the core time module
clocks used, so that a user _could_ add them into the pick-a-clock
decision easily.

Cheers,
--
Cameron Simpson <[hidden email]> DoD#743
http://www.cskk.ezoshosting.com/cs/

I'd be careful who you call smart or not smart. Smart isn't knowing how to
save six bytes. Smart is knowing WHEN. - Peter Cherna, Amiga O.S. Development
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: PEP 418 is too divisive and confusing and should be postponed

Cameron Simpson
In reply to this post by zooko
On 05Apr2012 13:39, Zooko Wilcox-O'Hearn <[hidden email]> wrote:
| Good job, Victor Stinner on baking the accumulated knowledge of this
| thread into PEP 418. Even though I'm very interested in the topic, I
| haven't been able to digest the whole thread(s) on the list and
| understand what the current collective understanding is.

There isn't a collective understanding :-) That's why all the noise!

| The detailed
| PEP document helps a lot.

Yes indeed, though like all of us I think it could (always) use more
detail on my pet concerns.

| I think there are still some mistakes, either in our collective
| understanding as reflected by the PEP, or in my own head.
|
| For starters, I still don't understand the first, most basic thing:
| what do people mean when they say "monotonic clock"? I don't
| understand the current text of PEP 418 with regard to the definition
| of that word.

A monotonic clock never returns t0 > t1 for t0, t1 being two adjacent
polls of the clock. On its own it says nothing about steadiness or
correlation with real world time.

_Quality of implementation_ says that the montonic() call should try to
return a close that is monotonic and _also_ steady and preferably
precise. How these things are balancer is a matter of policy.

| Allow me to resort to an analogy. There is an infinitely long,
| perfectly straight and flat racetrack. There is a flag that gets
| dragged along it at a constant rate, with the label "REAL TIME" on the
| flag. There are some runners, each with a different label on their
| chest:
|
| Runner A: a helicopter hovers over Runner A. Occasionally it picks him
| up and plops him down right next to the flag. Also, he wears a headset
| and listens to instructions from his coach to run a little faster or
| slower, as necessary, to remain abreast of the flag.

If he always runs forwards, it is montonic. Not very steady when the
helicopter comes to play.

| Runner B: a helicopter hovers over Runner B. If he is behind the flag,
| it will pick him up and plop him down right next to the flag. However,
| if he is ahead of the flag it will not pick him up.

Seems like runner A without instruction. Monotonic. Not very steady.

| Runner C: no helicopter ever picks up Runner C, but he does wear a
| headset and listens to instructions from his coach to run a little
| faster or a little slower. His coach tells him to run a little faster
| if he is behind the flag or run a little slower if he is in front of
| the flag, with the goal of eventually having him right next to the
| flag.

If he always runs forward, monotonic. And steady.

| Runner D: like Runner C, he never gets picked up, but he listens to
| instructions to run a little faster or a little slower. However,
| instead of telling him to run faster in order to catch up to the flag,
| or to run slower in order to "catch down" to the flag, his coach
| instead tells him to run a little faster if he is moving slower than
| the flag is moving, and to run a little slower if he is moving faster
| than the flag is moving. Note that this is very different from Runner
| C, in that it is not intended to cause him to eventually be right next
| to the flag, and indeed if it is done right it guarantees that he will
| *never* be right next to the flag, although he will be moving just as
| fast as the flag is moving.
|
| Runner E: no helicopter, no headset. He just proceeds at his own pace,
| blissfully unaware of the exhortations of others.
|
| Now: which ones of these five runners do you call "monotonic"? Which
| ones do you call "steady"?

If they all run forwards, they're all monotonic.

If their coach or helicopter can move then _backwards_ they're not
monotonic. If the helicopter can move them an arbitrary (but matching
the game plan) distance, they're not steady. Otherwise they are steady,
if the runner's speed is always sufficiently close to the flag speed
(this threshold and the criteria for measuring it as subject to debate,
forming policy).

And "high resolution" has its own flavours, though generally less
contentious.

Cheers,
--
Cameron Simpson <[hidden email]> DoD#743
http://www.cskk.ezoshosting.com/cs/

If you don't shoot the fish in your barrel, your barrel will soon be
full of fish. - Tim Mefford
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: PEP 418 is too divisive and confusing and should be postponed

Eric Snow-2
In reply to this post by Guido van Rossum

On Apr 5, 2012 11:01 AM, "Guido van Rossum" <[hidden email]> wrote:
> I think it would be better if the proposed algorithm (or whatever
> algorithm to "fix" timeouts) was implemented by the
> application/library code using the timeout (or provided as a separate
> library function), rather than by the clock, since the clock can't
> know what fallback behavior the app/lib needs.

+1

-eric


_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: PEP 418 is too divisive and confusing and should be postponed

Greg Ewing
In reply to this post by Cameron Simpson
Cameron Simpson wrote:

> A monotonic clock never returns t0 > t1 for t0, t1 being two adjacent
> polls of the clock. On its own it says nothing about steadiness or
> correlation with real world time.

No, no, no.

This is the strict mathematical meaning of the word "monotonic",
but the way it's used in relation to OS clocks, it seems to
mean rather more than that.

A clock whose only guarantee is that it never goes backwards
is next to useless. For things like benchmarks and timeouts,
the important thing about a clock that it *keeps going forward*
at a reasonably constant rate. On the other hand, it can have
an arbitrary starting point and doesn't have to be related
to any external time standard.

I'm assuming this is what Linux et al mean when they talk
about a "monotonic clock", because anything else doesn't make
sense.

So if we're going to use the term "monotonic" at all, I think we
should explicitly define it as having this meaning, i.e.
both mathematically monotonic and steady. Failure to be clear
about this has caused a huge amount of confusion in this thead
so far.

--
Greg
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: PEP 418 is too divisive and confusing and should be postponed

Cameron Simpson
On 06Apr2012 13:14, Greg Ewing <[hidden email]> wrote:
| Cameron Simpson wrote:
| > A monotonic clock never returns t0 > t1 for t0, t1 being two adjacent
| > polls of the clock. On its own it says nothing about steadiness or
| > correlation with real world time.
|
| No, no, no.
| This is the strict mathematical meaning of the word "monotonic",
| but the way it's used in relation to OS clocks, it seems to
| mean rather more than that.

It's like my next paragraph didn't exist...

  _Quality of implementation_ says that the montonic() call should try to
  return a close that is monotonic and _also_ steady and preferably precise.
  How these things are balancer is a matter of policy.

To forstall things, right at the bottom of this I'm going to say i agree
with you about objectives, but not about terminology. I maintain that
"monotonic" still means what I said, and that it is the combination of
the word with "clock" that brings in your other criteria.

| A clock whose only guarantee is that it never goes backwards
| is next to useless. For things like benchmarks and timeouts,
| the important thing about a clock that it *keeps going forward*
| at a reasonably constant rate. [...]
| I'm assuming this is what Linux et al mean when they talk
| about a "monotonic clock", because anything else doesn't make
| sense.

This is why I say there's no global understanding. Why assume?
On a Linux box, "man 3 clock_getres" says (sorry, this is a bit wordy):

   All  implementations  support the system-wide real-time clock, which
   is identified by CLOCK_REALTIME.  Its time represents seconds and
   nanoseconds  since the Epoch.  When its time is changed, timers for
   a relative interval are unaffected, but timers for an absolute point
   in time  are affected.

   More  clocks may be implemented.  The interpretation of the
   corresponding time values and the effect on timers is unspecified.

   Sufficiently recent versions of glibc and the Linux kernel support
   the following clocks:

   CLOCK_REALTIME
      System-wide real-time clock.  Setting this clock requires
      appropriate privileges.

   CLOCK_MONOTONIC
      Clock that cannot be set and  represents  monotonic  time since
      some unspecified starting point.

   CLOCK_MONOTONIC_RAW (since Linux 2.6.28; Linux-specific)
      Similar  to  CLOCK_MONOTONIC, but provides access to a raw hard-
      ware-based time that is not subject to NTP adjustments.

   CLOCK_PROCESS_CPUTIME_ID
      High-resolution per-process timer from the CPU.

   CLOCK_THREAD_CPUTIME_ID
      Thread-specific CPU-time clock.

The first paragraph is very clear about real time (wall clock, and what
time.time() does, being "seconds since the epoch"). The CLOCK_MONOTONIC*
modes clearly imply steadiness.

"man 3p clock_getres" (POSIX clock_getres) is even more verbose and general.

| So if we're going to use the term "monotonic" at all, I think we
| should explicitly define it as having this meaning, i.e.
| both mathematically monotonic and steady.

I think if it is too "unsteady" then it is not "time".

So I actually side with you in the requirement for a "clock", but
monotonic alone does not mean that. Quality of implementation _may_ mean
we don't offer something abjectly erratic.

| Failure to be clear
| about this has caused a huge amount of confusion in this thead
| so far.

And burdening the word "monotonic" itself with exciting new meanings
doesn't help. "monotonic clock", sure, _that_ has additional
connotations, but not just the word monotonic.

Cheers,
--
Cameron Simpson <[hidden email]> DoD#743
http://www.cskk.ezoshosting.com/cs/

We had the experience, but missed the meaning.  - T.S. Eliot
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: PEP 418 is too divisive and confusing and should be postponed

Steven D'Aprano-8
In reply to this post by Greg Ewing
Greg Ewing wrote:

> Cameron Simpson wrote:
>
>> A monotonic clock never returns t0 > t1 for t0, t1 being two adjacent
>> polls of the clock. On its own it says nothing about steadiness or
>> correlation with real world time.
>
> No, no, no.
>
> This is the strict mathematical meaning of the word "monotonic",
> but the way it's used in relation to OS clocks, it seems to
> mean rather more than that.
>
> A clock whose only guarantee is that it never goes backwards
> is next to useless. For things like benchmarks and timeouts,
> the important thing about a clock that it *keeps going forward*

That would be a *strictly* monotonic clock, as opposed to merely monotonic.

And yes, a merely monotonic clock could just return a constant value, forever:

9, 9, 9, 9, 9, ...

and yes, such a thing would be useless.

Various people have suggested caching the last value of time() and re-using it
if the new value is in the past. This will give a monotonic clock, but since
it can give constant timestamps for an indefinite period, it's usefulness is
limited.

I earlier put forward an alternate implementation which gives no more than one
such constant tick in a row. If you know the hardware resolution of the clock,
you can even avoid that single constant tick by always advancing the timestamp
by that minimum resolution:

_prev = _prev_raw = 0
_res = 1e-9  # nanosecond resolution
def monotonic():
     global _prev, _prev_raw
     raw = time()
     delta = max(_res, raw - _prev_raw)
     _prev_raw = raw
     _prev += delta
     return _prev

Even if time() jumps backwards, or stays constant, monotonic() here will be
strictly monotonic.



--
Steven
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: PEP 418 is too divisive and confusing and should be postponed

Greg Ewing
In reply to this post by Cameron Simpson
Cameron Simpson wrote:

> I maintain that
> "monotonic" still means what I said, and that it is the combination of
> the word with "clock" that brings in your other criteria.

I'm not trying to redefine the word "monotonic" in general.
All I'm saying is that *if* the PEP is going to talk about
a "monotonic clock" or "monotonic time", it should clearly
define what this means, because it's not obvious to everyone
that it implies something more than the mathematical meaning.

Alternatively, don't use the word "monotonic" at all, and
find a better term.

>    CLOCK_MONOTONIC
>       Clock that cannot be set and  represents  monotonic  time since
>       some unspecified starting point.

Which doesn't help very much, because it talks about "monotonic
time" without saying what that means. Googling for that phrase
doesn't seem turn up anything very useful. Apparently we're
supposed to just know.

--
Greg
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: PEP 418 is too divisive and confusing and should be postponed

Greg Ewing
In reply to this post by Steven D'Aprano-8
Steven D'Aprano wrote:
> Greg Ewing wrote:
>
>> the important thing about a clock that it *keeps going forward*
>
> That would be a *strictly* monotonic clock, as opposed to merely monotonic.

Well, yes, but even that's not enough -- it needs to go forward
at a reasonably constant rate, otherwise it's just as useless.
If it had enough resolution, it could go forward by one femtosecond
every hour for a while and still call itself strictly monotonic...

--
Greg
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: PEP 418 is too divisive and confusing and should be postponed

Paul Moore
In reply to this post by Cameron Simpson
On 6 April 2012 02:50, Cameron Simpson <[hidden email]> wrote:
(Quoted from the Linux manpage)
All  implementations  support the system-wide real-time clock, which
  is identified by CLOCK_REALTIME.  Its time represents seconds and
  nanoseconds  since the Epoch.  When its time is changed, timers for
  a relative interval are unaffected, but timers for an absolute point
  in time  are affected.

This made me think. They make a distinction between "timers for a relative interval" and "timers for an absolute point in time".

But that's not right - *all* of the clock calls we are talking about here return a *single* number. Interpreting that as an absolute time needs an epoch. Or to put it another way, clock values are always meaningless without context - whereas clock *differences* are what actually carry meaning (in terms of a duration).

On that basis, I'd say that

- A clock that doesn't get adjusted or slewed is a tick counter (which technically doesn't have any relationship to time, although the tick frequency can be used to convert to seconds, but see the next entry)
- A clock that gets slewed but not adjusted is a seconds counter (or at least, the nearest approximation the system can provide - presumably better than a tick counter)
- A clock that gets adjusted is not an interval timer at all, but an absolute timer (which therefore shouldn't really be used for benchmarking or timeouts)

It seems to me that what *I* would most often need are the second two of these (to at least as high a precision as my app needs, which may vary but "to the highest precision possible" would do :-)) I'd be happy for a seconds counter to fallback to a tick counter converted to seconds using its frequency - slewing is simply an accuracy improvement process, as far as I can see.

It seems to me that the current time.time() and time.wallclock() are the right names for my "absolute timer" and "seconds timer" above. Whether their implementations match my definitions I'm not sure, but that's what I'd hope. One thing I would expect is that time.wallclock() would never go backwards (so differences are always positive). The various other debates about monotonic, steady, etc, seem to me to be only relevant for specialist uses that I don't care about.

As regards suspension, if I'm timing intervals and the system suspends, I'd be happy to say all bets are off. Similarly with timeouts. If I cared, I'd simply make sure the system didn't suspend :-)

As far as comparability between different threads or processes are concerned, I would expect absolute time (time.time) to be the same across threads or processes (but wouldn't generally write apps that were affected if it weren't - at least by small amounts), but I wouldn't expect time.wallclock values obtained in different threads or processes to be comparable (mostly because I can't think of a case where I'd compare them). Where VMs or multiple machines are involved, I wouldn't even expect absolute time to match (but that's the job of NTP, and if time.time follows NTP, there's no reason why there would be an issue even there).

Summary: I'm happy with time.time and time.wallclock. The rest of this debate doesn't matter for my usecases (and I suspect many other people's in practice).

[Update, after I downloaded and installed 3.3a2] Bah, looks like time.wallclock is gone. (Actually, looks like it was documented but not implemented in 3.3a1!). Actually, the docs and the implementation don't match - clock_gettime is documented as available, but it's not (at least on Windows). I still prefer time.wallclock() as described above and in the 3.3a1 documentation. I thought I knew what was going on, but now I'm confused. My comments above still stand, though.

Paul.

_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: PEP 418 is too divisive and confusing and should be postponed

Lennart Regebro-2
In reply to this post by Victor Stinner-3
On Thu, Apr 5, 2012 at 12:32, Victor Stinner <[hidden email]> wrote:
> I prefer to use CLOCK_MONOTONIC, not because it is also available for
> older Linux kernels, but because it is more reliable. Even if the
> underlying clock source is unstable (unstable frequency), a delta of
> two reads of the CLOCK_MONOTONIC clock is a result in *seconds*,
> whereas CLOCK_MONOTONIC_RAW may use an unit a little bit bigger or
> smaller than a second.

Aha. OK, CLOCK_MONOTONIC it is then.

//Lennart
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
123
Loading...