[RFC] PEP 418: Add monotonic time, performance counter and process time functions

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
53 messages Options
123
Reply | Threaded
Open this post in threaded view
|

[RFC] PEP 418: Add monotonic time, performance counter and process time functions

Victor Stinner-3
Hi,

Here is a simplified version of the first draft of the PEP 418. The
full version can be read online.
http://www.python.org/dev/peps/pep-0418/

The implementation of the PEP can be found in this issue:
http://bugs.python.org/issue14428

I post a simplified version for readability and to focus on changes
introduced by the PEP. Removed sections: Existing Functions,
Deprecated Function, Glossary, Hardware clocks, Operating system time
functions, System Standby, Links.

---

PEP: 418
Title: Add monotonic time, performance counter and process time functions
Version: f2bb3f74298a
Last-Modified: 2012-04-15 17:06:07 +0200 (Sun, 15 Apr 2012)
Author: Cameron Simpson <[hidden email]>, Jim Jewett
<[hidden email]>, Victor Stinner <[hidden email]>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 26-March-2012
Python-Version: 3.3

Abstract
========

This PEP proposes to add ``time.get_clock_info(name)``,
``time.monotonic()``, ``time.perf_counter()`` and
``time.process_time()`` functions to Python 3.3.

Rationale
=========

If a program uses the system time to schedule events or to implement
a timeout, it will not run events at the right moment or stop the
timeout too early or too late when the system time is set manually or
adjusted automatically by NTP.  A monotonic clock should be used
instead to not be affected by system time updates:
``time.monotonic()``.

To measure the performance of a function, ``time.clock()`` can be used
but it is very different on Windows and on Unix.  On Windows,
``time.clock()`` includes time elapsed during sleep, whereas it does
not on Unix.  ``time.clock()`` precision is very good on Windows, but
very bad on Unix.  The new ``time.perf_counter()`` function should be
used instead to always get the most precise performance counter with a
portable behaviour (ex: include time spend during sleep).

To measure CPU time, Python does not provide directly a portable
function.  ``time.clock()`` can be used on Unix, but it has a bad
precision.  ``resource.getrusage()`` can also be used on Unix, but it
requires to get fields of a structure and compute the sum of time
spent in kernel space and user space.  The new ``time.process_time()``
function acts as a portable counter that always measures CPU time
(doesn't include time elapsed during sleep) and has the best available
precision.

Each operating system implements clocks and performance counters
differently, and it is useful to know exactly which function is used
and some properties of the clock like its resolution and its
precision.  The new ``time.get_clock_info()`` function gives access to
all available information of each Python time function.

New functions:

* ``time.monotonic()``: timeout and scheduling, not affected by system
  clock updates
* ``time.perf_counter()``: benchmarking, most precise clock for short
  period
* ``time.process_time()``: profiling, CPU time of the process

Users of new functions:

* time.monotonic(): concurrent.futures, multiprocessing, queue, subprocess,
  telnet and threading modules to implement timeout
* time.perf_counter(): trace and timeit modules, pybench program
* time.process_time(): profile module
* time.get_clock_info(): pybench program to display information about the
  timer like the precision or the resolution

The ``time.clock()`` function is deprecated because it is not
portable: it behaves differently depending on the operating system.
``time.perf_counter()`` or ``time.process_time()`` should be used
instead, depending on your requirements. ``time.clock()`` is marked as
deprecated but is not planned for removal.


Python functions
================

New Functions
-------------

time.get_clock_info(name)
^^^^^^^^^^^^^^^^^^^^^^^^^

Get information on the specified clock.  Supported clock names:

* ``"clock"``: ``time.clock()``
* ``"monotonic"``: ``time.monotonic()``
* ``"perf_counter"``: ``time.perf_counter()``
* ``"process_time"``: ``time.process_time()``
* ``"time"``: ``time.time()``

Return a dictionary with the following keys:

* Mandatory keys:

  * ``"implementation"`` (str): name of the underlying operating system
    function.  Examples: ``"QueryPerformanceCounter()"``,
    ``"clock_gettime(CLOCK_REALTIME)"``.
  * ``"resolution"`` (float): resolution in seconds of the clock.
  * ``"is_monotonic"`` (bool): True if the clock cannot go backward.

* Optional keys:

  * ``"precision"`` (float): precision in seconds of the clock
    reported by the operating system.
  * ``"is_adjusted"`` (bool): True if the clock is adjusted (e.g. by a
    NTP daemon).


time.monotonic()
^^^^^^^^^^^^^^^^

Monotonic clock, i.e. cannot go backward.  It is not affected by system
clock updates.  The reference point of the returned value is
undefined, so that only the difference between the results of
consecutive calls is valid and is a number of seconds.

On Windows versions older than Vista, ``time.monotonic()`` detects
``GetTickCount()`` integer overflow (32 bits, roll-over after 49.7
days): it increases a delta by 2\ :sup:`32` each time than an overflow
is detected.  The delta is stored in the process-local state and so
the value of ``time.monotonic()`` may be different in two Python
processes running for more than 49 days. On more recent versions of
Windows and on other operating systems, ``time.monotonic()`` is
system-wide.

Availability: Windows, Mac OS X, Unix, Solaris. Not available on
GNU/Hurd.

Pseudo-code [#pseudo]_::

    if os.name == 'nt':
        # GetTickCount64() requires Windows Vista, Server 2008 or later
        if hasattr(time, '_GetTickCount64'):
            def monotonic():
                return _time.GetTickCount64() * 1e-3
        else:
            def monotonic():
                ticks = _time.GetTickCount()
                if ticks < monotonic.last:
                    # Integer overflow detected
                    monotonic.delta += 2**32
                monotonic.last = ticks
                return (ticks + monotonic.delta) * 1e-3
            monotonic.last = 0
            monotonic.delta = 0

    elif os.name == 'mac':
        def monotonic():
            if monotonic.factor is None:
                factor = _time.mach_timebase_info()
                monotonic.factor = timebase[0] / timebase[1]
            return _time.mach_absolute_time() * monotonic.factor
        monotonic.factor = None

    elif hasattr(time, "clock_gettime") and hasattr(time, "CLOCK_HIGHRES"):
        def monotonic():
            return time.clock_gettime(time.CLOCK_HIGHRES)

    elif hasattr(time, "clock_gettime") and hasattr(time, "CLOCK_MONOTONIC"):
        def monotonic():
            return time.clock_gettime(time.CLOCK_MONOTONIC)


On Windows, ``QueryPerformanceCounter()`` is not used even though it
has a better precision than ``GetTickCount()``.  It is not reliable
and has too many issues.


time.perf_counter()
^^^^^^^^^^^^^^^^^^^

Performance counter with the highest available precision to measure a
duration.  It does include time elapsed during sleep and is
system-wide.  The reference point of the returned value is undefined,
so that only the difference between the results of consecutive calls
is valid and is a number of seconds.

Pseudo-code::

    def perf_counter():
        if perf_counter.use_performance_counter:
            if perf_counter.performance_frequency is None:
                try:
                    perf_counter.performance_frequency =
_time.QueryPerformanceFrequency()
                except OSError:
                    # QueryPerformanceFrequency() fails if the installed
                    # hardware does not support a high-resolution performance
                    # counter
                    perf_counter.use_performance_counter = False
                else:
                    return _time.QueryPerformanceCounter() /
perf_counter.performance_frequency
            else:
                return _time.QueryPerformanceCounter() /
perf_counter.performance_frequency
        if perf_counter.use_monotonic:
            # The monotonic clock is preferred over the system time
            try:
                return time.monotonic()
            except OSError:
                perf_counter.use_monotonic = False
        return time.time()
    perf_counter.use_performance_counter = (os.name == 'nt')
    if perf_counter.use_performance_counter:
        perf_counter.performance_frequency = None
    perf_counter.use_monotonic = hasattr(time, 'monotonic')


time.process_time()
^^^^^^^^^^^^^^^^^^^

Sum of the system and user CPU time of the current process. It does
not include time elapsed during sleep. It is process-wide by
definition.  The reference point of the returned value is undefined,
so that only the difference between the results of consecutive calls
is valid.

It is available on all platforms.

Pseudo-code [#pseudo]_::

    if os.name == 'nt':
        def process_time():
            handle = win32process.GetCurrentProcess()
            process_times = win32process.GetProcessTimes(handle)
            return (process_times['UserTime'] +
process_times['KernelTime']) * 1e-7
    else:
        import os
        try:
            import resource
        except ImportError:
            has_resource = False
        else:
            has_resource = True

        def process_time():
            if process_time.use_process_cputime:
                try:
                    return time.clock_gettime(time.CLOCK_PROCESS_CPUTIME_ID)
                except OSError:
                    process_time.use_process_cputime = False
            if process_time.use_getrusage:
                try:
                    usage = resource.getrusage(resource.RUSAGE_SELF)
                    return usage[0] + usage[1]
                except OSError:
                    process_time.use_getrusage = False
            if process_time.use_times:
                try:
                    times = os.times()
                    return times[0] + times[1]
                except OSError:
                    process_time.use_getrusage = False
            return _time.clock()
        process_time.use_process_cputime = (
            hasattr(time, 'clock_gettime')
            and hasattr(time, 'CLOCK_PROCESS_CPUTIME_ID'))
        process_time.use_getrusage = has_resource
        # On OS/2, only the 5th field of os.times() is set, others are zeros
        process_time.use_times = (hasattr(os, 'times') and os.name != 'os2')


Alternatives: API design
========================

Other names for time.monotonic()
--------------------------------

* time.counter()
* time.metronomic()
* time.seconds()
* time.steady(): "steady" is ambiguous: it means different things to
  different people. For example, on Linux, CLOCK_MONOTONIC is
  adjusted. If we uses the real time as the reference clock, we may
  say that CLOCK_MONOTONIC is steady.  But CLOCK_MONOTONIC gets
  suspended on system suspend, whereas real time includes any time
  spent in suspend.
* time.timeout_clock()
* time.wallclock(): time.monotonic() is not the system time aka the
  "wall clock", but a monotonic clock with an unspecified starting
  point.

The name "time.try_monotonic()" was also proposed for an older
proposition of time.monotonic() which was falling back to the system
time when no monotonic clock was available.

Other names for time.perf_counter()
-----------------------------------

* time.hires()
* time.highres()
* time.timer()

Only expose operating system clocks
-----------------------------------

To not have to define high-level clocks, which is a difficult task, a
simpler approach is to only expose operating system clocks.
time.clock_gettime() and related clock identifiers were already added
to Python 3.3 for example.


time.monotonic(): Fallback to system time
-----------------------------------------

If no monotonic clock is available, time.monotonic() falls back to the
system time.

Issues:

* It is hard to define correctly such function in the documentation:
  is it monotonic? Is it steady? Is it adjusted?
* Some user want to decide what to do when no monotonic clock is
  available: use another clock, display an error, or do something
  else?

Different APIs were proposed to define such function.

One function with a flag: time.monotonic(fallback=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

* time.monotonic(fallback=True) falls back to the system time if no
  monotonic clock is available or if the monotonic clock failed.
* time.monotonic(fallback=False) raises OSError if monotonic clock
  fails and NotImplementedError if the system does not provide a
  monotonic clock

A keyword argument that gets passed as a constant in the caller is
usually poor API.

Raising NotImplementedError for a function is something uncommon in
Python and should be avoided.


One time.monotonic() function, no flag
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

time.monotonic() returns (time: float, is_monotonic: bool).

An alternative is to use a function attribute:
time.monotonic.is_monotonic.  The attribute value would be None before
the first call to time.monotonic().


Choosing the clock from a list of constraints
---------------------------------------------

The PEP as proposed offers a few new clocks, but their guarentees
are deliberately loose in order to offer useful clocks on different
platforms. This inherently embeds policy in the calls, and the
caller must thus choose a policy.

The "choose a clock" approach suggests an additional API to let
callers implement their own policy if necessary
by making most platform clocks available and letting the caller pick
amongst them.
The PEP's suggested clocks are still expected to be available for the common
simple use cases.

To do this two facilities are needed:
an enumeration of clocks, and metadata on the clocks to enable the user to
evaluate their suitability.

The primary interface is a function make simple choices easy:
the caller can use ``time.get_clock(*flags)`` with some combination of flags.
This include at least:

* time.MONOTONIC: clock cannot go backward
* time.STEADY: clock rate is steady
* time.ADJUSTED: clock may be adjusted, for example by NTP
* time.HIGHRES: clock with the highest precision

It returns a clock object with a .now() method returning the current time.
The clock object is annotated with metadata describing the clock feature set;
its .flags field will contain at least all the requested flags.

time.get_clock() returns None if no matching clock is found and so calls can
be chained using the or operator.  Example of a simple policy decision::

    T = get_clock(MONOTONIC) or get_clock(STEADY) or get_clock()
    t = T.now()

The available clocks always at least include a wrapper for ``time.time()``,
so a final call with no flags can always be used to obtain a working clock.

Example of flags of system clocks:

* QueryPerformanceCounter: MONOTONIC | HIGHRES
* GetTickCount: MONOTONIC | STEADY
* CLOCK_MONOTONIC: MONOTONIC | STEADY (or only MONOTONIC on Linux)
* CLOCK_MONOTONIC_RAW: MONOTONIC | STEADY
* gettimeofday(): (no flag)

The clock objects contain other metadata including the clock flags
with additional feature flags above those listed above, the name
of the underlying OS facility, and clock precisions.

``time.get_clock()`` still chooses a single clock; an enumeration
facility is also required.
The most obvious method is to offer ``time.get_clocks()`` with the
same signature as ``time.get_clock()``, but returning a sequence
of all clocks matching the requested flags.
Requesting no flags would thus enumerate all available clocks,
allowing the caller to make an arbitrary choice amongst them based
on their metadata.

Example partial implementation:
`clockutils.py <http://hg.python.org/peps/file/tip/pep-0418/clockutils.py>`_.

Working around operating system bugs?
-------------------------------------

Should Python ensure manually that a monotonic clock is truly
monotonic by computing the maximum with the clock value and the
previous value?

Since it's relatively straightforward to cache the last value returned
using a static variable, it might be interesting to use this to make
sure that the values returned are indeed monotonic.

* Virtual machines provide less reliable clocks.
* QueryPerformanceCounter() has known bugs (only one is not fixed yet)

Python may only work around a specific known operating system bug:
`KB274323`_ contains a code example to workaround the bug (use
GetTickCount() to detect QueryPerformanceCounter() leap).

Issues of a hacked monotonic function:

* if the clock is accidentally set forward by an hour and then back
  again, you wouldn't have a useful clock for an hour
* the cache is not shared between processes so different processes
  wouldn't see the same clock value
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: [RFC] PEP 418: Add monotonic time, performance counter and process time functions

Victor Stinner-3
> Here is a simplified version of the first draft of the PEP 418. The
> full version can be read online.
> http://www.python.org/dev/peps/pep-0418/

FYI there is no time.thread_time() function. It would only be
available on Windows and Linux. It does not use seconds but CPU
cycles. No module or program of the Python source code need such
function, whereas all other functions added by the PEP already have
users in the Python source code, see the Rationale section. For Linux,
CLOCK_THREAD_CPUTIME_ID is already available in Python 3.3. For
Windows, you can get GetThreadTimes() using ctypes or win32process.

> time.process_time()
> ^^^^^^^^^^^^^^^^^^^
>
> Pseudo-code [#pseudo]_::
>
>    if os.name == 'nt':
>        def process_time():
>            handle = win32process.GetCurrentProcess()
>            process_times = win32process.GetProcessTimes(handle)
>            return (process_times['UserTime'] +
> process_times['KernelTime']) * 1e-7
>    else:
>        import os
>        ...
>
>        def process_time():
>            ...
>            return _time.clock()

Is the C clock() function available on all platforms? timemodule.c
checks for HAVE_CLOCK, but test_time checks that time.clock() is
defined and does not fail since the changeset 4de05cbf5ad1, Dec 06
1996. If clock() is not available on all platforms,
time.process_time() documentation should be fixed.

Victor
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: [RFC] PEP 418: Add monotonic time, performance counter and process time functions

M.-A. Lemburg
In reply to this post by Victor Stinner-3
Victor Stinner wrote:

> Hi,
>
> Here is a simplified version of the first draft of the PEP 418. The
> full version can be read online.
> http://www.python.org/dev/peps/pep-0418/
>
> The implementation of the PEP can be found in this issue:
> http://bugs.python.org/issue14428
>
> I post a simplified version for readability and to focus on changes
> introduced by the PEP. Removed sections: Existing Functions,
> Deprecated Function, Glossary, Hardware clocks, Operating system time
> functions, System Standby, Links.

Looks good.

I'd suggest to also include a tool or API to determine the
real resolution of a time function (as opposed to the advertised
one). See pybench's clockres.py helper as example. You often
find large differences between the advertised resolution and
the available one, e.g. while process timers often advertise
very good resolution, they are in fact often only updated
at very coarse rates.

E.g. compare the results of clockres.py on Linux:

Clock resolution of various timer implementations:
time.clock:            10000.000us
time.time:                 0.954us
systimes.processtime:    999.000us

and FreeBSD:

Clock resolution of various timer implementations:
time.clock:             7812.500us
time.time:                 1.907us
systimes.processtime:      1.000us

and Mac OS X:

Clock resolution of various timer implementations:
time.clock:                1.000us
time.time:                 0.954us
systimes.processtime:      1.000us

Regarding changing pybench:
pybench has to stay backwards incompatible with
earlier releases to make it possible to compare timings.
You can add support for new timers to pybench, but please leave
the existing timers and defaults in place.

> ---
>
> PEP: 418
> Title: Add monotonic time, performance counter and process time functions
> Version: f2bb3f74298a
> Last-Modified: 2012-04-15 17:06:07 +0200 (Sun, 15 Apr 2012)
> Author: Cameron Simpson <[hidden email]>, Jim Jewett
> <[hidden email]>, Victor Stinner <[hidden email]>
> Status: Draft
> Type: Standards Track
> Content-Type: text/x-rst
> Created: 26-March-2012
> Python-Version: 3.3
>
> Abstract
> ========
>
> This PEP proposes to add ``time.get_clock_info(name)``,
> ``time.monotonic()``, ``time.perf_counter()`` and
> ``time.process_time()`` functions to Python 3.3.
>
> Rationale
> =========
>
> If a program uses the system time to schedule events or to implement
> a timeout, it will not run events at the right moment or stop the
> timeout too early or too late when the system time is set manually or
> adjusted automatically by NTP.  A monotonic clock should be used
> instead to not be affected by system time updates:
> ``time.monotonic()``.
>
> To measure the performance of a function, ``time.clock()`` can be used
> but it is very different on Windows and on Unix.  On Windows,
> ``time.clock()`` includes time elapsed during sleep, whereas it does
> not on Unix.  ``time.clock()`` precision is very good on Windows, but
> very bad on Unix.  The new ``time.perf_counter()`` function should be
> used instead to always get the most precise performance counter with a
> portable behaviour (ex: include time spend during sleep).
>
> To measure CPU time, Python does not provide directly a portable
> function.  ``time.clock()`` can be used on Unix, but it has a bad
> precision.  ``resource.getrusage()`` can also be used on Unix, but it
> requires to get fields of a structure and compute the sum of time
> spent in kernel space and user space.  The new ``time.process_time()``
> function acts as a portable counter that always measures CPU time
> (doesn't include time elapsed during sleep) and has the best available
> precision.
>
> Each operating system implements clocks and performance counters
> differently, and it is useful to know exactly which function is used
> and some properties of the clock like its resolution and its
> precision.  The new ``time.get_clock_info()`` function gives access to
> all available information of each Python time function.
>
> New functions:
>
> * ``time.monotonic()``: timeout and scheduling, not affected by system
>   clock updates
> * ``time.perf_counter()``: benchmarking, most precise clock for short
>   period
> * ``time.process_time()``: profiling, CPU time of the process
>
> Users of new functions:
>
> * time.monotonic(): concurrent.futures, multiprocessing, queue, subprocess,
>   telnet and threading modules to implement timeout
> * time.perf_counter(): trace and timeit modules, pybench program
> * time.process_time(): profile module
> * time.get_clock_info(): pybench program to display information about the
>   timer like the precision or the resolution
>
> The ``time.clock()`` function is deprecated because it is not
> portable: it behaves differently depending on the operating system.
> ``time.perf_counter()`` or ``time.process_time()`` should be used
> instead, depending on your requirements. ``time.clock()`` is marked as
> deprecated but is not planned for removal.
>
>
> Python functions
> ================
>
> New Functions
> -------------
>
> time.get_clock_info(name)
> ^^^^^^^^^^^^^^^^^^^^^^^^^
>
> Get information on the specified clock.  Supported clock names:
>
> * ``"clock"``: ``time.clock()``
> * ``"monotonic"``: ``time.monotonic()``
> * ``"perf_counter"``: ``time.perf_counter()``
> * ``"process_time"``: ``time.process_time()``
> * ``"time"``: ``time.time()``
>
> Return a dictionary with the following keys:
>
> * Mandatory keys:
>
>   * ``"implementation"`` (str): name of the underlying operating system
>     function.  Examples: ``"QueryPerformanceCounter()"``,
>     ``"clock_gettime(CLOCK_REALTIME)"``.
>   * ``"resolution"`` (float): resolution in seconds of the clock.
>   * ``"is_monotonic"`` (bool): True if the clock cannot go backward.
>
> * Optional keys:
>
>   * ``"precision"`` (float): precision in seconds of the clock
>     reported by the operating system.
>   * ``"is_adjusted"`` (bool): True if the clock is adjusted (e.g. by a
>     NTP daemon).
>
>
> time.monotonic()
> ^^^^^^^^^^^^^^^^
>
> Monotonic clock, i.e. cannot go backward.  It is not affected by system
> clock updates.  The reference point of the returned value is
> undefined, so that only the difference between the results of
> consecutive calls is valid and is a number of seconds.
>
> On Windows versions older than Vista, ``time.monotonic()`` detects
> ``GetTickCount()`` integer overflow (32 bits, roll-over after 49.7
> days): it increases a delta by 2\ :sup:`32` each time than an overflow
> is detected.  The delta is stored in the process-local state and so
> the value of ``time.monotonic()`` may be different in two Python
> processes running for more than 49 days. On more recent versions of
> Windows and on other operating systems, ``time.monotonic()`` is
> system-wide.
>
> Availability: Windows, Mac OS X, Unix, Solaris. Not available on
> GNU/Hurd.
>
> Pseudo-code [#pseudo]_::
>
>     if os.name == 'nt':
>         # GetTickCount64() requires Windows Vista, Server 2008 or later
>         if hasattr(time, '_GetTickCount64'):
>             def monotonic():
>                 return _time.GetTickCount64() * 1e-3
>         else:
>             def monotonic():
>                 ticks = _time.GetTickCount()
>                 if ticks < monotonic.last:
>                     # Integer overflow detected
>                     monotonic.delta += 2**32
>                 monotonic.last = ticks
>                 return (ticks + monotonic.delta) * 1e-3
>             monotonic.last = 0
>             monotonic.delta = 0
>
>     elif os.name == 'mac':
>         def monotonic():
>             if monotonic.factor is None:
>                 factor = _time.mach_timebase_info()
>                 monotonic.factor = timebase[0] / timebase[1]
>             return _time.mach_absolute_time() * monotonic.factor
>         monotonic.factor = None
>
>     elif hasattr(time, "clock_gettime") and hasattr(time, "CLOCK_HIGHRES"):
>         def monotonic():
>             return time.clock_gettime(time.CLOCK_HIGHRES)
>
>     elif hasattr(time, "clock_gettime") and hasattr(time, "CLOCK_MONOTONIC"):
>         def monotonic():
>             return time.clock_gettime(time.CLOCK_MONOTONIC)
>
>
> On Windows, ``QueryPerformanceCounter()`` is not used even though it
> has a better precision than ``GetTickCount()``.  It is not reliable
> and has too many issues.
>
>
> time.perf_counter()
> ^^^^^^^^^^^^^^^^^^^
>
> Performance counter with the highest available precision to measure a
> duration.  It does include time elapsed during sleep and is
> system-wide.  The reference point of the returned value is undefined,
> so that only the difference between the results of consecutive calls
> is valid and is a number of seconds.
>
> Pseudo-code::
>
>     def perf_counter():
>         if perf_counter.use_performance_counter:
>             if perf_counter.performance_frequency is None:
>                 try:
>                     perf_counter.performance_frequency =
> _time.QueryPerformanceFrequency()
>                 except OSError:
>                     # QueryPerformanceFrequency() fails if the installed
>                     # hardware does not support a high-resolution performance
>                     # counter
>                     perf_counter.use_performance_counter = False
>                 else:
>                     return _time.QueryPerformanceCounter() /
> perf_counter.performance_frequency
>             else:
>                 return _time.QueryPerformanceCounter() /
> perf_counter.performance_frequency
>         if perf_counter.use_monotonic:
>             # The monotonic clock is preferred over the system time
>             try:
>                 return time.monotonic()
>             except OSError:
>                 perf_counter.use_monotonic = False
>         return time.time()
>     perf_counter.use_performance_counter = (os.name == 'nt')
>     if perf_counter.use_performance_counter:
>         perf_counter.performance_frequency = None
>     perf_counter.use_monotonic = hasattr(time, 'monotonic')
>
>
> time.process_time()
> ^^^^^^^^^^^^^^^^^^^
>
> Sum of the system and user CPU time of the current process. It does
> not include time elapsed during sleep. It is process-wide by
> definition.  The reference point of the returned value is undefined,
> so that only the difference between the results of consecutive calls
> is valid.
>
> It is available on all platforms.
>
> Pseudo-code [#pseudo]_::
>
>     if os.name == 'nt':
>         def process_time():
>             handle = win32process.GetCurrentProcess()
>             process_times = win32process.GetProcessTimes(handle)
>             return (process_times['UserTime'] +
> process_times['KernelTime']) * 1e-7
>     else:
>         import os
>         try:
>             import resource
>         except ImportError:
>             has_resource = False
>         else:
>             has_resource = True
>
>         def process_time():
>             if process_time.use_process_cputime:
>                 try:
>                     return time.clock_gettime(time.CLOCK_PROCESS_CPUTIME_ID)
>                 except OSError:
>                     process_time.use_process_cputime = False
>             if process_time.use_getrusage:
>                 try:
>                     usage = resource.getrusage(resource.RUSAGE_SELF)
>                     return usage[0] + usage[1]
>                 except OSError:
>                     process_time.use_getrusage = False
>             if process_time.use_times:
>                 try:
>                     times = os.times()
>                     return times[0] + times[1]
>                 except OSError:
>                     process_time.use_getrusage = False
>             return _time.clock()
>         process_time.use_process_cputime = (
>             hasattr(time, 'clock_gettime')
>             and hasattr(time, 'CLOCK_PROCESS_CPUTIME_ID'))
>         process_time.use_getrusage = has_resource
>         # On OS/2, only the 5th field of os.times() is set, others are zeros
>         process_time.use_times = (hasattr(os, 'times') and os.name != 'os2')
>
>
> Alternatives: API design
> ========================
>
> Other names for time.monotonic()
> --------------------------------
>
> * time.counter()
> * time.metronomic()
> * time.seconds()
> * time.steady(): "steady" is ambiguous: it means different things to
>   different people. For example, on Linux, CLOCK_MONOTONIC is
>   adjusted. If we uses the real time as the reference clock, we may
>   say that CLOCK_MONOTONIC is steady.  But CLOCK_MONOTONIC gets
>   suspended on system suspend, whereas real time includes any time
>   spent in suspend.
> * time.timeout_clock()
> * time.wallclock(): time.monotonic() is not the system time aka the
>   "wall clock", but a monotonic clock with an unspecified starting
>   point.
>
> The name "time.try_monotonic()" was also proposed for an older
> proposition of time.monotonic() which was falling back to the system
> time when no monotonic clock was available.
>
> Other names for time.perf_counter()
> -----------------------------------
>
> * time.hires()
> * time.highres()
> * time.timer()
>
> Only expose operating system clocks
> -----------------------------------
>
> To not have to define high-level clocks, which is a difficult task, a
> simpler approach is to only expose operating system clocks.
> time.clock_gettime() and related clock identifiers were already added
> to Python 3.3 for example.
>
>
> time.monotonic(): Fallback to system time
> -----------------------------------------
>
> If no monotonic clock is available, time.monotonic() falls back to the
> system time.
>
> Issues:
>
> * It is hard to define correctly such function in the documentation:
>   is it monotonic? Is it steady? Is it adjusted?
> * Some user want to decide what to do when no monotonic clock is
>   available: use another clock, display an error, or do something
>   else?
>
> Different APIs were proposed to define such function.
>
> One function with a flag: time.monotonic(fallback=True)
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
> * time.monotonic(fallback=True) falls back to the system time if no
>   monotonic clock is available or if the monotonic clock failed.
> * time.monotonic(fallback=False) raises OSError if monotonic clock
>   fails and NotImplementedError if the system does not provide a
>   monotonic clock
>
> A keyword argument that gets passed as a constant in the caller is
> usually poor API.
>
> Raising NotImplementedError for a function is something uncommon in
> Python and should be avoided.
>
>
> One time.monotonic() function, no flag
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
> time.monotonic() returns (time: float, is_monotonic: bool).
>
> An alternative is to use a function attribute:
> time.monotonic.is_monotonic.  The attribute value would be None before
> the first call to time.monotonic().
>
>
> Choosing the clock from a list of constraints
> ---------------------------------------------
>
> The PEP as proposed offers a few new clocks, but their guarentees
> are deliberately loose in order to offer useful clocks on different
> platforms. This inherently embeds policy in the calls, and the
> caller must thus choose a policy.
>
> The "choose a clock" approach suggests an additional API to let
> callers implement their own policy if necessary
> by making most platform clocks available and letting the caller pick
> amongst them.
> The PEP's suggested clocks are still expected to be available for the common
> simple use cases.
>
> To do this two facilities are needed:
> an enumeration of clocks, and metadata on the clocks to enable the user to
> evaluate their suitability.
>
> The primary interface is a function make simple choices easy:
> the caller can use ``time.get_clock(*flags)`` with some combination of flags.
> This include at least:
>
> * time.MONOTONIC: clock cannot go backward
> * time.STEADY: clock rate is steady
> * time.ADJUSTED: clock may be adjusted, for example by NTP
> * time.HIGHRES: clock with the highest precision
>
> It returns a clock object with a .now() method returning the current time.
> The clock object is annotated with metadata describing the clock feature set;
> its .flags field will contain at least all the requested flags.
>
> time.get_clock() returns None if no matching clock is found and so calls can
> be chained using the or operator.  Example of a simple policy decision::
>
>     T = get_clock(MONOTONIC) or get_clock(STEADY) or get_clock()
>     t = T.now()
>
> The available clocks always at least include a wrapper for ``time.time()``,
> so a final call with no flags can always be used to obtain a working clock.
>
> Example of flags of system clocks:
>
> * QueryPerformanceCounter: MONOTONIC | HIGHRES
> * GetTickCount: MONOTONIC | STEADY
> * CLOCK_MONOTONIC: MONOTONIC | STEADY (or only MONOTONIC on Linux)
> * CLOCK_MONOTONIC_RAW: MONOTONIC | STEADY
> * gettimeofday(): (no flag)
>
> The clock objects contain other metadata including the clock flags
> with additional feature flags above those listed above, the name
> of the underlying OS facility, and clock precisions.
>
> ``time.get_clock()`` still chooses a single clock; an enumeration
> facility is also required.
> The most obvious method is to offer ``time.get_clocks()`` with the
> same signature as ``time.get_clock()``, but returning a sequence
> of all clocks matching the requested flags.
> Requesting no flags would thus enumerate all available clocks,
> allowing the caller to make an arbitrary choice amongst them based
> on their metadata.
>
> Example partial implementation:
> `clockutils.py <http://hg.python.org/peps/file/tip/pep-0418/clockutils.py>`_.
>
> Working around operating system bugs?
> -------------------------------------
>
> Should Python ensure manually that a monotonic clock is truly
> monotonic by computing the maximum with the clock value and the
> previous value?
>
> Since it's relatively straightforward to cache the last value returned
> using a static variable, it might be interesting to use this to make
> sure that the values returned are indeed monotonic.
>
> * Virtual machines provide less reliable clocks.
> * QueryPerformanceCounter() has known bugs (only one is not fixed yet)
>
> Python may only work around a specific known operating system bug:
> `KB274323`_ contains a code example to workaround the bug (use
> GetTickCount() to detect QueryPerformanceCounter() leap).
>
> Issues of a hacked monotonic function:
>
> * if the clock is accidentally set forward by an hour and then back
>   again, you wouldn't have a useful clock for an hour
> * the cache is not shared between processes so different processes
>   wouldn't see the same clock value
> _______________________________________________
> Python-Dev mailing list
> [hidden email]
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/mal%40egenix.com

--
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Apr 15 2012)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2012-04-28: PythonCamp 2012, Cologne, Germany              13 days to go

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: [RFC] PEP 418: Add monotonic time, performance counter and process time functions

Victor Stinner-3
2012/4/15 M.-A. Lemburg <[hidden email]>:
> I'd suggest to also include a tool or API to determine the
> real resolution of a time function (as opposed to the advertised
> one). See pybench's clockres.py helper as example.

The PEP includes such tool, but I forgot to mention it in the PEP:
http://hg.python.org/peps/file/tip/pep-0418/clock_precision.py

It is based on clockres.py from pybench. I used this tools to fill the
"Precision in Python" column of the different tables. The "Precision"
is the precision announced by the OS, whereas the "Precision in
Python" is the effictive precision in Python.

The full PEP includes results of different benchmarks: performance of
hardware clocks and performance of the different OS time functions.

> E.g. compare the results of clockres.py on Linux:
>
> Clock resolution of various timer implementations:
> time.clock:            10000.000us
> time.time:                 0.954us
> systimes.processtime:    999.000us
>
> and FreeBSD:
>
> Clock resolution of various timer implementations:
> time.clock:             7812.500us
> time.time:                 1.907us
> systimes.processtime:      1.000us

Cool, I found similar numbers :-)

> and Mac OS X:
>
> Clock resolution of various timer implementations:
> time.clock:                1.000us
> time.time:                 0.954us
> systimes.processtime:      1.000us

I will include these numbers on Mac OS X to the PEP.

> Regarding changing pybench:
> pybench has to stay backwards incompatible with
> earlier releases to make it possible to compare timings.
> You can add support for new timers to pybench, but please leave
> the existing timers and defaults in place.

I suppose that you are talking about this change:

-# Choose platform default timer
-if sys.platform[:3] == 'win':
-    # On WinXP this has 2.5ms resolution
-    TIMER_PLATFORM_DEFAULT = TIMER_TIME_CLOCK
-else:
-    # On Linux this has 1ms resolution
-    TIMER_PLATFORM_DEFAULT = TIMER_TIME_TIME
+TIMER_PLATFORM_DEFAULT = TIMER_TIME_PERF_COUNTER

from http://bugs.python.org/file25202/perf_counter_process_time.patch

It does not change the OS clock on Windows, only on Unix:
CLOCK_REALTIME (gettimeofday() for Python 3.2 and earlier) is replaced
with CLOCK_MONOTONIC. This change should only give a different result
if the system time is changed during the benchmark.

I'm ok to keep the default timer if you consider the change as incompatible.

Victor
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: [RFC] PEP 418: Add monotonic time, performance counter and process time functions

Victor Stinner-3
In reply to this post by Victor Stinner-3
> time.perf_counter()
> ^^^^^^^^^^^^^^^^^^^
>
> Performance counter with the highest available precision to measure a
> duration.  It does include time elapsed during sleep and is
> system-wide.  The reference point of the returned value is undefined,
> so that only the difference between the results of consecutive calls
> is valid and is a number of seconds.

It's maybe time for bikeshedding! Glyph wrote me in private:
"IMHO, perf_counter should be performance_counter() or
high_precision(); the abbreviation is silly :)"

The time module has other abbreviated names. I don't have a preference
between time.perf_counter() or time.performance_counter().

Solaris provides CLOCK_HIGHRES, "the nonadjustable, high-resolution
clock." If we map CLOCK_xxx names to functions name, we have:

 * CLOCK_MONOTONIC: time.monotonic()
 * CLOCK_HIGHRES: time.highres()

(whereas Windows provides QueryPerformanceCounter -> performance_counter)

I suppose that most people don't care that "resolution" and
"precision" are different things.

Victor
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: [RFC] PEP 418: Add monotonic time, performance counter and process time functions

Matt Joiner

This is becoming the Manhattan Project of bike sheds.


_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: [RFC] PEP 418: Add monotonic time, performance counter and process time functions

Victor Stinner-3
2012/4/16 Matt Joiner <[hidden email]>:
> This is becoming the Manhattan Project of bike sheds.

FreeBSD FAQ contains an entry "Why should I care what color the
bikeshed is?" which mention  a "sleep(1) should take fractional second
arguments" saga in 1999.
http://www.freebsd.org/doc/en/books/faq/misc.html#BIKESHED-PAINTING

Bikeshedding is maybe a common issue with the discussion around time
function? :-)

Victor
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: [RFC] PEP 418: Add monotonic time, performance counter and process time functions

Oleg Broytman
On Mon, Apr 16, 2012 at 12:38:41PM +0200, Victor Stinner <[hidden email]> wrote:
> Bikeshedding is maybe a common issue with the discussion around time
> function? :-)

   Perhaps because everyone of us lives in a different Time-Space
Continuum? ;-)

Oleg.
--
     Oleg Broytman            http://phdru.name/            [hidden email]
           Programmers don't die, they just GOSUB without RETURN.
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: [RFC] PEP 418: Add monotonic time, performance counter and process time functions

Antoine Pitrou
In reply to this post by Victor Stinner-3
On Mon, 16 Apr 2012 01:25:42 +0200
Victor Stinner <[hidden email]> wrote:
>
> I suppose that most people don't care that "resolution" and
> "precision" are different things.

Don't they? Actually, they don't care about resolution since they
receive a Python float.

Regards

Antoine.


_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: [RFC] PEP 418: Add monotonic time, performance counter and process time functions

Victor Stinner-3
In reply to this post by Victor Stinner-3
> Here is a simplified version of the first draft of the PEP 418. The
> full version can be read online.
> http://www.python.org/dev/peps/pep-0418/

I wrote an implementation in pure Python using ctypes for Python < 3.3:

https://bitbucket.org/haypo/misc/src/tip/python/pep418.py

I tested it on Linux, OpenBSD, FreeBSD and Windows. It's more a
proof-of-concept to test the PEP than a library written to be reused
by programs.

Victor
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: PEP 418: Add monotonic time, performance counter and process time functions

Cameron Simpson
In reply to this post by Victor Stinner-3
On 16Apr2012 01:25, Victor Stinner <[hidden email]> wrote:
| I suppose that most people don't care that "resolution" and
| "precision" are different things.

If we're using the same definitions we discussed offline, where

  - resolution is the units the clock call (underneath) works in (for
    example, nanoseconds)

  - precision is the effective precision of the results, for example
    milliseconds

I'd say people would care if they knew, and mostly care about
"precision".
--
Cameron Simpson <[hidden email]> DoD#743
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: [RFC] PEP 418: Add monotonic time, performance counter and process time functions

k3xji
In reply to this post by Victor Stinner-3


On Sun, Apr 15, 2012 at 6:18 PM, Victor Stinner <[hidden email]> wrote:
> Here is a simplified version of the first draft of the PEP 418. The
> full version can be read online.
> http://www.python.org/dev/peps/pep-0418/

FYI there is no time.thread_time() function. It would only be
available on Windows and Linux. It does not use seconds but CPU
cycles. No module or program of the Python source code need such
function,

Just FYI: in MACOSx, you can use  thread_info() to get that information. Also you can get that information in Solaris,too. In yappi profiler I use all of these approaches together to have an OS independent thread_times() functionality. Here is the relevant code:http://bitbucket.org/sumerc/yappi/src/7c7dc11e8728/timing.c

I also think that you are right about Python really not have any use case for this functionality, ...


--
Sümer Cip

_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: PEP 418: Add monotonic time, performance counter and process time functions

R. David Murray
In reply to this post by Cameron Simpson
On Tue, 17 Apr 2012 14:48:22 +1000, Cameron Simpson <[hidden email]> wrote:

> On 16Apr2012 01:25, Victor Stinner <[hidden email]> wrote:
> | I suppose that most people don't care that "resolution" and
> | "precision" are different things.
>
> If we're using the same definitions we discussed offline, where
>
>   - resolution is the units the clock call (underneath) works in (for
>     example, nanoseconds)
>
>   - precision is the effective precision of the results, for example
>     milliseconds
>
> I'd say people would care if they knew, and mostly care about
> "precision".

I think what the user cares about is "what is the smallest tick that
this clock result will faithfully represent?".  If the number of bits
returned is larger than the clock accuracy, you want the clock accuracy.
If the number of bits returned is smaller than the clock accuracy,
you want the number of bits.

(Yes, I'm using accuracy in a slightly different sense here...I think
we don't have the right words for this.)

To use other words, what the users cares about are the error bars on
the returned result.

--David
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: PEP 418: Add monotonic time, performance counter and process time functions

Chris Angelico
In reply to this post by Cameron Simpson
On Tue, Apr 17, 2012 at 2:48 PM, Cameron Simpson <[hidden email]> wrote:

> On 16Apr2012 01:25, Victor Stinner <[hidden email]> wrote:
> | I suppose that most people don't care that "resolution" and
> | "precision" are different things.
>
> If we're using the same definitions we discussed offline, where
>
>  - resolution is the units the clock call (underneath) works in (for
>    example, nanoseconds)
>
>  - precision is the effective precision of the results, for example
>    milliseconds
>
> I'd say people would care if they knew, and mostly care about
> "precision".

Meaning that resolution is a matter of format and API, not of clock.
If you take a C clock API that returns a value in nanoseconds and
return it as a Python float, you've changed the resolution. I don't
think resolution matters much, beyond that (for example) nanosecond
resolution allows a clock to be subsequently upgraded as far as
nanosecond precision without breaking existing code, even if currently
it's only providing microsecond precision. But it makes just as much
sense for your resolution to be 2**64ths-of-a-second or
quarters-of-the-internal-CPU-clock-speed as it does for nanoseconds.
As long as it's some fraction of the SI second, every different
resolution is just a fixed ratio away from every other one.

ChrisA
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: PEP 418: Add monotonic time, performance counter and process time functions

Victor Stinner-3
In reply to this post by R. David Murray
> I think what the user cares about is "what is the smallest tick that
> this clock result will faithfully represent?".  If the number of bits
> returned is larger than the clock accuracy, you want the clock accuracy.
> If the number of bits returned is smaller than the clock accuracy,
> you want the number of bits.
>
> (Yes, I'm using accuracy in a slightly different sense here...I think
> we don't have the right words for this.)
>
> To use other words, what the users cares about are the error bars on
> the returned result.

Ok ok, resolution / accuracy / precision are confusing (or at least
not well known concepts). So it's better to keep the name:
time.perf_counter() :-)

Victor
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: PEP 418: Add monotonic time, performance counter and process time functions

Cameron Simpson
In reply to this post by Chris Angelico
On 18Apr2012 00:18, Chris Angelico <[hidden email]> wrote:
| On Tue, Apr 17, 2012 at 2:48 PM, Cameron Simpson <[hidden email]> wrote:
| > On 16Apr2012 01:25, Victor Stinner <[hidden email]> wrote:
| > | I suppose that most people don't care that "resolution" and
| > | "precision" are different things.
| >
| > If we're using the same definitions we discussed offline, where
| >
| >  - resolution is the units the clock call (underneath) works in (for
| >    example, nanoseconds)
| >
| >  - precision is the effective precision of the results, for example
| >    milliseconds
| >
| > I'd say people would care if they knew, and mostly care about
| > "precision".
|
| Meaning that resolution is a matter of format and API, not of clock.
| If you take a C clock API that returns a value in nanoseconds and
| return it as a Python float, you've changed the resolution. I don't
| think resolution matters much, beyond that (for example) nanosecond
| resolution allows a clock to be subsequently upgraded as far as
| nanosecond precision without breaking existing code, even if currently
| it's only providing microsecond precision.

Yes; as stated, resolution is largely irrelevant to the user; it really
only places an upper bound on the precision. But it _is_ easy to know
from the unlying API doco, so it is easy to annotate the clocks with its
metadata.

Annoyingly, the more useful precision value is often harder to know.
--
Cameron Simpson <[hidden email]> DoD#743
http://www.cskk.ezoshosting.com/cs/

If anyone disagrees with anything I say, I am quite prepared not only
to retract it, but also to deny under oath that I ever said it.
        - Tom Lehrer
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: PEP 418: Add monotonic time, performance counter and process time functions

Cameron Simpson
In reply to this post by R. David Murray
On 17Apr2012 08:35, R. David Murray <[hidden email]> wrote:
| On Tue, 17 Apr 2012 14:48:22 +1000, Cameron Simpson <[hidden email]> wrote:
| > On 16Apr2012 01:25, Victor Stinner <[hidden email]> wrote:
| > | I suppose that most people don't care that "resolution" and
| > | "precision" are different things.
| >
| > If we're using the same definitions we discussed offline, where
| >
| >   - resolution is the units the clock call (underneath) works in (for
| >     example, nanoseconds)
| >
| >   - precision is the effective precision of the results, for example
| >     milliseconds
| >
| > I'd say people would care if they knew, and mostly care about
| > "precision".
|
| I think what the user cares about is "what is the smallest tick that
| this clock result will faithfully represent?".

That is what "precision" is supposed to mean above. I suspect we're all in
agreement here about its purpose.

| To use other words, what the users cares about are the error bars on
| the returned result.

Yes. And your discussion about the hw clock exceeding the API resulution
means we mean "the error bars as they escape from the OS API".

I still think we're all in agreement about the meaning here.
--
Cameron Simpson <[hidden email]> DoD#743
http://www.cskk.ezoshosting.com/cs/

Often the good of the many is worth more than the good of the few. Saying
"if they have saved one life then they are worthwhile" places the good of the
few above the good of the many and past a certain threshold that's a
reprehensible attitude, for which I have utter contempt.
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: [RFC] PEP 418: Add monotonic time, performance counter and process time functions

Victor Stinner-3
In reply to this post by Victor Stinner-3
> Here is a simplified version of the first draft of the PEP 418. The
> full version can be read online.
> http://www.python.org/dev/peps/pep-0418/
>
> The implementation of the PEP can be found in this issue:
> http://bugs.python.org/issue14428

The PEP is now fully ready: I just finished the implementation.

It looks like people, who complained on older versions of the PEP,
don't have new complain. Am I wrong? Everybody agree with the PEP 418?

I created http://hg.python.org/features/pep418/ repository for the
implementation. I tested it on Linux 3.3, FreeBSD 8.2, OpenBSD 5.0 and
Windows Seven. The implementation is now waiting your review!

There is also the toy implementation in pure Python for Python < 3.3:
https://bitbucket.org/haypo/misc/src/tip/python/pep418.py

Antoine asked "Is there a designated dictator for this PEP?". Nobody
answered. Maybe Guido van Rossum?

Victor
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: [RFC] PEP 418: Add monotonic time, performance counter and process time functions

Guido van Rossum
I'll do it. Give me a few days (tomorrow is fully booked with horrible
meetings).

On Tue, Apr 17, 2012 at 5:06 PM, Victor Stinner
<[hidden email]> wrote:

>> Here is a simplified version of the first draft of the PEP 418. The
>> full version can be read online.
>> http://www.python.org/dev/peps/pep-0418/
>>
>> The implementation of the PEP can be found in this issue:
>> http://bugs.python.org/issue14428
>
> The PEP is now fully ready: I just finished the implementation.
>
> It looks like people, who complained on older versions of the PEP,
> don't have new complain. Am I wrong? Everybody agree with the PEP 418?
>
> I created http://hg.python.org/features/pep418/ repository for the
> implementation. I tested it on Linux 3.3, FreeBSD 8.2, OpenBSD 5.0 and
> Windows Seven. The implementation is now waiting your review!
>
> There is also the toy implementation in pure Python for Python < 3.3:
> https://bitbucket.org/haypo/misc/src/tip/python/pep418.py
>
> Antoine asked "Is there a designated dictator for this PEP?". Nobody
> answered. Maybe Guido van Rossum?
>
> Victor
> _______________________________________________
> Python-Dev mailing list
> [hidden email]
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org



--
--Guido van Rossum (python.org/~guido)
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: PEP 418: Add monotonic time, performance counter and process time functions

Stephen J. Turnbull
In reply to this post by Victor Stinner-3
On Wed, Apr 18, 2012 at 8:25 AM, Victor Stinner
<[hidden email]> wrote:

> Ok ok, resolution / accuracy / precision are confusing (or at least
> not well known concepts).

Maybe not to us, but in fields like astronomy and mechanical
engineering there are commonly accepted definitions:

Resolution: the smallest difference between two physical values that
results in a different measurement by a given instrument.

Precision: the amount of deviation among measurements of the same
physical value by a single instrument.

Accuracy: the amount of deviation of measurements by a given
instrument from true values.

As usual there are issues of average vs. worst case, different
resolution/precision/accuracy over the instrument's range, etc. which
need to be considered in reporting values for these properties.

A typical application to clocks would be the duration of one tick.  If
the clock ticks once per second and time values are reported in
nanoseconds, the /resolution/ is *1 billion* nanoseconds, not *1*
nanosecond.    /Precision/ corresponds to the standard deviation of
tick durations.  It is not necessarily the case that a precise
instrument will be accurate; if every tick is *exactly* 59/60 seconds,
the clock is infinitely precise but horribly inaccurate for most
purposes (it loses an hour every three days, and you'll miss your
favorite TV show!)  And two /accurate/ clocks should report the same
times and the same durations when measuring the same things.

I don't really care if Python decides to use idiosyncratic
definitions, but the above are easy enough to find (eg
http://en.wikipedia.org/wiki/Accuracy_and_precision).
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
123