Testing the tests by modifying the ordering of dict items.

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Testing the tests by modifying the ordering of dict items.

Mark Shannon-3
Hi,

Python code should not depend upon the ordering of items in a dict.
Unfortunately it seems that a number of tests in the standard library do
just that.

Changing PyDict_MINSIZE from 8 to either 4 or 16 causes the following
tests to fail:

test_dis test_email test_inspect test_nntplib test_packaging
test_plistlib test_pprint test_symtable test_trace

test_sys also fails, but this is a legitimate failure in sys.getsizeof()

Changing the collision resolution function from f(n) = 5n + 1 to
f(n) = n + 1 results in the same failures, except for test_packaging and
test_symtable which pass.

Finally, changing the seed in unicode_hash() from (implicit) 0 to an
arbitrary value (12345678) causes the above tests to fail plus:

test_json test_set test_ttk_textonly test_urllib test_urlparse

I think this is a real issue as the unicode_hash() function is likely to
change soon due to http://bugs.python.org/issue13703.

Should I:

1. Submit one big bug report?

2. Submit a bug report for each "failing" test separately?

3. Ignore it, since the tests only fail when I start messing about?

Cheers,
Mark.
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Testing the tests by modifying the ordering of dict items.

Antoine Pitrou
On Thu, 05 Jan 2012 13:46:52 +0000
Mark Shannon <[hidden email]> wrote:
>
> Should I:
>
> 1. Submit one big bug report?
>
> 2. Submit a bug report for each "failing" test separately?

I would say a separate bug report for each failing test file, i.e. one
report for test_dis, one for test_email etc.
Hope this doesn't eat too much of your time :)

Regards

Antoine.


_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Testing the tests by modifying the ordering of dict items.

Barry Warsaw
In reply to this post by Mark Shannon-3
On Jan 05, 2012, at 01:46 PM, Mark Shannon wrote:

>2. Submit a bug report for each "failing" test separately?

I'm sure it will be a pain, but this is really the best thing to do.

-Barry
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Testing the tests by modifying the ordering of dict items.

csebasha
This post has NOT been accepted by the mailing list yet.
In reply to this post by Mark Shannon-3
Hello Mark,

Did you raise bug for this?