Re: [issue2609] strptime in Jython2.7.1 not working correctly in multithreaded environment

[Jim Baker-2]

+jython-dev because of the seriousness of this issue

Jeff,

I increasingly believe this issue is almost certainly http://bugs.jython.org/issue2487 as Ivan pointed out earlier in this thread. Note that the synchronization in PyType.fromClass, which is responsible for constructing this map, was causing deadlocks was removed in 2.7.1b3 and replaced with a busy wait in 2.7.1rc1.

I want to ask this question: should we revisit our original decision to remove this synchronization? (In https://hg.python.org/jython/rev/cffe5b824a21 and at least another followup commit set). It is possible that we got this mixed up with the low memory problem, and how it had a nasty action at a distance problem, see http://bugs.jython.org/issue2536 

At the very least this is a low cost thing to try out, although it may not be conclusive.

I should point out that I spent too much time trying to revisit how this publishes, and I never found a workable solution that didn't do the busy waiting that Darjus proposed as an alternative. Clearly we don't care about this code path being synchronized, only that it doesn't deadlock; it is not going to be hot except in some pathological usage scenario (constant reloading of Java classes through distinct class loaders, then used in some sort of callback scenario is the only thing that I can think of; otherwise this cost is only a startup cost).

- Jim

On Sun, Jul 30, 2017 at 3:35 AM, Jeff Allen <[hidden email]> wrote:

Jeff Allen added the comment:

Hmmm ... I'm not sure that actual concurrency (large numbers of parsers working at the same time) has much to do with this. I run my test as:

> while ../inst/bin/jython parse3a.py ; do echo "ok"; done

which lets me fiddle with the source as it runs. It can be quite a wait. The times I've caught this, the numbers were small. Here's one:

REPEATS=2, THREADS=10,  LENGTH=2.
barfoo
foofoo
foobar
barfoo
foofoo
barfoo
barfoo
barfoo
barfoo
barfoo
barfoo
barfoo
foofoo
foobar
foobar
barfoofoofoo

barfoo
barfoo
 *** Caught one! ***
Traceback (most recent call last):
  File "parse3a.py", line 25, in parse
    if m.groups(): short_print(m.group(1))
AttributeError: 'org.python.modules.sre.MatchObject' object has no attribute 'groups'
> /home/jeff/Jython/iss2609/work/parse3a.py(25)parse()
-> if m.groups(): short_print(m.group(1))
(Pdb) p m
<org.python.modules.sre.MatchObject object at 0x2>
(Pdb) m.groups()
('foobar', '52536')
(Pdb) p tests
['barfoo84894', 'barfoo38345E', 'foofoo8017', 'barfoo42953', 'foobar52536E', 'foobar18661', 'barfoo35695E', 'foofoo89916', 'barfoo94921', 'barfoo32379']
(Pdb) p i
0

Now, as you can see, the object m behaved in the thread as if it had no groups() method, but later in the debugger it is present, and it yields the right result. This happens in the first block of threads.

My theory is that use of this object by the thread is running ahead of initialising the object or (quite likely) its class dictionary. I'll test that, see if I can get an incriminating snapshot.

----------
assignee:  -> jeff.allen
components: +Core -Library
Added file: http://bugs.jython.org/file1594/parse3a.py

_______________________________________
Jython tracker <[hidden email]>
<http://bugs.jython.org/issue2609>
_______________________________________

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Jython-dev mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jython-dev