Jython 2.7.1 became unusable because of multithreading issues with regexps

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Jython 2.7.1 became unusable because of multithreading issues with regexps

jython-dev mailing list
Hello all!

We are using jython heavily from Java application in multiple threads.
This didn't cause any problems in 2.7.1b3 and earlier versions.

After upgrading to 2.7.1 we faced the following:  a simple call to
datetime.strptime(...) or json.loads(...) can lead to
"'org.python.modules.sre.MatchObject' object has no attribute..." error.

As I suspect, this is related to the fact that RegexObject is not
thread-safe anymore. It never was in Java, and never promised to be in
Python, though in earlier versions of Jython this was possible:

##############################################
import thread
import re
import time
REGEX = re.compile('((?:foo|bar)+)(\d*)E?')

def parse(line):
   m = REGEX.search(line)
   if m.groups():
     print m.group(1)

thread.start_new_thread(parse,('foofoofoofoofoofoofoofoofoo234',))
thread.start_new_thread(parse, ('foobarbarbarbarbarbarbarfoo4E',))
....
thread.start_new_thread(parse, ('foobarbarbarbarbarbarbarfoo4E',))
time.sleep(1)
###############################################

In Jython 2.7.1 this (sometimes! you might need to run this script a
couple of times) cause "'MatchObject' object has no attribute" errors.

Instead of calling shared RegexObject one can simply do this:

def printtime(time):
   print datetime.datetime.strptime(time, '%Y-%m-%dT%H:%M:%S')
thread.start_new_thread(printtime, ('2017-07-01T11:11:11',))
...
thread.start_new_thread(printtime, ('2017-07-01T11:11:11',))


The same with json.loads.

OK, we never share RegexObjects between threads in our code. But we
cannot refuse to use json.loads and datetime.strptime!

This became a major issue for us (http://bugs.jython.org/issue2609).
What would you suggest? How can we help to solve this?

Regards,

Ivan Ponomarev

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Jython-dev mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jython-dev
Reply | Threaded
Open this post in threaded view
|

Re: Jython 2.7.1 became unusable because of multithreading issues with regexps

Jeff Allen-2
Working with Ivan's demonstration, and a more stressful version (see
tickethttp://bugs.jython.org/issue2609), I was unable to reproduce the
problem on my main dev box (Windows 10).

I'm thinking hardware multiprocessing might be a factor.  Anyone with
lots of CPUs (and a different OS) like to take this on?

Jeff


On 22/07/2017 09:01, Ivan Ponomarev via Jython-dev wrote:

> Hello all!
>
> We are using jython heavily from Java application in multiple threads.
> This didn't cause any problems in 2.7.1b3 and earlier versions.
>
> After upgrading to 2.7.1 we faced the following:  a simple call to
> datetime.strptime(...) or json.loads(...) can lead to
> "'org.python.modules.sre.MatchObject' object has no attribute..." error.
>
> As I suspect, this is related to the fact that RegexObject is not
> thread-safe anymore. It never was in Java, and never promised to be in
> Python, though in earlier versions of Jython this was possible:
>
> ##############################################
> import thread
> import re
> import time
> REGEX = re.compile('((?:foo|bar)+)(\d*)E?')
>
> def parse(line):
>   m = REGEX.search(line)
>   if m.groups():
>     print m.group(1)
>
> thread.start_new_thread(parse,('foofoofoofoofoofoofoofoofoo234',))
> thread.start_new_thread(parse, ('foobarbarbarbarbarbarbarfoo4E',))
> ....
> thread.start_new_thread(parse, ('foobarbarbarbarbarbarbarfoo4E',))
> time.sleep(1)
> ###############################################
>
> In Jython 2.7.1 this (sometimes! you might need to run this script a
> couple of times) cause "'MatchObject' object has no attribute" errors.
>
> Instead of calling shared RegexObject one can simply do this:
>
> def printtime(time):
>   print datetime.datetime.strptime(time, '%Y-%m-%dT%H:%M:%S')
> thread.start_new_thread(printtime, ('2017-07-01T11:11:11',))
> ...
> thread.start_new_thread(printtime, ('2017-07-01T11:11:11',))
>
>
> The same with json.loads.
>
> OK, we never share RegexObjects between threads in our code. But we
> cannot refuse to use json.loads and datetime.strptime!
>
> This became a major issue for us (http://bugs.jython.org/issue2609).
> What would you suggest? How can we help to solve this?
>
> Regards,
>
> Ivan Ponomarev
>
> ------------------------------------------------------------------------------
>
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Jython-dev mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/jython-dev
>


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Jython-dev mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jython-dev