Quantcast

Is cherrypy's thread pool adding a delay during the first requests?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Is cherrypy's thread pool adding a delay during the first requests?

Gosku
Hi,

I have a cherrypy project that uses thread pool of 10 threads. The server.conf is like this:

[global]
server.socket_port = 8049
server.socket_host = "192.168.0.164"
server.thread_pool = 10

tools.gzip.on = True
tools.auth.on = True
tools.Functions.on = True

request.show_tracebacks = False
request.error_response = 'handle_error'

log.error_file = 'error.log'
log.access_file = 'access.log'
tools.secureheaders.on = True

and webapp.py uses like this:

conf_path = os.path.join(os.getcwd(), 'server.conf')
cherrypy.config.update(conf_path)

app = cherrypy.tree.mount(WEBSITE(), "/", config=conf_path)

The thing is when I start cherrypy (or when I restart it with uwsgi because I deployed something) the first requests are taking an increasing amount of time, later going back to "normal" times.
Today I realized that the number of increasing-time requests is related to the number of threads. For example, here you have a screenshot of this behavior when cherrypy is set up with a thread pool of 10:

In the example, asking for the same resource is taking more and more time (the resource is not important). After asking for it 10 times, the server settles down and everything works as it should.

The same thing happens if I configure a thread pool of 13:


After the 13th request, everything goes back to normal.
This behavior is very annoying, specially because in production I have 40 threads (where the 40th request is taking around 20 seconds) and every time I deploy changes, the website slows down for a couple of minutes.

How can I fix it whithout setting the thread pool to 1?

Thank you on advance.

Regards,
Gosku

--
You received this message because you are subscribed to the Google Groups "cherrypy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/cherrypy-users.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Is cherrypy's thread pool adding a delay during the first requests?

Joseph S. Tate
This is not something you can fix without changing the way that CherryPy loads its applications. You simply have to plan for it. Perhaps there is a way to preload the application, or immediately exercise each thread so that they load what they need to, but this is not part of CherryPy itself (an engine plugin that does this would be a great contribution). It should be noted that most web frameworks suffer this problem, even PHP. Your problem sounds like something you've made worse though by increasing the number of threads.

I'm not sure what kind of hardware you're running on, but 40 threads on a single CPU (Python is CPU Locked by the GIL, so only one CPU can be used per python application process in most cases) is probably not going to perform well under load.

1. make sure you have auto-reloading turned off; do all the other performance tuning you can to reduce startup time (reduce the # of objects, classes, etc. Turn off lazy loading for ORMs and other libraries that use it.)
1a. Check to make sure you're not suffering a deadlock or other race condition with so many threads.
1b. Make sure that your database connection pool settings matches your httpd thread pool settingsĀ + some margin so that you're not suffering resource contention when creating database connections.
2. Run a script to hit the web server on restart to eat the performance penalties so your users don't see it.
3. Convert your 40 thread single process application to 4-10 thread application processes by running 4 cherrypy servers on separate ports and reverse_proxying them with nginx or Apache HTTPD.
3a. make sure you're offloading all static file handling to an httpd server written in C. The static file handler in CherryPy is great for development, and really well optimized, but it's still much slower than a web server written in C.
3b. Use gzipping on this proxying server to keep load off the python code and out of the threads.
4. Stagger or sequentially execute restarts so that this loading does not all happen at the same time on a single page load.
5. Make use of browser caching and perhaps server side caching too to run fewer threads so that you can minimize these restarting penalties.

On Tue, Sep 20, 2016 at 1:52 PM Gosku <[hidden email]> wrote:
Hi,

I have a cherrypy project that uses thread pool of 10 threads. The server.conf is like this:

[global]
server.socket_port = 8049
server.socket_host = "192.168.0.164"
server.thread_pool = 10

tools.gzip.on = True
tools.auth.on = True
tools.Functions.on = True

request.show_tracebacks = False
request.error_response = 'handle_error'

log.error_file = 'error.log'
log.access_file = 'access.log'
tools.secureheaders.on = True

and webapp.py uses like this:

conf_path = os.path.join(os.getcwd(), 'server.conf')
cherrypy.config.update(conf_path)

app = cherrypy.tree.mount(WEBSITE(), "/", config=conf_path)

The thing is when I start cherrypy (or when I restart it with uwsgi because I deployed something) the first requests are taking an increasing amount of time, later going back to "normal" times.
Today I realized that the number of increasing-time requests is related to the number of threads. For example, here you have a screenshot of this behavior when cherrypy is set up with a thread pool of 10:

In the example, asking for the same resource is taking more and more time (the resource is not important). After asking for it 10 times, the server settles down and everything works as it should.

The same thing happens if I configure a thread pool of 13:


After the 13th request, everything goes back to normal.
This behavior is very annoying, specially because in production I have 40 threads (where the 40th request is taking around 20 seconds) and every time I deploy changes, the website slows down for a couple of minutes.

How can I fix it whithout setting the thread pool to 1?

Thank you on advance.

Regards,
Gosku

--
You received this message because you are subscribed to the Google Groups "cherrypy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/cherrypy-users.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "cherrypy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/cherrypy-users.
For more options, visit https://groups.google.com/d/optout.
Loading...