Quantcast

Django 1.11 doesn't work well behind pgBouncer

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Django 1.11 doesn't work well behind pgBouncer

Сергей Фурсов
Hi!

In our application stack we connect from our web/background workers to postgres database through dedicated pgBouncer service. 
pgBouncer use transaction pooling 
Connections work in autocommit mode.

After upgrading to 1.11 version with new server-side cursors for iterator method, many of our DB requests started failing with "Cursor _django_curs_<id> does not exist'. 

According to pgBouncer docs, it doesn't support using WITH HOLD cursors with transaction pooling mode. But it doesn't seems reasonable to run pgBouncer in session pooling mode, we can just switch to direct persistent connections to postgres DB. On the other hand in this case we have to increase minimum number of DB connection to number of our workers (from 5 to ~30) to prevent blocking blocking requests. And this is violate recommendation to have (core_count * 2) + effective_spindle_count max connections to DB.

So is there some recommendations how efficiently work with postgres DB in django 1.11?

Thanks,
Sergey


--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAG-zk-BM%3D2U7WZTOdDuE%2BT0Lqgh-2K4jNTo6xscL%2Bj349%3DYEnQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Django 1.11 doesn't work well behind pgBouncer

Сергей Фурсов

2017-04-06 22:30 GMT+03:00 Sergey Fursov <[hidden email]>:
Hi!

In our application stack we connect from our web/background workers to postgres database through dedicated pgBouncer service. 
pgBouncer use transaction pooling 
Connections work in autocommit mode.

After upgrading to 1.11 version with new server-side cursors for iterator method, many of our DB requests started failing with "Cursor _django_curs_<id> does not exist'. 

According to pgBouncer docs, it doesn't support using WITH HOLD cursors with transaction pooling mode. But it doesn't seems reasonable to run pgBouncer in session pooling mode, we can just switch to direct persistent connections to postgres DB. On the other hand in this case we have to increase minimum number of DB connection to number of our workers (from 5 to ~30) to prevent blocking blocking requests. And this is violate recommendation to have (core_count * 2) + effective_spindle_count max connections to DB.

So is there some recommendations how efficiently work with postgres DB in django 1.11?

Thanks,
Sergey



--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAG-zk-ACbEWXYVDXW-N%2B6eDBm1j-7L8M24D9tK1E0OOzSHMUJw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Django 1.11 doesn't work well behind pgBouncer

serjant
In reply to this post by Сергей Фурсов
Yep, we are experiencing the same issue after the upgrade, downgraded back to the working version - 1.10.7

On Thursday, April 6, 2017 at 10:32:03 PM UTC+3, Sergey Fursov wrote:
Hi!

In our application stack we connect from our web/background workers to postgres database through dedicated pgBouncer service. 
pgBouncer use transaction pooling 
Connections work in autocommit mode.

After upgrading to 1.11 version with new server-side cursors for iterator method, many of our DB requests started failing with "Cursor _django_curs_<id> does not exist'. 

According to pgBouncer docs, it doesn't support using WITH HOLD cursors with transaction pooling mode. But it doesn't seems reasonable to run pgBouncer in session pooling mode, we can just switch to direct persistent connections to postgres DB. On the other hand in this case we have to increase minimum number of DB connection to number of our workers (from 5 to ~30) to prevent blocking blocking requests. And this is violate recommendation to have (core_count * 2) + effective_spindle_count max connections to DB.

So is there some recommendations how efficiently work with postgres DB in django 1.11?

Thanks,
Sergey


--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/f9e143dd-4c27-4232-bf2e-be12bade83a2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Loading...