Quantcast

Alternative to celery

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Alternative to celery

guettli-3
Some years ago I used celery for a project.

Today I face roughly the same use case again.

The last time I used it, celery felt to heavy weighted, too complicated.

I found this alternative: http://python-rq.org/

There are two things which speak agains python-rq:

 - redis is a cache. I need a persistent queue. If a power-failure happens, no job must get lost.

 - AFAIK python-rq uses python pickle to serialize data. I want a language neutral data exchange format.

At the moment my favourite protocol is gRPC (protocol-buffers). I never used it up to now, but this would be a nice use case.

Maybe I am missing the right terms. What does celery implement? According to wikipedia "asynchronous task queue". But
my favorite search engine could not reveal a gRPC based server implemented in Python ....

What do you think?


--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/d43d2cef-f1a1-418d-9d62-ecce85961fbb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Alternative to celery

Andréas Kühne
I don't know anything about the python-rq stuff - however redis can be configured to save to disk (see https://redis.io/topics/persistence) - so you wouldn't loose anything on a power-failure (if configured correctly). Redis is so much more than a cache nowadays.

Regards,

Andréas

2017-04-20 11:16 GMT+02:00 guettli <[hidden email]>:
Some years ago I used celery for a project.

Today I face roughly the same use case again.

The last time I used it, celery felt to heavy weighted, too complicated.

I found this alternative: http://python-rq.org/

There are two things which speak agains python-rq:

 - redis is a cache. I need a persistent queue. If a power-failure happens, no job must get lost.

 - AFAIK python-rq uses python pickle to serialize data. I want a language neutral data exchange format.

At the moment my favourite protocol is gRPC (protocol-buffers). I never used it up to now, but this would be a nice use case.

Maybe I am missing the right terms. What does celery implement? According to wikipedia "asynchronous task queue". But
my favorite search engine could not reveal a gRPC based server implemented in Python ....

What do you think?


--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/d43d2cef-f1a1-418d-9d62-ecce85961fbb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAK4qSCefVFpgZEw4Ar4AGDUSdOH-A-7-kvgysqeHY9zET1DHZQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Alternative to celery

Avraham Serour
try googling 'python task queue'


On Thu, Apr 20, 2017 at 12:25 PM, Andréas Kühne <[hidden email]> wrote:
I don't know anything about the python-rq stuff - however redis can be configured to save to disk (see https://redis.io/topics/persistence) - so you wouldn't loose anything on a power-failure (if configured correctly). Redis is so much more than a cache nowadays.

Regards,

Andréas

2017-04-20 11:16 GMT+02:00 guettli <[hidden email]>:
Some years ago I used celery for a project.

Today I face roughly the same use case again.

The last time I used it, celery felt to heavy weighted, too complicated.

I found this alternative: http://python-rq.org/

There are two things which speak agains python-rq:

 - redis is a cache. I need a persistent queue. If a power-failure happens, no job must get lost.

 - AFAIK python-rq uses python pickle to serialize data. I want a language neutral data exchange format.

At the moment my favourite protocol is gRPC (protocol-buffers). I never used it up to now, but this would be a nice use case.

Maybe I am missing the right terms. What does celery implement? According to wikipedia "asynchronous task queue". But
my favorite search engine could not reveal a gRPC based server implemented in Python ....

What do you think?


--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/d43d2cef-f1a1-418d-9d62-ecce85961fbb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAK4qSCefVFpgZEw4Ar4AGDUSdOH-A-7-kvgysqeHY9zET1DHZQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAFWa6t%2B7WTzt7o1SoOV45YrL%2B%3DaD_X30E1yNFj3-ki54ivNL2A%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Loading...