Thread-management middleware components?

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Thread-management middleware components?

Jim Fulton

Has anyone written any thread-management middleware components for WSGI?
Many web applications need to run application code in separate threads.
Often, the number of threads needs to be limited, either by throttling
the rate of thread creation, or by dispatching requests to a thread pool.
This is a capability that could be provided by a server, however, it seems
that it might be functionality better provided at an intermediate layer to
make it more pluggable.

Jim

--
Jim Fulton           mailto:[hidden email]       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org
_______________________________________________
Web-SIG mailing list
[hidden email]
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/lists%40nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Thread-management middleware components?

ianb
Jim Fulton wrote:
> Has anyone written any thread-management middleware components for WSGI?
> Many web applications need to run application code in separate threads.
> Often, the number of threads needs to be limited, either by throttling
> the rate of thread creation, or by dispatching requests to a thread pool.
> This is a capability that could be provided by a server, however, it seems
> that it might be functionality better provided at an intermediate layer to
> make it more pluggable.

Right now all threading and generally concurrency is handled by the
server.  Since it *has* to be handled by the server, I'm not sure what
the advantage would be to duplicating that functionality?  Well,
strictly speaking you could have a server with wsgi.threaded and
wsgi.multiprocess both being false, and the server presumably being
asynchronous, but I think that's challenging to fit into the WSGI spec
-- there was some discussion some time ago that dwindled off, and I
don't think there was ever any resolution on handling asynchronous
servers/apps in WSGI.

I don't see a need for a lot of interchangeable thread pools, a handful
at most should do.

--
Ian Bicking  /  [hidden email]  /  http://blog.ianbicking.org
_______________________________________________
Web-SIG mailing list
[hidden email]
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/lists%40nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Thread-management middleware components?

Jim Fulton
Ian Bicking wrote:

> Jim Fulton wrote:
>
>> Has anyone written any thread-management middleware components for WSGI?
>> Many web applications need to run application code in separate threads.
>> Often, the number of threads needs to be limited, either by throttling
>> the rate of thread creation, or by dispatching requests to a thread pool.
>> This is a capability that could be provided by a server, however, it
>> seems
>> that it might be functionality better provided at an intermediate
>> layer to
>> make it more pluggable.
>
>
> Right now all threading and generally concurrency is handled by the
> server.  Since it *has* to be handled by the server,

Why does it have to be handled by the server?

 > I'm not sure what
> the advantage would be to duplicating that functionality?

The advantage is that it gives people deploying an application
more control.  We've recently switched to using WSGI for
HTTP in Zope.  Our default "out of the box" server of choice
is Twisted, however, the current thread-management strategy used by
Twisted's WSGI server doesn't meet out needs.  I could try to get
Twisted to change it's stragegy, and I probably will, but it would
be more flexible to be able to plug something in.


 > Well,
> strictly speaking you could have a server with wsgi.threaded and
> wsgi.multiprocess both being false, and the server presumably being
> asynchronous, but I think that's challenging to fit into the WSGI spec
> -- there was some discussion some time ago that dwindled off, and I
> don't think there was ever any resolution on handling asynchronous
> servers/apps in WSGI.

We have long experience with combining an asynchronous network server
with a threaded application server. Asynchronous network servers can
handle I/O with lots of network clients very efficiently, but only
if an application doesn't block.  Real applications often take
significant time to compute results.  A thread-management facility that
bridges asychronous servers with threaded application can work very well.

It's possible that my need is specific to using asynchronous servers,
but I consider working well with asynchronous servers to be a pretty
important requirement.

> I don't see a need for a lot of interchangeable thread pools, a handful
> at most should do.

I'm not sure what you mean by this.

On the one hand, I'd like to be free to choose my own thread-management
stragegy.  On the other hand, if there are multiple asynchronous servers,
I don't see why they should each have to maintain their own thread-management
subsystems if one can be shared among the different servers.

Jim

--
Jim Fulton           mailto:[hidden email]       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org
_______________________________________________
Web-SIG mailing list
[hidden email]
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/lists%40nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Thread-management middleware components?

ianb
Jim Fulton wrote:
>> Right now all threading and generally concurrency is handled by the
>> server.  Since it *has* to be handled by the server,
>
>
> Why does it have to be handled by the server?

Because most WSGI apps are blocking, so unless you want the server to be
non-concurrent it has to handle this.  Of course you design a
non-concurrent WSGI server that *had* to be used with some threading
middleware.  WSGI doesn't seem like a good fit for that, though.

>  > I'm not sure what
>
>> the advantage would be to duplicating that functionality?
>
>
> The advantage is that it gives people deploying an application
> more control.  We've recently switched to using WSGI for
> HTTP in Zope.  Our default "out of the box" server of choice
> is Twisted, however, the current thread-management strategy used by
> Twisted's WSGI server doesn't meet out needs.  I could try to get
> Twisted to change it's stragegy, and I probably will, but it would
> be more flexible to be able to plug something in.

I think in this particular case -- barring direct changes to Twisted --
it would make more sense to build on Twisted's non-WSGI asyncronous
application support, and build a threadpool that calls WSGI from there.

>  > Well,
>> strictly speaking you could have a server with wsgi.threaded and
>> wsgi.multiprocess both being false, and the server presumably being
>> asynchronous, but I think that's challenging to fit into the WSGI spec
>> -- there was some discussion some time ago that dwindled off, and I
>> don't think there was ever any resolution on handling asynchronous
>> servers/apps in WSGI.
>
>
> We have long experience with combining an asynchronous network server
> with a threaded application server. Asynchronous network servers can
> handle I/O with lots of network clients very efficiently, but only
> if an application doesn't block.  Real applications often take
> significant time to compute results.  A thread-management facility that
> bridges asychronous servers with threaded application can work very well.
>
> It's possible that my need is specific to using asynchronous servers,
> but I consider working well with asynchronous servers to be a pretty
> important requirement.

I think the server has to be synchronous by the time it calls a WSGI
app.  There's nothing saying that the WSGI support in Twisted is the
WSGI support you have to use.

My impression is that it is hard to standardize anything async-related
because they use slightly different conventions on how to do async
(e.g., Deferred vs. ad hoc callbacks).  So... whatever standardization
there is to be done there is probably below WSGI.

>> I don't see a need for a lot of interchangeable thread pools, a
>> handful at most should do.
>
>
> I'm not sure what you mean by this.
>
> On the one hand, I'd like to be free to choose my own thread-management
> stragegy.  On the other hand, if there are multiple asynchronous servers,
> I don't see why they should each have to maintain their own
> thread-management
> subsystems if one can be shared among the different servers.

Sure, but if there's only, say, 4 viable strategies and 3 serious async
servers (are there even that many of either?) then it's easier just to
figure out how to plug each strategy in on a case-by-case basis, and
discuss the concrete issues with the server developers.

--
Ian Bicking  /  [hidden email]  /  http://blog.ianbicking.org
_______________________________________________
Web-SIG mailing list
[hidden email]
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/lists%40nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Thread-management middleware components?

Jim Fulton
Ian Bicking wrote:

> Jim Fulton wrote:
>
>>> Right now all threading and generally concurrency is handled by the
>>> server.  Since it *has* to be handled by the server,
>>
>>
>>
>> Why does it have to be handled by the server?
>
>
> Because most WSGI apps are blocking, so unless you want the server to be
> non-concurrent it has to handle this.  Of course you design a
> non-concurrent WSGI server that *had* to be used with some threading
> middleware.

Actually, I suggest a WSGI server that *can* be used with
threading middleware.

 > WSGI doesn't seem like a good fit for that, though.

...

> I think in this particular case -- barring direct changes to Twisted --
> it would make more sense to build on Twisted's non-WSGI asyncronous
> application support, and build a threadpool that calls WSGI from there.

I don't want to maintain a non-WSGI interface and I don't want to
maintain my own WSGI Twisted interface.

...

> I think the server has to be synchronous by the time it calls a WSGI
> app.  There's nothing saying that the WSGI support in Twisted is the
> WSGI support you have to use.

No, but I have good reasons for wanting to use it.

> My impression is that it is hard to standardize anything async-related
> because they use slightly different conventions on how to do async
> (e.g., Deferred vs. ad hoc callbacks).  So... whatever standardization
> there is to be done there is probably below WSGI.

...

> Sure, but if there's only, say, 4 viable strategies and 3 serious async
> servers (are there even that many of either?) then it's easier just to
> figure out how to plug each strategy in on a case-by-case basis, and
> discuss the concrete issues with the server developers.

That's what I'll do if I have to.

Jim

--
Jim Fulton           mailto:[hidden email]       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org
_______________________________________________
Web-SIG mailing list
[hidden email]
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/lists%40nabble.com