Accessing deeply nested multiple doc roots for WSGI apps

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Accessing deeply nested multiple doc roots for WSGI apps

Ben Bangert
As I'm working on some tools to facilitate the easy use and  
distribution of WSGI apps and middleware, a problem is starting to  
crop up. Most of these WSGI apps come with their own little set of  
static files (images, javascript, etc.) that need to be delivered  
should they be executed. Of course, Apache and such only allow for a  
single doc-root so I can't exactly through in each WSGI app's path to  
the static files.

These static paths are also very deep as the WSGI apps are installed  
as Python egg's. I've been using Ian's StaticURLParser from Python  
Paste, but its speed concerns me plus the fact that it means my  
webapp is essentially doing little more than relay filesystem data.

Several thoughts occur to me to deal with this:

1) Have a faster version of StaticURLParser, perhaps written in C
2) Create some sort of specification for a single static docroot  
where each WSGI egg gets its own symlink into

#1 still leaves me with the WSGI app sending static data, which isn't  
ideal but so far it works and I can put the WSGI app under any URL  
prefix without a problem. It also requires me to dynamically generate  
the URL to all static information.

#2 is probably easier in some respects, since if the scheme is a  
given (ie, /media/PACKAGE/VERSION/FILE.GIF) then I don't need to  
generate all the URL's and the webserver can be pointed to the static  
files root.

Has anyone else thought of ways to deal with this? If I missed some  
prior thread about this exact topic that solves it, sorry.

- Ben
_______________________________________________
Web-SIG mailing list
[hidden email]
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/lists%40nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Accessing deeply nested multiple doc roots for WSGI apps

Jason Stitt
On Oct 14, 2005, at 7:30 PM, Ben Bangert wrote:

> 1) Have a faster version of StaticURLParser, perhaps written in C
> 2) Create some sort of specification for a single static docroot
> where each WSGI egg gets its own symlink into
>
> #1 still leaves me with the WSGI app sending static data, which isn't
> ideal but so far it works and I can put the WSGI app under any URL
> prefix without a problem. It also requires me to dynamically generate
> the URL to all static information.

I agree with you that sending static files over WSGI is not ideal, in  
general. Depending on the server implementation you are using it is  
theoretically possible to take advantage of sendfile() for performance:

http://www.python.org/peps/pep-0333.html#optional-platform-specific- 
file-handling

But if you are using FastCGI or SCGI (e.g. with Flup) instead of  
running within the HTTP server itself (e.g. with mod_python), I'm not  
sure if that will work. (Anybody more familiar with Flup's  
capabilities, or whether any currently available server implements  
that part of the PEP?)

Can you set up Squid or a similar proxy and cache static URLs? That  
would probably get rid of any performance problems.

> #2 is probably easier in some respects, since if the scheme is a
> given (ie, /media/PACKAGE/VERSION/FILE.GIF) then I don't need to
> generate all the URL's and the webserver can be pointed to the static
> files root.

Personally, I wouldn't base the URL/filename on the package name and  
version. What if you wanted to run multiple instances of the same  
application, each with its own files? Some of them could be  
dynamically generated, or maybe the header graphic just needs to be  
different for each instance. And if any of these files might be  
linked to externally, you don't want a changing version # in there.

The publishing engine I'm planning (yeah, I know, everyone has one  
these days, don't they?) needs to be able to run as many different  
sites as I want on one server. Anything to do with filesystem paths  
and base URLs/hostnames will be configured per site, so I don't have  
a standard naming scheme in the code itself. This does mean  
"generating" URLs, but you're going to have to do that for links  
between pages anyway, right?

Jason

P.S. New on this list. Is there an introduction protocol, and is it  
more or less complicated than HTTP? ;-)

_______________________________________________
Web-SIG mailing list
[hidden email]
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/lists%40nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Accessing deeply nested multiple doc roots for WSGI apps

ianb
In reply to this post by Ben Bangert
Ben Bangert wrote:

> As I'm working on some tools to facilitate the easy use and  
> distribution of WSGI apps and middleware, a problem is starting to  
> crop up. Most of these WSGI apps come with their own little set of  
> static files (images, javascript, etc.) that need to be delivered  
> should they be executed. Of course, Apache and such only allow for a  
> single doc-root so I can't exactly through in each WSGI app's path to  
> the static files.
>
> These static paths are also very deep as the WSGI apps are installed  
> as Python egg's. I've been using Ian's StaticURLParser from Python  
> Paste, but its speed concerns me plus the fact that it means my  
> webapp is essentially doing little more than relay filesystem data.
>
> Several thoughts occur to me to deal with this:
>
> 1) Have a faster version of StaticURLParser, perhaps written in C
> 2) Create some sort of specification for a single static docroot  
> where each WSGI egg gets its own symlink into

I've thought about this too.  I think a little of both is best -- have
some configurable location where files can be dumped (using
Package/Version/) and a URL that goes with that.  Then I figure the
application, on first startup, will either copy or symlink files into
that location -- though sometimes the user the application runs as
couldn't do that, so maybe there should be a way to make it happen
separately from running the application.  In fact, maybe it could simply
be some seperate piece of metadata, so the application doesn't have to
do it at all.

Then the app should generate all the URLs based on that configured
static location, or if no location has been configured it should fall
back on StaticURLParser.  The fallback is useful, especially in a
situation like a standalone HTTP server where there isn't any static
serving (or at least none that is particularly better than StaticURLParser).

--
Ian Bicking  |  [hidden email]  |  http://blog.ianbicking.org
_______________________________________________
Web-SIG mailing list
[hidden email]
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/lists%40nabble.com