A view associated with a lot of urls

classic Classic list List threaded Threaded
17 messages Options
Reply | Threaded
Open this post in threaded view
|

A view associated with a lot of urls

Nolhian
Hello,

I've got a view associated with a lot of urls which have 2 variables
( like /computer/keyboard, /animal/dog and so on ). I thought of doing
a regex like [a-z]/a-z]/$  and make 2 lists or dicts at the beginning
of the view and check if the 2 variables are in these dicts or respond
with a 404 error page. Obviously it will be huge dicts let's say 100
keys for the first and maybe 50 for the second one. Is it the best way
to go ?  Won't it be slow / consume a lot of ram ( because of the
dicts )  if a lot of people  are viewing the pages simultaneously ?

Thanks,
Nolhian

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.

Reply | Threaded
Open this post in threaded view
|

Re: A view associated with a lot of urls

Nan-8

Django's regular-expression based URL dispatcher is designed to do
this directly; and you might want to use a model (database table) for
each of your lookups.  The third part of the tutorial [1] addresses
this.

[1] https://docs.djangoproject.com/en/dev/intro/tutorial03/

On Nov 29, 3:30 am, Nolhian <[hidden email]> wrote:

> Hello,
>
> I've got a view associated with a lot of urls which have 2 variables
> ( like /computer/keyboard, /animal/dog and so on ). I thought of doing
> a regex like [a-z]/a-z]/$  and make 2 lists or dicts at the beginning
> of the view and check if the 2 variables are in these dicts or respond
> with a 404 error page. Obviously it will be huge dicts let's say 100
> keys for the first and maybe 50 for the second one. Is it the best way
> to go ?  Won't it be slow / consume a lot of ram ( because of the
> dicts )  if a lot of people  are viewing the pages simultaneously ?
>
> Thanks,
> Nolhian

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.

Reply | Threaded
Open this post in threaded view
|

Re: A view associated with a lot of urls

Nolhian
Thanks you for your answer, yes I plan on using different models for
the lookups. But I'm confused, is that a yes to my question, Lists/
dicts are the way to go here ?

On Nov 29, 7:03 pm, Nan <[hidden email]> wrote:

> Django's regular-expression based URL dispatcher is designed to do
> this directly; and you might want to use a model (database table) for
> each of your lookups.  The third part of the tutorial [1] addresses
> this.
>
> [1]https://docs.djangoproject.com/en/dev/intro/tutorial03/
>
> On Nov 29, 3:30 am, Nolhian <[hidden email]> wrote:
>
>
>
>
>
>
>
> > Hello,
>
> > I've got a view associated with a lot of urls which have 2 variables
> > ( like /computer/keyboard, /animal/dog and so on ). I thought of doing
> > a regex like [a-z]/a-z]/$  and make 2 lists or dicts at the beginning
> > of the view and check if the 2 variables are in these dicts or respond
> > with a 404 error page. Obviously it will be huge dicts let's say 100
> > keys for the first and maybe 50 for the second one. Is it the best way
> > to go ?  Won't it be slow / consume a lot of ram ( because of the
> > dicts )  if a lot of people  are viewing the pages simultaneously ?
>
> > Thanks,
> > Nolhian

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.

Reply | Threaded
Open this post in threaded view
|

Re: A view associated with a lot of urls

Nolhian
In reply to this post by Nan-8
Thanks you for your answer, yes I plan on using different models for
the lookups. But I'm confused, is that a yes to my question, Lists/
dicts are the way to go here ?

On Nov 29, 7:03 pm, Nan <[hidden email]> wrote:

> Django's regular-expression based URL dispatcher is designed to do
> this directly; and you might want to use a model (database table) for
> each of your lookups.  The third part of the tutorial [1] addresses
> this.
>
> [1]https://docs.djangoproject.com/en/dev/intro/tutorial03/
>
> On Nov 29, 3:30 am, Nolhian <[hidden email]> wrote:
>
>
>
>
>
>
>
> > Hello,
>
> > I've got a view associated with a lot of urls which have 2 variables
> > ( like /computer/keyboard, /animal/dog and so on ). I thought of doing
> > a regex like [a-z]/a-z]/$  and make 2 lists or dicts at the beginning
> > of the view and check if the 2 variables are in these dicts or respond
> > with a 404 error page. Obviously it will be huge dicts let's say 100
> > keys for the first and maybe 50 for the second one. Is it the best way
> > to go ?  Won't it be slow / consume a lot of ram ( because of the
> > dicts )  if a lot of people  are viewing the pages simultaneously ?
>
> > Thanks,
> > Nolhian

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.

Reply | Threaded
Open this post in threaded view
|

Re: A view associated with a lot of urls

DrBloodmoney
In reply to this post by Nolhian
On Tue, Nov 29, 2011 at 2:19 PM, Nolhian <[hidden email]> wrote:
> Thanks you for your answer, yes I plan on using different models for
> the lookups. But I'm confused, is that a yes to my question, Lists/
> dicts are the way to go here ?

I would not do it that way. Seems incredibly brittle and you are
mixing up you model with your view (so to speak).

I would do it like so:

url(r'^(?P<category>\w+)/(?P<item>\w+)/?(?P<subitem>\w+)/', view,
name='complicated_view')

and then in the view:

def view(request, category, item, subitem=None):
      MyModel.objects.get(category=category, item=item)

or whatever.

Then you could have urls that look like /computer/keyboard/ or
/animal/dog/Fido, etc

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.

Reply | Threaded
Open this post in threaded view
|

Re: A view associated with a lot of urls

Nolhian
Thanks !

So you'll not check if the category/item exists ( if a user enter a
bad url ), you'll just hit the database to check if it does in fact
exists ?

However with that, there is no way for example to make a difference
between "there is no items in this category" and "this category
doesn't exist" !

Well that could be a solution but only 1 collection for a lot of
different unrelated categories seems really odd to me, why not a model
per category ?

On Nov 29, 8:30 pm, DrBloodmoney <[hidden email]> wrote:

> On Tue, Nov 29, 2011 at 2:19 PM, Nolhian <[hidden email]> wrote:
> > Thanks you for your answer, yes I plan on using different models for
> > the lookups. But I'm confused, is that a yes to my question, Lists/
> > dicts are the way to go here ?
>
> I would not do it that way. Seems incredibly brittle and you are
> mixing up you model with your view (so to speak).
>
> I would do it like so:
>
> url(r'^(?P<category>\w+)/(?P<item>\w+)/?(?P<subitem>\w+)/', view,
> name='complicated_view')
>
> and then in the view:
>
> def view(request, category, item, subitem=None):
>       MyModel.objects.get(category=category, item=item)
>
> or whatever.
>
> Then you could have urls that look like /computer/keyboard/ or
> /animal/dog/Fido, etc

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.

Reply | Threaded
Open this post in threaded view
|

Re: A view associated with a lot of urls

DrBloodmoney
On Tue, Nov 29, 2011 at 3:37 PM, Nolhian <[hidden email]> wrote:
> Thanks !
>
> So you'll not check if the category/item exists ( if a user enter a
> bad url ), you'll just hit the database to check if it does in fact
> exists ?

No I don't know what your business logic for the view/model would be.
I was just showing an example of how you could take keywords from your
request url and map them to your models to display results to the user
without having to hard-code them in your urls or settings or whatever.


> However with that, there is no way for example to make a difference
> between "there is no items in this category" and "this category
> doesn't exist" !
>
> Well that could be a solution but only 1 collection for a lot of
> different unrelated categories seems really odd to me, why not a model
> per category ?

Again it's however you want to model your data. For the sake of
normalization (and typing/database size/etc) I'd probably have one
model for categories:

class Category(models.Model):
...

class Item(models.Model):
    category = models.ForeignKey(Category, related_name='items')
...

and so forth.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.

Reply | Threaded
Open this post in threaded view
|

Re: A view associated with a lot of urls

Nolhian
>No I don't know what your business logic for the view/model would be.
>I was just showing an example of how you could take keywords from your
>request url and map them to your models to display results to the user
>without having to hard-code them in your urls or settings or whatever.

Oh okay :) I'll test with dicts and lists to see if it that works
well.


>Again it's however you want to model your data. For the sake of
>normalization (and typing/database size/etc) I'd probably have one
>model for categories:

Yes me too. Plus I totally forgot to say that I use a NoSQL database,
so I'll put every item directly in the category 'table'.

Thanks for your help !
Nolhian

On Nov 29, 10:22 pm, DrBloodmoney <[hidden email]> wrote:

> On Tue, Nov 29, 2011 at 3:37 PM, Nolhian <[hidden email]> wrote:
> > Thanks !
>
> > So you'll not check if the category/item exists ( if a user enter a
> > bad url ), you'll just hit the database to check if it does in fact
> > exists ?
>
> No I don't know what your business logic for the view/model would be.
> I was just showing an example of how you could take keywords from your
> request url and map them to your models to display results to the user
> without having to hard-code them in your urls or settings or whatever.
>
> > However with that, there is no way for example to make a difference
> > between "there is no items in this category" and "this category
> > doesn't exist" !
>
> > Well that could be a solution but only 1 collection for a lot of
> > different unrelated categories seems really odd to me, why not a model
> > per category ?
>
> Again it's however you want to model your data. For the sake of
> normalization (and typing/database size/etc) I'd probably have one
> model for categories:
>
> class Category(models.Model):
> ...
>
> class Item(models.Model):
>     category = models.ForeignKey(Category, related_name='items')
> ...
>
> and so forth.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.

Reply | Threaded
Open this post in threaded view
|

Re: A view associated with a lot of urls

Reinout van Rees
In reply to this post by Nolhian
On 29-11-11 21:37, Nolhian wrote:
> However with that, there is no way for example to make a difference
> between "there is no items in this category" and "this category
> doesn't exist" !

Probably not a problem. How are people ending up on those URLS? Not by
guessing, I think. So you probably have a start page that lists the
categories and then a category page that lists the items.

A 404 on /non/existing leads people to try /non/ which would also 404.


Reinout

--
Reinout van Rees                    http://reinout.vanrees.org/
[hidden email]             http://www.nelen-schuurmans.nl/
"If you're not sure what to do, make something. -- Paul Graham"

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.

Reply | Threaded
Open this post in threaded view
|

Re: A view associated with a lot of urls

Nolhian
>Probably not a problem. How are people ending up on those URLS? Not by
>guessing.

That's just the point here, to avoid crashing because some "smart"
people tried to play with urls. So with only a get or 404 there is no
way to know if the category/items does indeed exist or if the user
played with the url and gave a non existent category or category/
items. For example in one case we should display "No items in this
category" and in  the other respond a 404 error page. That's why I'm
considering checking at the beginning of the view with a list/dict
filled with category names and one filled with item names to make sure
that the category is in the list/dict and same for the item before
hitting the database, and if not respond with a 404.


On Nov 30, 8:53 am, Reinout van Rees <[hidden email]> wrote:

> On 29-11-11 21:37, Nolhian wrote:
>
> > However with that, there is no way for example to make a difference
> > between "there is no items in this category" and "this category
> > doesn't exist" !
>
> Probably not a problem. How are people ending up on those URLS? Not by
> guessing, I think. So you probably have a start page that lists the
> categories and then a category page that lists the items.
>
> A 404 on /non/existing leads people to try /non/ which would also 404.
>
> Reinout
>
> --
> Reinout van Rees                    http://reinout.vanrees.org/
> [hidden email]            http://www.nelen-schuurmans.nl/
> "If you're not sure what to do, make something. -- Paul Graham"

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.

Reply | Threaded
Open this post in threaded view
|

Re: A view associated with a lot of urls

Ivo Brodien-2
Hi,

That's why I'm
considering checking at the beginning of the view with a list/dict
filled with category names and one filled with item names to make sure
that the category is in the list/dict and same for the item before
hitting the database, and if not respond with a 404.

As Reinout suggested you should have a CategoryModel which you can check if a Category exists and the get the items of show that the category does not exist or 404 if somebody entered a wrong URL/Category. Checking against the DB is probably the same thing you want to do, but it uses the DB and not a List/Dict which you have to maintain manually somehow.

I think you care too much about people entering wrong URLs. People are clicking links on your webpage, right? Or do you give him instructions to write urls in the adress field?

As long as you produce correct urls the code works and only should show 404 if someone messed with the URL.

Consider doing this:

1) use the url tag or get_absolute_url of your models to produce the urls in your template
2) configure your site to send you 404 errors which have a referer (read below and [1])

404 errors

Django can also be configured to e-mail errors about broken links (404 “page not found” errors). Django sends e-mails about 404 errors when:
If those conditions are met, Django will e-mail the users listed in the MANAGERS setting whenever your code raises a 404 and the request has a referer. (It doesn’t bother to e-mail for 404s that don’t have a referer – those are usually just people typing in broken URLs or broken Web ‘bots).


bye


--
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Reply | Threaded
Open this post in threaded view
|

Re: A view associated with a lot of urls

Nolhian
Hello,


>As Reinout suggested you should have a CategoryModel which you can check >if a Category exists and the get the items of show that the category does not >exist or 404 if somebody entered a wrong URL/Category. Checking against the >DB is probably the same thing you want to do, but it uses the DB and not a >List/Dict which you have to maintain manually somehow.

I don't quite understand what you mean here. You want me to create a
CategoryModel of which all my Category models should be derived ? If
that's it, how can it help me to determine if URL is valid or not, I
can't see how that works.


>I think you care too much about people entering wrong URLs. People are >clicking links on your webpage, right? Or do you give him instructions to >write urls in the adress field?

Yes they click links on the webpages :)

1) use the url tag or get_absolute_url of your models to produce the
urls in your template
2) configure your site to send you 404 errors which have a referer
(read below and [1])

>404 errors
>Django can also be configured to e-mail errors about broken links (404 “page >not found” errors). Django sends e-mails about 404 errors when:
>DEBUG is False
>SEND_BROKEN_LINK_EMAILS is True
>Your MIDDLEWARE_CLASSES setting includes CommonMiddleware (which it >does by default).
>If those conditions are met, Django will e-mail the users listed in the >MANAGERS setting whenever your code raises a 404 and the request has a >referer. (It doesn’t bother to e-mail for 404s that don’t have a referer – those >are usually just people typing in broken URLs or broken Web ‘bots).

Wow that's nice to know, thanks for all these infos !


On Nov 30, 11:15 am, Ivo Brodien <[hidden email]> wrote:

> Hi,
>
> > That's why I'm
> > considering checking at the beginning of the view with a list/dict
> > filled with category names and one filled with item names to make sure
> > that the category is in the list/dict and same for the item before
> > hitting the database, and if not respond with a 404.
>
> As Reinout suggested you should have a CategoryModel which you can check if a Category exists and the get the items of show that the category does not exist or 404 if somebody entered a wrong URL/Category. Checking against the DB is probably the same thing you want to do, but it uses the DB and not a List/Dict which you have to maintain manually somehow.
>
> I think you care too much about people entering wrong URLs. People are clicking links on your webpage, right? Or do you give him instructions to write urls in the adress field?
>
> As long as you produce correct urls the code works and only should show 404 if someone messed with the URL.
>
> Consider doing this:
>
> 1) use the url tag or get_absolute_url of your models to produce the urls in your template
> 2) configure your site to send you 404 errors which have a referer (read below and [1])
>
> 404 errors
> Django can also be configured to e-mail errors about broken links (404 “page not found” errors). Django sends e-mails about 404 errors when:
> DEBUG is False
> SEND_BROKEN_LINK_EMAILS is True
> Your MIDDLEWARE_CLASSES setting includes CommonMiddleware (which it does by default).
> If those conditions are met, Django will e-mail the users listed in the MANAGERS setting whenever your code raises a 404 and the request has a referer. (It doesn’t bother to e-mail for 404s that don’t have a referer – those are usually just people typing in broken URLs or broken Web ‘bots).
>
> bye
>
> [1]https://docs.djangoproject.com/en/1.3/howto/error-reporting/

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.

Reply | Threaded
Open this post in threaded view
|

Re: A view associated with a lot of urls

Ivo Brodien-2

I don't quite understand what you mean here. You want me to create a
CategoryModel of which all my Category models should be derived ? If
that's it, how can it help me to determine if URL is valid or not, I
can't see how that works.

Sorry, it was DrBloodmoney, who wrote this:

class Category(models.Model):
   name = models.CharField.... # should be unique

class Item(models.Model):
 name = models.CharField()
  category = models.ForeignKey(Category, related_name='items’)

in your view you do so:

def view(request, category, item):
     items= get_object_or_404(Category,name=category).items().filter(name=item)
     ....

That’s it.

Now if someone enters a category by hand which does not exist, he gets a 404. If the category exists, but there is no item with such a name items will be empty.

Bye
Ivo

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Reply | Threaded
Open this post in threaded view
|

Re: A view associated with a lot of urls

Nolhian
> Sorry, it was DrBloodmoney, who wrote this:
>
> > class Category(models.Model):
> >    name = models.CharField.... # should be unique
>
> > class Item(models.Model):
> >  name = models.CharField()
> >   category = models.ForeignKey(Category, related_name='items’)
>

Oh that, yes I intended to do so.

>Now if someone enters a category by hand which does not exist, he gets a >404. If the category exists, but there is no item with such a name items will >be empty.

Well I agree that if the category doesn't exist he will get a 404. But
if the Category exists and there are no items in the 'table', it will
not be empty, it will also raise a 404 too.  That's why I wrote :

>However with that, there is no way for example to make a difference
>between "there is no items in this category" and "this category
>doesn't exist" !

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.

Reply | Threaded
Open this post in threaded view
|

Re: A view associated with a lot of urls

Nolhian
In reply to this post by Ivo Brodien-2
Well that should work pretty well with an SQL database. But as I said
my database is  a NoSQL one, here I need to have one "table" per
category. So creating another Categorylist table just for fun and put
all the category names into, then hit it in order to check if the
category exists just seems wrong to me.
I thought of retrieving a list of the table names from the database
and check if the category the user want is in that list but for that I
must import another module.

On Nov 30, 1:41 pm, Ivo Brodien <[hidden email]> wrote:

> > I don't quite understand what you mean here. You want me to create a
> > CategoryModel of which all my Category models should be derived ? If
> > that's it, how can it help me to determine if URL is valid or not, I
> > can't see how that works.
>
> Sorry, it was DrBloodmoney, who wrote this:
>
> > class Category(models.Model):
> >    name = models.CharField.... # should be unique
>
> > class Item(models.Model):
> >  name = models.CharField()
> >   category = models.ForeignKey(Category, related_name='items’)
>
> in your view you do so:
>
> def view(request, category, item):
>      items= get_object_or_404(Category,name=category).items().filter(name=item)
>      ....
>
> That’s it.
>
> Now if someone enters a category by hand which does not exist, he gets a 404. If the category exists, but there is no item with such a name items will be empty.
>
> Bye
> Ivo

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.

Reply | Threaded
Open this post in threaded view
|

Re: A view associated with a lot of urls

Ivo Brodien-2
In reply to this post by Nolhian

> Well I agree that if the category doesn't exist he will get a 404. But
> if the Category exists and there are no items in the 'table', it will
> not be empty, it will also raise a 404 too.  That's why I wrote :
>
>> However with that, there is no way for example to make a difference
>> between "there is no items in this category" and "this category
>> doesn't exist” !

No. with the line below items will be empty ant not 404.

items= get_object_or_404(Category,name=category).items().filter(name=item)

or  items without the brackets. (not sure about the syntax at the moment)

The above is the same as:

# This could produce 404
category = get_object_or_404(Category,name=category)

# This will result in an empty queryset and not 404
items = category.items().filter(name=item)

It is also obvious that the last line does not produce 404 since 404 is HTML related and the line above is a DB query which at most should result in something empty.

I just saw your answer about using NoSQL. Sorry, forgot about that.

In that case I cannot help you.

Good Luck

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.

Reply | Threaded
Open this post in threaded view
|

Re: A view associated with a lot of urls

Nolhian
> > Well I agree that if the category doesn't exist he will get a 404. But
> > if the Category exists and there are no items in the 'table', it will
> > not be empty, it will also raise a 404 too.  That's why I wrote :
>
> >> However with that, there is no way for example to make a difference
> >> between "there is no items in this category" and "this category
> >> doesn't exist” !
>
> No. with the line below items will be empty ant not 404.
>
> items= get_object_or_404(Category,name=category).items().filter(name=item)
>
> or  items without the brackets. (not sure about the syntax at the moment)
>
> The above is the same as:
>
> # This could produce 404
> category = get_object_or_404(Category,name=category)
>
> # This will result in an empty queryset and not 404
> items = category.items().filter(name=item)
>
> It is also obvious that the last line does not produce 404 since 404 is HTML related and the line above is a DB query which at most should result in something empty.

Yes, sorry I misread what you said. I read it correctly shortly after
posting and that's why I deleted my post like 1 min after posting it,
didn't think you would still get it though !

> I just saw your answer about using NoSQL. Sorry, forgot about that.
> In that case I cannot help you.

Okay ! Anyway thank you for your help, your infos about the urls and
configuring the site to send 404 errors are also really appreciated ;)

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.