Serialization on part of a model

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Serialization on part of a model

Daniel Roseman

I'm trying to use the serialization feature to drive a javascript
"selector" popup on my CMS. Basically, what I want to do is this: get
an XML or JSON list of all the article names and headlines in a certain
section. The serialization code is easy:

-- from models.py:
class Section(models.Model):
    name = models.CharField(maxlength=50)
    parent_section = models.ForeignKey("Section", null=True,
blank=True, related_name="subsection")

class Article(models.Model):
    name = models.SlugField()
    create_date = models.DateTimeField('Creation date', auto_now_add =
True)
    section = models.ForeignKey(Section)
    headline = models.CharField(maxlength=200, blank = True)
    article_text = models.TextField(blank = True)

-- from views.py
    sec = Section.objects.get(name="mysection")
    stories = sec.article_set.all()
    data = serializers.serialize("xml", stories)

But this returns _all_ the model fields - including the entire article
text. Clearly, if there are lots of long articles in a section, I'm
potentially transmitting megabytes of unwanted information.

What I really want to do is something like:
    stories = sec.article_set.values("name", "headline")
    data = serializers.serialize("xml", stories)
But that doesn't work because values() returns a dictionary, not a
QuerySet.

Is there a way to do this, or have I misunderstood the point of
serialization - and if so, what's the alternative?

Thanks,
Daniel.


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/django-users
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: Serialization on part of a model

Rajesh Dhawan

Hi Daniel,

As you've discovered, the built-in serializers require a Queryset of
model objects.

You could use the simplejson library directly on your "values"
dictionary.

Basically,
1. import simplejson
2. data = simplejson.dumps(your_dictionary_here)

See the documentation at the top of this code for more:
http://code.djangoproject.com/browser/django/trunk/django/utils/simplejson/__init__.py

-Raj


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/django-users
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: Serialization on part of a model

Daniel Roseman

RajeshD wrote:

> Hi Daniel,
>
> As you've discovered, the built-in serializers require a Queryset of
> model objects.
>
> You could use the simplejson library directly on your "values"
> dictionary.
>
> Basically,
> 1. import simplejson
> 2. data = simplejson.dumps(your_dictionary_here)
>

Thanks for the tip. I had already tried this but was getting an "is not
serializable" TypeError. I hadn't realised that the list returned from
a values() call is not actually a list, but a ValuesQuerySet. So to
make it work you have to do simplejson.dumps(list(valuedictionary))

--
Daniel.


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/django-users
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: Serialization on part of a model

Deryck Hodge

Hi, Daniel.

> > As you've discovered, the built-in serializers require a Queryset of
> > model objects.
> >
> > You could use the simplejson library directly on your "values"
> > dictionary.

I submitted http://code.djangoproject.org/ticket/2701 which provides a
small patch to limit by field for the serializers module.  Something
like -- serializers.serialize('xml', stories, fields=('headline',
'name'))

Perhaps you could try the patch attached to the ticket.  Also, I
mention it just to draw the core dev's attention to the patch.

Cheers,
deryck

--
Deryck Hodge                            http://www.devurandom.org/
Samba Team                                         http://www.samba.org/

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/django-users
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: Serialization on part of a model

Malcolm Tredinnick

Hey Deryck,

On Thu, 2006-10-05 at 19:00 -0500, Deryck Hodge wrote:

> Hi, Daniel.
>
> > > As you've discovered, the built-in serializers require a Queryset of
> > > model objects.
> > >
> > > You could use the simplejson library directly on your "values"
> > > dictionary.
>
> I submitted http://code.djangoproject.org/ticket/2701 which provides a
> small patch to limit by field for the serializers module.  Something
> like -- serializers.serialize('xml', stories, fields=('headline',
> 'name'))
>
> Perhaps you could try the patch attached to the ticket.  Also, I
> mention it just to draw the core dev's attention to the patch.

It already has my attention. I looked it over a week ago when I had some
time to put into bugs and it didn't seem obviously insane (which is the
first hurdle to get over). I wanted to think about the API a little bit
-- it's not obviously wrong in any way; I just wanted to think before
going ahead. In principle, I think the idea in that patch is worthwhile
and either it or something very similar will go in shortly (I have some
more bug fixing time set aside next week).

Regards,
Malcolm



--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/django-users
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: Serialization on part of a model

Nikolaus Schlemm
In reply to this post by Deryck Hodge

> I submitted http://code.djangoproject.org/ticket/2701 which provides a
> small patch to limit by field for the serializers module.  Something
> like -- serializers.serialize('xml', stories, fields=('headline',
> 'name'))

excellent, definetely a "+1" from me!
--
cheers,

        Nikl

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/django-users
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: Serialization on part of a model

Daniel Roseman
In reply to this post by Deryck Hodge

Deryck Hodge wrote:
> I submitted http://code.djangoproject.org/ticket/2701 which provides a
> small patch to limit by field for the serializers module.  Something
> like -- serializers.serialize('xml', stories, fields=('headline',
> 'name'))
>
> Perhaps you could try the patch attached to the ticket.  Also, I
> mention it just to draw the core dev's attention to the patch.

Thanks a lot, this is exactly the sort of thing I was after.
Unfortunately, it doesn't seem to work on JSON. Running from the shell,
I get the following:

>>> serializers.serialize('json', sec.article_set.all(), fields=('id', 'name'))
Traceback (most recent call last):
  File "<console>", line 1, in ?
  File "c:\mydown~1\django_mr\django\core\serializers\__init__.py",
line 55, in serialize
    s.serialize(queryset, **options)
  File "c:\mydown~1\django_mr\django\core\serializers\base.py", line
52, in serialize
    self.end_serialization()
  File "c:\mydown~1\django_mr\django\core\serializers\json.py", line
19, in end_serialization
    simplejson.dump(self.objects, self.stream,
cls=DateTimeAwareJSONEncoder, **self.options)
  File "c:\mydown~1\django_mr\django\utils\simplejson\__init__.py",
line 115, in dump
    check_circular=check_circular, allow_nan=allow_nan,
TypeError: __init__() got an unexpected keyword argument 'fields'

It works OK on XML, though. Any ideas?

--
Daniel.


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/django-users
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: Serialization on part of a model

Deryck Hodge

On 10/6/06, Daniel Roseman <[hidden email]> wrote:

> Thanks a lot, this is exactly the sort of thing I was after.
> Unfortunately, it doesn't seem to work on JSON. Running from the shell,
> I get the following:
>
> >>> serializers.serialize('json', sec.article_set.all(), fields=('id', 'name'))
> Traceback (most recent call last):
>   File "<console>", line 1, in ?
>   File "c:\mydown~1\django_mr\django\core\serializers\__init__.py",
> line 55, in serialize
>     s.serialize(queryset, **options)
>   File "c:\mydown~1\django_mr\django\core\serializers\base.py", line
> 52, in serialize
>     self.end_serialization()
>   File "c:\mydown~1\django_mr\django\core\serializers\json.py", line
> 19, in end_serialization
>     simplejson.dump(self.objects, self.stream,
> cls=DateTimeAwareJSONEncoder, **self.options)
>   File "c:\mydown~1\django_mr\django\utils\simplejson\__init__.py",
> line 115, in dump
>     check_circular=check_circular, allow_nan=allow_nan,
> TypeError: __init__() got an unexpected keyword argument 'fields'
>
> It works OK on XML, though. Any ideas?
>

Thanks, Daniel, for the traceback.  I think I know why this is
happening -- it's related to the relationship and the conditional I
used.  I'll test with a similar example and update the patch.

Sorry for the delay, too.  Been offline with the birth of a new baby.
:-)  Should have a new patch in a day or so, as time allows.

Cheers,
deryck

--
Deryck Hodge                            http://www.devurandom.org/
Samba Team                                        http://www.samba.org/

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/django-users
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: Serialization on part of a model

Deryck Hodge
In reply to this post by Malcolm Tredinnick

On 10/5/06, Malcolm Tredinnick <[hidden email]> wrote:
> It already has my attention. I looked it over a week ago when I had some
> time to put into bugs and it didn't seem obviously insane (which is the
> first hurdle to get over). I wanted to think about the API a little bit
> -- it's not obviously wrong in any way; I just wanted to think before
> going ahead. In principle, I think the idea in that patch is worthwhile
> and either it or something very similar will go in shortly (I have some
> more bug fixing time set aside next week).
>

Thanks, Malcolm.  If you have any ideas about the API, or would like
some help shaping this patch into what you think may be ideal, I'm
glad to continue helping on this.

Cheers,
deryck

--
Deryck Hodge                            http://www.devurandom.org/
Samba Team                                        http://www.samba.org/

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/django-users
-~----------~----~----~----~------~----~------~--~---