In-memory queryset

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

In-memory queryset

paul-5
Hi,

It would be really convenient for me if there was an implementation of the QuerySet API which instead of using a database as its data source, used in-memory model instance that had not been persisted to the database at all.

I looked around and found nothing like this.

Is this because nothing like this exists? Is it because it's a terrible idea for a reason that's not obvious to me? Would it not be awesome for the API for interacting with collections of in-memory instances and with a database-backed collection to be the same?

Many thanks,

Paul

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/d4f7c142-5d3f-4158-98bb-1e2171f0e63e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: In-memory queryset

ludovic coues
This look like a question for django user.

The QuerySet API is only an abstraction layer to SQL code. That's why
it require a database.
If you really don't want to store your object on a filesystem, sqlite
support database living only in memory. That's mainly for testing
purpose but that could fit your use-case. And django support multiple
database.

2017-03-07 15:13 GMT+01:00  <[hidden email]>:

> Hi,
>
> It would be really convenient for me if there was an implementation of the
> QuerySet API which instead of using a database as its data source, used
> in-memory model instance that had not been persisted to the database at all.
>
> I looked around and found nothing like this.
>
> Is this because nothing like this exists? Is it because it's a terrible idea
> for a reason that's not obvious to me? Would it not be awesome for the API
> for interacting with collections of in-memory instances and with a
> database-backed collection to be the same?
>
> Many thanks,
>
> Paul
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [hidden email].
> To post to this group, send email to [hidden email].
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/d4f7c142-5d3f-4158-98bb-1e2171f0e63e%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.



--

Cordialement, Coues Ludovic
+336 148 743 42

--
You received this message because you are subscribed to the Google Groups "Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAEuG%2BTZbj6hR%3DyBi56QSxatK1X1XmxuBfHTw39LEVwgWe_B%2BuA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: In-memory queryset

Jacob Kaplan-Moss-2
In reply to this post by paul-5
Hey Paul -

There are a couple of implementations I'm aware of:


Neither are a complete implementation of the QuerySet API, and I think this is because the QuerySet API surface is _huge_. Implementing the entire API would be a monumental task (especially if you start wanting to implement some of the more complex operations, like annotate/aggregate). At a certain point, it because easier to just use a real database -- most people use SQLite at that point.

It's certainly _possible_ to implement a in-memory datastore, and it might be pretty useful. It's just pretty dang hard to do more than the first 20% or so.

Jacob

On Tue, Mar 7, 2017 at 6:13 AM, <[hidden email]> wrote:
Hi,

It would be really convenient for me if there was an implementation of the QuerySet API which instead of using a database as its data source, used in-memory model instance that had not been persisted to the database at all.

I looked around and found nothing like this.

Is this because nothing like this exists? Is it because it's a terrible idea for a reason that's not obvious to me? Would it not be awesome for the API for interacting with collections of in-memory instances and with a database-backed collection to be the same?

Many thanks,

Paul

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/d4f7c142-5d3f-4158-98bb-1e2171f0e63e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAK8PqJEW_OiKjHjusVOEgSSKjzHy5gXZXV9Hi8yB2E0RmZGsPg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: In-memory queryset

Tim Chase-7
On 2017-03-07 07:17, Jacob Kaplan-Moss wrote:
> It's certainly _possible_ to implement a in-memory datastore, and
> it might be pretty useful. It's just pretty dang hard to do more
> than the first 20% or so.

Would there also be issues with WSGI spinning up/down various Django
processes?  Any in-memory (in-process) data would be lost when the
process goes down.  I could see it working for an individual dev
server or if you could guarantee that a single WSGI process remains
running indefinitely. But for those, I'd just go with sqlite's
":memory:" option.

-tkc



--
You received this message because you are subscribed to the Google Groups "Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/20170307101615.48bb1a00%40bigbox.christie.dr.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: In-memory queryset

neal
In reply to this post by paul-5
Hi Paul,

Depending on your use cases django-modelcluster might be of some use:

https://github.com/wagtail/django-modelcluster

It's primarly for dealing with a bunch of models that have foreign key relationships, but before they're written to a database (i.e before they have primary keys that allow the foreign keys to work). At some point they are typically committed to a database but they also could be ephemeral or serialised and handed off to something else.

It only supports a subset of the QuerySet API (pragmatically that which allows it to serve the purpose it was created for).

Cheers, Neal  

On Tuesday, March 7, 2017 at 2:57:57 PM UTC, [hidden email] wrote:

It would be really convenient for me if there was an implementation of the QuerySet API which instead of using a database as its data source, used in-memory model instance that had not been persisted to the database at all.

I looked around and found nothing like this.

Is this because nothing like this exists? Is it because it's a terrible idea for a reason that's not obvious to me? Would it not be awesome for the API for interacting with collections of in-memory instances and with a database-backed collection to be the same?

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/e5749e59-167f-48fd-9563-7cb69b5e9f6b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Loading...