Multi Database use with statement

classic Classic list List threaded Threaded
6 messages Options
vcc
Reply | Threaded
Open this post in threaded view
|

Multi Database use with statement

vcc
Hi,

I write a patch for using with statement in python 2.5 and up for multi database access, example code like this:

# test.py
from __future__ import with_statement
from django.contrib.auth.models import User
from django.db import using

with using('db1'):
    print User.objects.count() # db1 user count
    with using('db2'):
        print User.objects.count() # db2 user count
print User.objects.count() # default db user count

#settings.py

# default db
DATABASE_ENGINE = 'sqlite3'
DATABASE_NAME = 'test.db'

# others
DATABASES = {
 'db1': dict(
            DATABASE_ENGINE = 'sql_server.pyodbc',
            DATABASE_NAME = 'db1',
            DATABASE_USER = 'user',
            DATABASE_PASSWORD = 'password',
            DATABASE_ODBC_DSN = 'DSN1',
            DATABASE_COLLATION = 'Chinese_PRC_CI_AS',
 ),
 'db2': dict(
            DATABASE_ENGINE = 'sqlite3',
            DATABASE_NAME = 'db2.db',
 ),
}

Please see patch in the attchments, comment welcome.

Regards,
Wei guangjing

--
http://code.google.com/p/django-pyodbc



--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Django developers" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en
-~----------~----~----~----~------~----~------~--~---


multidb.diff (6K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Multi Database use with statement

mtrier

Hi Wei,

On Dec 27, 6:23 am, "wei guangjing" <[hidden email]> wrote:

> Hi,
>
> I write a patch for using with statement in python 2.5 and up for multi database access, example code like this:
>
> # test.py
> from __future__ import with_statement
> from django.contrib.auth.models import User
> from django.db import using
>
> with using('db1'):
>     print User.objects.count() # db1 user count
>     with using('db2'):
>         print User.objects.count() # db2 user count
> print User.objects.count() # default db user count
>

A couple of ideas to throw in the mix.  I'd like to see a Meta
option.  Like:

Meta:
    using = 'db2'

So you have a default for your model.

Additionally what about just using the generative syntax for the
using.  It doesn't seem that a lot is to be gained by using the
context manager.  For instance:

User.objects.using('db2').all()

Also when thinking about third-party plugable applications you have to
consider how you're going to handle that. If you have a third party
app and you want it to all be directed to a particular database, how
does that get handled in your scenario above?

Just a couple of things to consider.

Michael Trier
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Django developers" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: Multi Database use with statement

Alex Gaynor

Personally I think I like Michael's suggestion of a using method on a
queryset, and a default in Meta(pretty much the way ordering works).
To his question of reusable applications I think the solution to that
is actually something outside the scope of Django itself, a reusable's
apps views should take a db kwarg callback that takes whatever the
appropriate params and returns the db to be used.

Alex

On Dec 27, 8:37 pm, mtrier <[hidden email]> wrote:

> Hi Wei,
>
> On Dec 27, 6:23 am, "wei guangjing" <[hidden email]> wrote:
>
> > Hi,
>
> > I write a patch for using with statement in python 2.5 and up for multi database access, example code like this:
>
> > # test.py
> > from __future__ import with_statement
> > from django.contrib.auth.models import User
> > from django.db import using
>
> > with using('db1'):
> >     print User.objects.count() # db1 user count
> >     with using('db2'):
> >         print User.objects.count() # db2 user count
> > print User.objects.count() # default db user count
>
> A couple of ideas to throw in the mix.  I'd like to see a Meta
> option.  Like:
>
> Meta:
>     using = 'db2'
>
> So you have a default for your model.
>
> Additionally what about just using the generative syntax for the
> using.  It doesn't seem that a lot is to be gained by using the
> context manager.  For instance:
>
> User.objects.using('db2').all()
>
> Also when thinking about third-party plugable applications you have to
> consider how you're going to handle that. If you have a third party
> app and you want it to all be directed to a particular database, how
> does that get handled in your scenario above?
>
> Just a couple of things to consider.
>
> Michael Trier
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Django developers" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en
-~----------~----~----~----~------~----~------~--~---

vcc
Reply | Threaded
Open this post in threaded view
|

Re: Multi Database use with statement

vcc
In reply to this post by mtrier
Please see the attached patch for using in models's Meta option, I also like this option.

This patch export some helper function from django.db,

get_connection(name) get database connection by name
get_current_connection() get current connection

you can use follow function to ensure use special db connection:
enter_connection_management(connection)  
leave_connection_management()

Wei guangjing


----- Original Message -----
From: "mtrier" <[hidden email]>
To: "Django developers" <[hidden email]>
Sent: Sunday, December 28, 2008 10:37 AM
Subject: Re: Multi Database use with statement



Hi Wei,

On Dec 27, 6:23 am, "wei guangjing" <[hidden email]> wrote:

> Hi,
>
> I write a patch for using with statement in python 2.5 and up for multi database access, example code like this:
>
> # test.py
> from __future__ import with_statement
> from django.contrib.auth.models import User
> from django.db import using
>
> with using('db1'):
> print User.objects.count() # db1 user count
> with using('db2'):
> print User.objects.count() # db2 user count
> print User.objects.count() # default db user count
>
A couple of ideas to throw in the mix.  I'd like to see a Meta
option.  Like:

Meta:
    using = 'db2'

So you have a default for your model.

Additionally what about just using the generative syntax for the
using.  It doesn't seem that a lot is to be gained by using the
context manager.  For instance:

User.objects.using('db2').all()

Also when thinking about third-party plugable applications you have to
consider how you're going to handle that. If you have a third party
app and you want it to all be directed to a particular database, how
does that get handled in your scenario above?

Just a couple of things to consider.

Michael Trier


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Django developers" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en
-~----------~----~----~----~------~----~------~--~---


using.diff (10K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Multi Database use with statement

Ludvig Ericson
In reply to this post by vcc

On Dec 27, 2008, at 12:23, wei guangjing wrote:
> from __future__ import with_statement
> from django.contrib.auth.models import User
> from django.db import using
>
> with using('db1'):
>    print User.objects.count() # db1 user count
>    with using('db2'):
>        print User.objects.count() # db2 user count
> print User.objects.count() # default db user count

A few thoughts:
  - How will Django know which database is "chosen"? It would need  
some kind of thread-global storage to know. That's, IMO, not very nice.
  - Is "using" really such a great name? If I look at it, I don't  
really associate to databases or anything like it.
  - Isn't the QuerySet object responsible for knowing what to query?  
Like:
      db1_count = User.objects.at("db2").count()

As for reusability, it becomes sort of a tough situation for app  
developers if they're responsible entirely for defining which database  
takes care of what - if you'd use the QuerySet-centered way, you'd be  
able to pass that to generic views, for example.

These are just my thoughts, though.

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Django developers" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en
-~----------~----~----~----~------~----~------~--~---

vcc
Reply | Threaded
Open this post in threaded view
|

Re: Multi Database use with statement

vcc
In reply to this post by vcc
I setup a git repository http://github.com/weigj/django-multidb/tree/master for it.

I patched fields, query and manager, so Model.objects.using('db1') is work now, like this:

users = User.objects.using('db1').filter(....)
for user in users:
      user.save(using='db2')

Wei guangjing

----- Original Message -----
From: "wei guangjing" <[hidden email]>
To: <[hidden email]>
Sent: Sunday, December 28, 2008 12:00 PM
Subject: Re: Multi Database use with statement


> Please see the attached patch for using in models's Meta option, I also like this option.
>
> This patch export some helper function from django.db,
>
> get_connection(name) get database connection by name
> get_current_connection() get current connection
>
> you can use follow function to ensure use special db connection:
> enter_connection_management(connection)  
> leave_connection_management()
>
> Wei guangjing
>
>
> ----- Original Message -----
> From: "mtrier" <[hidden email]>
> To: "Django developers" <[hidden email]>
> Sent: Sunday, December 28, 2008 10:37 AM
> Subject: Re: Multi Database use with statement
>
>
>
> Hi Wei,
>
> On Dec 27, 6:23 am, "wei guangjing" <[hidden email]> wrote:
>> Hi,
>>
>> I write a patch for using with statement in python 2.5 and up for multi database access, example code like this:
>>
>> # test.py
>> from __future__ import with_statement
>> from django.contrib.auth.models import User
>> from django.db import using
>>
>> with using('db1'):
>> print User.objects.count() # db1 user count
>> with using('db2'):
>> print User.objects.count() # db2 user count
>> print User.objects.count() # default db user count
>>
>
> A couple of ideas to throw in the mix.  I'd like to see a Meta
> option.  Like:
>
> Meta:
>    using = 'db2'
>
> So you have a default for your model.
>
> Additionally what about just using the generative syntax for the
> using.  It doesn't seem that a lot is to be gained by using the
> context manager.  For instance:
>
> User.objects.using('db2').all()
>
> Also when thinking about third-party plugable applications you have to
> consider how you're going to handle that. If you have a third party
> app and you want it to all be directed to a particular database, how
> does that get handled in your scenario above?
>
> Just a couple of things to consider.
>
> Michael Trier
>
>
> >
>
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Django developers" group.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to [hidden email]
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en
-~----------~----~----~----~------~----~------~--~---