Some obscurity with paramstyle

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
34 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Re: Some obscurity with paramstyle

Michael Bayer

On Jul 18, 2011, at 3:33 AM, M.-A. Lemburg wrote:

>
> For the next version, we're likely going to reduce the number
> of allowed formats to just two (%s and ?, IIRC), since having
> lots of different formats has resulted in much confusion.

Why have both %s and ? (both positional?)  If there is to only be positional, what purpose is served by having two choices for the format ?

Also is it accurate that named parameters, support for dictionaries passed to execute(), will be dropped ?   This leads things back in the "more difficult to use" direction IMHO.


_______________________________________________
DB-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/db-sig
Reply | Threaded
Open this post in threaded view
|

Re: Some obscurity with paramstyle

M.-A. Lemburg
Michael Bayer wrote:
>
> On Jul 18, 2011, at 3:33 AM, M.-A. Lemburg wrote:
>
>>
>> For the next version, we're likely going to reduce the number
>> of allowed formats to just two (%s and ?, IIRC), since having
>> lots of different formats has resulted in much confusion.
>
> Why have both %s and ? (both positional?)  If there is to only be positional, what purpose is served by having two choices for the format ?

It's well possible that I've misremembered the ones we picked
in a discussion some time ago. Could well have been named and
qmark.

> Also is it accurate that named parameters, support for dictionaries passed to execute(), will be dropped ?   This leads things back in the "more difficult to use" direction IMHO.

Depends on which parameter types we choose. Note that I don't think
we'll actually drop the support, just require support for two types
and leave the others optionally supported as well.

--
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jul 18 2011)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/
_______________________________________________
DB-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/db-sig
Reply | Threaded
Open this post in threaded view
|

Re: Some obscurity with paramstyle

Michael Bayer
In reply to this post by M.-A. Lemburg

On Jul 18, 2011, at 8:54 AM, M.-A. Lemburg wrote:

>
> For data input/output, the situation is difficult, since not
> all backends support Unicode, some only configuration dependent
> character sets and some others are basically ASCII/binary data
> only.

OK, but even those DBAPIs can present Python unicode objects on the user-facing side, at least as an option (the module global you mention speaks to this - I would suggest the parameter is available as a global and on the connection object).   If the only encoding available is "ascii", then that's all you have.

Of course bytes should be accepted as well, though some DBAPIs have problems with this.   I think passing unicode + an encoding is known at the global/connection level gives the widest latitude to the DBAPI itself in dealing with textual data.

>
> We will likely have to introduce a new TEXT() constructor
> which maps data objects to text data and then takes care
> of the database specific encoding issues.
>
> There are similar issues with float/decimal and date/time
> values.
>
> It would be great if we could resolve all of these using
> a data type mapping facility that defines input and output
> mappings in an efficient and flexible way.

Several DBAPIs have these already, but particularly for the Decimal issue, it's critical that numerics aren't presented as Python floats before there is even a chance to extract the information.

>
> --
> Marc-Andre Lemburg
> eGenix.com
>
> Professional Python Services directly from the Source  (#1, Jul 18 2011)
>>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
> ________________________________________________________________________
>
> ::: Try our new mxODBC.Connect Python Database Interface for free ! ::::
>
>
>   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
>    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
>           Registered at Amtsgericht Duesseldorf: HRB 46611
>               http://www.egenix.com/company/contact/
> _______________________________________________
> DB-SIG maillist  -  [hidden email]
> http://mail.python.org/mailman/listinfo/db-sig

_______________________________________________
DB-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/db-sig
Reply | Threaded
Open this post in threaded view
|

Re: Some obscurity with paramstyle

Michael Bayer
In reply to this post by M.-A. Lemburg

On Jul 18, 2011, at 3:52 AM, M.-A. Lemburg wrote:

> Vernon Cole wrote:
>> On Sun, Jul 17, 2011 at 8:54 AM, Michael Bayer <[hidden email]>wrote:
>>> Is there a path to changes being made in the DBAPI?   i.e. would there be a
>>> DBAPI 3 ?
>>
>>
>> That possibility has been discussed before, and is particularly timely given
>> that it is impossible to write a PEP 249 compliant module in Python 3.  [For
>> example, the PEP states that an Error exception "must be a subclass of the
>> Python StandardError" -- which Python 3 does not support.]
>>
>> Marc-André Lemburg (the author of PEP 249) came out against an update --
>> mostly due to performance reasons.
>
> Not sure, where you read that :-)
>
> I'm not opposed to a DB API 3, but since there are only a few
> things on the table for DB API 3 and the adoption of Python 3.x
> has just started, so it's not all that urgent.

Do you have an idea in mind when effort on the new spec might be happening ?   If I can find the time I'd want to contribute a compliance suite, and/or update the old one (since I'd like DBAPI 3 to be a lot more specific about some things).


_______________________________________________
DB-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/db-sig
Reply | Threaded
Open this post in threaded view
|

Re: Some obscurity with paramstyle

M.-A. Lemburg
Michael Bayer wrote:

>
> On Jul 18, 2011, at 3:52 AM, M.-A. Lemburg wrote:
>
>> Vernon Cole wrote:
>>> On Sun, Jul 17, 2011 at 8:54 AM, Michael Bayer <[hidden email]>wrote:
>>>> Is there a path to changes being made in the DBAPI?   i.e. would there be a
>>>> DBAPI 3 ?
>>>
>>>
>>> That possibility has been discussed before, and is particularly timely given
>>> that it is impossible to write a PEP 249 compliant module in Python 3.  [For
>>> example, the PEP states that an Error exception "must be a subclass of the
>>> Python StandardError" -- which Python 3 does not support.]
>>>
>>> Marc-André Lemburg (the author of PEP 249) came out against an update --
>>> mostly due to performance reasons.
>>
>> Not sure, where you read that :-)
>>
>> I'm not opposed to a DB API 3, but since there are only a few
>> things on the table for DB API 3 and the adoption of Python 3.x
>> has just started, so it's not all that urgent.
>
> Do you have an idea in mind when effort on the new spec might be happening ?   If I can find the time I'd want to contribute a compliance suite, and/or update the old one (since I'd like DBAPI 3 to be a lot more specific about some things).

We might as well start now. I've been wanting to kick off the process
for about a year now, but never really got around to it (too many
other projects).

The talks I gave at EuroPython mentions a few things I think should
go into the DB API 3. The list archives will likely have a few more.

--
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jul 19 2011)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/
_______________________________________________
DB-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/db-sig
Reply | Threaded
Open this post in threaded view
|

Re: Some obscurity with paramstyle

Vernon D. Cole
In reply to this post by Federico Di Gregorio-5

Yes, please. I would like to see that. Adodbapi has an output conversion mechanism, but I don't like it particularly well.
  I would be happy to create a fork of adodbapi for a reference implementation. It will talk to any major database engine, but unfortunately, only on Windows.
Vernon Cole
(sent from my 'droid phone)

On Jul 19, 2011 3:44 AM, "Federico Di Gregorio" <[hidden email]> wrote:
> On 18/07/11 14:54, M.-A. Lemburg wrote:
> [snip]
>> We will likely have to introduce a new TEXT() constructor
>> which maps data objects to text data and then takes care
>> of the database specific encoding issues.
>>
>> There are similar issues with float/decimal and date/time
>> values.
>>
>> It would be great if we could resolve all of these using
>> a data type mapping facility that defines input and output
>> mappings in an efficient and flexible way.
>
> At least 2 drivers (psycopg and pysqlite) provide a Python->backend
> mechanism based on PEP 246, "Object Adaptation". If other implementors
> are interested I can write a short explanation about how it works and
> why it was chosen only for the Python->backend path and not for the reverse.
>
> federico
>
> --
> Federico Di Gregorio [hidden email]
> Studio Associato Di Nunzio e Di Gregorio http://dndg.it
> Se sai che hai un ***** di file così, lo manovri subito. -- vodka
> _______________________________________________
> DB-SIG maillist - [hidden email]
> http://mail.python.org/mailman/listinfo/db-sig


_______________________________________________
DB-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/db-sig
Reply | Threaded
Open this post in threaded view
|

Re: Some obscurity with paramstyle

Daniele Varrazzo-2
> On Jul 19, 2011 3:44 AM, "Federico Di Gregorio"
> <[hidden email]> wrote:
>> At least 2 drivers (psycopg and pysqlite) provide a Python->backend
>> mechanism based on PEP 246, "Object Adaptation". If other implementors
>> are interested I can write a short explanation about how it works and
>> why it was chosen only for the Python->backend path and not for the
>> reverse.

On Tue, Jul 19, 2011 at 11:05 AM, Vernon Cole <[hidden email]> wrote:
> Yes, please. I would like to see that. Adodbapi has an output conversion
> mechanism, but I don't like it particularly well.

For a description:

http://initd.org/psycopg/docs/advanced.html#adapting-new-python-types-to-sql-syntax
http://initd.org/psycopg/docs/advanced.html#type-casting-of-sql-types-into-python-objects

-- Daniele
_______________________________________________
DB-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/db-sig
Reply | Threaded
Open this post in threaded view
|

Re: Some obscurity with paramstyle

M.-A. Lemburg
Daniele Varrazzo wrote:

>> On Jul 19, 2011 3:44 AM, "Federico Di Gregorio"
>> <[hidden email]> wrote:
>>> At least 2 drivers (psycopg and pysqlite) provide a Python->backend
>>> mechanism based on PEP 246, "Object Adaptation". If other implementors
>>> are interested I can write a short explanation about how it works and
>>> why it was chosen only for the Python->backend path and not for the
>>> reverse.
>
> On Tue, Jul 19, 2011 at 11:05 AM, Vernon Cole <[hidden email]> wrote:
>> Yes, please. I would like to see that. Adodbapi has an output conversion
>> mechanism, but I don't like it particularly well.
>
> For a description:
>
> http://initd.org/psycopg/docs/advanced.html#adapting-new-python-types-to-sql-syntax
> http://initd.org/psycopg/docs/advanced.html#type-casting-of-sql-types-into-python-objects

While this is a nice system, it's also very slow. It uses function
calls and string parsing/conversion for adapting each value. This
works if you only have to insert/fetch a few rows, but won't be
feasible for larger volumes.

I think we need something more low-level, which tries to
avoid (Python) function calls if possible, e.g. it should be
possible to write adapters in C and only point to them using
a symbol (sketching here):

cursor.setinputconverter(BINARY, module.BINARY_INPUT_CONVERTER)
cursor.setoutputconverter(BINARY, module.BINARY_OUTPUT_CONVERTER)

For mxODBC we'd then use something like this:

# Convert Python unicode object data to SQLWCHAR data
cursor.setinputconverter(unicode, module.SQLWCHAR_INPUT_CONVERTER)

# Convert SQL_WCHAR type code data to Python unicode
cursor.setoutputconverter(SQL.WCHAR, module.SQLCHAR_OUTPUT_CONVERTER_UNICODE)

The advantage here is that the database module could work
directly on the internal data structure to implement
the conversion rather than having to round-trip to Python.

In order to provide the psycopg style adaption mechanism,
the converter functions would have to be registered with
converter codes, e.g.

module.registerinputconverter(module.POINT_INPUT_CONVERTER, adapt_point)
module.registeroutputconverter(module.POINT_OUTPUT_CONVERTER, cast_point)

Lookup would then be a simple dictionary lookup on output
(via the column type code) and a dictionary lookup/isinstance()
loop for input mappings.

--
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jul 19 2011)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/
_______________________________________________
DB-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/db-sig
Reply | Threaded
Open this post in threaded view
|

Re: Some obscurity with paramstyle

Federico Di Gregorio-5
On 19/07/11 14:18, M.-A. Lemburg wrote:
> Daniele Varrazzo wrote:
[snip]
> While this is a nice system, it's also very slow. It uses function
> calls and string parsing/conversion for adapting each value. This
> works if you only have to insert/fetch a few rows, but won't be
> feasible for larger volumes.

It depends on the backend, too. Sincerely when I need to load hudreds of
thousands of rows I just avoid INSERT and go directly for mechanisms
like PostgreSQL COPY. INSERT is (usually) so slooow that the drivers
doesn't add that much overhead.

> I think we need something more low-level, which tries to
> avoid (Python) function calls if possible, e.g. it should be
> possible to write adapters in C and only point to them using
> a symbol (sketching here):
>
> cursor.setinputconverter(BINARY, module.BINARY_INPUT_CONVERTER)
> cursor.setoutputconverter(BINARY, module.BINARY_OUTPUT_CONVERTER)
>
> For mxODBC we'd then use something like this:
>
> # Convert Python unicode object data to SQLWCHAR data
> cursor.setinputconverter(unicode, module.SQLWCHAR_INPUT_CONVERTER)
>
> # Convert SQL_WCHAR type code data to Python unicode
> cursor.setoutputconverter(SQL.WCHAR, module.SQLCHAR_OUTPUT_CONVERTER_UNICODE)
>
> The advantage here is that the database module could work
> directly on the internal data structure to implement
> the conversion rather than having to round-trip to Python.

But the disavantage is that you can't easily work with composite types
where you need to convert the primitive types and pack them into the
composite (arrays, geographic data, ...) while adapt() is perfect for
nested types.

Anyway, happy that the discussion started :)

federico

--
Federico Di Gregorio                         [hidden email]
Studio Associato Di Nunzio e Di Gregorio                  http://dndg.it
 I porcellini di terra sono davvero Crostacei! Non lo sapevo!
  Certo che sono crostacei, hanno la crosta!
  Allora la pizza è un crostaceo?!               -- discorso all'ESC2k07
_______________________________________________
DB-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/db-sig
Reply | Threaded
Open this post in threaded view
|

Re: Some obscurity with paramstyle

Chris Clark-2
In reply to this post by M.-A. Lemburg
M.-A. Lemburg wrote:

> Daniele Varrazzo wrote:
>  
>>> On Jul 19, 2011 3:44 AM, "Federico Di Gregorio"
>>> <[hidden email]> wrote:
>>>      
>>>> At least 2 drivers (psycopg and pysqlite) provide a Python->backend
>>>> mechanism based on PEP 246, "Object Adaptation". If other implementors
>>>> are interested I can write a short explanation about how it works and
>>>> why it was chosen only for the Python->backend path and not for the
>>>> reverse.
>>>>        
>> On Tue, Jul 19, 2011 at 11:05 AM, Vernon Cole <[hidden email]> wrote:
>>    
>>> Yes, please. I would like to see that. Adodbapi has an output conversion
>>> mechanism, but I don't like it particularly well.
>>>      
>> For a description:
>>
>> http://initd.org/psycopg/docs/advanced.html#adapting-new-python-types-to-sql-syntax
>> http://initd.org/psycopg/docs/advanced.html#type-casting-of-sql-types-into-python-objects
>>    
>
> While this is a nice system, it's also very slow. It uses function
> calls and string parsing/conversion for adapting each value. This
> works if you only have to insert/fetch a few rows, but won't be
> feasible for larger volumes.
>
> I think we need something more low-level, which tries to
> avoid (Python) function calls if possible, .....

I have to confess the only piece of pep-0246 I read was:

>
>       Rejection Notice
>
>     I'm rejecting this PEP.  Something much better is about to happen;
>     it's too early to say exactly what, but it's not going to resemble
>     the proposal in this PEP too closely so it's better to start a new
>     PEP.  GvR.
>  

Does anyone know what the "better" option was? The date on the pep is a
number of years ago.

Chris

_______________________________________________
DB-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/db-sig
Reply | Threaded
Open this post in threaded view
|

Re: Some obscurity with paramstyle

James Henstridge
In reply to this post by Federico Di Gregorio-5
On Mon, Jul 18, 2011 at 5:28 PM, Federico Di Gregorio
<[hidden email]> wrote:

> On 18/07/11 11:25, James Henstridge wrote:
>> If all adapters that support transactions function like that, then I
>> think it would be sensible to require them to raise an error in that
>> case.  If they don't raise an error, you know that someone somewhere
>> is going to write code like the following:
>>
>>     with transaction:
>>         # do stuff
>>         with transaction:
>>             # do more stuff
>>
>> ... and wonder why things break.
>
> This can make sense for backends that support check points
> (subtransactions). PostgreSQL is an example (even if we don't yet
> support checkpoints in psycopg).

While it is definitely possible to simulate some kind of
"sub-transaction" through the use of save points, I'm not sure whether
it is actually a good idea to magically substitute one for the other.

One feature of committing a transaction is that it makes the work has
been saved permanently and is available to other connections, which
won't be the case with a save point based sub-transaction.  If I wrote
some code that relied on those semantics (e.g. one that posts a
message to another process after completing its work), then it would
be an error to call it from within transaction context and I'd want to
see an error explaining the fact.

There probably are uses where a recursive transaction
decorator/context manager might be useful, but I don't think it is
necessarily something users would want by default.

James.
_______________________________________________
DB-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/db-sig
Reply | Threaded
Open this post in threaded view
|

Re: Some obscurity with paramstyle

M.-A. Lemburg
In reply to this post by James Henstridge
James Henstridge wrote:

> On Mon, Jul 18, 2011 at 3:52 PM, M.-A. Lemburg <[hidden email]> wrote:
>> Vernon Cole wrote:
>>> A cursor and/or a
>>> connection should probably be context managers, so that they will work in a
>>> "with" statement.
>>
>> This is was discussed before and it should probably go into
>> the DB API in some form.
>>
>> I'm no particular fan of hiding transactions in context
>> managers, though. In practice this often causes problems, since
>> you usually want to apply error handling logic in case of
>> problems, other than simply issuing a .rollback().
>
> For most of the applications I work with, the transaction handling has
> been delegated to function decorators, or have it hidden in the
> framework (e.g. Django's TransactionMiddleware).  If there are clean
> up tasks that need to happen on transaction commit or roll back (e.g.
> deleting a file on a failed transaction), then using a global
> transaction manager like Zope's transaction module seems to be a good
> fit.
>
> For transaction retry or error reporting, I haven't seen much benefit
> in leaving the transaction in an "open by broken" state over cleaning
> up with a rollback.

In more detail: When letting a context
manager do a .rollback() on a connection in case of
an error, the error still propagates up the call chain.

However, since the transaction was rolled back, access
to cursors on the connection and the connection itself
will no longer work, so your error reporting and cleanup
possibilities are left with just the error object.

Likewise, when having the context manager do an implicit
.commit() when leaving the block, you are giving away
control and have to add extra code in case you want to
prevent the changes from being committed, e.g. in a debug
situation.

Things can get really interesting in multi-threaded
applications, if you're sharing connections
between threads - but it's better to avoid that anyway,
so I won't go into detail.

Most of these issues can be resolved by coding explicit
context managers that deal with your particular case, e.g.

with TransactionalContext(connection):
    ...

Overall, I think using a transaction manager that explicitly
controls the .commit() and .rollback() calls is a lot
cleaner and the direct "with connection:" construct causes more
problems than it solves in more complex applications.

On the plus side, in simple applications it can help
the user to not forget the .commit() call (the
.rollback() is implicit at garbage collection time
anyway).

So in conclusion, I think we should add it, but with
a warning to the user that it's often better to write
your own context manager.

>> Another problem is that the connections used in a with
>> statement will usually have already started a transaction
>> before entering the context, so the .rollback() would
>> remove more than just the things added in the with
>> context.
>
> None of the databases I've worked with fit the Python DB API's
> "implicit begin" behaviour for transactions, so their adapters have
> all needed to run a simple state machine to determine when it is
> necessary to start a transaction.  So it would be pretty easy for them
> to raise an error if an attempt was made to use the context manager
> when a transaction was already in progress.

The implicit start of transactions originates from the ODBC
standard which is used by quite a few database as native API,
so the above cannot easily be generalized.

There's also a difference between starting a transaction and
actually making changes on that transaction. An error should
only be raised in case changes were made, but that would
require a lot tedious internal state keeping by the database
package, unless the backend provides an API to query whether
the transaction actually contains any changes.

I don't think there's much we can do about this particular
aspect of using connections as context managers.

> If all adapters that support transactions function like that, then I
> think it would be sensible to require them to raise an error in that
> case.  If they don't raise an error, you know that someone somewhere
> is going to write code like the following:
>
>     with transaction:
>         # do stuff
>         with transaction:
>             # do more stuff
>
> ... and wonder why things break.

Indeed.

>> There's also the problem of intuitive use: some users will
>> likely expect the connection to also be closed when leaving
>> the context - much like what happens with files.
>
> If that is a concern, you could require a method call to get the
> context manager rather than making the connection itself a context
> manager.  For example:
>
>     with connection.begin_transaction():
>         ...

True, but as discussed above, what would that method do ?

It could do an implicit rollback, but this would then likely
cause other problems such as clearing out changes the user
actually wants to commit later on.

The safe way I know is opening a new connection:

with db.connect(...) as connection:
    ...

--
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jul 20 2011)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/
_______________________________________________
DB-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/db-sig
Reply | Threaded
Open this post in threaded view
|

Re: Some obscurity with paramstyle

M.-A. Lemburg
In reply to this post by Chris Clark-2
Chris Clark wrote:

> M.-A. Lemburg wrote:
>> Daniele Varrazzo wrote:
>>  
>>>> On Jul 19, 2011 3:44 AM, "Federico Di Gregorio"
>>>> <[hidden email]> wrote:
>>>>      
>>>>> At least 2 drivers (psycopg and pysqlite) provide a Python->backend
>>>>> mechanism based on PEP 246, "Object Adaptation". If other implementors
>>>>> are interested I can write a short explanation about how it works and
>>>>> why it was chosen only for the Python->backend path and not for the
>>>>> reverse.
>>>>>        
>>> On Tue, Jul 19, 2011 at 11:05 AM, Vernon Cole <[hidden email]>
>>> wrote:
>>>    
>>>> Yes, please. I would like to see that. Adodbapi has an output
>>>> conversion
>>>> mechanism, but I don't like it particularly well.
>>>>      
>>> For a description:
>>>
>>> http://initd.org/psycopg/docs/advanced.html#adapting-new-python-types-to-sql-syntax
>>>
>>> http://initd.org/psycopg/docs/advanced.html#type-casting-of-sql-types-into-python-objects
>>>
>>>    
>>
>> While this is a nice system, it's also very slow. It uses function
>> calls and string parsing/conversion for adapting each value. This
>> works if you only have to insert/fetch a few rows, but won't be
>> feasible for larger volumes.
>>
>> I think we need something more low-level, which tries to
>> avoid (Python) function calls if possible, .....
>
> I have to confess the only piece of pep-0246 I read was:
>
>>
>>       Rejection Notice
>>
>>     I'm rejecting this PEP.  Something much better is about to happen;
>>     it's too early to say exactly what, but it's not going to resemble
>>     the proposal in this PEP too closely so it's better to start a new
>>     PEP.  GvR.
>>  
>
> Does anyone know what the "better" option was? The date on the pep is a
> number of years ago.

Not that I'm aware off.

Python has moved on to ABCs, which address part of the intended
use in a different way (the outside objects have to implement the
protocol and announce this using an ABC).

The adapter pattern is still useful, though. I just think that
we need to provide a way for database modules to implement
such adapter in straight C (as well as in Python) rather than
requiring a round-trip to Python in all cases.

The sketch I drew up would allow for this. Most database modules
use such C adapters in some form anyway, so these would just
need to be refactored a little bit to make them available
to the Python application, which should help with the adoption
of the new techniques.

--
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jul 20 2011)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/
_______________________________________________
DB-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/db-sig
Reply | Threaded
Open this post in threaded view
|

Re: Some obscurity with paramstyle

William Dode
In reply to this post by M.-A. Lemburg
On 18-07-2011, M.-A. Lemburg wrote:
>> IMHO I would prefer to see the DBAPI have exactly two paramstyles,
>> named and qmark, and have all DBAPIs support both consistently.    
>> The (py)format styles continuously introduce the mixing of Python's
>> string formatting behavior with the presentation of bound parameters,
>> which are two completely different things.
>
> +1
>

+1 also

Too many formats are a risk that user will not bother and use string
formatting.

To help we could provide reference implementation of current formats
converters. It's not a bottleneck since the parsing is done only one
time.


--
William Dodé - http://flibuste.net
Informaticien Indépendant

_______________________________________________
DB-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/db-sig
12