It's a nightmare to transfer SqlAlchemy Object with PyAMF

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

It's a nightmare to transfer SqlAlchemy Object with PyAMF

一首诗
We use PyAmf 0.4.2 in our server to transfer SqlAlchemy objects to an Action Script application.

It used to work but today I decided to add some business logic to a SA mapped class.
Then I found PyAmf can not transfer it anymore.  It complains that "callable" can not be encoded, but doesn't tell me which callable could not be encoded.
So I modified PyAMF's code to give me more information.

Then I found it's because there is a variable in SA object who's value is a class.
It took me a lot of time to move the variable out.

--------------

Then I try to use PyAMF's client function to transfer this SA object to another server.
Sadly, it fails again and complains about  could not get static attributes in a "[]".
How can that be possible?

I guess maybe I should use AMF3, so I build client like this:

    client = RemotingService(url, pyamf.AMF3)

It doesn't work, same exception.

I decided to update to PyAMF's newest version, which is PyAMF 5.1.

OK, this time I found it failed to transfer a dozen SA objects .....
Client side got a http 500 error without any detailed information.
No log, no exception ...

--------------
 
Tomorrow I shall write code to convert the SA object to an ordinary python dictionary and convert it back on the side.
If I had already known it's this hard to make it work, I would have finished writing this kind of conversion many hours ago.

--------------

For anymore who would want to try to transfer SA object by PyAMF,
I strongly suggest that only do that if your SA object is quite simple.

_______________________________________________
PyAMF users mailing list - [hidden email]
http://lists.pyamf.org/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: It's a nightmare to transfer SqlAlchemy Object with PyAMF

Simon Bierbaum-2
Since PyAMF 0.5, you can exclude attributes from being encoded using the __amf__ property:

http://pyamf.org/architecture/attributecontrol.html

Before, pyamf.register_class had parameters to control this (I don't have any docs on this, but I used it with PyAMF 0.3.1 and it worked fine. I can provide code if you need it).

As for the 'could not get static attributes in a "[]"' error, you might have a many-to-many relationship configured in SQLAlchemy which PyAMF is trying to encode. However, the many-to-many attribute from a mapper() is not a regular list but an _AssociationList instance IIRC, so PyAMF does not fail on a list but in fact on a private SQLAlchemy class. I'm not sure how PyAMF handles this correctly, maybe someone else can elaborate on that?

If you get a HTTP error 500, you might want to look into you webserver's error_log.

Cheers, Simon

Am 17.04.2010 um 20:02 schrieb Peter Cai:

> We use PyAmf 0.4.2 in our server to transfer SqlAlchemy objects to an Action Script application.
>
> It used to work but today I decided to add some business logic to a SA mapped class.
> Then I found PyAmf can not transfer it anymore.  It complains that "callable" can not be encoded, but doesn't tell me which callable could not be encoded.
> So I modified PyAMF's code to give me more information.
>
> Then I found it's because there is a variable in SA object who's value is a class.
> It took me a lot of time to move the variable out.
>
> --------------
>
> Then I try to use PyAMF's client function to transfer this SA object to another server.
> Sadly, it fails again and complains about  could not get static attributes in a "[]".
> How can that be possible?
>
> I guess maybe I should use AMF3, so I build client like this:
>
>     client = RemotingService(url, pyamf.AMF3)
>
> It doesn't work, same exception.
>
> I decided to update to PyAMF's newest version, which is PyAMF 5.1.
>
> OK, this time I found it failed to transfer a dozen SA objects .....
> Client side got a http 500 error without any detailed information.
> No log, no exception ...
>
> --------------
>  
> Tomorrow I shall write code to convert the SA object to an ordinary python dictionary and convert it back on the side.
> If I had already known it's this hard to make it work, I would have finished writing this kind of conversion many hours ago.
>
> --------------
>
> For anymore who would want to try to transfer SA object by PyAMF,
> I strongly suggest that only do that if your SA object is quite simple.
> _______________________________________________
> PyAMF users mailing list - [hidden email]
> http://lists.pyamf.org/mailman/listinfo/users

_______________________________________________
PyAMF users mailing list - [hidden email]
http://lists.pyamf.org/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: It's a nightmare to transfer SqlAlchemy Object with PyAMF

Nick Joyce
In reply to this post by 一首诗
On 17 Apr 2010, at 19:02, Peter Cai wrote:

> We use PyAmf 0.4.2 in our server to transfer SqlAlchemy objects to an Action Script application.

This is quite an old version and SQLAlchemy support was simple at best. What version of SQLAlchemy are you using? There are a couple of specific api changes between versions that PyAMF tires to compensate for but we may have missed some things ..

>
> It used to work but today I decided to add some business logic to a SA mapped class.
> Then I found PyAmf can not transfer it anymore.  It complains that "callable" can not be encoded, but doesn't tell me which callable could not be encoded.
> So I modified PyAMF's code to give me more information.

PyAMF should give you as much information as you need to quickly find the error .. if it does not then it is a bug.

> Then I found it's because there is a variable in SA object who's value is a class.
> It took me a lot of time to move the variable out.
>
> --------------
>
> Then I try to use PyAMF's client function to transfer this SA object to another server.
> Sadly, it fails again and complains about  could not get static attributes in a "[]".
> How can that be possible?

If indeed as Simon says, that this is an _AssociationList rather than a pure list, pyamf may well fail on this too. Can you give an example of how this is failing so that we can test and better support m2m relationships in the future?

>
> I guess maybe I should use AMF3, so I build client like this:
>
>     client = RemotingService(url, pyamf.AMF3)
>
> It doesn't work, same exception.
>
> I decided to update to PyAMF's newest version, which is PyAMF 5.1.
>
> OK, this time I found it failed to transfer a dozen SA objects .....
> Client side got a http 500 error without any detailed information.
> No log, no exception ...

There was a change from 0.4 to 0.5 that meant you had to specifically provide a logger argument to the gateway, e.g.:

import logging

services = {}

gw = WSGIGateway(services, logger=logging)

This would explain why you were getting no exceptions. Perhaps the documentation could be better in this area ..

>
> --------------
>  
> Tomorrow I shall write code to convert the SA object to an ordinary python dictionary and convert it back on the side.
> If I had already known it's this hard to make it work, I would have finished writing this kind of conversion many hours ago.

I am sorry you feel frustrated with the lack of progress you are making :-( Providing excellent support SQLAlchemy is a goal of ours.

PyAMF 0.6 will provide further support, including non required static attributes (making multiple inheritance much easier) and support for the newer 0.6 line of SQLAlchemy.

>
> --------------
>
> For anymore who would want to try to transfer SA object by PyAMF,
> I strongly suggest that only do that if your SA object is quite simple.

I have built a couple of projects using PyAMF/SQLAlchemy and found the integration to work quite well - can you share more about your model so that I can see where we are failing?

Thanks!

_______________________________________________
PyAMF users mailing list - [hidden email]
http://lists.pyamf.org/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: It's a nightmare to transfer SqlAlchemy Object with PyAMF

一首诗
Maybe I am in a hurry to finish the job yesterday so I did' t think
carefully think about the problem
and not patient enough waiting for help from community.

Working at the weekend always made me nervous.

Simon might be right about the [] problem, there is m2m in that SA I
want to transfer.
But lazy load helped me to avoid cycles in objects I want to transfer.
And theses data could be send to Ascript Client.
They just don't work when PyAMF behaves as client not server.

Here's the call stack:

    --- <exception caught here> ---
      File "/usr/lib/python2.4/site-packages/twisted/internet/threads.py",
line 26, in _putResultInDeferred
        result = f(*args, **kwargs)
      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/remoting/client/__init__.py",
line 52, in __call__
        return self.service._call(self, *args)
      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/remoting/client/__init__.py",
line 101, in _call
        response = self._gw.execute_single(request)
      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/remoting/client/__init__.py",
line 401, in execute_single
        body = remoting.encode(self.getAMFRequest([request]),
strict=self.strict)
      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/remoting/__init__.py",
line 679, in encode
        _write_body(name, message, stream, encoder, strict)
      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/remoting/__init__.py",
line 520, in _write_body
        _encode_body(message)
      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/remoting/__init__.py",
line 493, in _encode_body
        encoder.writeElement(x)
      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/amf0.py",
line 534, in writeElement
        func(data)
      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/amf0.py",
line 578, in writeArray
        self.writeElement(data)
      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/amf0.py",
line 534, in writeElement
        func(data)
      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/amf0.py",
line 760, in writeObject
        self.writeElement(value)
      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/amf0.py",
line 534, in writeElement
        func(data)
      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/__init__.py",
line 698, in __call__
        self.encoder.writeElement(self.func(data, encoder=self.encoder))
      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/amf0.py",
line 534, in writeElement
        func(data)
      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/amf0.py",
line 578, in writeArray
        self.writeElement(data)
      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/amf0.py",
line 534, in writeElement
        func(data)
      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/amf0.py",
line 760, in writeObject
        self.writeElement(value)
      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/amf0.py",
line 534, in writeElement
        func(data)
      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/amf0.py",
line 760, in writeObject
        self.writeElement(value)
      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/amf0.py",
line 534, in writeElement
        func(data)
      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/amf0.py",
line 756, in writeObject
        obj_attrs = self._getObjectAttrs(o, alias)
      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/amf0.py",
line 718, in _getObjectAttrs
        raise pyamf.EncodeError('Unable to determine object attributes')
    pyamf.EncodeError: Unable to determine object attributes


You can see, it doesn't tell which object it could not "determine
object attributes".  I modified
       raise pyamf.EncodeError('Unable to determine object attributes')
to
       raise pyamf.EncodeError('Unable to determine object attributes %s' % o)
and found o is actually a [].

Every time I got a encode or decode error, I have to modify PyAMF to
find which object or attribute caused the problem.

Anyway, PyAMF does help me in the past, so I should do something for it:
Here is all my SA model.  You can use them to made some "out of lab" tests.

http://dl.dropbox.com/u/4420645/blf.7z

If you want to reproduce the error, create some
blf.model.blfdb.nvr.Nvr and try to transfer them as client.
Let me know if you found something.

On Sun, Apr 18, 2010 at 4:50 AM, Nick Joyce <[hidden email]> wrote:

>
> On 17 Apr 2010, at 19:02, Peter Cai wrote:
>
> > We use PyAmf 0.4.2 in our server to transfer SqlAlchemy objects to an Action Script application.
>
> This is quite an old version and SQLAlchemy support was simple at best. What version of SQLAlchemy are you using? There are a couple of specific api changes between versions that PyAMF tires to compensate for but we may have missed some things ..
>
> >
> > It used to work but today I decided to add some business logic to a SA mapped class.
> > Then I found PyAmf can not transfer it anymore.  It complains that "callable" can not be encoded, but doesn't tell me which callable could not be encoded.
> > So I modified PyAMF's code to give me more information.
>
> PyAMF should give you as much information as you need to quickly find the error .. if it does not then it is a bug.
>
> > Then I found it's because there is a variable in SA object who's value is a class.
> > It took me a lot of time to move the variable out.
> >
> > --------------
> >
> > Then I try to use PyAMF's client function to transfer this SA object to another server.
> > Sadly, it fails again and complains about  could not get static attributes in a "[]".
> > How can that be possible?
>
> If indeed as Simon says, that this is an _AssociationList rather than a pure list, pyamf may well fail on this too. Can you give an example of how this is failing so that we can test and better support m2m relationships in the future?
>
> >
> > I guess maybe I should use AMF3, so I build client like this:
> >
> >     client = RemotingService(url, pyamf.AMF3)
> >
> > It doesn't work, same exception.
> >
> > I decided to update to PyAMF's newest version, which is PyAMF 5.1.
> >
> > OK, this time I found it failed to transfer a dozen SA objects .....
> > Client side got a http 500 error without any detailed information.
> > No log, no exception ...
>
> There was a change from 0.4 to 0.5 that meant you had to specifically provide a logger argument to the gateway, e.g.:
>
> import logging
>
> services = {}
>
> gw = WSGIGateway(services, logger=logging)
>
> This would explain why you were getting no exceptions. Perhaps the documentation could be better in this area ..
>
> >
> > --------------
> >
> > Tomorrow I shall write code to convert the SA object to an ordinary python dictionary and convert it back on the side.
> > If I had already known it's this hard to make it work, I would have finished writing this kind of conversion many hours ago.
>
> I am sorry you feel frustrated with the lack of progress you are making :-( Providing excellent support SQLAlchemy is a goal of ours.
>
> PyAMF 0.6 will provide further support, including non required static attributes (making multiple inheritance much easier) and support for the newer 0.6 line of SQLAlchemy.
>
> >
> > --------------
> >
> > For anymore who would want to try to transfer SA object by PyAMF,
> > I strongly suggest that only do that if your SA object is quite simple.
>
> I have built a couple of projects using PyAMF/SQLAlchemy and found the integration to work quite well - can you share more about your model so that I can see where we are failing?
>
> Thanks!
>
> _______________________________________________
> PyAMF users mailing list - [hidden email]
> http://lists.pyamf.org/mailman/listinfo/users



--
look to the things around you,the immediate world around you, if you
are alive,it will mean something to you ——Paul Strand
_______________________________________________
PyAMF users mailing list - [hidden email]
http://lists.pyamf.org/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: It's a nightmare to transfer SqlAlchemy Object with PyAMF

Thijs Triemstra
Could you try to upgrade to 0.5.1, you're using a very old version of PyAMF..

Thijs

On 18 Apr 2010, at 04:08, Peter Cai wrote:

> Maybe I am in a hurry to finish the job yesterday so I did' t think
> carefully think about the problem
> and not patient enough waiting for help from community.
>
> Working at the weekend always made me nervous.
>
> Simon might be right about the [] problem, there is m2m in that SA I
> want to transfer.
> But lazy load helped me to avoid cycles in objects I want to transfer.
> And theses data could be send to Ascript Client.
> They just don't work when PyAMF behaves as client not server.
>
> Here's the call stack:
>
>    --- <exception caught here> ---
>      File "/usr/lib/python2.4/site-packages/twisted/internet/threads.py",
> line 26, in _putResultInDeferred
>        result = f(*args, **kwargs)
>      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/remoting/client/__init__.py",
> line 52, in __call__
>        return self.service._call(self, *args)
>      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/remoting/client/__init__.py",
> line 101, in _call
>        response = self._gw.execute_single(request)
>      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/remoting/client/__init__.py",
> line 401, in execute_single
>        body = remoting.encode(self.getAMFRequest([request]),
> strict=self.strict)
>      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/remoting/__init__.py",
> line 679, in encode
>        _write_body(name, message, stream, encoder, strict)
>      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/remoting/__init__.py",
> line 520, in _write_body
>        _encode_body(message)
>      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/remoting/__init__.py",
> line 493, in _encode_body
>        encoder.writeElement(x)
>      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/amf0.py",
> line 534, in writeElement
>        func(data)
>      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/amf0.py",
> line 578, in writeArray
>        self.writeElement(data)
>      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/amf0.py",
> line 534, in writeElement
>        func(data)
>      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/amf0.py",
> line 760, in writeObject
>        self.writeElement(value)
>      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/amf0.py",
> line 534, in writeElement
>        func(data)
>      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/__init__.py",
> line 698, in __call__
>        self.encoder.writeElement(self.func(data, encoder=self.encoder))
>      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/amf0.py",
> line 534, in writeElement
>        func(data)
>      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/amf0.py",
> line 578, in writeArray
>        self.writeElement(data)
>      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/amf0.py",
> line 534, in writeElement
>        func(data)
>      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/amf0.py",
> line 760, in writeObject
>        self.writeElement(value)
>      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/amf0.py",
> line 534, in writeElement
>        func(data)
>      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/amf0.py",
> line 760, in writeObject
>        self.writeElement(value)
>      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/amf0.py",
> line 534, in writeElement
>        func(data)
>      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/amf0.py",
> line 756, in writeObject
>        obj_attrs = self._getObjectAttrs(o, alias)
>      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/amf0.py",
> line 718, in _getObjectAttrs
>        raise pyamf.EncodeError('Unable to determine object attributes')
>    pyamf.EncodeError: Unable to determine object attributes
>
>
> You can see, it doesn't tell which object it could not "determine
> object attributes".  I modified
>       raise pyamf.EncodeError('Unable to determine object attributes')
> to
>       raise pyamf.EncodeError('Unable to determine object attributes %s' % o)
> and found o is actually a [].
>
> Every time I got a encode or decode error, I have to modify PyAMF to
> find which object or attribute caused the problem.
>
> Anyway, PyAMF does help me in the past, so I should do something for it:
> Here is all my SA model.  You can use them to made some "out of lab" tests.
>
> http://dl.dropbox.com/u/4420645/blf.7z
>
> If you want to reproduce the error, create some
> blf.model.blfdb.nvr.Nvr and try to transfer them as client.
> Let me know if you found something.
>
> On Sun, Apr 18, 2010 at 4:50 AM, Nick Joyce <[hidden email]> wrote:
>>
>> On 17 Apr 2010, at 19:02, Peter Cai wrote:
>>
>>> We use PyAmf 0.4.2 in our server to transfer SqlAlchemy objects to an Action Script application.
>>
>> This is quite an old version and SQLAlchemy support was simple at best. What version of SQLAlchemy are you using? There are a couple of specific api changes between versions that PyAMF tires to compensate for but we may have missed some things ..
>>
>>>
>>> It used to work but today I decided to add some business logic to a SA mapped class.
>>> Then I found PyAmf can not transfer it anymore.  It complains that "callable" can not be encoded, but doesn't tell me which callable could not be encoded.
>>> So I modified PyAMF's code to give me more information.
>>
>> PyAMF should give you as much information as you need to quickly find the error .. if it does not then it is a bug.
>>
>>> Then I found it's because there is a variable in SA object who's value is a class.
>>> It took me a lot of time to move the variable out.
>>>
>>> --------------
>>>
>>> Then I try to use PyAMF's client function to transfer this SA object to another server.
>>> Sadly, it fails again and complains about  could not get static attributes in a "[]".
>>> How can that be possible?
>>
>> If indeed as Simon says, that this is an _AssociationList rather than a pure list, pyamf may well fail on this too. Can you give an example of how this is failing so that we can test and better support m2m relationships in the future?
>>
>>>
>>> I guess maybe I should use AMF3, so I build client like this:
>>>
>>>     client = RemotingService(url, pyamf.AMF3)
>>>
>>> It doesn't work, same exception.
>>>
>>> I decided to update to PyAMF's newest version, which is PyAMF 5.1.
>>>
>>> OK, this time I found it failed to transfer a dozen SA objects .....
>>> Client side got a http 500 error without any detailed information.
>>> No log, no exception ...
>>
>> There was a change from 0.4 to 0.5 that meant you had to specifically provide a logger argument to the gateway, e.g.:
>>
>> import logging
>>
>> services = {}
>>
>> gw = WSGIGateway(services, logger=logging)
>>
>> This would explain why you were getting no exceptions. Perhaps the documentation could be better in this area ..
>>
>>>
>>> --------------
>>>
>>> Tomorrow I shall write code to convert the SA object to an ordinary python dictionary and convert it back on the side.
>>> If I had already known it's this hard to make it work, I would have finished writing this kind of conversion many hours ago.
>>
>> I am sorry you feel frustrated with the lack of progress you are making :-( Providing excellent support SQLAlchemy is a goal of ours.
>>
>> PyAMF 0.6 will provide further support, including non required static attributes (making multiple inheritance much easier) and support for the newer 0.6 line of SQLAlchemy.
>>
>>>
>>> --------------
>>>
>>> For anymore who would want to try to transfer SA object by PyAMF,
>>> I strongly suggest that only do that if your SA object is quite simple.
>>
>> I have built a couple of projects using PyAMF/SQLAlchemy and found the integration to work quite well - can you share more about your model so that I can see where we are failing?
>>
>> Thanks!
>>
>> _______________________________________________
>> PyAMF users mailing list - [hidden email]
>> http://lists.pyamf.org/mailman/listinfo/users
>
>
>
> --
> look to the things around you,the immediate world around you, if you
> are alive,it will mean something to you ——Paul Strand
> _______________________________________________
> PyAMF users mailing list - [hidden email]
> http://lists.pyamf.org/mailman/listinfo/users


_______________________________________________
PyAMF users mailing list - [hidden email]
http://lists.pyamf.org/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: It's a nightmare to transfer SqlAlchemy Object with PyAMF

一首诗
I can't take the risk of breaking other parts the our application.
I've already finished the code of convert SA objects to dictionary, that works very well.

On Tue, Apr 20, 2010 at 5:11 AM, Thijs Triemstra | Collab <[hidden email]> wrote:
Could you try to upgrade to 0.5.1, you're using a very old version of PyAMF..

Thijs

On 18 Apr 2010, at 04:08, Peter Cai wrote:

> Maybe I am in a hurry to finish the job yesterday so I did' t think
> carefully think about the problem
> and not patient enough waiting for help from community.
>
> Working at the weekend always made me nervous.
>
> Simon might be right about the [] problem, there is m2m in that SA I
> want to transfer.
> But lazy load helped me to avoid cycles in objects I want to transfer.
> And theses data could be send to Ascript Client.
> They just don't work when PyAMF behaves as client not server.
>
> Here's the call stack:
>
>    --- <exception caught here> ---
>      File "/usr/lib/python2.4/site-packages/twisted/internet/threads.py",
> line 26, in _putResultInDeferred
>        result = f(*args, **kwargs)
>      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/remoting/client/__init__.py",
> line 52, in __call__
>        return self.service._call(self, *args)
>      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/remoting/client/__init__.py",
> line 101, in _call
>        response = self._gw.execute_single(request)
>      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/remoting/client/__init__.py",
> line 401, in execute_single
>        body = remoting.encode(self.getAMFRequest([request]),
> strict=self.strict)
>      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/remoting/__init__.py",
> line 679, in encode
>        _write_body(name, message, stream, encoder, strict)
>      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/remoting/__init__.py",
> line 520, in _write_body
>        _encode_body(message)
>      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/remoting/__init__.py",
> line 493, in _encode_body
>        encoder.writeElement(x)
>      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/amf0.py",
> line 534, in writeElement
>        func(data)
>      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/amf0.py",
> line 578, in writeArray
>        self.writeElement(data)
>      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/amf0.py",
> line 534, in writeElement
>        func(data)
>      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/amf0.py",
> line 760, in writeObject
>        self.writeElement(value)
>      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/amf0.py",
> line 534, in writeElement
>        func(data)
>      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/__init__.py",
> line 698, in __call__
>        self.encoder.writeElement(self.func(data, encoder=self.encoder))
>      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/amf0.py",
> line 534, in writeElement
>        func(data)
>      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/amf0.py",
> line 578, in writeArray
>        self.writeElement(data)
>      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/amf0.py",
> line 534, in writeElement
>        func(data)
>      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/amf0.py",
> line 760, in writeObject
>        self.writeElement(value)
>      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/amf0.py",
> line 534, in writeElement
>        func(data)
>      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/amf0.py",
> line 760, in writeObject
>        self.writeElement(value)
>      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/amf0.py",
> line 534, in writeElement
>        func(data)
>      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/amf0.py",
> line 756, in writeObject
>        obj_attrs = self._getObjectAttrs(o, alias)
>      File "/usr/lib/python2.4/site-packages/PyAMF-0.4.2-py2.4-linux-i686.egg/pyamf/amf0.py",
> line 718, in _getObjectAttrs
>        raise pyamf.EncodeError('Unable to determine object attributes')
>    pyamf.EncodeError: Unable to determine object attributes
>
>
> You can see, it doesn't tell which object it could not "determine
> object attributes".  I modified
>       raise pyamf.EncodeError('Unable to determine object attributes')
> to
>       raise pyamf.EncodeError('Unable to determine object attributes %s' % o)
> and found o is actually a [].
>
> Every time I got a encode or decode error, I have to modify PyAMF to
> find which object or attribute caused the problem.
>
> Anyway, PyAMF does help me in the past, so I should do something for it:
> Here is all my SA model.  You can use them to made some "out of lab" tests.
>
> http://dl.dropbox.com/u/4420645/blf.7z
>
> If you want to reproduce the error, create some
> blf.model.blfdb.nvr.Nvr and try to transfer them as client.
> Let me know if you found something.
>
> On Sun, Apr 18, 2010 at 4:50 AM, Nick Joyce <[hidden email]> wrote:
>>
>> On 17 Apr 2010, at 19:02, Peter Cai wrote:
>>
>>> We use PyAmf 0.4.2 in our server to transfer SqlAlchemy objects to an Action Script application.
>>
>> This is quite an old version and SQLAlchemy support was simple at best. What version of SQLAlchemy are you using? There are a couple of specific api changes between versions that PyAMF tires to compensate for but we may have missed some things ..
>>
>>>
>>> It used to work but today I decided to add some business logic to a SA mapped class.
>>> Then I found PyAmf can not transfer it anymore.  It complains that "callable" can not be encoded, but doesn't tell me which callable could not be encoded.
>>> So I modified PyAMF's code to give me more information.
>>
>> PyAMF should give you as much information as you need to quickly find the error .. if it does not then it is a bug.
>>
>>> Then I found it's because there is a variable in SA object who's value is a class.
>>> It took me a lot of time to move the variable out.
>>>
>>> --------------
>>>
>>> Then I try to use PyAMF's client function to transfer this SA object to another server.
>>> Sadly, it fails again and complains about  could not get static attributes in a "[]".
>>> How can that be possible?
>>
>> If indeed as Simon says, that this is an _AssociationList rather than a pure list, pyamf may well fail on this too. Can you give an example of how this is failing so that we can test and better support m2m relationships in the future?
>>
>>>
>>> I guess maybe I should use AMF3, so I build client like this:
>>>
>>>     client = RemotingService(url, pyamf.AMF3)
>>>
>>> It doesn't work, same exception.
>>>
>>> I decided to update to PyAMF's newest version, which is PyAMF 5.1.
>>>
>>> OK, this time I found it failed to transfer a dozen SA objects .....
>>> Client side got a http 500 error without any detailed information.
>>> No log, no exception ...
>>
>> There was a change from 0.4 to 0.5 that meant you had to specifically provide a logger argument to the gateway, e.g.:
>>
>> import logging
>>
>> services = {}
>>
>> gw = WSGIGateway(services, logger=logging)
>>
>> This would explain why you were getting no exceptions. Perhaps the documentation could be better in this area ..
>>
>>>
>>> --------------
>>>
>>> Tomorrow I shall write code to convert the SA object to an ordinary python dictionary and convert it back on the side.
>>> If I had already known it's this hard to make it work, I would have finished writing this kind of conversion many hours ago.
>>
>> I am sorry you feel frustrated with the lack of progress you are making :-( Providing excellent support SQLAlchemy is a goal of ours.
>>
>> PyAMF 0.6 will provide further support, including non required static attributes (making multiple inheritance much easier) and support for the newer 0.6 line of SQLAlchemy.
>>
>>>
>>> --------------
>>>
>>> For anymore who would want to try to transfer SA object by PyAMF,
>>> I strongly suggest that only do that if your SA object is quite simple.
>>
>> I have built a couple of projects using PyAMF/SQLAlchemy and found the integration to work quite well - can you share more about your model so that I can see where we are failing?
>>
>> Thanks!
>>
>> _______________________________________________
>> PyAMF users mailing list - [hidden email]
>> http://lists.pyamf.org/mailman/listinfo/users
>
>
>
> --
> look to the things around you,the immediate world around you, if you
> are alive,it will mean something to you ——Paul Strand
> _______________________________________________
> PyAMF users mailing list - [hidden email]
> http://lists.pyamf.org/mailman/listinfo/users


_______________________________________________
PyAMF users mailing list - [hidden email]
http://lists.pyamf.org/mailman/listinfo/users



--
look to the things around you,the immediate world around you, if you are alive,it will mean something to you ——Paul Strand

_______________________________________________
PyAMF users mailing list - [hidden email]
http://lists.pyamf.org/mailman/listinfo/users