Accessing DataSocket Server with Python

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Accessing DataSocket Server with Python

Garrone, Corrado
Dear Python Team,

currently I am working on a research project for my bachelor degree. A
LabVIEW application is used for current and power measurements, whereas the
measured data are sent to DataSocket Server, a technology by National
Instruments used for data exchange between computers and applications.
DataSocket is based on TCP/IP and thus requesting data from DataSocket
should be similar to an internet request.
I know with the socket library in Python it is possible with to establish
sockets, send internet requests and communicate between clients and
servers.
Is there a possibility to access NI DataSocket and get measurement data
with Python on the same computer where Python is installed and the codes
are executed? Can you maybe send me an example code where such a connection
with DataSocket is established?

If you got any queries, please do not hesitate to contact me.

Thank you very much for your efforts.

Kind regards,

 *Corrado Garrone*
DH-Student Fachrichtung Elektrotechnik / Co-op Student B.Eng. Electrical
Engineering

Roche Diagnostics GmbH
DFGHMV8Y6164
Sandhofer Strasse 116
68305 Mannheim / Germany

Phone: apprentice
mailto:corrado.garrone at roche.com <corrado.garrone at roche.com>

 *Roche Diagnostics GmbH*
Sandhofer Stra?e 116; D?68305 Mannheim; Telefon +49?621?759?0;
Telefax +49?621?759?2890
Sitz der Gesellschaft: Mannheim - Registergericht: AG Mannheim HRB 3962 -
Gesch?ftsf?hrung: Dr. Ursula Redeker, Sprecherin; Edgar Vieth -
Aufsichtsratsvorsitzender: Dr. Severin Schwan

 *Confidentiality Note*
This message is intended only for the use of the named recipient(s) and may
contain confidential and/or privileged information. If you are not the
intended recipient, please contact the sender and delete the message. Any
unauthorized use of the information contained in this message is prohibited.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150527/34a12c5f/attachment.html>

Reply | Threaded
Open this post in threaded view
|

Accessing DataSocket Server with Python

Dan Stromberg-2
I have no idea about the protocol used by NI DataSockets, but you
might be able to reverse engineer the protocol by using the official
client with a sniffer.

Also, be aware that TCP/IP guarantees that you get the correct data in
the correct order, but it doesn't guarantee anything about the sizes
of the chunks in which that data arrives.  So you could send 100
bytes, 100 bytes, 100 bytes, but on the other end receive 100 bytes,
50 bytes, 75 bytes, 75 bytes.  When you catenate them all together,
it's still the same data though.

HTH.

On Wed, May 27, 2015 at 4:30 AM, Garrone, Corrado
<corrado.garrone at roche.com> wrote:

> Dear Python Team,
>
> currently I am working on a research project for my bachelor degree. A
> LabVIEW application is used for current and power measurements, whereas the
> measured data are sent to DataSocket Server, a technology by National
> Instruments used for data exchange between computers and applications.
> DataSocket is based on TCP/IP and thus requesting data from DataSocket
> should be similar to an internet request.
> I know with the socket library in Python it is possible with to establish
> sockets, send internet requests and communicate between clients and servers.
> Is there a possibility to access NI DataSocket and get measurement data with
> Python on the same computer where Python is installed and the codes are
> executed? Can you maybe send me an example code where such a connection with
> DataSocket is established?
>
> If you got any queries, please do not hesitate to contact me.
>
> Thank you very much for your efforts.
>
> Kind regards,
>
> Corrado Garrone
> DH-Student Fachrichtung Elektrotechnik / Co-op Student B.Eng. Electrical
> Engineering
>
> Roche Diagnostics GmbH
> DFGHMV8Y6164
> Sandhofer Strasse 116
> 68305 Mannheim / Germany
>
> Phone: apprentice
> mailto:corrado.garrone at roche.com
>
> Roche Diagnostics GmbH
> Sandhofer Stra?e 116; D?68305 Mannheim; Telefon +49?621?759?0; Telefax
> +49?621?759?2890
> Sitz der Gesellschaft: Mannheim - Registergericht: AG Mannheim HRB 3962 -
> Gesch?ftsf?hrung: Dr. Ursula Redeker, Sprecherin; Edgar Vieth -
> Aufsichtsratsvorsitzender: Dr. Severin Schwan
>
> Confidentiality Note
> This message is intended only for the use of the named recipient(s) and may
> contain confidential and/or privileged information. If you are not the
> intended recipient, please contact the sender and delete the message. Any
> unauthorized use of the information contained in this message is prohibited.
>
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>


Reply | Threaded
Open this post in threaded view
|

Accessing DataSocket Server with Python

William Ray Wing

> On May 28, 2015, at 6:17 PM, Dan Stromberg <drsalists at gmail.com> wrote:
>
> I have no idea about the protocol used by NI DataSockets, but you
> might be able to reverse engineer the protocol by using the official
> client with a sniffer.
>
> Also, be aware that TCP/IP guarantees that you get the correct data in
> the correct order, but it doesn't guarantee anything about the sizes
> of the chunks in which that data arrives.  So you could send 100
> bytes, 100 bytes, 100 bytes, but on the other end receive 100 bytes,
> 50 bytes, 75 bytes, 75 bytes.  When you catenate them all together,
> it's still the same data though.
>
> HTH.
>
> On Wed, May 27, 2015 at 4:30 AM, Garrone, Corrado

While that?s certainly possible in a routed network (and even then can be overridden with the ?do not fragment? bit), it won?t happen in a LAN or self-contained instrument set-up.  These days, even routed networks tend to deliver anything less than a 1500 byte packet as a single entity.  With fiber backbones and high-speed LANs, it is more work for a router to fragment a packet then to simply pass it on.  The days of 480 byte packets pretty much went away with dial-up modems.

Bill



> <corrado.garrone at roche.com> wrote:
>> Dear Python Team,
>>
>> currently I am working on a research project for my bachelor degree. A
>> LabVIEW application is used for current and power measurements, whereas the
>> measured data are sent to DataSocket Server, a technology by National
>> Instruments used for data exchange between computers and applications.
>> DataSocket is based on TCP/IP and thus requesting data from DataSocket
>> should be similar to an internet request.
>> I know with the socket library in Python it is possible with to establish
>> sockets, send internet requests and communicate between clients and servers.
>> Is there a possibility to access NI DataSocket and get measurement data with
>> Python on the same computer where Python is installed and the codes are
>> executed? Can you maybe send me an example code where such a connection with
>> DataSocket is established?
>>
>> If you got any queries, please do not hesitate to contact me.
>>
>> Thank you very much for your efforts.
>>
>> Kind regards,
>>
>> Corrado Garrone
>> DH-Student Fachrichtung Elektrotechnik / Co-op Student B.Eng. Electrical
>> Engineering
>>
>> Roche Diagnostics GmbH
>> DFGHMV8Y6164
>> Sandhofer Strasse 116
>> 68305 Mannheim / Germany
>>
>> Phone: apprentice
>> mailto:corrado.garrone at roche.com
>>
>> Roche Diagnostics GmbH
>> Sandhofer Stra?e 116; D?68305 Mannheim; Telefon +49?621?759?0; Telefax
>> +49?621?759?2890
>> Sitz der Gesellschaft: Mannheim - Registergericht: AG Mannheim HRB 3962 -
>> Gesch?ftsf?hrung: Dr. Ursula Redeker, Sprecherin; Edgar Vieth -
>> Aufsichtsratsvorsitzender: Dr. Severin Schwan
>>
>> Confidentiality Note
>> This message is intended only for the use of the named recipient(s) and may
>> contain confidential and/or privileged information. If you are not the
>> intended recipient, please contact the sender and delete the message. Any
>> unauthorized use of the information contained in this message is prohibited.
>>
>>
>> --
>> https://mail.python.org/mailman/listinfo/python-list
>>
> --
> https://mail.python.org/mailman/listinfo/python-list



Reply | Threaded
Open this post in threaded view
|

Accessing DataSocket Server with Python

Chris Angelico
On Fri, May 29, 2015 at 11:37 PM, William Ray Wing <wrw at mac.com> wrote:

>> On May 28, 2015, at 6:17 PM, Dan Stromberg <drsalists at gmail.com> wrote:
>>
>> I have no idea about the protocol used by NI DataSockets, but you
>> might be able to reverse engineer the protocol by using the official
>> client with a sniffer.
>>
>> Also, be aware that TCP/IP guarantees that you get the correct data in
>> the correct order, but it doesn't guarantee anything about the sizes
>> of the chunks in which that data arrives.  So you could send 100
>> bytes, 100 bytes, 100 bytes, but on the other end receive 100 bytes,
>> 50 bytes, 75 bytes, 75 bytes.  When you catenate them all together,
>> it's still the same data though.
>>
>> HTH.
>>
>> On Wed, May 27, 2015 at 4:30 AM, Garrone, Corrado
>
> While that?s certainly possible in a routed network (and even then can be overridden with the ?do not fragment? bit), it won?t happen in a LAN or self-contained instrument set-up.  These days, even routed networks tend to deliver anything less than a 1500 byte packet as a single entity.  With fiber backbones and high-speed LANs, it is more work for a router to fragment a packet then to simply pass it on.  The days of 480 byte packets pretty much went away with dial-up modems.
>

It's uncommon to receive 100, 50, 75, 75, to be sure, but it's
certainly possible for multiple writes to be gathered together into
single reads; there are buffers at both ends, and if there's any delay
anywhere, stuff can get combined while it's waiting. (Even stuff that
had previously been sent out as multiple packets, if the data has to
be re-sent, might get grouped.) So you still need to consider your
protocol to be a stream, and delimit messages explicitly rather than
depending on the length of a socket-read. Most of these kinds of
protocols seem to have some kind of message boundary system, though,
so this shouldn't be a problem. (I'm not familiar with DataSocket
itself, but I've used several others that are like that. Even
venerable TELNET effectively does - a "message" is a line of text, so
it's delimited by newline.)

ChrisA


Reply | Threaded
Open this post in threaded view
|

Accessing DataSocket Server with Python

Grant Edwards-7
In reply to this post by Dan Stromberg-2
On 2015-05-29, William Ray Wing <wrw at mac.com> wrote:

> While that?s certainly possible in a routed network (and even then
> can be overridden with the ?do not fragment? bit), it won?t happen in
> a LAN or self-contained instrument set-up.

You don't know that.

> These days, even routed networks tend to deliver anything less than a
> 1500 byte packet as a single entity.

Doesn't matter.

It can still happen in the network stack on either end.  If you
quickly call send() which small amounts of data, it's quite possible
that the network stack will combine them into a single packet.  Under
some conditions this is done intentionally to reduce the overall
network overhead.

On the receiving end, if several small packets are received between
calls to recv(), then a single call to recv() will return data from
multiple packets.

> With fiber backbones and high-speed LANs, it is more work for a
> router to fragment a packet then to simply pass it on.  The days of
> 480 byte packets pretty much went away with dial-up modems.

But modern TCP/IP stacks will still combine packets to save on network
overhead.

If you assume TCP read/write operations are atomic and "message"
boundaries are preserved, your code is wrong.  It will eventually
fail.  Period.

--
Grant Edwards               grant.b.edwards        Yow! Inside, I'm already
                                  at               SOBBING!
                              gmail.com            


Reply | Threaded
Open this post in threaded view
|

Accessing DataSocket Server with Python

Chris Angelico
On Sat, May 30, 2015 at 2:29 AM, Grant Edwards <invalid at invalid.invalid> wrote:
> If you assume TCP read/write operations are atomic and "message"
> boundaries are preserved, your code is wrong.  It will eventually
> fail.  Period.

Indeed. That said, though, if your writes are all smaller than one
packet, and you perfectly alternate a write and a read, a write and a
read, at both ends, then you can go a very long way without ever
running into this. In that case, you could have proper parsing and
buffering as a failure case, with the normal case expecting a
termination mark at the end of the socket-read - something like this:

write(query)
data = read(as_much_as_possible)
if data[-1] != "\n":
    # Huh, stuff got split.
    while True:
        more_data = read(as_much_as_possible)
        data += moredata
        if data[-1] == "\n": break
        cope_with_possibility_of(socket_disconnection)
handle_response(data)

Alternating writes and reads ensures that two messages can't be
combined, and if a single message fits inside a packet, it'll usually
be sent that way. But you still can't guarantee that.

ChrisA


Reply | Threaded
Open this post in threaded view
|

Accessing DataSocket Server with Python

Grant Edwards-7
In reply to this post by Grant Edwards-7
On 2015-05-29, Chris Angelico <rosuav at gmail.com> wrote:

> On Sat, May 30, 2015 at 2:29 AM, Grant Edwards <invalid at invalid.invalid> wrote:
>
>> If you assume TCP read/write operations are atomic and "message"
>> boundaries are preserved, your code is wrong.  It will eventually
>> fail.  Period.
>
> Indeed. That said, though, if your writes are all smaller than one
> packet, and you perfectly alternate a write and a read, a write and a
> read, at both ends, then you can go a very long way without ever
> running into this.

That's true.  You probably won't see a failure until you do something
like run through some sort of WAN connection (satellite and PPP links
are excellent for exposing bad network code), or somebody configures a
switch, router, or IP interface in a goofy (but entire valid) way, or
somebody changes the app such that it writes more than 1500 bytes, or
it violates the strict alternation of reads/writes.

But someday, somehow, (IME) one of those always happens. And whoever
has to fix it will wish bad things upon you and your descendants.

I've lost count of the times I've had to fix somebody else's code that
broke because they assumed TCP was a datagram service rather than a
byte-stream service.

--
Grant Edwards               grant.b.edwards        Yow! UH-OH!!  I put on
                                  at               "GREAT HEAD-ON TRAIN
                              gmail.com            COLLISIONS of the 50's"
                                                   by mistake!!!


Reply | Threaded
Open this post in threaded view
|

Accessing DataSocket Server with Python

Marko Rauhamaa
In reply to this post by Grant Edwards-7
Chris Angelico <rosuav at gmail.com>:

> Indeed. That said, though, if your writes are all smaller than one
> packet, and you perfectly alternate a write and a read, a write and a
> read, at both ends, then you can go a very long way without ever
> running into this.

Rare errors are worse than consistent errors.

TCP streams are intercepted by address translators, load balancers,
proxies, protocol translators and what not. They will eagerly take
advantage of the fact that TCP is a full-duplex octet stream.

Also, programs could be plugged into other types of octet stream with
other stream chopping characteristics.

Finally, the MTU size is very unpredictable. You have all kinds of
encapsulation (6to4, VPN, VLAN, tunneling etc) that make it difficult to
be sure about what's a "small" write.

What's more, you should be on the lookout for partial writes not block
until they finish. Typically, it is not enough to rely on TCP's flow
control but also impose a separate application-level, end-to-end flow
control to avoid deadlocks on the one hand and buffer overflows on the
other hand.


Marko