tonights meeting - (my feelings won't be hurt if you delete)

classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

tonights meeting - (my feelings won't be hurt if you delete)

snowwrite
Nice to meet everyone!! (Stephen McInerney I meant to leave you the rest
of the strawberries!! but that's ok I just ate the rest of them..
heehee..mm yummy)

Just a couple things I was thinking about when I got home.. I'm a part
of the Linux Picnic volunteer group... every year it's the Saturday
AFTER the Linux World Conference in SF...this year the Linux Picnic is
August 19, 2006 ... why don't we plan on going together as a group?? Or
at least meet each other there...
Just an idea for a social event outside of our meetings.

It was the first time I've attended a meeting in a year.. last time was
Alex's presentation last May..(I can't believe it's been THAT long)...

Dennis.. the mapping and random access was absolutely awesome, we
definitely need to keep that going at every meeting (my good friend
Stephen Hindle was a first time attendee and he thoroughly enjoyed that
part of the meeting).

Stephen, the data you provided us was meticulous and it was obvious you
put a lot of hard work into gathering these statistics for us, Thank
You! I know we didn't have an opportunity to discuss your findings but
it definitely left us much to think about in the coming months as we
plan out the meetings.

I bought toooo much food! But that's ok..I have six kids ages 9 yo - 21
yo.. they'll eat what was left in two days.. so no big loss here
:-)..but it does bring up the fact that we most, absolutely definitely
need an RSVP system for attendance and food. I spent about $100 (maybe a
bit more) but that included a good supply of paper plates, cups,
napkins.. etc that I can reuse for future meetings.. I collected $45..
(actually more than I thought I'd collect) but there was quite a bit
leftover so it wasn't a huge loss. It will help if we have an rsvp
system in place so I know exactly who is or is not eating and can
purchase accordingly..

I'd like to propose that we start the three lists as indicated..
baypiggies-announce.. etc.. but add one for the new website..
baypiggies-web .. so I can get feedback.. along with Paul McGavin (and
anyone else who wants to help) on what we want/need on the new Plone CMS
website in the works.

OK that's it from me for awhile.. won't be at ironport.. but I'll be
avail offlist :-)

Donna M. Snow



 


_______________________________________________
Baypiggies mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/baypiggies
Reply | Threaded
Open this post in threaded view
|

ctypes presentation material online

Dennis Reinhardt

I have now posted my ctypes presentation material online linked at
http://www.spamai.com/present/  Format is zipped PDF, about 270 KB.

This is the same URL as given in the baypiggies.net website describing the
talk so if you want to look this up later without saving this email, you
can find it there.

The presentation posted contains material I did not discuss: prototypes for
several useful functions.
------------------------------------
| Dennis    | [hidden email]     |
| Reinhardt | Powerful Anti-Spam   |
| http://dair.com/py/planmeet.html |
------------------------------------

_______________________________________________
Baypiggies mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/baypiggies
Reply | Threaded
Open this post in threaded view
|

Re: ctypes presentation material online

Tung Wai Yip
Couldn't make it to the meeting. Thanks for sharing the experience with  
ctype.

The risk of crashing seems so scary. Have you evaluated pywin32? I wonder  
how they compare in the context of your project.

Wai Yip

> I have now posted my ctypes presentation material online linked at
> http://www.spamai.com/present/  Format is zipped PDF, about 270 KB.
>
> This is the same URL as given in the baypiggies.net website describing  
> the
> talk so if you want to look this up later without saving this email, you
> can find it there.
>
> The presentation posted contains material I did not discuss: prototypes  
> for
> several useful functions.


_______________________________________________
Baypiggies mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/baypiggies
Reply | Threaded
Open this post in threaded view
|

Re: ctypes presentation material online

Marilyn Davis
In reply to this post by Dennis Reinhardt
On Fri, 12 May 2006, Dennis Reinhardt wrote:

>
> I have now posted my ctypes presentation material online linked at
> http://www.spamai.com/present/  Format is zipped PDF, about 270 KB.

Thank you Dennis.  That was an interesting little talk.  And you
started 1 minute late.  That was only because the first group of us
wasn't let in until 1 minute late.  You were ready!

(I'll also comment that Alex helped me find the bathroom, which was
close by, so I didn't miss very much.  AND the toilet seat was warm!
I never experienced anything so luxurious.  Shocking at first though.)

But, back to the talk, I particularly appreciate that you exposed the
big problem you had.  You defined something as 2-longs, and then used
it as 4-longs, crashing malloc like a C program.  Ouch.

You know, there's a whole industry devoted to finding that sort of bug
in C code, because it is so hard to find.  You're not really doing
python when things get that hard.

If you had it to do over again, would you use ctypes?  Or try one of
the other options, even though they take more memory?

Marilyn

>
> This is the same URL as given in the baypiggies.net website describing the
> talk so if you want to look this up later without saving this email, you
> can find it there.
>
> The presentation posted contains material I did not discuss: prototypes for
> several useful functions.
> ------------------------------------
> | Dennis    | [hidden email]     |
> | Reinhardt | Powerful Anti-Spam   |
> | http://dair.com/py/planmeet.html |
> ------------------------------------
>
> _______________________________________________
> Baypiggies mailing list
> [hidden email]
> http://mail.python.org/mailman/listinfo/baypiggies
>

--

_______________________________________________
Baypiggies mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/baypiggies
Reply | Threaded
Open this post in threaded view
|

Re: ctypes presentation material online

Marilyn Davis
In reply to this post by Tung Wai Yip
On Fri, 12 May 2006, Tung Wai Yip wrote:

> Couldn't make it to the meeting. Thanks for sharing the experience with  
> ctype.
>
> The risk of crashing seems so scary. Have you evaluated pywin32? I wonder  

Yeh.  Memory bugs are just terrible to find.

I once had one in C code where I was mallocing space for a new string:

buf = malloc(len(string + 1));

My eyeball looked at that a jillion times before I realized that I had
a misplaced ')' and it should be:

buf = malloc(len(string) + 1);

The problem is that it crashed much later, after I wrote to the space
(and overwrote the memory), and then it finally crashed in another
call to malloc.  This is typical.  Between the bug and the crash were
many many instructions and many many times of writing into malloc'ed
space.

Anyhow, Python, I kiss your feet, for taking this stuff out of my life.

Marilyn



> how they compare in the context of your project.
>
> Wai Yip
>
> > I have now posted my ctypes presentation material online linked at
> > http://www.spamai.com/present/  Format is zipped PDF, about 270 KB.
> >
> > This is the same URL as given in the baypiggies.net website describing  
> > the
> > talk so if you want to look this up later without saving this email, you
> > can find it there.
> >
> > The presentation posted contains material I did not discuss: prototypes  
> > for
> > several useful functions.
>
>
> _______________________________________________
> Baypiggies mailing list
> [hidden email]
> http://mail.python.org/mailman/listinfo/baypiggies
>

--

_______________________________________________
Baypiggies mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/baypiggies
Reply | Threaded
Open this post in threaded view
|

Re: ctypes presentation material online

Dennis Reinhardt
In reply to this post by Dennis Reinhardt
At 10:21 AM 5/12/2006, Marilyn Davis wrote:
>If you had it to do over again, would you use ctypes?  Or try one of
>the other options, even though they take more memory?

Tough question, because memory is but one part of the tradeoff.

First, let me say that what I am doing is building a ready-to-run program,
an exe which when run, well, just runs.  This is in contrast to other
deployments where you first need to install Python and a number of other
components first.  BitTorrent install IIRC is of this type, for example.

Adding other components can greatly complicate the user install.  There is
a major cost borne by the end user in how well integrated the components
are.  Adding more components creates problems beyond just download
size.  In order to be ready-to-run, you need some scheme for finding
components you have installed which does not conflict with any Python
components the user may have installed already.  I solve this by altering
sys.path at startup.  I would have to include additional components and
know that there are compatible with this (i.e. that they can operate
independent of registry settings).

I never got far enough with WxPython to see if this was a problem.  There
is a major learning curve.

I also had a learning curve implementing DialogDevil (my program) with
ctypes.  Even with the 40-50 hours I spent tracking down the crash and the
learning curve with ctypes itself (non-trivial), I would still write
DialogDevil using this approach.  Here is why:

         I could be reasonably assured I would not have
         packaging and deployment issues downstream

         I could be assured that the entire non-COM
         Windows API was available to me and that I
         would not get stuck late in the project because
         some interface was missing or not usable.

         The simple functionality was manageable.  I
         could isolate problems with ctypes.  With more
        complex packages, my support burden for
        isolating problems is increased.

         The download size was a very nice bonus.

However, in a larger sense, no, I have not done it again.  Currently, I am
using a web-browser as a user interface.  The idea here is that I have
written an http server and user interactions are via web forms.  In so
doing, I get client-server architecture nearly for free with an obvious
upgrade to multi-user implementations.    This is not a benefit for
DialogDevil type products but is compelling for others.

Regards, Dennis


----------------------------------
| Dennis    | [hidden email]   |
| Reinhardt | Powerful Anti-Spam |
----------------------------------

_______________________________________________
Baypiggies mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/baypiggies
Reply | Threaded
Open this post in threaded view
|

Re: tonights meeting - (my feelings won't be hurt if you delete)

Shannon -jj Behrens
In reply to this post by snowwrite
> I'd like to propose that we start the three lists as indicated..
> baypiggies-announce.. etc.. but add one for the new website..
> baypiggies-web .. so I can get feedback ...

Stephen, Dennis, and I were talking about a baypiggies-announce
mailing list.  We reached concensus that a baypiggies-announce mailing
list is an absolute must for people who don't want to deal with the
huge amount of traffic on the mailing list.

Aahz, who can do this?  Do you object strongly?

Best Regards,
-jj
_______________________________________________
Baypiggies mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/baypiggies
Reply | Threaded
Open this post in threaded view
|

Re: ctypes presentation material online

Marilyn Davis
In reply to this post by Dennis Reinhardt
On Fri, 12 May 2006, Dennis Reinhardt wrote:

> At 10:21 AM 5/12/2006, Marilyn Davis wrote:
> >If you had it to do over again, would you use ctypes?  Or try one of
> >the other options, even though they take more memory?
>
> Tough question, because memory is but one part of the tradeoff.
>
> First, let me say that what I am doing is building a ready-to-run program,
> an exe which when run, well, just runs.  This is in contrast to other
> deployments where you first need to install Python and a number of other
> components first.  BitTorrent install IIRC is of this type, for example.
>

I see.

What about C for this?  I mean since you're looking for a ready-to-run
and you have to deal with typing and be careful about memory anyway?

I guess that's a huge learning curve if you don't know it already.

Maybe sticking to Python and reaching back to C through ctypes is
still faster to develop than a total C solution.  With C, you'd have
to worry about memory for every piece of it you use.

> Adding other components can greatly complicate the user install.  There is
> a major cost borne by the end user in how well integrated the components
> are.  Adding more components creates problems beyond just download
> size.  In order to be ready-to-run, you need some scheme for finding
> components you have installed which does not conflict with any Python
> components the user may have installed already.  I solve this by altering
> sys.path at startup.  I would have to include additional components and
> know that there are compatible with this (i.e. that they can operate
> independent of registry settings).
>
> I never got far enough with WxPython to see if this was a problem.  There
> is a major learning curve.
>
> I also had a learning curve implementing DialogDevil (my program) with
> ctypes.  Even with the 40-50 hours I spent tracking down the crash and the
> learning curve with ctypes itself (non-trivial), I would still write
> DialogDevil using this approach.  Here is why:
>
>          I could be reasonably assured I would not have
>          packaging and deployment issues downstream
>
>          I could be assured that the entire non-COM
>          Windows API was available to me and that I
>          would not get stuck late in the project because
>          some interface was missing or not usable.
>
>          The simple functionality was manageable.  I
>          could isolate problems with ctypes.  With more
>         complex packages, my support burden for
>         isolating problems is increased.
>
>          The download size was a very nice bonus.
>
> However, in a larger sense, no, I have not done it again.  Currently, I am
> using a web-browser as a user interface.  The idea here is that I have
> written an http server and user interactions are via web forms.  In so
> doing, I get client-server architecture nearly for free with an obvious
> upgrade to multi-user implementations.    This is not a benefit for
> DialogDevil type products but is compelling for others.

Thank you so much Dennis.  This is valuable insight for us.

Marilyn

>
> Regards, Dennis
>
>
> ----------------------------------
> | Dennis    | [hidden email]   |
> | Reinhardt | Powerful Anti-Spam |
> ----------------------------------
>
> _______________________________________________
> Baypiggies mailing list
> [hidden email]
> http://mail.python.org/mailman/listinfo/baypiggies
>

--

_______________________________________________
Baypiggies mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/baypiggies
Reply | Threaded
Open this post in threaded view
|

Re: ctypes presentation material online

Stephen McInerney
In reply to this post by Marilyn Davis

The case below is a good argument for static type checking?
Is there any type checker smart enough to understand that
string + 1 is unlikely to have semantic meaning.
(I know this is C but the concept is generic.)

>From: Marilyn Davis <[hidden email]>
>
>Yeh.  Memory bugs are just terrible to find.
>
>I once had one in C code where I was mallocing space for a new string:
>
>buf = malloc(len(string + 1));
>
>My eyeball looked at that a jillion times before I realized that I had
>a misplaced ')' and it should be:
>
>buf = malloc(len(string) + 1);
>
>The problem is that it crashed much later, after I wrote to the space
>(and overwrote the memory), and then it finally crashed in another
>call to malloc.


_______________________________________________
Baypiggies mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/baypiggies
Reply | Threaded
Open this post in threaded view
|

Re: ctypes presentation material online

Dennis Reinhardt
In reply to this post by Tung Wai Yip
At 09:52 AM 5/12/2006, Tung Wai Yip wrote:
>The risk of crashing seems so scary. Have you evaluated pywin32? I wonder
>how they compare in the context of your project.

I knew of pywin32.  More than anything, I needed access to the list
control.  This is a very complicated object whose printed documentation
amounts to about 200 pages.

I didn't see a lot of benefit in a wrapper for such a complex object.  IMO,
I really needed to talk to it directly.

I recently investigated writing a COM object.  I might look at pywin32 for
that.  However, the author of ctypes (Thomas Heller) is in development of
comtypes and I looked at that first.  I have suspended the COM project
because it is clear to me that the overall scope is beyond what a 1 person
company can bring to market.



----------------------------------
| Dennis    | [hidden email]   |
| Reinhardt | Powerful Anti-Spam |
----------------------------------

_______________________________________________
Baypiggies mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/baypiggies
Reply | Threaded
Open this post in threaded view
|

Re: ctypes presentation material online

Dennis Reinhardt
In reply to this post by Dennis Reinhardt
At 11:48 AM 5/12/2006, you wrote:
>What about C for this?  I mean since you're looking for a ready-to-run
>and you have to deal with typing and be careful about memory anyway?
>I guess that's a huge learning curve if you don't know it already.

The phase of the program I discussed was in the initialization, where there
are no real reasons to choose Python over C.

I have written and still use my own native exe compiler for my own PLD2
language (another l-o-ng talk?).  Suffice it to say, learning curve for a
compiled program is not a problem.  Memory is not an issue since PLD2
memory is laid out statically.

I could do C and have done some modest coding with it.  It is impossible to
program for Windows without being able to constantly read C.  I am not fond
of the typing.  Over 95% of types are 32 bit integers.  More than once, the
correct response to a type error is to cast the type (i.e. ignore).

I could use typing in ctypes but I didn't so I don't have to deal with typing.

------------------------------------
| Dennis    | [hidden email]     |
| Reinhardt | Powerful Anti-Spam   |
| http://dair.com/py/planmeet.html |
------------------------------------

_______________________________________________
Baypiggies mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/baypiggies
Reply | Threaded
Open this post in threaded view
|

Re: ctypes presentation material online

Marilyn Davis
In reply to this post by Stephen McInerney
On Fri, 12 May 2006, Stephen McInerney wrote:

>
> The case below is a good argument for static type checking?
> Is there any type checker smart enough to understand that
> string + 1 is unlikely to have semantic meaning.

Good thought.

But although it is unlikely, it is entirely possible.  string + 1 in C
is string[1:] in Python.  It trims off the first character.

string is defined as a char *, which is the address of the first
character.  All string processing is convention, not a built-in data
type.  The string is assumed to reside at the memory address in
contiguous bytes until there is a '\0', marking the end.  That is why
it takes strlen(string + 1) to malloc memory for it: the length of the
string plus one byte for that '\0' flag to mark the end.

And that is why, string + 1, which is the address of the second
character, is string[1:].

When I teach C, I encourage students memorize only 3 things:

1.  The point of a programming language is to communicate with other
engineers that the computer also understands.

2.  The name of an array is shorthand for the address of the first
element of the array.

3.  A string is an array of characters with a '\0' at the end.

2 and 3 are very hard.

Teaching Python, they only get #1.

I love it.

Marilyn

> (I know this is C but the concept is generic.)
>
> >From: Marilyn Davis <[hidden email]>
> >
> >Yeh.  Memory bugs are just terrible to find.
> >
> >I once had one in C code where I was mallocing space for a new string:
> >
> >buf = malloc(len(string + 1));
> >
> >My eyeball looked at that a jillion times before I realized that I had
> >a misplaced ')' and it should be:
> >
> >buf = malloc(len(string) + 1);
> >
> >The problem is that it crashed much later, after I wrote to the space
> >(and overwrote the memory), and then it finally crashed in another
> >call to malloc.
>
>
>

--

_______________________________________________
Baypiggies mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/baypiggies
Reply | Threaded
Open this post in threaded view
|

Re: ctypes presentation material online

KenSeehart
In reply to this post by Stephen McInerney
Unfortunately there is nothing wrong with the expression string + 1.  It's just pointer arithmatic.
If string is a char *, then string+1 is that string with the first character ommitted (hence one character /shorter/ than the original string).

The problem below doesn't have a clear analogy in python.  And static type checking in python would not pertain to any analogous problem in python.  Everything about this kind of problem is very language specific.

I'm actually a bit of a skeptic of the idea of using static type checking in python.  In practice it is rarely possible to obtain /any/ reliable type information on objects at compile time in python.

- Ken

Stephen McInerney wrote:
The case below is a good argument for static type checking?
Is there any type checker smart enough to understand that
string + 1 is unlikely to have semantic meaning.
(I know this is C but the concept is generic.)

  
From: Marilyn Davis [hidden email]

Yeh.  Memory bugs are just terrible to find.

I once had one in C code where I was mallocing space for a new string:

buf = malloc(len(string + 1));

My eyeball looked at that a jillion times before I realized that I had
a misplaced ')' and it should be:

buf = malloc(len(string) + 1);

The problem is that it crashed much later, after I wrote to the space
(and overwrote the memory), and then it finally crashed in another
call to malloc.
    


_______________________________________________
Baypiggies mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/baypiggies


  


_______________________________________________
Baypiggies mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/baypiggies
Reply | Threaded
Open this post in threaded view
|

Re: ctypes presentation material online

Stephen McInerney
In reply to this post by Marilyn Davis
Hi Marilyn,

(Sure, most people have fond experiences of debugging in C/C++, a language
where weakly typed pointer arithmetic plus implicit type conversions gives
you enough rope to hang yourself a thousand times over.
Before I settled on Python in the year 2000, one of my motivations was
avoiding C pointer debugging andn memory management. In fact my formative
evangelizing experience in finally deciding to go to Python was a
frustrating 2 months attempting to use CompuWare Boundschecker (overpriced
at $499, underfeatured and brain-dead) on some memory leaks in some of my
MSEE project code under a tight deadline and getting nowhere fast. Never
again.)

My point in this case is that although 'len (str + 1)' is syntactically
legal, if you had any sort of heuristic in a type checker, it would flag
'str + 1' as being semantically dubious (it might be legal, but then it
might not be, so it merits closer scrutiny).
I know exactly what the C effect of len(str + 1) will be: you'll undercount
the length of the string by 1, which is a creeping bug, (unless *str ==
'\0', in which case you get a stray pointer and a nonsense length, then you
get a blatant bug).
The problem here is that the language types and the compiler do not
discriminate between length, length + 1 being  integers, whereas str, str +
1 are addresses - it's all represented by 32bits. It would be interesting to
see a heuristic chart of the probability that each of different types of
pointer arithmetic are semantically correct.

I've used and scripted linters in several languages (C, Verilog) in
production flows and a key lesson I learned is to search for and flag not
just the outright illegal stuff, but the dubious stuff. Intermediate
variables, explicit casts and/or linter pragmas are a good practice (in
moderation).

Static type checking is neat, and if done across language wrapper boundaries
could be even neater.  I think this is a topic of potential interest to a
lot of people? (GUI folks too, for sure)
Wrapping Fortran libraries seems to be pretty similarly painful (e.g.
Jeffrey Whitaker's NAGpy
http://www.cdc.noaa.gov/people/jeffrey.s.whitaker/python/nagpy/

Stephen


_______________________________________________
Baypiggies mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/baypiggies