More API thoughts (policy objects)

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

More API thoughts (policy objects)

R. David Murray
I've been mulling on policy objects since talking with Barry about them at
the sprint.  Today I sat down and started on the "real" API documentation
(ie: in Sphinx, fully written out not just sketched).  So far I have
the stub of the main email doc copied from the current package, with the
intro turned into an XXX, and the new changes sections marked with an XXX,
and a cut down table of contents listing only the new docs I've written.

There are two doc sections currently, Policy Objects, and Headers.
Headers is somewhat revised from both my original proposal and from the
code actually implemented in the bzr branch.  This is because I realized
as I was writing the docs that I'd failed to think through how header
subclasses were going to get created.  Creating the correct subclass
requires knowing the field name of the header for which an object is
being instantiated, which means that the parser can't just gather header
lines and pass them to the header class to parse the way I'd originally
visualized.  Instead we need a factory function that will look up the
field name in the registry and instantiate the correct class, passing it
the name and value.

Since the registry is going to be in the policy object, that led me
to write the policy object docs.  After several iterations over both
documents, I have something that I think at least hangs together, and
which I think is fairly decent from a usability standpoint (I hope...that
opinion is based on thought experiments, not real code, after all!).
I'm not sure yet about the higher levels (the parser...or is the
Message object the parser?), but I've got a vision of how the Message
objects, the Header objects, and Message sub-objects plug together.
The plugging-together isn't documented yet, although the existing docs
give some good glimpses.

One potentially controversial bit of this design is that I'm planning
to have the header factory pull together three independent classes
and dynamically build a composite class out of them to instantiate an
object from.  This is not something I've ever done before in code, so I'm
not sure it will even work.  But it seems to make sense as a design,
so I've speced it that way.  I'll be curious if anyone has any insights,
opinions, or better ways of accomplishing the same goal.

This design arose from the fact that in my overall design the Bytes and
String classes share a common base class and that most of the code is
common between the two.  When thinking about the registry of structured
headers, it felt rather yucky to have either two registries (bytes and
string) or a registry with (bytes class, string class) tuples.  Add to
this the fact that it would be really nice for an application to have a
way to substitute in its own base class without having to build a whole
new subclass tree, and I decided to give dynamic class building a try.
So the design I've speced has a header factory method looking up the
base class, the string or byte class, and the specific header class in
the registry, and then composing the three of them in order to return
a header instance.

Am I crazy?

Anyway, if you want to look over the docs (most of which are probably
valid even if I end up having to give up on my composition-based
registry), they are up on my web site here:

http://www.bitdance.com/projects/email6/doc/email/

--David

PS: it's not clear to me that policy objects make thread safety natural,
although they do make it *possible*.
_______________________________________________
Email-SIG mailing list
[hidden email]
Your options: http://mail.python.org/mailman/options/email-sig/lists%40nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: More API thoughts (policy objects)

Barry Warsaw
This looks pretty good David.  Nice to see you run with it.

(Small nit: there are a few typos here and there.)

On Mar 05, 2010, at 11:27 PM, R. David Murray wrote:

>Since the registry is going to be in the policy object, that led me
>to write the policy object docs.  After several iterations over both
>documents, I have something that I think at least hangs together, and
>which I think is fairly decent from a usability standpoint (I hope...that
>opinion is based on thought experiments, not real code, after all!).

I've read the document through a few times, and while I'm sure my lack of
sleep is a contributing factor (the other part being my denseness ;), I'm not
quite getting how the Policy and PolicySet classes work together, nor how the
policy_overlay argument comes into play.

>One potentially controversial bit of this design is that I'm planning
>to have the header factory pull together three independent classes
>and dynamically build a composite class out of them to instantiate an
>object from.  This is not something I've ever done before in code, so I'm
>not sure it will even work.  But it seems to make sense as a design,
>so I've speced it that way.  I'll be curious if anyone has any insights,
>opinions, or better ways of accomplishing the same goal.
>
>This design arose from the fact that in my overall design the Bytes and
>String classes share a common base class and that most of the code is
>common between the two.  When thinking about the registry of structured
>headers, it felt rather yucky to have either two registries (bytes and
>string) or a registry with (bytes class, string class) tuples.  Add to
>this the fact that it would be really nice for an application to have a
>way to substitute in its own base class without having to build a whole
>new subclass tree, and I decided to give dynamic class building a try.
>So the design I've speced has a header factory method looking up the
>base class, the string or byte class, and the specific header class in
>the registry, and then composing the three of them in order to return
>a header instance.
>
>Am I crazy?
Not necessarily!  It's an interesting take on the design problem that makes
some sense.  The other way to handle it is through composition and delegation,
but honestly IMO Python does not have a great delegation story (no,
__getattr*__ does not cut it ;).  It's also pretty easy to do in Python, if I
understand what you're trying to get at:

Python 3.1.2rc1+ (release31-maint:78807, Mar  8 2010, 22:02:30)
[GCC 4.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> class Foo: pass
...
>>> class Bar: pass
...
>>> class Baz: pass
...
>>> Qux = type('Qux', (Foo, Bar, Baz), {})
>>> type(Qux)
<class 'type'>
>>> Qux()
<__main__.Qux object at 0x7f0f7e62a750>
>>> Qux.__bases__
(<class '__main__.Foo'>, <class '__main__.Bar'>, <class '__main__.Baz'>)


A few other thoughts.

* Do we need raise_on_defect, handle_defect() and register_defect().  Seems a
  bit of overkill; what's the use case?

* Why does header_indent default to 8 spaces?  Shouldn't it be a tab or single
  space?

* Do you think newline should be renamed end_of_line, linesep, or end?  The
  latter two are evocative of os.linesep and print(..., end='\n', ...).
  "newline" specifically is actually misleading.

That's it for now.  Cool stuff!
-Barry

_______________________________________________
Email-SIG mailing list
[hidden email]
Your options: http://mail.python.org/mailman/options/email-sig/lists%40nabble.com

signature.asc (853 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: More API thoughts (policy objects)

R. David Murray
On Mon, 08 Mar 2010 22:07:48 -0500, Barry Warsaw <[hidden email]> wrote:
> This looks pretty good David.  Nice to see you run with it.
>
> (Small nit: there are a few typos here and there.)

Yes, I've caught a few, feel free to point out any others :)

> On Mar 05, 2010, at 11:27 PM, R. David Murray wrote:
> >Since the registry is going to be in the policy object, that led me
> >to write the policy object docs.  After several iterations over both
> >documents, I have something that I think at least hangs together, and
> >which I think is fairly decent from a usability standpoint (I hope...that
> >opinion is based on thought experiments, not real code, after all!).
>
> I've read the document through a few times, and while I'm sure my lack of
> sleep is a contributing factor (the other part being my denseness ;), I'm not
> quite getting how the Policy and PolicySet classes work together, nor how the
> policy_overlay argument comes into play.

No, you are right.  What I speced works, I think, but in the light of
morning it is way too confusing, and users would get it "wrong", which
means the API design is flawed.  I haven't quite wrapped my head around
how to do it right, but I do plan to fix it.

Here are what I think the design requirements are:

    1) a complete set of defaults has to come from somewhere and be
       accessible to every object that uses policy.

    2) an object needs to be able to provide local overrides to these
       defaults that are normally obeyed.

    3) an application should to be able to specify changes to the
       global defaults without having to specify all the defaults,
       and those changes should not affect the object-local overrides.

    4) an application should be able to locally override the application
       global defaults on specific method calls that support doing so.
       I think these overrides *should* override the object-local
       overrides.

    5) ideally there should be one set of pre-defined 'policy objects'
       that can be used as application level defaults and as
       local-command overrides (this is where I got into trouble,
       I think).

Whatever else is true, the docs need examples (ie: use cases).  If I
write those next, I'll probably figure out a better API.

> >One potentially controversial bit of this design is that I'm planning
> >to have the header factory pull together three independent classes
> >and dynamically build a composite class out of them to instantiate an
> >object from.  This is not something I've ever done before in code, so I'm
> >not sure it will even work.  But it seems to make sense as a design,
> >so I've speced it that way.  I'll be curious if anyone has any insights,
> >opinions, or better ways of accomplishing the same goal.
> >
> >This design arose from the fact that in my overall design the Bytes and
> >String classes share a common base class and that most of the code is
> >common between the two.  When thinking about the registry of structured
> >headers, it felt rather yucky to have either two registries (bytes and
> >string) or a registry with (bytes class, string class) tuples.  Add to
> >this the fact that it would be really nice for an application to have a
> >way to substitute in its own base class without having to build a whole
> >new subclass tree, and I decided to give dynamic class building a try.
> >So the design I've speced has a header factory method looking up the
> >base class, the string or byte class, and the specific header class in
> >the registry, and then composing the three of them in order to return
> >a header instance.
> >
> >Am I crazy?
>
> Not necessarily!  It's an interesting take on the design problem that
> makes some sense.  The other way to handle it is through composition
> and delegati on, but honestly IMO Python does not have a great
> delegation story (no, __getattr*__ does not cut it ;).  It's also

Right, this is a Python weak spot.

> pretty easy to do in Python, if  I understand what you're trying to
> get at:
>
> Python 3.1.2rc1+ (release31-maint:78807, Mar  8 2010, 22:02:30)=20
> [GCC 4.4.3] on linux2 Type "help", "copyright", "credits" or "license"
> for more information.
> >>> class Foo: pass
> ...=20
> >>> class Bar: pass
> ...=20
> >>> class Baz: pass
> ...=20
> >>> Qux = type('Qux', (Foo, Bar, Baz), {}) type(Qux)
> <class 'type'>
> >>> Qux()
> <__main__.Qux object at 0x7f0f7e62a750>
> >>> Qux.__bases__
> (<class '__main__.Foo'>, <class '__main__.Bar'>, <class
> '__main__.Baz'>)

Ah, good, that is what I thought it would look like, but I hadn't run
the experiment yet.

> A few other thoughts.
>
> * Do we need raise_on_defect, handle_defect() and register_defect().
> Seems a bit of overkill; what's the use case?

Perhaps I'm over-designing :).  My thought was that this gave the policy
object centralized control over what happens to defects.  The flag
controls the raise-or-not.  handle_defect is on the policy object so
that every method that needs to handle a defect doesn't have to implement
its own raise-or-not logic.  register_defect is the hook to do something
non-standard with defects that aren't raised.  The most obvious use case
for this would be to log the defects instead of/in addition to putting
them on the object.

Perhaps handle_defect should be private method.  Since the application
can control raise-or-not via the flag, there's no real need to have that
method be part of the public API.

> * Why does header_indent default to 8 spaces?  Shouldn't it be a tab
> or single space?

You are right, it should be tab.  I will fix it.  I was carrying
Python's anti-tab sentiment a little too far ;)

> * Do you think newline should be renamed end_of_line, linesep, or end?
> The latter two are evocative of os.linesep and print(..., end='\n',
> ...).  "newline" specifically is actually misleading.

Yes, I would like to rename newline to *something* else.  I was following
the lead of the io module docs in this, but I don't like it.  Maybe we
could change the io docs, too? :)

I think I like using 'linesep' better than 'end'.  In the print function,
'end' comes literally at the end of the string being output, whereas in
the email package case we are talking about what goes in between lines,
as well as at the end.

> That's it for now.  Cool stuff!  -Barry

Thanks for the feedback!

--David
_______________________________________________
Email-SIG mailing list
[hidden email]
Your options: http://mail.python.org/mailman/options/email-sig/lists%40nabble.com