autogenerating API docs using sphinx

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

autogenerating API docs using sphinx

Ondrej Certik
Hi,

I wrote a short script for autogenerating API from sources, the output
are .rst files that sphinx can parse and generate the docs in the
modules/index sections.

Unfortunately, if the docstrings are not written in rst, it's not
going to work properly. Also it uses python inspection (it's easy to
implement), instead of parsing the source files directly.

I am putting it here in case you find it useful. I wrote it for sympy
originally, but we are not using it yet, because the output contains
too much noise.

Please let me know if you improve it/fix it. :)

Ondrej

_______________________________________________
Doc-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/doc-sig

generate_reference.py (6K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: autogenerating API docs using sphinx

Gael Varoquaux
On Mon, Apr 14, 2008 at 11:51:32PM +0200, Ondrej Certik wrote:
> I wrote a short script for autogenerating API from sources, the output
> are .rst files that sphinx can parse and generate the docs in the
> modules/index sections.

I just did something similar this week end.

I would like to get it in our SVN, but first Georg needs to implement a
missing directive in the LaTeX writer, because I can't commit my changes
to SVN as long as we can't build PDFs. So to allow people to have a look
at it, I have uploaded the whole documentation directory to
http://gael-varoquaux.info/docs.tar.gz , including the generated rest,
and the html. The two interesting files are render_image.py, which is
kind of pushing the notion of doctests to something with a graphical
output (this is for a 3D plotting application), except that it doesn't
check the output is valid yet, and "mlab_reference", which is an attempt
to make something quite general.

This is very taylored toward the documentation of mlab (a scripting API
for the plotting application). In mlab the docstring are partially
generated automaticaly, with addition of the keyword arguments, so I
could control them well to be good rest. In addition mlab is a namespace
that exposes functions to the users coming from different module, each
giving different fonctionnality. I use this to structure the
documentation.

I am not entirely happy with the code, as there might be to much code
specific to my use. I would like to see some thing like this quite
general, but I also want to keep the high quality output this gives me.
Anybody is welcomed to adapt it to your purposes, while trying to keep it
general, and we can see what comes out of it.

Cheers,

Gaël
_______________________________________________
Doc-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/doc-sig
Reply | Threaded
Open this post in threaded view
|

Re: autogenerating API docs using sphinx

Ondrej Certik
On Tue, Apr 15, 2008 at 12:20 AM, Gael Varoquaux
<[hidden email]> wrote:

> On Mon, Apr 14, 2008 at 11:51:32PM +0200, Ondrej Certik wrote:
>
> > I wrote a short script for autogenerating API from sources, the output
>  > are .rst files that sphinx can parse and generate the docs in the
>  > modules/index sections.
>
>  I just did something similar this week end.
>
>  I would like to get it in our SVN, but first Georg needs to implement a
>
> missing directive in the LaTeX writer, because I can't commit my changes
>  to SVN as long as we can't build PDFs. So to allow people to have a look
>  at it, I have uploaded the whole documentation directory to
>
> http://gael-varoquaux.info/docs.tar.gz , including the generated rest,
>  and the html. The two interesting files are render_image.py, which is
>  kind of pushing the notion of doctests to something with a graphical
>  output (this is for a 3D plotting application), except that it doesn't
>
> check the output is valid yet, and "mlab_reference", which is an attempt
>  to make something quite general.
>
>  This is very taylored toward the documentation of mlab (a scripting API
>  for the plotting application). In mlab the docstring are partially
>
> generated automaticaly, with addition of the keyword arguments, so I
>  could control them well to be good rest. In addition mlab is a namespace
>  that exposes functions to the users coming from different module, each
>  giving different fonctionnality. I use this to structure the
>  documentation.
>
>  I am not entirely happy with the code, as there might be to much code
>  specific to my use. I would like to see some thing like this quite
>  general, but I also want to keep the high quality output this gives me.
>  Anybody is welcomed to adapt it to your purposes, while trying to keep it
>
> general, and we can see what comes out of it.

Thanks Gael for sharing it with us. I looked at it and it looks really great!

I want a code, that goes through all modules (or files) and produces
API doc for each function and each class + methods it finds. When I
have some time, I'll try to look at your code and see if I can reuse
something.

Ondrej
_______________________________________________
Doc-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/doc-sig
Reply | Threaded
Open this post in threaded view
|

Re: autogenerating API docs using sphinx

Skip Montanaro-3

Would it perhaps be better to create a Sphinx backend for an existing
source-to-documentation tool like doxygen?  I have no idea how hard or
feasible that would be or if there are better candidate tools.  Doxygen just
happens to be one I am grudingly familiar with because the C++ programmers
at work use it.  If successful it would have the added benefit of exposing
Sphinx and restructured text outside the Python world.

Skip
_______________________________________________
Doc-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/doc-sig
Reply | Threaded
Open this post in threaded view
|

Re: autogenerating API docs using sphinx

Gael Varoquaux
On Mon, Apr 14, 2008 at 09:16:34PM -0500, [hidden email] wrote:

> Would it perhaps be better to create a Sphinx backend for an existing
> source-to-documentation tool like doxygen?  I have no idea how hard or
> feasible that would be or if there are better candidate tools.  Doxygen just
> happens to be one I am grudingly familiar with because the C++ programmers
> at work use it.  If successful it would have the added benefit of exposing
> Sphinx and restructured text outside the Python world.

That would be great, but it wouldn't replace what I am doing. I have
taylored in my autogeneration of docs a lot of smart guessing (special
cases for my module?) to have a usable and readable doc generated. I hate
the docs generated by tools like doxygen. They are just not meant for end
users, but for programmers.

Two purposes, two tools; fair enough.

Gaël
_______________________________________________
Doc-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/doc-sig
Reply | Threaded
Open this post in threaded view
|

Re: autogenerating API docs using sphinx

Ondrej Certik
On Tue, Apr 15, 2008 at 11:26 AM, Gael Varoquaux
<[hidden email]> wrote:

>
> On Mon, Apr 14, 2008 at 09:16:34PM -0500, [hidden email] wrote:
>
>  > Would it perhaps be better to create a Sphinx backend for an existing
>  > source-to-documentation tool like doxygen?  I have no idea how hard or
>  > feasible that would be or if there are better candidate tools.  Doxygen just
>  > happens to be one I am grudingly familiar with because the C++ programmers
>  > at work use it.  If successful it would have the added benefit of exposing
>  > Sphinx and restructured text outside the Python world.
>
>  That would be great, but it wouldn't replace what I am doing. I have
>  taylored in my autogeneration of docs a lot of smart guessing (special
>  cases for my module?) to have a usable and readable doc generated. I hate
>  the docs generated by tools like doxygen. They are just not meant for end
>  users, but for programmers.
>
>  Two purposes, two tools; fair enough.

Yep. I myself am looking for something that Gael is doing -- simple
API docs for endusers (and myself too!), that quickly shows me, what
kind of stuff I can do. For everything else, I can either use epydoc,
or just look into the sources directly (that's what I prefer).

Ondrej
_______________________________________________
Doc-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/doc-sig
Reply | Threaded
Open this post in threaded view
|

Re: autogenerating API docs using sphinx

Georg Brandl-2
In reply to this post by Ondrej Certik
Ondrej Certik schrieb:

> Hi,
>
> I wrote a short script for autogenerating API from sources, the output
> are .rst files that sphinx can parse and generate the docs in the
> modules/index sections.
>
> Unfortunately, if the docstrings are not written in rst, it's not
> going to work properly. Also it uses python inspection (it's easy to
> implement), instead of parsing the source files directly.
>
> I am putting it here in case you find it useful. I wrote it for sympy
> originally, but we are not using it yet, because the output contains
> too much noise.
>
> Please let me know if you improve it/fix it. :)

What does it do differently from autodoc, and could autodoc be extended
to cover your usecase as well?

Georg

--
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.

_______________________________________________
Doc-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/doc-sig
Reply | Threaded
Open this post in threaded view
|

Re: autogenerating API docs using sphinx

Ondrej Certik
On Sun, Apr 20, 2008 at 11:47 AM, Georg Brandl <[hidden email]> wrote:

> Ondrej Certik schrieb:
>
>
> > Hi,
>  >
>  > I wrote a short script for autogenerating API from sources, the output
>  > are .rst files that sphinx can parse and generate the docs in the
>  > modules/index sections.
>  >
>  > Unfortunately, if the docstrings are not written in rst, it's not
>  > going to work properly. Also it uses python inspection (it's easy to
>  > implement), instead of parsing the source files directly.
>  >
>  > I am putting it here in case you find it useful. I wrote it for sympy
>  > originally, but we are not using it yet, because the output contains
>  > too much noise.
>  >
>  > Please let me know if you improve it/fix it. :)
>
>  What does it do differently from autodoc, and could autodoc be extended
>  to cover your usecase as well?

You mean autodoc in sphinx?

I want to get a documentation of all classes+methods and functions in
all files in sympy automatically (we have a lot of .py files and I
don't want to handle each of them manually). I didn't figure it out
how this could be done in autodoc. But if you look at my script, it's
really simple, so the autodoc definitely can be extended.

But the script currently produces quite a lot of noise (some empty
classes, etc.) in the output. It needs to be improved.

Ondrej
_______________________________________________
Doc-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/doc-sig
Reply | Threaded
Open this post in threaded view
|

Re: autogenerating API docs using sphinx

Georg Brandl-2
Ondrej Certik schrieb:

> On Sun, Apr 20, 2008 at 11:47 AM, Georg Brandl <[hidden email]> wrote:
>> Ondrej Certik schrieb:
>>
>>
>> > Hi,
>>  >
>>  > I wrote a short script for autogenerating API from sources, the output
>>  > are .rst files that sphinx can parse and generate the docs in the
>>  > modules/index sections.
>>  >
>>  > Unfortunately, if the docstrings are not written in rst, it's not
>>  > going to work properly. Also it uses python inspection (it's easy to
>>  > implement), instead of parsing the source files directly.
>>  >
>>  > I am putting it here in case you find it useful. I wrote it for sympy
>>  > originally, but we are not using it yet, because the output contains
>>  > too much noise.
>>  >
>>  > Please let me know if you improve it/fix it. :)
>>
>>  What does it do differently from autodoc, and could autodoc be extended
>>  to cover your usecase as well?
>
> You mean autodoc in sphinx?

Yes.

> I want to get a documentation of all classes+methods and functions in
> all files in sympy automatically (we have a lot of .py files and I
> don't want to handle each of them manually). I didn't figure it out
> how this could be done in autodoc. But if you look at my script, it's
> really simple, so the autodoc definitely can be extended.
>
> But the script currently produces quite a lot of noise (some empty
> classes, etc.) in the output. It needs to be improved.

autodoc needs currently at least one entry per module in some file.
It does everything else your script does, as far as I can see.

Georg


--
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.

_______________________________________________
Doc-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/doc-sig