docs.python.org, .txt and search engines

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

docs.python.org, .txt and search engines

jason kirtland-2
The reST sources on http://docs.python.org/ are getting indexed in
Google and ":mod:`whatnot`" is showing up in search results.  That's
been more distracting than valuable to me in my searches.  Maybe add a
robots.txt exclusion for /_sources and the other _sources directories
on docs.python.org?

  User-agent: *
  Disallow: /_sources/
  Disallow: /3.0/_sources/

Cheers,
Jason
_______________________________________________
Doc-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/doc-sig
Reply | Threaded
Open this post in threaded view
|

Re: docs.python.org, .txt and search engines

Aahz
On Fri, Dec 12, 2008, jason kirtland wrote:

>
> The reST sources on http://docs.python.org/ are getting indexed in
> Google and ":mod:`whatnot`" is showing up in search results.  That's
> been more distracting than valuable to me in my searches.  Maybe add a
> robots.txt exclusion for /_sources and the other _sources directories
> on docs.python.org?
>
>   User-agent: *
>   Disallow: /_sources/
>   Disallow: /3.0/_sources/

This is the first time I've heard this complaint, and nobody else has
spoken up in favor.  Overall, I think I'm -0; somebody may well want to
search the sources.
--
Aahz ([hidden email])           <*>         http://www.pythoncraft.com/

"It is easier to optimize correct code than to correct optimized code."
--Bill Harlan
_______________________________________________
Doc-SIG maillist  -  [hidden email]
http://mail.python.org/mailman/listinfo/doc-sig