perl-www-robotrules
Perl - WWW::RobotRules
3.6.0
2026-01-10
eisfair-noarch
the eisfair team, team(at)eisfair(dot)org
stable
perl 3.6.0
perl-uri 3.6.0
Internal Program Version: WWW::RobotRules 6.02.5.42.0
This module parses _/robots.txt_ files as specified in "A Standard for
Robot Exclusion", at http://www.robotstxt.org/wc/norobots.html Webmasters
can use the _/robots.txt_ file to forbid conforming robots from accessing
parts of their web site.
http://search.cpan.org/dist/WWW-RobotRules