Class
RobotRules
In:
lib/spider/robot_rules.rb
Parent:
Object
Based on Perl‘s WWW::RobotRules module, by Gisle Aas.
Methods
allowed?
new
parse
Public Class methods
new
( user_agent )
Public Instance methods
allowed?
( text_uri )
parse
( text_uri, robots_data )
[Validate]