[ # comment string NL ]*
User-Agent: [ [ WS ]+ agent_id ]+ [ [ 
WS ]* # comment string ]? NL
[ # comment string NL ]*
Disallow: [ [ WS ]+ patch_root ]* [ [
WS ]* # comment string ]? NL
[
# comment string NL
|
Disallow: [ [ WS ]+ patch_root ]* [ [
WS ]* # comment string ]? NL
]*
[ NL ]+