tree ed997318ba59c1901d338885d5bcce7850716036
parent aa0c75e72725598e7bd8b9715734b4463bdc7828
parent 46db3ed71355fefda42a008ece75094f51859ab2
author Sebastian Nagel <snagel@apache.org> 1588687144 +0200
committer GitHub <noreply@github.com> 1588687144 +0200
gpgsig -----BEGIN PGP SIGNATURE-----
 
 wsBcBAABCAAQBQJesXEoCRBK7hj4Ov3rIwAAdHIIAJJ6EPzh4mPC8yPxPHa+UEVP
 +DPK3BZJbbJZ8f91rYO3Wg5OLj3gGz8MSqmCv4QqaDex5AzYrWyxa2qe9qrJ7igT
 5FIa8OZZWNDbZFBwyFf05nKC3tnva7Kk4uUGT0A08wloFDv1a9EzKgl5iuMGjBzl
 z7cuT7PbhzLVARSVOl9nMcSjXjj3dVmGVNOopZJA4f86JfINrw1nkJsY1ocKPV5d
 xg4m6BRzN/mguU6YiM866f9nZ3G0dSbk9XaK/LhxVK4NVcicC9JHYz1x1rHnbLY1
 elxqZzLgWnMhlfrMTkRnkZI1L8QCb+oKYVKTwX5YwzCyj5k52FVSo5dAFA60z2s=
 =uAXq
 -----END PGP SIGNATURE-----
 

Merge pull request #521 from sebastian-nagel/NUTCH-2002-checkers-robotstxt

NUTCH-2002 parse and index checkers to check robots.txt