| commit | e6d3e5781e27bb1d58bd7631444e3291af358baf | [log] [tgz] |
|---|---|---|
| author | Sebastian Nagel <snagel@apache.org> | Fri Apr 10 13:41:47 2020 +0200 |
| committer | GitHub <noreply@github.com> | Fri Apr 10 13:41:47 2020 +0200 |
| tree | 05afc8b3ec41c0f2569ff7008eef7321aae29c86 | |
| parent | e9dd18015d00ad731656c8288faff6376db93f07 [diff] | |
| parent | e6bc45181369ced98fa7d9df23685620938b0c9c [diff] |
Merge pull request #506 from sebastian-nagel/NUTCH-2775-robots-min-delay NUTCH-2775 Fetcher to guarantee minimum delay even if robots.txt defines shorter Crawl-delay