commit | 19ee562a4f917dcb3adf9603ce816ac50b50202f | [log] [tgz] |
---|---|---|
author | Daniel Gruno <humbedooh@apache.org> | Wed Jul 17 05:54:38 2019 +0200 |
committer | Daniel Gruno <humbedooh@apache.org> | Wed Jul 17 05:54:38 2019 +0200 |
tree | 8312ea59972e9236561e946639c58c504e51b5eb | |
parent | 31ebcf3fcca0a40fa3205bf5e9a58022dc9f4fa3 [diff] |
work around rate limits by sleeping it off If we get hit by rate limits, first try to sleep it off in 5 minute intervals till we hopefully get refreshed limits, otherwise bail after a max of 2 hours of waiting.
The Kibble Scanners collect information for the Kibble Suite.
python3 src/kibble-scanner.py
.usage: kibble-scanner.py [-h] [-o ORG] [-f CONFIG] [-a AGE] [-s SOURCE] [-n NODES] [-t TYPE] [-e EXCLUDE [EXCLUDE ...]] [-v VIEW] optional arguments: -h, --help show this help message and exit -o ORG, --org ORG The organisation to gather stats for. If left out, all organisations will be scanned. -f CONFIG, --config CONFIG Location of the yaml config file (full path) -a AGE, --age AGE Minimum age in hours before performing a new scan on an already processed source. --age 12 will not process any source that was processed less than 12 hours ago, but will process new sources. -s SOURCE, --source SOURCE A specific source (wildcard) to run scans on. -n NODES, --nodes NODES Number of nodes in the cluster (used for load balancing) -t TYPE, --type TYPE Specific type of scanner to run (default is run all scanners) -e EXCLUDE [EXCLUDE ...], --exclude EXCLUDE [EXCLUDE ...] Specific type of scanner(s) to exclude -v VIEW, --view VIEW Specific source view to scan (default is scan all sources)
conf/
: Config filessrc/
:kibble-scanner.py
: Main script for launching scansplugins/
:brokers
: The various database brokers (ES or JSON API)utils
: Utility librariesscanners
: The individual scanner applicationsplugins/scanners/ponymail.py
)plugins/scanners/jira.py
)plugins/scanners/bugzilla.py
)plugins/scanners/gerrit.py
)plugins/scanners/git-sync.py
)plugins/scanners/git-census.py
)plugins/scanners/git-evolution.py
)plugins/scanners/git-sloc.py
)plugins/scanners/github.py
)plugins/scanners/github-stats.py
)plugins/scanners/pipermail.py
)(optional)
TBD. Please see https://kibble.apache.org/ for details!