Google Git
Sign in
apache / kibble-scanners / cd0819008e0d55a11345ff90d76238f1a87fca04
commitcd0819008e0d55a11345ff90d76238f1a87fca04[log] [tgz]
authorDaniel Gruno <humbedooh@apache.org>Thu Dec 07 11:35:24 2017 +0100
committerDaniel Gruno <humbedooh@apache.org>Thu Dec 07 11:35:24 2017 +0100
treed68c3352aa2117001a3019701d9c459e8c4a1c16
parentecc80ce1e3c4dee8afb0cc9a2cda434021686c65 [diff]
rework tone lib to accept an array of bodies

Azure accepts up to 1000 bodies at the same time to
speed up and prevent rate limits, so let's make use of that.
also rework the watson to accept an array, even through
it's still one call per body.
1 file changed
tree: d68c3352aa2117001a3019701d9c459e8c4a1c16
  1. conf/
  2. src/
  3. CODE_OF_CONDUCT.md
  4. CONTRIBUTING.md
  5. LICENSE
  6. mapping.json
  7. README.md
  8. requirements.txt
README.md

Kibble Scanner Application

The Kibble Scanners collect information for the Kibble Suite.

Setup instructions:

  • Edit conf/config.yaml to match your Kibble service

How to run:

  • On a daily/weekly/whatever basis, run: python3 src/kibble-scanner.py.

Directory structure:

  • conf/: Config files
  • src/:
    • kibble-scanner.py: Main script for launching scans
    • plugins/:
      • brokers: The various database brokers (ES or JSON API)
      • utils: Utility libraries
      • scanners: The individual scanner applications

Currently available scanner plugins:

  • Apache Pony Mail (plugins/scanners/ponymail.py)
  • Atlassian JIRA (plugins/scanners/jira.py)
  • BugZilla Issue Tracker (plugins/scanners/bugzilla.py)
  • Gerrit Code Review (plugins/scanners/gerrit.py)
  • Git Repository Fetcher (plugins/scanners/git-sync.py)
  • Git Census Counter (plugins/scanners/git-census.py)
  • Git Code Evolution Counter (plugins/scanners/git-evolution.py)
  • Git SLoC Counter (plugins/scanners/git-sloc.py)
  • GitHub Issues/PRs (plugins/scanners/github.py)
  • GitHub Traffic Statistics (plugins/scanners/github-stats.py)
  • GNU Mailman Pipermail (plugins/scanners/pipermail.py)

Requirements:

  • cloc version 1.70 or later
  • git binaries
  • python3 (3.3 or later)
  • python3-elasticsearch
  • python3-dateutils

Multi-threaded CLoC counter

To speed up things, you can use the multi-threaded version of cloc found at this location. On machines with many cores, you may experience a tenfold speed increase in the SLoC and evolution scans. It requires ForkManager, which may be installed via CPAN: cpan Parallel::ForkManager

Get involved

TBD. Please see https://kibble.apache.org/ for details!