Google Git
Sign in
apache / kibble-scanners / d4750da886f78e41a3d6ff9df8003d6bd1fa33bf
commitd4750da886f78e41a3d6ff9df8003d6bd1fa33bf[log] [tgz]
authorDaniel Gruno <humbedooh@apache.org>Fri Mar 02 12:07:45 2018 +0100
committerDaniel Gruno <humbedooh@apache.org>Fri Mar 02 12:07:45 2018 +0100
tree0d1f4d78000f8e6ee42ab970e67eab4e1a0ddb14
parent9a94bde0e097247522bb2f0a169ed00f7b8b4072 [diff]
Alignm with new DB format

if the DB is typeless, write to it and fetch from it
accordingly.
1 file changed
tree: 0d1f4d78000f8e6ee42ab970e67eab4e1a0ddb14
  1. conf/
  2. src/
  3. CODE_OF_CONDUCT.md
  4. CONTRIBUTING.md
  5. LICENSE
  6. mapping.json
  7. README.md
  8. requirements.txt
README.md

Kibble Scanner Application

The Kibble Scanners collect information for the Kibble Suite.

Setup instructions:

  • Edit conf/config.yaml to match your Kibble service

How to run:

  • On a daily/weekly/whatever basis, run: python3 src/kibble-scanner.py.

Directory structure:

  • conf/: Config files
  • src/:
    • kibble-scanner.py: Main script for launching scans
    • plugins/:
      • brokers: The various database brokers (ES or JSON API)
      • utils: Utility libraries
      • scanners: The individual scanner applications

Currently available scanner plugins:

  • Apache Pony Mail (plugins/scanners/ponymail.py)
  • Atlassian JIRA (plugins/scanners/jira.py)
  • BugZilla Issue Tracker (plugins/scanners/bugzilla.py)
  • Gerrit Code Review (plugins/scanners/gerrit.py)
  • Git Repository Fetcher (plugins/scanners/git-sync.py)
  • Git Census Counter (plugins/scanners/git-census.py)
  • Git Code Evolution Counter (plugins/scanners/git-evolution.py)
  • Git SLoC Counter (plugins/scanners/git-sloc.py)
  • GitHub Issues/PRs (plugins/scanners/github.py)
  • GitHub Traffic Statistics (plugins/scanners/github-stats.py)
  • GNU Mailman Pipermail (plugins/scanners/pipermail.py)

Requirements:

  • cloc version 1.70 or later
  • git binaries
  • python3 (3.3 or later)
  • python3-elasticsearch
  • python3-dateutils

Multi-threaded CLoC counter

To speed up things, you can use the multi-threaded version of cloc found at this location. On machines with many cores, you may experience a tenfold speed increase in the SLoC and evolution scans. It requires ForkManager, which may be installed via CPAN: cpan Parallel::ForkManager

Get involved

TBD. Please see https://kibble.apache.org/ for details!