Google Git
Sign in
apache / kibble-scanners / ac30427ace2767750c905883b7495304852190d3
commitac30427ace2767750c905883b7495304852190d3[log] [tgz]
authorDaniel Gruno <humbedooh@apache.org>Mon Dec 31 05:50:32 2018 +0100
committerDaniel Gruno <humbedooh@apache.org>Mon Dec 31 05:50:32 2018 +0100
treed094efd73b227848a294e36f6f586148ec0ee014
parent2c32eea1ee4e9d81bddac286709290d65a127dab [diff]
Fake user email if nto provided by JIRA

In some instances, JIRA will have email visibility turned
off for the REST API. In such instances, we should use the
domain of the JIRA instance and the username (which is still
visible to us) to fake an email address. While not perfect,
this still allows us to get a good unique count of actors.
1 file changed
tree: d094efd73b227848a294e36f6f586148ec0ee014
  1. conf/
  2. src/
  3. CODE_OF_CONDUCT.md
  4. CONTRIBUTING.md
  5. LICENSE
  6. mapping.json
  7. README.md
  8. requirements.txt
README.md

Kibble Scanner Application

The Kibble Scanners collect information for the Kibble Suite.

Setup instructions:

  • Edit conf/config.yaml to match your Kibble service

How to run:

  • On a daily/weekly/whatever basis, run: python3 src/kibble-scanner.py.

Directory structure:

  • conf/: Config files
  • src/:
    • kibble-scanner.py: Main script for launching scans
    • plugins/:
      • brokers: The various database brokers (ES or JSON API)
      • utils: Utility libraries
      • scanners: The individual scanner applications

Currently available scanner plugins:

  • Apache Pony Mail (plugins/scanners/ponymail.py)
  • Atlassian JIRA (plugins/scanners/jira.py)
  • BugZilla Issue Tracker (plugins/scanners/bugzilla.py)
  • Gerrit Code Review (plugins/scanners/gerrit.py)
  • Git Repository Fetcher (plugins/scanners/git-sync.py)
  • Git Census Counter (plugins/scanners/git-census.py)
  • Git Code Evolution Counter (plugins/scanners/git-evolution.py)
  • Git SLoC Counter (plugins/scanners/git-sloc.py)
  • GitHub Issues/PRs (plugins/scanners/github.py)
  • GitHub Traffic Statistics (plugins/scanners/github-stats.py)
  • GNU Mailman Pipermail (plugins/scanners/pipermail.py)

Requirements:

  • cloc version 1.70 or later
  • git binaries
  • python3 (3.3 or later)
  • python3-elasticsearch
  • python3-dateutils

Multi-threaded CLoC counter

To speed up things, you can use the multi-threaded version of cloc found at this location. On machines with many cores, you may experience a tenfold speed increase in the SLoC and evolution scans. It requires ForkManager, which may be installed via CPAN: cpan Parallel::ForkManager

Get involved

TBD. Please see https://kibble.apache.org/ for details!