Camel Kit Knowledge

Knowledge layer for Camel Kit -- semantic search over Apache Camel documentation, release notes, CVE advisories, and component metadata.

Release 0.0.2 bundles the Camel 4.22.1 catalog for endpoint validation. Development continues at 0.0.3-SNAPSHOT. Unless explicitly marked 0.0.2, this README describes the current monorepo sources. Their downloadable knowledge index uses independent knowledge-index-v1-* releases; the published 0.0.2 server keeps its original download URL and does not support the new release-discovery or pinning options.

Modules

Prerequisites

• Java 17+
• Use the Maven wrapper at the repository root. All commands below run from that root.

No Docker. No Docling. No external tokens.

Data Sources

Dynamic Version Resolution

An explicit index rebuild discovers the documentation versions to ingest. Normal builds use the committed index without refreshing its content. This selection does not change the server's separately pinned endpoint-validation catalog.

1. Active versions -- the indexer refreshes apache/camel-website and parses release frontmatter (content/releases/release-*.md). It selects every supported LTS line (kind: lts, eol > today), plus the latest release line only when it is non-LTS. A latest LTS release therefore leaves no non-LTS line selected. Draft and future-dated releases are ignored; the highest published patch represents each selected line.
2. Release tags -- latest release tags are resolved via git ls-remote (JGit) for camel, camel-spring-boot, and camel-quarkus
3. Quarkus-to-Camel mapping -- resolved by fetching each Quarkus release tag's pom.xml from GitHub and reading the camel.major.minor property
4. Tag-aware cloning -- repos are cloned at immutable release tags (not moving branches). .fetched-tag marker files enable cache reuse between builds

The selected lines control component documentation, runtime documentation, and Camel Catalog metadata. Historical release notes from the oldest selected line onward remain searchable, so older non-LTS versions can still appear in camel_docs_index_info through their release history. CVE advisories are indexed independently of this version selection.

AsciiDoc Conversion

Component documentation is converted directly from AsciiDoc to Markdown via a custom AsciidoctorJ converter (MarkdownConverter). No HTML intermediate, no Docling container.

• Antora partial$ includes resolved by inlining partial files with attribute substitution
• Antora jsonpath$ / jsonpathcount$ extensions reimplemented in Java (JsonPathIncludeProcessor, pre-processing pipeline)
• jsonpathTable:: block macros render option tables from component JSON data
• Component docs render with real data (e.g. “The Kafka component supports 127 options”)

Camel Catalog Integration

Endpoint validation uses the bundled Camel 4.22.1 catalog. The downloadable index is refreshed independently; updating it does not change the validator catalog.

CamelCatalogIndexer downloads the camel-catalog JAR from Maven Central for each active version, extracts component and EIP JSON metadata, and creates document chunks with structured option data (properties, types, defaults, descriptions). If the exact version JAR is not on Maven Central (unreleased), it falls back to previous patch versions.

Build

Normal build (uses pre-built index)

./mvnw -B -N install
./mvnw -B -f knowledge/pom.xml clean install

The MCP module downloads the ONNX embedding model from HuggingFace during generate-resources (skipped if already cached).

All Knowledge modules use Maven group org.apache.camel and Java packages under org.apache.camel.kit.knowledge. Java consumers must update dependencies and imports from the previous io.github.luigidemasi coordinates and io.github.luigidemasi.camelkit package prefix. Knowledge and Kit inherit build conventions from the root parent, but retain independent versions and release schedules. The MCP server also works with clients that do not use Kit.

Rebuild index from source

./mvnw install -pl knowledge/index -Prebuild-index -am -B

This will:

1. Clone/update Git repos at latest release tags
2. Render AsciiDoc to Markdown with resolved option tables
3. Download Camel Catalog JARs for structured option data
4. Parse release notes and CVE advisories
5. Enrich release notes with Apache JIRA issue details (public, no auth required)
6. Generate embeddings (384-dim vectors; cached in knowledge/indexer/src/main/resources/apache-camel/embedding-cache/, so incremental rebuilds only embed new or changed chunks)
7. Write the Lucene index to knowledge/index/src/main/resources/knowledge-index/ plus an index.json manifest skeleton

CVE enrichment and fallback

Apache Camel advisories remain the source for affected versions, fixes, severity, and descriptions. For optional CVSS scores/vectors and CWE classifications, the indexer reuses valid cached NVD records, then queries NVD. If NVD returns an HTTP error (including 429), times out, or returns an invalid or missing record, it falls back to CIRCL Vulnerability-Lookup, using its FKIE mirror of NVD data. A mirror can lag behind NVD; only a record matching the requested CVE ID is accepted.

No API key is required. Uncached requests are spaced at least 6.1 seconds apart across both services. Each request has a 15-second deadline including its response body, with a 5-second connection timeout. HTTP 429/503 responses put that service on cooldown according to Retry-After (30 seconds when absent or invalid); the other service is still eligible. Each lookup attempts each available service once.

Successful responses are cached under knowledge/indexer/src/main/resources/apache-camel/cve-cache/, together with their source, URL, and retrieval time. Existing NVD caches remain usable. If both services fail, the Apache advisory is still indexed without the optional enrichment; failures are not cached, so a later rebuild retries them. Rerun the rebuild command after updating the indexer to enrich previously skipped CVEs; completed index files are not changed automatically.

Index Distribution

The index is data, not code — it is published as a GitHub Release asset, not a Maven artifact. Reindexing is performed on a branch; the generated index is committed with a signed commit and reviewed through a PR. After that PR is merged, the Index Release workflow (manually dispatched from main with the reviewed commit SHA, independently of server releases) builds the reviewed index without regenerating it, runs the retrieval-quality gate (RetrievalQualityTest, with working vectors required), verifies that the build did not modify the index, and publishes the reviewed knowledge-index.zip + index.json to a release tagged knowledge-index-v1-YYYY.MM.DD-HHMMSS.

On the first knowledge query, the MCP server resolves the index in this order:

1. knowledge.index.path — a local index directory, used in place (dev, tests, air-gapped)
2. Release discovery — query https://api.github.com/repos/apache/camel-kit/releases and select the newest published knowledge-index-v1-YYYY.MM.DD-HHMMSS release. Product releases, drafts, prereleases and other format channels are ignored. knowledge.index.version=YYYY.MM.DD-HHMMSS selects an exact index tag instead. An explicit knowledge.index.url bypasses discovery and supports legacy or mirrored manifests. The manifest must match the selected tag, v1 format, embedding model and 384 dimensions. The archive is SHA-256 verified, unzipped and activated atomically in ~/.camel-kit/knowledge-index/. A failed update retains the previous working index; a pinned client may fall back only to that exact cached version. The index is opened directly from the cache, without per-startup extraction.
3. Classpath — legacy fallback for uber-jars bundling the index, only when no exact index version is pinned

Discovery runs on the first knowledge query in each server process. It follows at most 10 release-list pages (100 releases per page); configure an explicit manifest URL if that bound is exceeded. Each page has the same five-second response deadline as a manifest. The v1 manifest contract is formatVersion: 1, model granite-embedding-small-english-r2-q8, and embeddingDimensions: 384. Legacy manifests may omit format and dimension fields, but must identify that model. New publishers include all fields. A breaking format or model change requires a new channel.

The first monorepo index release must be published before default discovery is usable. Until then, set knowledge.index.path to the committed local index, or use the old manifest explicitly: https://github.com/luigidemasi/camel-kit-knowledge/releases/latest/download/index.json. Existing released servers continue to use their original artifact and download URLs.

The MCP handshake and tool listing do not initialize the index; call camel_docs_index_info to check readiness. With no usable cache, a failed first installation requires a reachable manifest or knowledge.index.path. The cache filesystem must support advisory file locks and atomic moves for both version directories and metadata. If acquiring the lock or an atomic move fails, a readable active cache remains usable; without one, the download attempt fails. A later attempt can activate an update once the failure clears; retrying alone cannot add missing advisory-lock or atomic-move support. Atomic moves do not guarantee durability across power loss.

Cache updates are serialized across threads and processes, from reading the active version through pruning. Cache sharing requires all clients to use this locking protocol; older clients without it must use a separate cache. When another updater holds the lock, a caller uses a readable active cache immediately. Without a usable cache, lock acquisition waits at most five seconds in total across threads and processes before the download attempt fails. Each download uses its own staging directory. The next locked attempt cleans up recognized interrupted staging files, including legacy <version>.part entries only when they are real directories and the version prefix follows the safe-name rules below. New metadata files honor the process umask; replacements preserve existing POSIX permissions. HTTP connections have a three-second timeout. Manifest requests must finish within five seconds and archive requests within five minutes, including the response body; a timeout falls back to the readable active cache when available. The manifest ETag is saved after activation and bound to the manifest URL and version; saving it is best effort. An HTTP 200 response without an ETag removes the old validator, and legacy unbound validators trigger a fresh check.

Custom manifest versions must be 1–128 ASCII letters, digits, dots, underscores or hyphens, beginning with a letter or digit. Trailing dots, .part suffixes, current, etag, and Windows device names are reserved (case-insensitive). Version directories must be real directories inside the cache, not symbolic links.

Two runtime guards protect the vector leg: the index carries an embedding-model stamp (__index_meta__), and a startup self-check re-embeds a few stored chunks and verifies the stored vectors match — on any mismatch, vector search is disabled and search degrades to BM25 + reranker with a loud log line.

Running the MCP Server

Current source build

After the build above, run the server from the repository root with the committed index:

java -Dknowledge.index.path=knowledge/index/src/main/resources/knowledge-index \
  -jar knowledge/mcp/target/camel-kit-knowledge-mcp-0.0.3-SNAPSHOT-runner.jar

The installed runner's Maven coordinates are org.apache.camel:camel-kit-knowledge-mcp:0.0.3-SNAPSHOT:runner. Kit still pins the released 0.0.2 runner until explicitly configured otherwise; building this source does not alter that pin. See using the local runner with Kit.

The server uses stdio by default and does not open an HTTP port. To enable the HTTP and SSE transports explicitly on localhost:

java -Dquarkus.http.host-enabled=true -Dquarkus.http.host=127.0.0.1 \
  -Dquarkus.http.port=9090 \
  -Dknowledge.index.path=knowledge/index/src/main/resources/knowledge-index \
  -jar knowledge/mcp/target/camel-kit-knowledge-mcp-0.0.3-SNAPSHOT-runner.jar

Published release 0.0.2

The existing release remains available at its original coordinates:

jbang io.github.luigidemasi:camel-kit-knowledge-mcp:0.0.2:runner

It uses the standalone repository's original index download URL. The monorepo release-discovery and exact-version pinning behavior below requires the current source build.

Configure index selection in current source builds

Choose one of these configurations in an external config/application.properties relative to the server's working directory, or pass each property as a -Dkey=value argument before -jar. Remove the local-path argument from the examples above when using discovery or a manifest: knowledge.index.path takes precedence over both.

To select an exact monorepo release, replace this illustrative version with an existing published index version:

# Exact published tag suffix; example only
knowledge.index.version=2026.10.26-120000

To bypass release discovery with a mirrored manifest (which must reference an accessible archive), use:

knowledge.index.url=file:///opt/mirrors/camel-kit/index.json

To use an unpacked local index without downloading it, use:

knowledge.index.path=/opt/camel-kit/knowledge-index

An explicit manifest URL may also be combined with a version pin; its manifest version must match the pin. A local path is used directly and bypasses release-version selection. Environment equivalents include KNOWLEDGE_INDEX_PATH, KNOWLEDGE_INDEX_URL and KNOWLEDGE_INDEX_VERSION in the MCP server process.

MCP Tools

7 tools with camel_docs_* prefix:

Search Architecture

Hybrid search combining:

• BM25 text search (20% weight) -- keyword matching via Lucene's standard analyzer
• Vector search (80% weight) -- semantic similarity via KNN float vectors (384-dim Granite embeddings)

Full-text results are reranked with a local cross-encoder when its model is available. If embeddings are unavailable or fail the compatibility checks, retrieval falls back to BM25, with reranking when available; camel_docs_index_info reports the active mode.

Component lookups (camel_docs_component_info) first query the exact structured component field. If there is no exact hit, they use full-text search and label the response match: "fuzzy"; exact hits use match: "exact", and no results use match: "none".

Configuration Reference

System properties

MCP application properties

Cache locations (gitignored)

Apache JIRA (CAMEL-* issues) is public and requires no authentication.

See the shared release guide for independent parent, product and maintainer-triggered index publication.

License

Apache License 2.0