Apache ManifoldCF™

An open-source enterprise content integration framework designed to crawl, extract, and index documents from disparate content repositories into search engines while preserving native document Access Control Lists (ACLs).

🚀 Key Features

• Security-Aware Ingestion: Preserves native document Access Control Lists (ACLs) and security trimming to ensure users only search content they are authorized to view.
• Extensive Repository Connectors: Pre-built connectors for SharePoint, Google Drive, Amazon S3, CMIS, Web, File Systems, JDBC, Nuxeo, REST APIs, and more.
• Flexible Output Targets: Native output support for Apache Solr, OpenSearch, Elasticsearch.
• High-Throughput Concurrency: Powered by Java 25 & Virtual Threads (Project Loom) for ultra-scalable, non-blocking asynchronous document fetching.
• Modern Web Stack: Embedded Jetty 12.1 application container.
• Cloud Native: Container-ready with official Docker images and Helm/Kubernetes integration support.

🛠️ Prerequisites & Building from Source

Requirements

• Java Development Kit (JDK): Java 25 or higher (java -version).
• Apache Ant: Version 1.10.0 or higher (ant -version).
• Apache Maven: Version 3.8.0 or higher (for Maven builds).

Building with Apache Ant

# Build complete release package
ant clean build

# Run single-process example
cd dist/example
java --enable-preview -jar start.jar

Building with Apache Maven

mvn clean install

🐳 Docker Quickstart

Pull and run the official Apache ManifoldCF container:

docker pull apache/manifoldcf:2.31
docker run -d -p 8345:8345 --name manifoldcf apache/manifoldcf:2.31

Access the UI at http://localhost:8345/mcf-crawler-ui.

📂 Source Code Structure

• framework/: Core Apache ManifoldCF crawler framework and execution agents.
• connectors/: Repository, authority, transform, and output connector modules.
• distribution/: Packaging build targets and example setups.
• site/: Documentation and website source files.

🤝 Community & Support

• Website: https://manifoldcf.apache.org/
• Issue Tracker: Apache JIRA (CONNECTORS)
• Mailing Lists:
  • Developer List: dev@manifoldcf.apache.org
  • User List: user@manifoldcf.apache.org

📄 Licensing

Apache ManifoldCF is licensed under the Apache License 2.0.