added DeepWiki badge
1 file changed
tree: 7e6a009689f96fc3b1a5d0a111bd6f98e9819262
  1. .github/
  2. connectors/
  3. dist-license/
  4. distribution/
  5. framework/
  6. lib-license/
  7. lib-proprietary/
  8. mod-authz-annotate/
  9. site/
  10. src/
  11. test-materials/
  12. .asf.yaml
  13. .geminiignore
  14. .gitignore
  15. build.xml
  16. CHANGES.txt
  17. common-build.xml
  18. connectors.xml
  19. create-release-candidate-only-artifacts.sh
  20. create-release-candidate.sh
  21. DEPENDENCIES.txt
  22. Dockerfile
  23. KEYS
  24. LICENSE.txt
  25. logging.xml
  26. NOTICE.txt
  27. pom.xml
  28. README.md
  29. README.txt
  30. sign-release-artifacts.sh
README.md

Apache ManifoldCFโ„ข

Apache ManifoldCF Release Java Baseline Concurrency Web Stack License Docker Pulls Issue Tracker DeepWiki

An open-source enterprise content integration framework designed to crawl, extract, and index documents from disparate content repositories into search engines while preserving native document Access Control Lists (ACLs).


๐Ÿš€ Key Features

  • Security-Aware Ingestion: Preserves native document Access Control Lists (ACLs) and security trimming to ensure users only search content they are authorized to view.
  • Extensive Repository Connectors: Pre-built connectors for SharePoint, Google Drive, Amazon S3, CMIS, Web, File Systems, JDBC, Nuxeo, REST APIs, and more.
  • Flexible Output Targets: Native output support for Apache Solr, OpenSearch, Elasticsearch.
  • High-Throughput Concurrency: Powered by Java 25 & Virtual Threads (Project Loom) for ultra-scalable, non-blocking asynchronous document fetching.
  • Modern Web Stack: Embedded Jetty 12.1 application container.
  • Cloud Native: Container-ready with official Docker images and Helm/Kubernetes integration support.

๐Ÿ› ๏ธ Prerequisites & Building from Source

Requirements

  • Java Development Kit (JDK): Java 25 or higher (java -version).
  • Apache Ant: Version 1.10.0 or higher (ant -version).
  • Apache Maven: Version 3.8.0 or higher (for Maven builds).

Building with Apache Ant

# Build complete release package
ant clean build

# Run single-process example
cd dist/example
java --enable-preview -jar start.jar

Building with Apache Maven

mvn clean install

๐Ÿณ Docker Quickstart

Pull and run the official Apache ManifoldCF container:

docker pull apache/manifoldcf:2.31
docker run -d -p 8345:8345 --name manifoldcf apache/manifoldcf:2.31

Access the UI at http://localhost:8345/mcf-crawler-ui.


๐Ÿ“‚ Source Code Structure

  • framework/: Core Apache ManifoldCF crawler framework and execution agents.
  • connectors/: Repository, authority, transform, and output connector modules.
  • distribution/: Packaging build targets and example setups.
  • site/: Documentation and website source files.

๐Ÿค Community & Support


๐Ÿ“„ Licensing

Apache ManifoldCF is licensed under the Apache License 2.0.