| Apache Ignite 3 is a distributed database for high-performance computing. |
| |
| ## Common Development Commands |
| |
| ### Building the Project |
| - **Full build**: `./gradlew clean build` |
| - **Fast build (skip tests)**: `./gradlew clean build -x test -x integrationTest` |
| - **Fastest build (skip everything)**: `./gradlew clean build -x assembleDist -x distTar -x distZip -x check` |
| |
| ### Testing |
| - **Unit tests only**: `./gradlew clean test` |
| - **Integration tests only**: `./gradlew clean integrationTest` |
| - **Both unit + integration**: `./gradlew clean test integrationTest` |
| - **Run single test class**: `./gradlew :ignite-<module>:test --tests "*TestClassName*"` |
| |
| ### Code Quality Checks |
| - **All checks**: `./gradlew clean check` |
| - **Checkstyle only**: `./gradlew checkstyleMain checkstyleIntegrationTest checkstyleTest checkstyleTestFixtures` |
| - **Spotbugs only**: `./gradlew spotbugsMain` |
| - **PMD only**: `./gradlew pmdMain pmdTest` |
| |
| ### Packaging and Distribution |
| - **Create distribution packages**: `./gradlew clean allDistZip` |
| - **CLI package only**: `./gradlew clean packaging-cli:distZip` |
| - **DB package only**: `./gradlew clean packaging-db:distZip` |
| - **Docker image**: `./gradlew clean docker -x test -x check` |
| |
| ### Running Ignite (Manual) |
| - **Start node from distribution**: `./bin/ignite3db start` (from unpacked distribution) |
| - **Connect CLI**: `./bin/ignite3` (from CLI distribution) |
| - **Docker Compose cluster**: `docker compose -f packaging/docker/docker-compose.yml up -d` |
| |
| ### Benchmarks |
| - **Run module benchmarks**: `./gradlew clean :ignite-<module>:jmh` |
| - **Run specific benchmark**: `./gradlew clean :ignite-<module>:jmh -PjmhBench=BenchmarkName.methodName` |
| - **JFR profiling**: `./gradlew :ignite-<module>:jmh -PjmhProfileJfr` |
| |
| ### Documentation |
| - **Generate Javadoc**: `./gradlew aggregateJavadoc` |
| - **Module-specific Javadoc**: `./gradlew javadoc` |
| |
| ## High-Level Architecture |
| |
| ### Core Components |
| Apache Ignite 3 is built with a modular, component-based architecture where components form an acyclic dependency graph: |
| |
| 1. **Runner Module** (`modules/runner/`): Main entry point that wires up all components and handles node lifecycle |
| 2. **Network Module** (`modules/network/`): Group membership, messaging, and cluster communication |
| 3. **Metastorage Module** (`modules/metastorage/`): Distributed key-value storage for cluster metadata using Raft consensus |
| 4. **Catalog Module** (`modules/catalog/`): Schema management and DDL operations |
| 5. **Table Module** (`modules/table/`): Table API implementation and partition management |
| 6. **SQL Engine** (`modules/sql-engine/`): Distributed SQL query processing with Apache Calcite |
| 7. **Storage Engines**: Pluggable storage with RocksDB (`modules/storage-rocksdb/`) and in-memory (`modules/storage-page-memory/`) options |
| 8. **Transactions** (`modules/transactions/`): ACID transactions with MVCC and serializable isolation |
| 9. **Compute** (`modules/compute/`): Distributed compute framework for job execution |
| 10. **Configuration** (`modules/configuration/`): Dynamic configuration management system |
| |
| ### Component Initialization |
| Components are initialized in topological sort order based on dependencies. Each component: |
| - Has clearly defined dependencies provided at construction time |
| - Receives metastorage watch notifications in dependency order |
| - Must not create cyclic dependencies |
| - Uses the Vault (`modules/vault/`) for persistent local state |
| |
| ### Key Architectural Patterns |
| - **Schema-Driven**: All operations are based on explicit schemas defined through DDL |
| - **Watch-Based Updates**: Components react to metastorage changes via reliable watch processing |
| - **Pluggable Storage**: Storage engines can be swapped (RocksDB, in-memory, custom) |
| - **Partition-Based**: Data is distributed using partition-based sharding |
| - **Raft for Consensus**: Critical components (metastorage, table partitions) use Raft for consistency |
| |
| ### Testing Structure |
| - **Unit tests**: Located in `src/test/java/` - use JUnit 5 with `@Test` annotations |
| - **Integration tests**: Located in `src/integrationTest/java/` - test component interactions |
| - **Test fixtures**: Located in `src/testFixtures/java/` - shared test utilities |
| - **Benchmarks**: Located in `src/jmh/java/` - JMH performance benchmarks |
| |
| ### Module Organization |
| Each module follows standard Gradle structure: |
| - `src/main/java/` - Main source code |
| - `src/test/java/` - Unit tests |
| - `src/integrationTest/java/` - Integration tests |
| - `src/testFixtures/java/` - Test utilities |
| - `build.gradle` - Module-specific build configuration |
| |
| Key modules are organized by functional area (storage, network, sql-engine, etc.) with clear API boundaries and minimal inter-module dependencies. |
| |
| ## Code Style & Commit Rules |
| |
| ### Before Every Commit (MANDATORY) |
| Run checkstyle and PMD on the modified module(s): |
| ```bash |
| ./gradlew :ignite-<module>:checkstyleMain :ignite-<module>:checkstyleTest :ignite-<module>:pmdMain :ignite-<module>:pmdTest |
| ``` |
| |
| Run IDEA inspections on the modified module(s): |
| ```bash |
| idea inspect . .idea/inspectionProfiles/Project_Default.xml /tmp/results -d modules/<module> |
| ``` |
| Notes: |
| - IntelliJ IDEA must be closed for command-line inspections to work. |
| - If `idea` command is not available, ask the user to install it via: **Tools > Create Command-line Launcher** in IntelliJ IDEA. |
| - Check results: `find /tmp/results -name "*.xml" ! -name ".descriptions.xml" -exec cat {} \;` |
| |
| ### Before Every PR (MANDATORY) |
| Run IDEA inspections on the modified module(s) and fix all issues: |
| ```bash |
| idea inspect . .idea/inspectionProfiles/Project_Default.xml /tmp/results -d modules/<module> |
| ``` |
| **IMPORTANT**: Always run IDEA inspections before creating or updating a PR. The inspections catch issues that checkstyle/PMD miss (e.g., methods that can be static, redundant suppressions, etc.). |
| |
| ### Git Push |
| **Never use `git push` without specifying the target branch.** Always push explicitly: |
| ```bash |
| git push origin <branch-name> |
| ``` |
| |
| ### Design Preferences |
| - **Prefer objects over static methods** - Use constructor injection everywhere possible |
| - **Always include Jira ticket link in PRs** - Link to https://issues.apache.org/jira/browse/IGNITE-XXXXX |
| |
| ## Jira Workflow |
| |
| - All tickets **must** have the `ignite-3` label |
| - When resolving a ticket: |
| - **Reviewer** must be set (ask if unsure whom to set) |
| - **Fix Version** must be set |