Merge pull request #4 from sleeepyjack/cccl-pin-synthetic-version
diff --git a/.github/workflows/pre-commit.yml b/.github/workflows/pre-commit.yml
index 3f5b9a1..d4843d6 100644
--- a/.github/workflows/pre-commit.yml
+++ b/.github/workflows/pre-commit.yml
@@ -19,5 +19,14 @@
with:
python-version: '3.x'
+ - name: Cache pre-commit environments
+ uses: actions/cache@v4
+ with:
+ path: ~/.cache/pre-commit
+ key: pre-commit-${{ runner.os }}-${{ hashFiles('.pre-commit-config.yaml') }}
+
+ - name: Install pre-commit
+ run: python -m pip install pre-commit
+
- name: Run pre-commit
- uses: pre-commit/action@v3.0.1
+ run: pre-commit run --all-files
diff --git a/README.md b/README.md
index ddab10a..a563271 100644
--- a/README.md
+++ b/README.md
@@ -17,9 +17,11 @@
under the License.
-->
-[](https://central.sonatype.com/artifact/org.apache.datasketches/datasketches-cuda)
-
# Apache® DataSketches™ Core CUDA Library Component
+
+> **Note:** This project is experimental and under active development. APIs and
+> behavior may change without notice.
+
This is the core CUDA component of the DataSketches library. It contains sketching algorithms that can be accessed directly from user applications.
Note that we have parallel core library components for Java, C++, Python, GO, and Rush implementations of many of the same sketch algorithms:
@@ -79,7 +81,7 @@
Fetched automatically via CPM at configure time (no manual install required):
-- [NVIDIA/cccl](https://github.com/NVIDIA/cccl) — currently fetched from `main` while this library develops against unreleased cudax HLL APIs. This should move to a versioned CCCL release once the required APIs are tagged.
+- [NVIDIA/cccl](https://github.com/NVIDIA/cccl) — pinned to commit `c95f99757cf95044ce82b905eec88ff40c851f7b` as synthetic version `3.5.1` while this library develops against unreleased cudax HLL APIs. This should move to a real CCCL release once the required APIs are tagged.
- [apache/datasketches-cpp](https://github.com/apache/datasketches-cpp) `5.2.0` (fall-back if `find_package(DataSketches 5.0.0 CONFIG)` does not locate a system install)
- [Catch2](https://github.com/catchorg/Catch2) `3.5.3` (test-only)
@@ -136,5 +138,5 @@
- **HLL_8 only.** `HLL_4` and `HLL_6` packing are not yet implemented; constructing with those throws `std::invalid_argument`. `AuxHashMap` (the HLL_4 exception table) is also pending.
- **No LIST / SET deserialization.** The wire format's small-cardinality modes are rejected at parse. Sketches must already be in HLL mode.
- **Round-trip diverges on `FLAGS` (oooFlag) and `hipAccum`.** GPU output always sets `oooFlag=1` (pins CPU side to the Composite estimator) and `hipAccum=0` (no HIP tracking on parallel atomic update). All other bytes round-trip exactly.
-- **CCCL tracks `main` during active development.** Until upstream tags a CCCL release containing the required cudax HLL policy and explicit stream / memory-resource APIs, `cmake/thirdparty/get_cccl.cmake` CPM-fetches `NVIDIA/cccl:main` and emits a warning at configure time. `get_cccl.cmake` carries a `TODO(find_package)` block for the future switch to a version-guarded release lookup.
+- **CCCL uses a synthetic development version.** Until upstream tags a CCCL release containing the required cudax HLL policy and explicit stream / memory-resource APIs, `cmake/thirdparty/get_cccl.cmake` uses `CPMFindPackage` with synthetic version `3.5.1` and a pinned CCCL main commit. This prevents automatically accepting older CCCL installs from disk while keeping an explicit `CPM_CCCL_SOURCE` override available for development.
- **No driver on some dev hosts.** CI gates the runtime parity test (`parity_test.cu`); host-only tests (preamble, reduction state, normalizing hasher, composite finalizer, policy compile) pass without a GPU.
diff --git a/cmake/thirdparty/get_cccl.cmake b/cmake/thirdparty/get_cccl.cmake
index 630335f..3fcd202 100644
--- a/cmake/thirdparty/get_cccl.cmake
+++ b/cmake/thirdparty/get_cccl.cmake
@@ -15,13 +15,14 @@
# specific language governing permissions and limitations
# under the License.
-# NVIDIA/cccl provides CCCL (which includes cudax). This project is still under
-# active development against unreleased cudax HyperLogLog APIs, so use CCCL main
-# for now instead of a released tag.
+# NVIDIA/cccl provides CCCL (which includes cudax). This project currently needs
+# unreleased cudax HyperLogLog APIs. Pin a known-good CCCL main commit and assign
+# it a synthetic version newer than the latest real CCCL release, so
+# CPMFindPackage will not silently accept an older CCCL install from disk.
#
# TODO(find_package): once NVIDIA/cccl ships a tagged release containing the
# required cudax HyperLogLog policy and explicit stream / memory-resource APIs,
-# replace the CPMAddPackage call below with the find_package-first pattern, e.g.:
+# replace the synthetic version and commit pin with that release version, e.g.:
#
# find_package(CCCL X.Y.Z CONFIG QUIET COMPONENTS cudax)
# if(CCCL_FOUND)
@@ -34,19 +35,23 @@
# behind it, and hll/include/hll_sketch.hpp uses cuda::experimental::cuco::hyperloglog.
#
# Developer override: -DCPM_CCCL_SOURCE=/path/to/local/cccl (CPM-native).
-if(NOT COMMAND CPMAddPackage)
+if(NOT COMMAND CPMFindPackage)
include(${CMAKE_CURRENT_LIST_DIR}/../get_cpm.cmake)
endif()
function(find_and_configure_cccl)
+ set(_cccl_version 3.5.1)
+ set(_cccl_tag c95f99757cf95044ce82b905eec88ff40c851f7b)
message(WARNING
- "datasketches_cuda: fetching CCCL@main via CPM "
+ "datasketches_cuda: using CCCL@${_cccl_tag} as synthetic version ${_cccl_version} "
"(TODO switch to a released CCCL version once required cudax HLL APIs are tagged)")
- CPMAddPackage(
+ CPMFindPackage(
NAME CCCL
+ VERSION ${_cccl_version}
GITHUB_REPOSITORY NVIDIA/cccl
- GIT_TAG main
+ GIT_TAG ${_cccl_tag}
GIT_SHALLOW FALSE
+ FIND_PACKAGE_ARGUMENTS EXACT CONFIG COMPONENTS cudax
OPTIONS
"CCCL_ENABLE_TESTING OFF"
"CCCL_ENABLE_EXAMPLES OFF"
