Google Git
Sign in
apache / cassandra-analytics / 9523a38b3f1b5bc4313e2949896ddc1fff58afbe
commit9523a38b3f1b5bc4313e2949896ddc1fff58afbe[log] [tgz]
authorjkonisa <jkonisa@apple.com>Thu Jun 15 13:31:01 2023 -0700
committerYifan Cai <ycai@apache.org>Wed Jun 21 14:11:35 2023 -0700
tree53ce280a2b5f99c54c60c93732bff9579ecf97f5
parentf0fae2deeee20df15ac1105af2163af2a7e7953d [diff]
CASSANDRA-18605 Adding support for TTL & Timestamps for bulk writes

This commit introduces a new feature in Spark Bulk Writer to support writes with
constant/per_row based TTL & Timestamps.

Patch by Jyothsna Konisa; Reviewed by Dinesh Joshi, Francisco Guerrero, Yifan Cai for CASSANDRA-18605
22 files changed
tree: 53ce280a2b5f99c54c60c93732bff9579ecf97f5
  1. .circleci/
  2. cassandra-analytics-core/
  3. cassandra-analytics-core-example/
  4. cassandra-bridge/
  5. cassandra-four-zero/
  6. cassandra-three-zero/
  7. config/
  8. githooks/
  9. gradle/
  10. ide/
  11. profiles/
  12. .asf.yaml
  13. .gitignore
  14. build.gradle
  15. CHANGES.txt
  16. code_version.sh
  17. DEV-README.md
  18. gradle.properties
  19. gradlew
  20. LICENSE.txt
  21. README.md
  22. settings.gradle
README.md

Cassandra Analytics

Cassandra Spark Bulk Reader

The open-source repository for the Cassandra Spark Bulk Reader. This library allows integration between Cassandra and Spark job, allowing users to run arbitrary Spark jobs against a Cassandra cluster securely and consistently.

This project contains the necessary open-source implementations to connect to a Cassandra cluster and read the data into Spark.

For example usage, see the example repository; sample steps:

import org.apache.cassandra.spark.sparksql.CassandraDataSource
import org.apache.spark.sql.SparkSession

val sparkSession = SparkSession.builder.getOrCreate()
val df = sparkSession.read.format("org.apache.cassandra.spark.sparksql.CassandraDataSource")
                          .option("sidecar_instances", "localhost,localhost2,localhost3")
                          .option("keyspace", "sbr_tests")
                          .option("table", "basic_test")
                          .option("DC", "datacenter1")
                          .option("createSnapshot", true)
                          .option("numCores", 4)
                          .load()

Cassandra Spark Bulk Writer

The Cassandra Spark Bulk Writer allows for high-speed data ingest to Cassandra clusters running Cassandra 3.0 and 4.0.

If you are a consumer of the Cassandra Spark Bulk Writer, please see our end-user documentation: usage instructions, FAQs, troubleshooting guides, and release notes.

Developers interested in contributing to the SBW, please see the DEV-README.

Getting Started

For example usage, see the example repository. This example covers both setting up Cassandra 4.0, Apache Sidecar, and running a Spark Bulk Reader and Spark Bulk Writer job.