commit | fc08d45b283e701aa6d558e99cd18318394b0de7 | [log] [tgz] |
---|---|---|
author | Francisco Guerrero <frankgh@apache.org> | Wed Jan 31 14:35:34 2024 -0800 |
committer | Francisco Guerrero <frankgh@apache.org> | Wed Jan 31 15:12:13 2024 -0800 |
tree | dca9ff6f9f1614c14d99a53be4136692860ab702 | |
parent | e0ae9d7484e242f6af495aac2cb4d8dc121fba89 [diff] |
CASSANDRA-19351 No longer need to synchronize on Schema.instance after Cassandra 4.0.12 We no longer need to synchronize on the `Schema.instance` in Analytics after the release of Cassandra 4.0.12, that includes a synchronization fix in https://issues.apache.org/jira/browse/CASSANDRA-18317. This commit cleans up TODOs pending on that code being released. Patch by Francisco Guerrero; Reviewed by Yifan Cai for CASSANDRA-19351
The open-source repository for the Cassandra Spark Bulk Reader. This library allows integration between Cassandra and Spark job, allowing users to run arbitrary Spark jobs against a Cassandra cluster securely and consistently.
This project contains the necessary open-source implementations to connect to a Cassandra cluster and read the data into Spark.
For example usage, see the example repository; sample steps:
import org.apache.cassandra.spark.sparksql.CassandraDataSource import org.apache.spark.sql.SparkSession val sparkSession = SparkSession.builder.getOrCreate() val df = sparkSession.read.format("org.apache.cassandra.spark.sparksql.CassandraDataSource") .option("sidecar_instances", "localhost,localhost2,localhost3") .option("keyspace", "sbr_tests") .option("table", "basic_test") .option("DC", "datacenter1") .option("createSnapshot", true) .option("numCores", 4) .load()
The Cassandra Spark Bulk Writer allows for high-speed data ingest to Cassandra clusters running Cassandra 3.0 and 4.0.
Developers interested in contributing to the Analytics library, please see the DEV-README.
For example usage, see the example repository. This example covers both setting up Cassandra 4.0, Apache Sidecar, and running a Spark Bulk Reader and Spark Bulk Writer job.