scala> spark.sql("CREATE DATABASE IF NOT EXISTS mycatalog.testks WITH DBPROPERTIES (class='SimpleStrategy',replication_factor='1')") org.apache.spark.SparkException: Cannot find catalog plugin class for catalog 'mycatalog': com.datastax.spark.connector.datasource.CassandraCatalog
Environment
None
Pull Requests
None
Activity
Maher Saidi
July 7, 2020 at 3:51 PM
thanks Jaro - it worked with the beta.
Over the weekend I unpacked both 3.0 alphas and the class com.datastax.spark.connector.datasource.CassandraCatalog was missing
Jarek Grabowski
July 7, 2020 at 12:49 PM
Maher, there is a new SCC version available: 3.0.0-beta. It’s compatible with Spark 3.0.0 (Data Source V2). Please see the updated quick start.
Maher Saidi
July 3, 2020 at 6:52 PM
(edited)
thank you Jaroslaw,
I tried 2.5.0 with Spark 2.4.6, JDK 8 and the whole connect via catalog docs seems not to work.
I think the docs are missing some steps here.
I get the same issue when I try spark-cassandra-connector_2.11-3.0.0-alpha2
thank you
Jarek Grabowski
July 2, 2020 at 8:35 AM
Hi Maher! Data Source v2 support is added in Spark Cassandra Connector 3.0.0 (no released yet). We have a preview version for you to try out: 3.0.0-alpha2.
We need to update the quick_start once we 3.0.0 is released.
I'm trying to run the tutorial from
https://github.com/datastax/spark-cassandra-connector/blob/master/doc/0_quick_start.md
The steps are not working and I get a error message saying
"org.apache.spark.SparkException: Cannot find catalog plugin class for catalog 'mycatalog': com.datastax.spark.connector.datasource.CassandraCatalog"
Step to reproduce
spark-shell --packages com.datastax.spark:spark-cassandra-connector_2.12:2.5.0 --conf spark.cassandra.connection.host=127.0.0.1 --conf spark.sql.extensions=com.datastax.spark.connector.CassandraSparkExtensions
/ _/_ ___ ____/ /_
_\ \/ _ \/ _ `/ _/ '/
/__/ ./_,// //_\ version 3.0.0
/_/
Using Scala version 2.12.10 (OpenJDK 64-Bit Server VM, Java 11.0.7)
spark.conf.set(s"spark.sql.catalog.mycatalog", "com.datastax.spark.connector.datasource.CassandraCatalog")
scala> spark.sql("CREATE DATABASE IF NOT EXISTS mycatalog.testks WITH DBPROPERTIES (class='SimpleStrategy',replication_factor='1')")
org.apache.spark.SparkException: Cannot find catalog plugin class for catalog 'mycatalog': com.datastax.spark.connector.datasource.CassandraCatalog