cannot read system_schema keyspace with spark 3

The issue with the inaccessible system keyspaces seems to be introduced since spark-cassandra-connector:3.0.1 and it remains in spark-cassandra-connector:3.1.0.

Just tried this with spark-cassandra-connector:3.0.0 and it works, but not with 3.1.0

I tried that but I still get the same error, can it be related to using the embedded Cassandra for unit tests? this was not an issue with spark 2.4

You don’t need to include the Java driver since there is already one “embedded” with the connector. You only really need these dependencies:

Alternatively, in Spark shell with:

Then try: