Job failing because of the continuos paging when reading data from DataStax Astra

Description

When I'm trying to read data from DataStax Astra (from Databricks or from local Spark), the job is failing with following error

The workaround is to disable continuous paging on the cluster level with

Maybe we should automatically disable it when we detect that we’re working with Astra?

Internal jira: DAAS-608

Environment

None

Pull Requests

None

Activity

Jarek Grabowski 
July 1, 2020 at 8:56 AM

We agreed to throw a meaningful error if the error pops up.

Sebastian Estevez 
June 22, 2020 at 4:53 PM

Maybe we should automatically disable it when we detect that we’re working with Astra?

Rather than disabling, it would be better if we could tune the concurrency. Note, the astra free teir has continuous paging concurrency set to 2 but other tiers of the may be higher.

Fixed

Details

Assignee

Reporter

Fix versions

Labels

Components

Affects versions

Priority

Created June 19, 2020 at 9:05 AM
Updated July 1, 2020 at 1:07 PM
Resolved July 1, 2020 at 1:07 PM