Expose paging state in API

Description

Expose paging state in the public api and document security considerations with exposing it to end-users

Environment

None

Pull Requests

None

Activity

Show:
Greg Bestland
August 25, 2016, 3:34 PM
Adam Holmberg
November 4, 2015, 2:33 PM

sometimes there is a failure getting a page (e.g. timeout is one failure). However, the iterator's place does not seem to be kept when we retry the .next() call

Without doing any analysis, I can say I don't think the iterator is meant to be resumed following an exception. I don't think it would be impossible, but it's not something that's supported right now. My first reaction is that this should be handled by the retry policy. If you are not finding that mechanism adequate, I welcome you to open a ticket.

Adam Holmberg
November 4, 2015, 2:29 PM

Without this insight, I'm afraid automatic paging is not usable at the moment.

Many people are using it successfully now. Hopefully some details will clear some things up for you.

1. what is the lower level mechanism for automatic paging in the driver and Cassandra – or point me to documentation/code?

https://github.com/apache/cassandra/blob/cassandra-2.2/doc/native_protocol_v4.spec#L330-L335
https://github.com/apache/cassandra/blob/cassandra-2.2/doc/native_protocol_v4.spec#L551-L556
https://github.com/apache/cassandra/blob/cassandra-2.2/doc/native_protocol_v4.spec#L981-L1012

https://github.com/datastax/python-driver/blob/2.7.2/cassandra/cluster.py#L3478-L3525

2. what comprises the paging state? is it the compound primary key or some subset of it?

PagingState implementation here. What's in it is dependent on the query type. The state is treated as an opaque blob by clients.

how is it affected by node timeout/failures and the subsequent default or user-defined retry policies?

Retries for each page are handled in the same way as is configured for explicit requests in the driver.
http://datastax.github.io/python-driver/api/cassandra/policies.html#cassandra.policies.RetryPolicy

specifically does the paging state fully define the result returned in the sense that the content of the next page is guaranteed invariant conditioned on a particular paging state?

To the degree afforded by the Cassandra server. If your data is changing during the paging operation, you could get back different results with the same paging state.

Ying-zong Huang
November 3, 2015, 9:28 PM
Edited

Thanks Adam. What is observed is that when we get a result on a call such as

r is supposed to be an iterable. However, when the result has many pages of large data, sometimes there is a failure getting a page (e.g. timeout is one failure). However, the iterator's place does not seem to be kept when we retry the .next() call, so we get a different set of result back each time depending on the vagaries of where failures occurred. So then we also tried changing the retry policy in the hopes that the driver paging code was aware of failures at the lower level. This does not appear to be the case, either, as we still get back different sets of results each time.

The inconsistency is fairly reproducible in our environment – I'll be happy to submit a report of course, but on the content of this ticket: I can't say it is a driver bug, because as you say it could be Cassandra itself. This is why it would be nice to have insight into the paging state itself, even just for debugging. If this can't be implemented in the near future, could you shed some light on:

  1. what is the lower level mechanism for automatic paging, both in the driver and Cassandra – or point me to documentation/code?

  2. what comprises the paging state? is it the compound primary key or some subset of it?

  3. how is it affected by node timeout/failures and the subsequent built-in or user-defined retry policies? specifically does the paging state fully define the result returned in the sense that the content of the next page is guaranteed invariant conditioned on a particular paging state? if so, having the paging state will solve the problem for us. if not, then something more seriously wrong is with the design of automatic paging.

Without this insight, I'm afraid automatic paging is not usable for us. Hope you can help, thanks again!

Adam Holmberg
November 3, 2015, 8:58 PM

Can you describe what is happening? Is it reproducible? I'm interested to hear what you are seeing.

There have been Cassandra bugs related to repeated results, but I'm not aware of a bug on the driver side.

If you believe there is a bug I would appreciate a description in a new bug ticket. The intent of this ticket is simply to expose and document the internal paging state marker sent from the server.

Fixed

Assignee

Unassigned

Reporter

Adam Holmberg

Fix versions

Labels