This issue is related to CASSANDRA-10786.
In short, when the table is altered (column is added), the wildcard queries keep the old result metadata and never return the added column for all but the very first client (as the very first client is the only one that receives UNPREPARED statement.
After adding result metadata to md5 on Cassandra side, as the message metadata on the client is never updated and new client ID of the re-prepared message is not saved anywhere, client would fall into the infinite re-prepare loop:
(1) try running the query
fail with UNPREPARED
re-prepare (although without saving the new ID)
go to (1)
I first tried to transparently swap the request when getting UNPREPARED, although that would mean that the original "outdated" prepared query would re-prepare no matter how many times it runs (even though results it returns would be correct). On the other hand, making changes in the BoundStatement itself (or in it's PreparedStatement) might lead to undesired behaviour on the client-side or even be prone to races (when request is sent assuming one id and metadata and came back after it got swapped).
Question is what'd be a good solution for such situation? Should we try and update the metadata everywhere in all bound statements or simply throw an exception and force the client to re-prepare the query?
I've attached the "test" file to illustrate the problem a bit better.
See the updated description of CASSANDRA-10786.
I've made a proof-of-concept patch https://github.com/datastax/java-driver/tree/10786-3.0. In order for it to work, one would need corresponding Cassandra built from https://github.com/ifesdjeen/cassandra/tree/10786-trunk.
You can run C* either straight from the repo (ant && ./bin/cassandra -f`) or with CCM ccm create 10786-trunk --install-dir=path_to_repo.
And a little background/motivation for why the patch looks like it looks:
There was an idea to try substituting the old `preparedId` for DefaultPreparedStatement, but If we swap `DefaultPreparedStatement:reparedId`, we still have the instances of old `PreparedStatements` hanging (for example one we originally got here with).
So the "original" prepared statements won't be influenced. Every time we will bind and run `execute` with them again, the issue will be the same.
The test is there for sakes of illustration for now. I'll create a better suite if/when we agree on how we would like to proceed with the change.
I've changed the implementation a bit since we decided to take a different approach. Cassandra patch got some positive feedback. Could anyone also review the driver patch?
Removing from 3.2.0 since the Cassandra ticket is scheduled for 3.11.