We see an issue in production cluster
00:00 - Client starts out with 4 cassandra nodes : n1, n2, n3, n4
01:00 - Error creating connection pool to n1 due to connection time out. No attempt to re-stablish connection to n1.
01:02 - Same phenomenon for n2
02:01 - Same phenomenon for n3
02:04 - Same phenomenon for n4. Now we are out of nodes to send requests to and start throwing "no host was tried"
Key issue seems to be: No attempt to re-stablish connection when connection establishment fails with a time out.Attached the connection time out exception in conn-time-out.txt
Full log is sent to datatstax separately.