We are experiencing a periodic NoHostAvailableException (stacktrace attached) which probably starts happening after temporary glitch with connections to Cassandra but the driver is not able to fully recover from the situation. The application throws errors until it is restarted which immediately solves the problem.
I am attaching a file that contains number of such errors in given second and you can see that it happens every minute which seems to be strange. It would seem like there was some periodic task inside the driver which causes the errors at the same time.
Driver version: 2.0.8
Cassandra version: 1.2 and 2.0 (happens on both)
val poolingOptions = new PoolingOptions()
val socketOptions = new SocketOptions()
val clusterBuilder = Cluster.builder()
Any idea why this could happen? Thanks.
First, I recommend upgrading to 2.0.9, which was just released.
Read the release announcement which explains how to configure the read timeout. I think it is too low in your case.
You should also set core=max on the pool to avoid a known issue (also detailed in the link above). Note that each connection can handle up to 128 simultaneous requests, so 50 is probably more than you need. If you want to monitor how many connections you're really using, see the comments on JAVA-456.
Thank you for your suggestions. I will try to update the driver and tweak the configuration but probably not sooner than in January for obvious reasons
However, the periodicity seems to be too weird! Is there some kind of periodic task inside the driver? Can't you point us in some direction where we should look for some issues?
Please feel free to reopen if the issue still occurs.