Running with 3 separate C* clusters with identical application all using 3.4.0 version of the connector driver. After running for several days without issue, the application on one cluster threw the following exception:
Reading through the source tree, it seems like the only way this could occur is if the channel itself is null before it gets used here:
So perhaps it was a void future? Perhaps checking isVoid() on the ChannelFuture would be wise before attempting to use it.
I have not managed to reproduce this condition before or since, but maybe someone else can deduce the right course of action. Looking over the source tree, the code looks effectively identical (aside from some formatting differences) so it may be a problem through the 3.11.x release.
$ uname -a
Linux 4.15.0-45-generic #48~16.04.1-Ubuntu SMP Tue Jan 29 18:03:48 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
I agree with your assessment, it's future.channel() that is null. The future must be failed, we should probably only try to add to allChannels in the else block. I'll check that this has no unintended consequences, and try to get a reproduction.
The channel can indeed be null in certain error conditions, see the comments in Netty's AbstractBootstrap.initAndRegister.
I'll rework the code to guard against that.
Side note: you might be hitting SocketException("too many open files") as indicated in the comment, so maybe there's a resource leak in your application. That will become clearer after we let the original exception surface.
A fair point about the open file handles. I don’t think that is the case (as the other setups have been running for weeks without issue) but will make sure.