IllegalArgumentException: rpc_address is not a column defined in this metadata

Description

Server limits the number of connections to 5. Client bombards the server over one host. At some point control connection is marked down and re-established. You get the following error:

2014-11-13 14:42:33 ERROR Cluster:1564 - Unknown error during reconnection to /ip1:9042, scheduling retry in 16000 milliseconds
java.lang.IllegalArgumentException: rpc_address is not a column defined in this metadata
at com.datastax.driver.core.ColumnDefinitions.getAllIdx(ColumnDefinitions.java:273)
at com.datastax.driver.core.ColumnDefinitions.getFirstIdx(ColumnDefinitions.java:279)
at com.datastax.driver.core.ArrayBackedRow.getInet(ArrayBackedRow.java:239)
at com.datastax.driver.core.ControlConnection.refreshNodeInfo(ControlConnection.java:438)
at com.datastax.driver.core.Cluster$Manager$5.onReconnection(Cluster.java:1540)
at com.datastax.driver.core.AbstractReconnectionHandler.run(AbstractReconnectionHandler.java:124)

The client code is shown below:

PlainTextAuthProvider authProvider = new PlainTextAuthProvider("user", "pwd");
Collection<InetSocketAddress> whiteList = new ArrayList<InetSocketAddress>();
whiteList.add(new InetSocketAddress("ip1", 9042));
WhiteListPolicy policy = new WhiteListPolicy(new DCAwareRoundRobinPolicy("DC1"), whiteList);
Cluster cluster = Cluster.builder().withAuthProvider(authProvider).withLoadBalancingPolicy(policy).addContactPoint("1ip1").build();
cluster.getConfiguration().getPoolingOptions().setMinSimultaneousRequestsPerConnectionThreshold(HostDistance.LOCAL, 0);
cluster.getConfiguration().getPoolingOptions().setMaxSimultaneousRequestsPerConnectionThreshold(HostDistance.LOCAL, 1);
cluster.getConfiguration().getPoolingOptions().setMaxConnectionsPerHost(HostDistance.LOCAL, 100);
cluster.getConfiguration().getPoolingOptions().setCoreConnectionsPerHost(HostDistance.LOCAL, 3);
final Session session = cluster.connect("system");
ExecutorService executorService = Executors.newFixedThreadPool(10);
for (int i=0;i<100;++i) {
executorService.execute(new Runnable() {
@Override
public void run()
{ while (true) readSystemPeersAndAssert(session); }
});
}
Uninterruptibles.sleepUninterruptibly(1, TimeUnit.HOURS);

Full trace is attached.

Environment

None

Pull Requests

None

Status

Assignee

Olivier Michallat

Reporter

Vishy Kasar

Labels

None

PM Priority

None

Reproduced in

None

External issue ID

None

External issue ID

None

External issue ID

None

External issue ID

None

External issue ID

None

External issue ID

None

Doc Impact

None

Reviewer

None

Size

None

Components

Fix versions

Affects versions

2.0.7
2.0.8

Priority

Major
Configure