Prevent warning log that contact point is from different DC for unresolved addresses

Description

If contact points are added using InetSocketAddress.createUnresolved a warning log is generated.

20:00:07,565 WARN com.datastax.oss.driver.internal.core.loadbalancing.helper.OptionalLocalDcHelper - [my_session|default]
You specified DC1 as the local DC, but some contact points are from a different DC:
Node(endPoint=n1:9042, hostId=null, hashCode=344f7dd2)=null,
Node(endPoint=n2:9042, hostId=null, hashCode=277ee3d6)=null; please provide the correct local DC, or check your contact points

The queries execute successfully but the warning is misleading suggesting that something is not configured correctly.

OptionalLocalDcHelper.checkLocalDatacenterCompatibility does not take in consideration that datacenter can be null for unresolved contact points. A different warning can perhaps be shown for unresolved contact points when datacenter cannot be identified.

How to reproduce

Setup

  • DC1

    • n1

    • n2

    • n3

  • DC2

    • n4

    • n5

    • n6

Create session

Environment

None

Pull Requests

None

Activity

Show:
Bret McGuire
July 13, 2020, 4:22 PM

Worth noting that resolved and unresolved InetSocketAddresses aren’t considered equal:

 

org.junit.ComparisonFailure:
Expected :localhost/127.0.0.1:9042
Actual :localhost:9042

 

This makes sense as they are representing fundamentally different pieces of information.

 

I’m leaning towards the idea that the EndPoint level might not be the right place to fix this.

Olivier Michallat
July 13, 2020, 4:43 PM
Edited

I think the problem is in InitialNodeListRefresh. This is the first time we read the entries in system.peers, and we are trying to match them to the contact points. However we don't know the host_ids of the contact points yet, so we compare by endpoint:

As you noted the comparison fails. So not only do we get the warning, but the driver also considers the contact point as removed, and the system.peers entry as a new node, which is incorrect.

I can reproduce this locally with a 3-node cluster:

Whenever node 2 or 3 is used for the initial connection, this produces:

So we need to compare resolved inet addresses, at least in `findIn`. But this will vary across EndPoint implementations (for example SniEndPoint.resolve() returns the same address for all nodes), so we might have to do this in DefaultEndPoint.equals() indeed.

Olivier Michallat
July 13, 2020, 4:53 PM

I wonder if the implementation of DefaultEndPoint.resolve() isn’t misleading

The implementation does what it's supposed to, but it's the name that was a poor choice. The method returns the socket address that the driver will use to establish the connection. But that address might still be unresolved, so that has nothing to do with DNS resolution.

Olivier Michallat
July 13, 2020, 6:06 PM

PR is up and will be merged for 4.8.0.

Apart from the spurious warnings and onAdd/onRemove events, another negative consequence is that the "new" nodes will use a resolved address, so the driver won't resolve it again if it needs to reconnect in the future. This only affects the contact points that appear in the warning: the one that was used for the initial connection is refreshed properly.

Alexandre Dutra
July 21, 2020, 8:02 PM

Possibly related to this: .

Fixed

Assignee

Olivier Michallat

Reporter

Konstantin Nikolov

Labels

None

PM Priority

None

Reproduced in

None

Affects versions

None

Fix versions

Pull Request

None

Doc Impact

None

Size

None

External issue ID

None

External issue ID

None

Components

Priority

Minor
Configure