Support clusters with different native and storage ports at each node using CASSANDRA-7544

Description

CASSANDRA-7544 has the goal of allowing you to run multiple Cassandra instances on the same network interface (IP). Instances are disambiguated using the IP and port number.
The client wire protocol already communicated the port number in events. The system peers table has an additional column containing the port which clients need to be aware of if they want to route correctly and connect to a cluster that is configured to use different port numbers at each node.
By default the client won't using the discovered port numbers. When building the client you have to specify allowing port discovery. This nit is necessary for transitioning from a cluster where discovered port numbers are for the non-SSL port. Only the native port that supports both SSL and non-SSL traffic is propagated. In the future people will only use one port for both SSL and non-SSL, but during the transition the client needs to now if it should trust the discovered port number or use the one that is supplied in the client configuration.

Resolution

  • system.peers_v2 is used by default. It fallbacks to system.peers for older C* and DSE versions.

  • port discovery is enabled by default (if the hosts have a broacast_rpc_port) (require peers_v2)

Environment

None

Pull Requests

None

Activity

Show:
Ariel Weisberg
March 2, 2017, 6:57 AM

I just became aware that the client protocol supports a single socket for both secure and unsecure connections. So maybe there is no reason to send the SSL port over as well. The server supports a dedicated SSL port in addition to the unencrypted point, but it's not clear why. It's possible to set up a configuration where knowing both ports matters and where the server will not accept encrypted connections on the unencrypted port, but again it's not clear why you would do that.

The other work related to the additional information in system tables related to peers and token aware routing still exists.

Jim Witschey
April 21, 2018, 12:20 AM

Hey , I see that CASSANDRA-7544 has been merged, and that there's talk of a code freeze/release for 4.0 soon. Is the feature sufficiently fleshed-out and set in stone for us to review and merge your PR, or should we continue holding off for now?

Ariel Weisberg
April 21, 2018, 1:28 AM

I don't have time to work on before the 4.0 freeze, but I can after. Yes it's ready to review as the server support is available to develop and test against.

Without this change the driver will still work with 4.0 it just won't support clusters with different ports at each instance.

Jim Witschey
April 21, 2018, 4:00 AM

Great; we'll get in our initial review in the coming weeks and wait for your reply.

Ariel Weisberg
July 10, 2018, 8:25 AM

Apparently I don't currently have a CLA and one is not going to be forthcoming so I won't be able to contribute code for this.

Fixed

Assignee

Alan Boudreault

Reporter

Ariel Weisberg

Fix versions

Labels

None

PM Priority

None

External issue ID

None

Doc Impact

None

Reviewer

None

Size

None

Pull Request

None

Epic Link

Sprint

Py P-NEXT

Priority

Major