The most commonly used piece of information in KeyspaceTokenMap is replicasByToken: it's used by the LBP for token-aware routing.
tokenRangesByNode is only needed for TokenMap.getReplicas(CqlIdentifier, ByteBuffer). The driver never uses that method internally, it's only invoked by some analytics clients. A lot of regular transactional applications probably never call it.
tokenRangesByNode is basically an inversion of replicasByToken, so we're storing the same information twice (and since the size is probably more than doubled, because we can't do the same optimizations).
Precomputing that information wastes a lot of heap space if the client is never going to use it. We could compute it lazily, but that comes at the price of a higher latency for the first call to getReplicas (I think it can go up to a few hundred milliseconds for a very large cluster). We need to evaluate if this is a worthy tradeoff; there is some background on about why we precomputed it in the first place.