Bug when using decimal type columns
Description
Environment
splitly@rubydev2:~/splitly-api$ rails -v
Rails 5.1.4
splitly@rubydev2:~/splitly-api$ ruby -v
ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux]
amzsplit@cassandradev:~$ cqlsh 192.168.131.234
Connected to Test Cluster at 192.168.131.234:9042.
[cqlsh 5.0.1 | Cassandra 3.10 | CQL spec 3.4.4 | Native protocol v4]
Pull Requests
Activity
Sandeep Tamhankar February 15, 2018 at 9:04 PM
I've committed a fix in the RUBY-322 branch. We don't have a formal release planned yet, but you can use the bits in that branch.
Sandeep Tamhankar February 15, 2018 at 6:39 PM
Thanks for all the info. I couldn't repro at first because I was using Ruby 2.2.x. It turns out that 2.4.x changes BigDecimal behavior to no longer accept "2.", and your Rails version must be using Ruby 2.4.x.
James Andrews February 15, 2018 at 9:40 AM
A little more information. It seems when you save an integer value to a decimal column in Cassandra, it actually saves and returns an integer.
cqlsh:splitly> INSERT INTO reporting_history (user_id, product_id, reporting_day, ppc_spend) VALUES (2, 111508717, 1512086400, 3);
cqlsh:splitly> SELECT ppc_spend FROM reporting_history WHERE user_id=2 AND product_id=111508717 AND reporting_day=1512086400;
ppc_spend
-----------
3
Then in Ruby on Rails:
require 'cassandra'
cluster = Cassandra.cluster(
We are using IP tables to block out unwanted connections.
#username: username,
#password: password,
hosts: [ENV['cassandra_hosts']],
)
@@session = cluster.connect('splitly')
def hello
statement = @@session.prepare("
select user_id,product_id,reporting_day,ppc_spend from reporting_history where user_id=2 and product_id=111508717 and reporting_day=1512086400
")
return @@session.execute(
statement
#arguments: [2]
#type_hints: [::Cassandra::Types.dec]
)
end
Gives:
2.5.0 :040 > Bigdata.hello
Traceback (most recent call last):
ArgumentError (invalid value for BigDecimal(): "3.")
BigDecimal is not able to parse "3.". Instead of saving an integer, we will actually save a decimal:
cqlsh:splitly> INSERT INTO reporting_history (user_id, product_id, reporting_day, ppc_spend) VALUES (2, 111508717, 1512086400, 3.00);
cqlsh:splitly> SELECT ppc_spend FROM reporting_history WHERE user_id=2 AND product_id=111508717 AND reporting_day=1512086400;
ppc_spend
-----------
3.00
Running a select in Ruby:
2.5.0 :041 > Bigdata.hello
=> #<Cassandra::Result:0x25241ec @rows=[{"user_id"=>2, "product_id"=>111508717, "reporting_day"=>1512086400, "ppc_spend"=>0.3e1}] @last_page=true>
So it clearly looks like a bug with the Datastax driver.
The problem exists where Ruby tries to parse a string of "2." back to the Ruby driver, where a column is of type decimal. It seems as though the Ruby driver is unable to parse "2.":
2.5.0 :001 > BigDecimal("2.0")
=> 0.2e1
2.5.0 :002 > BigDecimal("2.")
Traceback (most recent call last):
2: from (irb):2
1: from (irb):2:in `BigDecimal'
ArgumentError (invalid value for BigDecimal(): "2.")
If a decimal type cell value is null then the problem doesn't exist, however, for non-null valued cell the error above happens. E.g.
module Bigdata
class << self
require 'cassandra'
cluster = Cassandra.cluster(
hosts: [ENV['cassandra_hosts']],
)
@@session = cluster.connect('splitly')
def hello
statement = @@session.prepare("
SELECT * FROM reporting_history WHERE user_id=2 LIMIT 11
")
return @@session.execute(
statement
#arguments: [2]
#type_hints: [::Cassandra::Types.dec]
)
end
End
splitly@rubydev2:~/splitly-api$ rails c
Running via Spring preloader in process 13649
Loading development environment (Rails 5.1.4)
2.5.0 :003 > Bigdata.hello
Traceback (most recent call last):
ArgumentError (invalid value for BigDecimal(): "2.")
On the 11th row returned, there is a non-null decimal type cell value.