从 Cassandra 中的 CSV 导入时没有在表中插入行

Question

我正在尝试将 CSV 文件导入 Cassandra 表，但是我遇到了问题。 插入成功后，至少 Cassandra 是这么说的，我还是看不到任何记录。这里有更多细节：

qlsh:recommendation_engine> COPY row_historical_game_outcome_data  FROM '/home/adelin/workspace/docs/re_raw_data2.csv' WITH DELIMITER='|';

2 rows imported in 0.216 seconds.
cqlsh:recommendation_engine> select * from row_historical_game_outcome_data;

 customer_id | game_id | time | channel | currency_code | game_code | game_name | game_type | game_vendor | progressive_winnings | stake_amount | win_amount
-------------+---------+------+---------+---------------+-----------+-----------+-----------+-------------+----------------------+--------------+------------

(0 rows)
cqlsh:recommendation_engine> 

这就是我的数据的样子

'SomeName'|673|'SomeName'|'SomeName'|'TYPE'|'M'|123123|0.20000000000000001|0.0|'GBP'|2015-07-01 00:01:42.19700|0.0|
'SomeName'|673|'SomeName'|'SomeName'|'TYPE'|'M'|456456|0.20000000000000001|0.0|'GBP'|2015-07-01 00:01:42.19700|0.0| 

这是 cassandra 版本 apache-cassandra-2.2.0

编辑：

CREATE TABLE row_historical_game_outcome_data (
    customer_id int,
    game_id int,
    time timestamp,
    channel text,
    currency_code text,
    game_code text,
    game_name text,
    game_type text,
    game_vendor text,
    progressive_winnings double,
    stake_amount double,
    win_amount double,
    PRIMARY KEY ((customer_id, game_id, time))
) WITH bloom_filter_fp_chance = 0.01
    AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'
    AND comment = ''
    AND compaction = {'class': 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'}
    AND compression = {'sstable_compression': 'org.apache.cassandra.io.compress.LZ4Compressor'}
    AND dclocal_read_repair_chance = 0.1
    AND default_time_to_live = 0
    AND gc_grace_seconds = 864000
    AND max_index_interval = 2048
    AND memtable_flush_period_in_ms = 0
    AND min_index_interval = 128
    AND read_repair_chance = 0.0
    AND speculative_retry = '99.0PERCENTILE';

我还按照 uri2x 的建议尝试了以下方法

但还是一无所获：

select * from row_historical_game_outcome_data;

 customer_id | game_id | time | channel | currency_code | game_code | game_name | game_type | game_vendor | progressive_winnings | stake_amount | win_amount
-------------+---------+------+---------+---------------+-----------+-----------+-----------+-------------+----------------------+--------------+------------

(0 rows)
cqlsh:recommendation_engine> COPY row_historical_game_outcome_data ("game_vendor","game_id","game_code","game_name","game_type","channel","customer_id","stake_amount","win_amount","currency_code","time","progressive_winnings")  FROM '/home/adelin/workspace/docs/re_raw_data2.csv' WITH DELIMITER='|';

2 rows imported in 0.192 seconds.
cqlsh:recommendation_engine> select * from row_historical_game_outcome_data;

 customer_id | game_id | time | channel | currency_code | game_code | game_name | game_type | game_vendor | progressive_winnings | stake_amount | win_amount
-------------+---------+------+---------+---------------+-----------+-----------+-----------+-------------+----------------------+--------------+------------

(0 rows)

Answer 1

好的，我必须更改有关您的数据文件的一些内容才能使其正常工作：

SomeName|673|SomeName|SomeName|TYPE|M|123123|0.20000000000000001|0.0|GBP|2015-07-01 00:01:42|0.0
SomeName|673|SomeName|SomeName|TYPE|M|456456|0.20000000000000001|0.0|GBP|2015-07-01 00:01:42|0.0

• 移除了尾管。
• 将时间缩短到秒。
• 删除了所有单引号。

一旦我这样做了，我就执行了：

aploetz@cqlsh:stackoverflow> COPY row_historical_game_outcome_data 
(game_vendor,game_id,game_code,game_name,game_type,channel,customer_id,stake_amount,
 win_amount,currency_code , time , progressive_winnings) 
FROM '/home/aploetz/cassandra_stack/re_raw_data3.csv' WITH DELIMITER='|';

Improper COPY command.

这个有点棘手。我终于发现COPY 不喜欢列名time。我将表格调整为使​​用名称game_time，然后重新运行COPY：

aploetz@cqlsh:stackoverflow> DROP TABLE row_historical_game_outcome_data ;
aploetz@cqlsh:stackoverflow> CREATE TABLE row_historical_game_outcome_data (
             ...     customer_id int,
             ...     game_id int,
             ...     game_time timestamp,
             ...     channel text,
             ...     currency_code text,
             ...     game_code text,
             ...     game_name text,
             ...     game_type text,
             ...     game_vendor text,
             ...     progressive_winnings double,
             ...     stake_amount double,
             ...     win_amount double,
             ...     PRIMARY KEY ((customer_id, game_id, game_time))
             ... );

aploetz@cqlsh:stackoverflow> COPY row_historical_game_outcome_data
(game_vendor,game_id,game_code,game_name,game_type,channel,customer_id,stake_amount,
 win_amount,currency_code , game_time , progressive_winnings)
FROM '/home/aploetz/cassandra_stack/re_raw_data3.csv' WITH DELIMITER='|';

3 rows imported in 0.738 seconds.
aploetz@cqlsh:stackoverflow> SELECT * FROM row_historical_game_outcome_data ;

 customer_id | game_id | game_time                | channel | currency_code | game_code | game_name | game_type | game_vendor | progressive_winnings | stake_amount | win_amount
-------------+---------+--------------------------+---------+---------------+-----------+-----------+-----------+-------------+----------------------+--------------+------------
      123123 |     673 | 2015-07-01 00:01:42-0500 |       M |           GBP |  SomeName |  SomeName |      TYPE |    SomeName |                    0 |          0.2 |          0
      456456 |     673 | 2015-07-01 00:01:42-0500 |       M |           GBP |  SomeName |  SomeName |      TYPE |    SomeName |                    0 |          0.2 |          0

(2 rows)

• 我不知道为什么它说“导入了 3 行”，所以我猜它是在计算标题行。
• 您的键都是分区键。不知道你是否真的明白这一点。我只指出这一点，因为我想不出一个理由来指定多个分区键同时指定一个集群键。
• 我在 DataStax 文档中找不到任何表明“时间”是保留字的内容。这可能是cqlsh中的一个错误。但说真的，您可能应该将基于时间的列名称指定为“时间”以外的名称。

Answer 2

另一条评论。 CQL 中的 COPY 添加了 WITH HEADER = TRUE，这将导致 CSV 文件的标题行（第一行）被忽略。 (http://docs.datastax.com/en/cql/3.3/cql/cql_reference/copy_r.html)

“时间”不是 CQL 中的保留字（相信我，因为我自己刚刚更新了 DataStax 文档中的 CQL 保留字）。但是，您确实在列名“时间”周围的 COPY 命令中的列名之间显示了空格，我认为这就是问题所在。没有空格，只有逗号；在 CSV 文件中对所有行执行相同操作。 (http://docs.datastax.com/en/cql/3.3/cql/cql_reference/keywords_r.html)

Answer 3

在您的 CSV 文件中有两件事会困扰 cqlsh：

1. 删除尾随 |在每个 CSV 行的末尾
2. 从您的时间值中删除微秒（精度最多应为毫秒​​）。