【问题标题】:How to have a SELECT query prepared in CQL using Python3?如何使用 Python3 在 CQL 中准备 SELECT 查询?
【发布时间】:2019-06-13 12:34:25
【问题描述】:

就像我可以准备 INSERT 查询一样,我如何在 CQL-Python 中准备 SELECT 查询? 插入语句:

insert_sql = self.session.prepare(
            (
                "INSERT INTO  {} ({}, {}, {}) VALUES (?,?,?)"
            ).format(
                self.table_name, "id", "version", "row"
            )
        )

然后执行它:

params = (key, "version_1", row, )
                futures.append(
                    (
                        insert_sql,
                        params
                    )
                )
            # Checking whether the formed keys already exists
            results = execute_concurrent(
                self.session, futures, concurrency=1000, raise_on_first_error=False
            )

当我对 SELECT 做同样的事情时:

select_sql = self.session.prepare(
            (
                "SELECT * FROM {} WHERE {}=%s AND {}=%s AND {}!=%s"
            ).format(
                self.table_name, "id", "version", "row"
            )
        )

然后执行它:

params = (key, "version_1", row, )
                futures.append(
                    (
                        select_sql,
                        params
                    )
                )
            # Checking whether the formed keys already exists
            results = execute_concurrent(
                self.session, futures, concurrency=1000, raise_on_first_error=False
            )

我遇到了一个错误:

cassandra.InvalidRequest: Error from server: code=2200 [Invalid query] message="Unsupported "!=" relation: row != ?"
Traceback (most recent call last):
  File "getting_started.py", line 145, in <module>
    example1.insert_data()
  File "getting_started.py", line 97, in insert_data
    self.table_name, "id", "version", "row"
  File "cassandra/cluster.py", line 2405, in cassandra.cluster.Session.prepare
  File "cassandra/cluster.py", line 2402, in cassandra.cluster.Session.prepare
  File "cassandra/cluster.py", line 4062, in cassandra.cluster.ResponseFuture.result
cassandra.InvalidRequest: Error from server: code=2200 [Invalid query] message="Unsupported "!=" relation: row != ?"

在代码中将!= 更改为NOT 后:

select_query = (
                "SELECT * FROM {} WHERE {}=%s AND {}=%s AND NOT {}=%s"
            ).format(
                self.table_name, "id", "version", "row"
            )
        print(select_query)
        select_sql = self.session.prepare(
            select_query
        )
        for df in chunks:
            futures = []
            df = df.to_dict(orient='records')
            chunk_counter += 1
            for row in df:
                key = str(row["0"])
                row = json.dumps(row, default=str)
                params = (key, "version_1", row, )
                futures.append(
                    (
                        select_sql,
                        params
                    )
                )
            # Checking whether the formed keys already exists
            results = execute_concurrent(
                self.session, futures, concurrency=1000, raise_on_first_error=False
            )
            for (success, result) in results:
                if not success:
                    self.handle_error(result)
                print(result)
                sys.exit()
            results = execute_concurrent(
                self.session, futures, concurrency=1000, raise_on_first_error=False)
            for (success, result) in results:
                if not success:
                    self.handle_error(result)  # result will be an Exception
            if (chunk_counter % 1000 == 0):
                self.log.info(
                    "{} 's Batch Insert Completed".format(chunk_counter))
            # print(
            #     str(chunk_counter*chunksize) + " : " +
            #     str(datetime.utcnow() - start_time)
            # )
        print("Complete task's duration is: {}".format(
            datetime.utcnow() - start_time))

我明白了:

etting_started.py:8: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
  from cassandra.cluster import BatchStatement, Cluster
2019-01-19 19:49:16,582 [INFO] root: setting keyspace...
SELECT * FROM TenMillion WHERE id=%s AND version=%s AND NOT row=%s
2019-01-19 19:49:16,590 [ERROR] cassandra.cluster: Error preparing query:
Traceback (most recent call last):
  File "cassandra/cluster.py", line 2402, in cassandra.cluster.Session.prepare
  File "cassandra/cluster.py", line 4062, in cassandra.cluster.ResponseFuture.result
cassandra.protocol.SyntaxException: <Error from server: code=2000 [Syntax error in CQL query] message="line 1:34 no viable alternative at character '%'">
Traceback (most recent call last):
  File "getting_started.py", line 147, in <module>
    example1.insert_data()
  File "getting_started.py", line 100, in insert_data
    select_query
  File "cassandra/cluster.py", line 2405, in cassandra.cluster.Session.prepare
  File "cassandra/cluster.py", line 2402, in cassandra.cluster.Session.prepare
  File "cassandra/cluster.py", line 4062, in cassandra.cluster.ResponseFuture.result
cassandra.protocol.SyntaxException: <Error from server: code=2000 [Syntax error in CQL query] message="line 1:34 no viable alternative at character '%'">

【问题讨论】:

    标签: python python-3.x cassandra cql


    【解决方案1】:

    Cassandra 似乎不支持 != 和 SELECT

    检查这里: https://docs.datastax.com/en/cql/3.3/cql/cql_reference/cqlSelect.html

    您必须编写一个不带!= 的查询并管理代码中的逻辑。

    【讨论】:

    • 这个操作的反面是什么?我没有达到目标或替代方案。
    • 您是否可以使用 IN 运算符并给出可接受的值集而不是给出“!=”?
    • 所以对于 1000 行,我需要在每个行参数中有 1000 个可能的值,以用于使用 IN 运算符的查询。这意味着 1000000 倍于 != 运算符的内存。这实际上会减慢我的速度,并且 OOM 错误不会太远。
    • 您的数据中存在多少“行”列的可能值?如果每一行列都有不同的值,那么为什么不使用 = 运算符呢?您能否更新一些有关您的 cassandra 表及其架构的更多信息?
    • 它是 csv 文件中的一行,任何值都是可能的。在我的版本 1 中,我得到 1000 万行并将其插入 Cassandra,下一次运行可能会带来 1400 万行,我需要知道其中已经存在哪些行。因此,我需要进行选择查询,以了解传入数据是否具有新行(因此更新)或相同行(然后无操作)。表架构为:id、版本、行,id和版本为PK。
    【解决方案2】:

    错误返回 message="Unsupported "!="。"!=" 不受支持。尝试将 SELECT 代码更改为:

    select_sql = self.session.prepare(
                (
                    "SELECT * FROM {} WHERE {}=%s AND {}=%s AND NOT {}=%s"
                ).format(
                    self.table_name, "id", "version", "row"
                )
            )
    

    【讨论】:

    • 嘿@Frans!为什么要将? 更改为%s?
    • 另外,您的方法返回错误:cassandra.protocol.SyntaxException: &lt;Error from server: code=2000 [Syntax error in CQL query] message="line 1:34 no viable alternative at character '%'"&gt;
    • 我建议将 AND {}!=%s 更改为 AND NOT ()=%s。假设 self.table_name = 'tablename' 这将导致如下查询:'SELECT * FROM tablename WHERE id=%s AND version=%s AND NOT row=%s'。
    • 我认为您在评论中打错了字。你写的是()而不是{}。
    • 最新的错误是关于查询中的%s。您不填写这些变量会导致错误。
    猜你喜欢
    • 1970-01-01
    • 1970-01-01
    • 2018-12-11
    • 1970-01-01
    • 1970-01-01
    • 2014-11-15
    • 2020-11-12
    • 1970-01-01
    • 1970-01-01
    相关资源
    最近更新 更多