是否可以更有效地使用 ScyllaDB for python 添加数据？

Question

我尝试将 ScyllaDB 与 python 一起使用，但是速度很慢。当我运行底部显示的示例代码时，我得到：

26:23:109998
26:23:112695

我关心的是最好的性能，不幸的是这次向数据库添加数据的时间肯定太长了。有什么方法可以加快这个过程？

        print(datetime.now().strftime("%M:%S:%f"))
        session.execute(
            """
            INSERT INTO log (id, date, message)
            VALUES (now(), %s, %s)
            """,
            (date, message)
        )
        print(datetime.now().strftime("%M:%S:%f"))

更新

根据本主题的建议，我决定根据官方文档使用准备好的语句和批处理来提高向 ScyllaDB 添加数据的性能。我目前的代码如下所示，但效率没有显着变化。还有其他想法吗？

print("time 0: " + str(datetime.now()))
query = "INSERT INTO message (id, message) VALUES (uuid(), ?)"
prepared = session.prepare(query)

for key in range(100):

    print(key)

    try:

        batch = BatchStatement(consistency_level=ConsistencyLevel.QUORUM)
        for key in range(100):

            batch.add(prepared, ("example message",))

        session.execute(batch)

    except Exception as e:
        print("An error occured : " + str(e))
        pass

print("time 1: " + str(datetime.now()))

运行此源代码后，结果如下所示：

test 0: 2018-06-19 11:10:13.990691
0
1
...
41
cAn error occured : Error from server: code=1100 [Coordinator node timed out waiting for replica nodes' responses] message="Operation timed out for messages.message - received only 1 responses from 2 CL=QUORUM." info={'write_type': 'BATCH', 'required_responses': 2, 'consistency': 'QUORUM', 'received_responses': 1}
42
...
52                                                                                                                                                                             An error occured : errors={'....0.3': 'Client request timeout. See Session.execute[_async](timeout)'}, last_host=.....0.3
53
An error occured : Error from server: code=1100 [Coordinator node timed out waiting for replica nodes' responses] message="Operation timed out for messages.message - received only 1 responses from 2 CL=QUORUM." info={'write_type': 'BATCH', 'required_responses': 2, 'consistency': 'QUORUM', 'received_responses': 1}
54
...
59
An error occured : Error from server: code=1100 [Coordinator node timed out waiting for replica nodes' responses] message="Operation timed out for messages.message - received only 1 responses from 2 CL=QUORUM." info={'write_type': 'BATCH', 'required_responses': 2, 'consistency': 'QUORUM', 'received_responses': 1}
60
61
62
...
69
70
71
An error occured : errors={'.....0.2': 'Client request timeout. See Session.execute[_async](timeout)'}, last_host=.....0.2
72
An error occured : errors={'....0.1': 'Client request timeout. See Session.execute[_async](timeout)'}, last_host=....0.1
73
74
...
98
99
test 1: 2018-06-19 11:11:03.494957

Answer 1

有几个因素会限制您的表现。从 Scylla 服务器配置开始。例如，如果您创建了一个具有非常小、慢速网络实例的集群。继续，客户端硬件和实例本身的工作负载，同时考虑每个主机的连接数、每个连接的线程数以及来自驱动程序/连接器端的其他可调参数。最后，通过使用准备好的语句，使用更有效的方式将信息写入 Scylla。

了解更多有关您正在使用的环境和工作负载的目的以推荐更具体的行动方案将很有帮助。

Answer 2

从使用准备好的语句开始，然后并行执行多个语句。