【问题标题】:Is Postgres caching our queries and how do we get around it?Postgres 是否缓存了我们的查询,我们如何绕过它?
【发布时间】:2018-09-28 08:18:58
【问题描述】:

我正在尝试运行以下 python3 代码:

import os
import psycopg2
import logging

# Set max attempts before giving up
MAX_ATTEMPTS = 5

# Set basic logging config to debug (i.e. log everything).
# By default, this will log te stdout (i.e. it will behave the same as print)
logging.basicConfig(level=logging.DEBUG)

# Grab DB url from env variable
database_url = os.environ.get('DATABASE_URL')

assert database_url is not None, 'DATABASE_URL env variable must be set to a postgres connection string.'

# Initiate psycopg2 and instantiate a cursor object
conn = psycopg2.connect(database_url)
cursor = conn.cursor()


# Define function to delete old records
def delete_old_records(cur):
    # execute a query to delete old records. We're going to refer to this as the "delete" command
    query = 'DELETE FROM my_table WHERE id NOT IN ( SELECT id FROM ( SELECT id FROM my_table ORDER BY id DESC LIMIT 1850 ) foo);'
    cur.execute(query)


# Set variables to keep track of loop
successful = False
attempts = 0

# While not successful and max attempts not reached
while not successful and attempts < MAX_ATTEMPTS:
    try:
        # Attempt to delete old records
        delete_old_records(cursor)
        # Set successful to True if no errors were encountered in the previous line
        successful = True
        # Log a message
        logging.info('Successfully truncated old records!')
    # If some psycopg2 error happens
    except psycopg2.Error as e:
        # Log the error
        logging.exception('Got exception when executing query')
        # Rollback the cursor and get ready to try again
        conn.rollback()
        # Increment attempts by 1
        attempts += 1

# If not able to perform operation after max attempts, log message to indicate failure
if not successful:
    logging.warning(f'Was not successfully able to truncate logs after {MAX_ATTEMPTS} retries. '
                    f'Check logs for traceback (console output by default).')

问题来了:

  1. 代码执行成功且没有错误。但是,当我们在 postico(Mac 的 Postgres GUI)上运行以下命令(以下称为“count”命令)时:

    SELECT count(*) from my_table;
    

    我们得到 1860 而不是 1850(即行没有被删除)。

  2. 在 psql 或 postico 中手动运行删除命令时,我们分别在 psql 或 postico 中运行 COUNT 命令时得到正确的结果。但是,在 ipython 中运行命令时,我们会得到不同的结果。

  3. 当我在计算机 A 上的 ipython 上打开了与数据库的连接,我运行了删除命令,我在计算机 B 上打开另一个连接到 ipython 上的数据库并运行 count 命令,我看到了db 行数没有改变,即仍然是 1860,没有减少到 1850。

我怀疑缓存/记忆,但我不确定我的命令是否真的有效。 psycopg2、postico 或 postgres 本身是否有可能导致这种情况的东西?我们如何绕过它?我们在 postico 或 psycopg2/postgres 上看不到任何清晰的缓存。

【问题讨论】:

    标签: python postgresql concurrency transactions postico


    【解决方案1】:

    不涉及缓存。 PostgreSQL 不缓存查询结果。

    你只是忘记COMMIT删除事务,所以它的效果在任何并发事务中都不可见。

    【讨论】:

    • 我试过同时做connection.commit()和connection as conn: with conn.cursor() as cur: cur.execute(query) 但postico仍然显示计数查询的结果作为 1860 年。@Laurenz Albe
    • 没关系,我是个白痴,像往常一样。错误的数据库连接字符串。非常感谢!实际上你能详细说明你的答案吗?哪些数据库引擎有或没有缓存,缓存是如何工作的?
    • 我只知道 MySQL 具有查询缓存,但我开始认为它是一个错误功能,最好在 99% 的情况下关闭它。我不知道它是如何工作的。
    猜你喜欢
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 2011-06-06
    • 1970-01-01
    • 1970-01-01
    • 2022-01-20
    相关资源
    最近更新 更多