当我连接到新的 SQLite 数据库时，MLFlow 运行 alembicruntime.migration答案

【问题标题】：MLFlow runs alembicruntime.migration when I connect to a new SQLite database当我连接到新的 SQLite 数据库时，MLFlow 运行 alembicruntime.migration
【发布时间】：2021-04-23 21:57:53
【问题描述】：

出于性能原因，我想使用 SQLite 后端而不是 MLFlow 中的默认 mlruns 文件夹。我将 tracking_uri 设置为 sqlite:///outputs/test.sqlite，然后使用默认 API（不是跟踪 API，但也会发生这种情况）创建一个新实验。

这是代码：

import mlflow
mlflow.set_tracking_uri("sqlite:///outputs/test.sqlite")
mlflow.create_experiment("experiment")

代码有效，但我得到以下输出：

2021/04/23 22:43:22 INFO mlflow.store.db.utils: Creating initial MLflow database tables...
2021/04/23 22:43:22 INFO mlflow.store.db.utils: Updating database tables
INFO  [alembic.runtime.migration] Context impl SQLiteImpl.
INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
INFO  [alembic.runtime.migration] Running upgrade  -> 451aebb31d03, add metric step
INFO  [alembic.runtime.migration] Running upgrade 451aebb31d03 -> 90e64c465722, migrate user column to tags
INFO  [alembic.runtime.migration] Running upgrade 90e64c465722 -> 181f10493468, allow nulls for metric values
INFO  [alembic.runtime.migration] Running upgrade 181f10493468 -> df50e92ffc5e, Add Experiment Tags Table
INFO  [alembic.runtime.migration] Running upgrade df50e92ffc5e -> 7ac759974ad8, Update run tags with larger limit
INFO  [alembic.runtime.migration] Running upgrade 7ac759974ad8 -> 89d4b8295536, create latest metrics table
INFO  [89d4b8295536_create_latest_metrics_table_py] Migration complete!
INFO  [alembic.runtime.migration] Running upgrade 89d4b8295536 -> 2b4d017a5e9b, add model registry tables to db
INFO  [2b4d017a5e9b_add_model_registry_tables_to_db_py] Adding registered_models and model_versions tables to database.
INFO  [2b4d017a5e9b_add_model_registry_tables_to_db_py] Migration complete!
INFO  [alembic.runtime.migration] Running upgrade 2b4d017a5e9b -> cfd24bdc0731, Update run status constraint with killed
INFO  [alembic.runtime.migration] Running upgrade cfd24bdc0731 -> 0a8213491aaa, drop_duplicate_killed_constraint
WARNI [0a8213491aaa_drop_duplicate_killed_constraint_py] Failed to drop check constraint. Dropping check constraints may not be supported by your SQL database. Exception content: No support for ALTER of constraints in SQLite dialectPlease refer to the batch mode feature which allows for SQLite migrations using a copy-and-move strategy.
INFO  [alembic.runtime.migration] Running upgrade 0a8213491aaa -> 728d730b5ebd, add registered model tags table
INFO  [alembic.runtime.migration] Running upgrade 728d730b5ebd -> 27a6a02d2cf1, add model version tags table
INFO  [alembic.runtime.migration] Running upgrade 27a6a02d2cf1 -> 84291f40a231, add run_link to model_version
INFO  [alembic.runtime.migration] Running upgrade 84291f40a231 -> a8c4a736bde6, allow nulls for run_id
INFO  [alembic.runtime.migration] Context impl SQLiteImpl.
INFO  [alembic.runtime.migration] Will assume non-transactional DDL.

当我第二次运行代码（使用不同的实验 ID）时，不会显示此信息。仅当我指定尚不存在的 SQLite 跟踪 URI 时才会显示。

我的问题是：

为什么会这样？
我能否以某种方式禁用此输出？它使我的输出过于混乱，因为如果我直接开始调用 log_params、log_metrics 等，我也会收到这些调用的警告。这些警告看起来像：

INFO  [alembic.runtime.migration] Context impl SQLiteImpl.
INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
INFO  [alembic.runtime.migration] Context impl SQLiteImpl.
INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
INFO  [alembic.runtime.migration] Context impl SQLiteImpl.
INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
INFO  [alembic.runtime.migration] Context impl SQLiteImpl.
INFO  [alembic.runtime.migration] Will assume non-transactional DDL.

在 mlflow 中创建新 SQLite 数据库的正确方法是什么？

加法：当我在同一个脚本中创建多个实验时也会发生这种情况，但是当我在脚本中创建一个实验然后再次运行脚本以创建第二个实验时不会发生这种情况。似乎某种全局状态在 MLFlow / SQLite 引擎中设置不正确？

for i in range(3):
    print(_i)
    mlflow.set_tracking_uri(tracking_uri)
    mlflow.create_experiment("experiment_%d" % _)

编辑

可以这样禁用：

import logging, sys
logging.disable(sys.maxsize)

不幸的是，这不是一个好的解决方案，因为它会全局禁用日志记录。

【问题讨论】：

标签： python database sqlite mlflow

【解决方案1】：

如果您可以只禁用 alembic.runtime.migration 模块，那么您可以在设置跟踪 url 的行之前插入以下内容：

import logging
logging.getLogger('alembic.runtime.migration').disabled = True

（不幸的是，logging.getLogger('alembic.runtime.migration').setLevel(logging.WARNING) 似乎并没有起到作用，因为在运行 this line 时级别会回到 logging.INFO 级别。）

【讨论】：