DBT 只运行一次模型

Question

我创建了一个模型来生成一个日历维度，我只想在明确指定运行它时运行它。

我尝试在 is_incremental() 块中使用没有任何内容的增量物化，希望如果没有查询来满足临时视图，dbt 不会做任何事情。不幸的是，这没有用。

任何关于我如何实现这一目标的建议或想法都非常感谢。

问候，

阿什利

Answer 1

我为此使用了标签。我们把这种东西称为“静态”模型。在您的模型中：

{{ config(tags=['static']) }}

然后在您的生产工作中：

dbt run --exclude tag:static

这并不能完全实现您想要的，因为您必须在命令行中添加选择器。但它很简单并且可以自我记录，这很好。

我觉得你应该能够破解增量实现来做到这一点。 dbt 会抱怨空模型，但您应该能够返回零记录的查询。这将取决于您的 RDBMS，这是否真的比仅运行模型更好/更快/更便宜，因为 dbt 仍将使用复杂的合并逻辑执行查询。

{{ config(materialized='incremental') }}

{% if is_incremental() %}
select * from {{ this }} limit 0
{% else %}
-- your model here, e.g.
{{ dbt_utils.date_spine( ... ) }}
{% endif %}

您最后/最好的选择可能是创建一个custom materialization 来检查现有关系，如果找到则无操作。您可以从incremental materialization 借用大部分代码来执行此操作。 （您可以将其作为宏添加到您的项目中）。没有测试过，但给你一个想法：

-- macros/static_materialization.sql
{% materialization static, default -%}

  -- relations
  {%- set existing_relation = load_cached_relation(this) -%}
  {%- set target_relation = this.incorporate(type='table') -%}
  {%- set temp_relation = make_temp_relation(target_relation)-%}
  {%- set intermediate_relation = make_intermediate_relation(target_relation)-%}
  {%- set backup_relation_type = 'table' if existing_relation is none else existing_relation.type -%}
  {%- set backup_relation = make_backup_relation(target_relation, backup_relation_type) -%}

  -- configs
  {%- set unique_key = config.get('unique_key') -%}
  {%- set full_refresh_mode = (should_full_refresh()  or existing_relation.is_view) -%}
  {%- set on_schema_change = incremental_validate_on_schema_change(config.get('on_schema_change'), default='ignore') -%}

  -- the temp_ and backup_ relations should not already exist in the database; get_relation
  -- will return None in that case. Otherwise, we get a relation that we can drop
  -- later, before we try to use this name for the current operation. This has to happen before
  -- BEGIN, in a separate transaction
  {%- set preexisting_intermediate_relation = load_cached_relation(intermediate_relation)-%}
  {%- set preexisting_backup_relation = load_cached_relation(backup_relation) -%}
   -- grab current tables grants config for comparision later on
  {% set grant_config = config.get('grants') %}
  {{ drop_relation_if_exists(preexisting_intermediate_relation) }}
  {{ drop_relation_if_exists(preexisting_backup_relation) }}

  {{ run_hooks(pre_hooks, inside_transaction=False) }}

  -- `BEGIN` happens here:
  {{ run_hooks(pre_hooks, inside_transaction=True) }}

  {% set to_drop = [] %}

  {% if existing_relation is none %}
      {% set build_sql = get_create_table_as_sql(False, target_relation, sql) %}
  {% elif full_refresh_mode %}
      {% set build_sql = get_create_table_as_sql(False, intermediate_relation, sql) %}
      {% set need_swap = true %}
  {% else %}
    {# ----- only changed the code between these comments ----- #}
    {# NO-OP. An incremental materialization would do a merge here #}
    {% set build_sql = "select 1" %}
    {# ----- only changed the code between these comments ----- #}

  {% endif %}

  {% call statement("main") %}
      {{ build_sql }}
  {% endcall %}

  {% if need_swap %}
      {% do adapter.rename_relation(target_relation, backup_relation) %}
      {% do adapter.rename_relation(intermediate_relation, target_relation) %}
      {% do to_drop.append(backup_relation) %}
  {% endif %}

  {% set should_revoke = should_revoke(existing_relation, full_refresh_mode) %}
  {% do apply_grants(target_relation, grant_config, should_revoke=should_revoke) %}

  {% do persist_docs(target_relation, model) %}

  {% if existing_relation is none or existing_relation.is_view or should_full_refresh() %}
    {% do create_indexes(target_relation) %}
  {% endif %}

  {{ run_hooks(post_hooks, inside_transaction=True) }}

  -- `COMMIT` happens here
  {% do adapter.commit() %}

  {% for rel in to_drop %}
      {% do adapter.drop_relation(rel) %}
  {% endfor %}

  {{ run_hooks(post_hooks, inside_transaction=False) }}

  {{ return({'relations': [target_relation]}) }}

{%- endmaterialization %}