如何访问 Amundsen 中的 Postgres 数据库

Question

您好，我正在尝试将 postgres 数据库中的数据加载到 Amundsen 元数据提取器中，因为我正在尝试使用，在为 amundsen 克隆 git 之后。

按照此链接中的以下步骤https://www.amundsen.io/amundsen/tutorials/index-postgres
我们将利用 postgres 元数据提取器从 postgres 数据库中提取元数据信息，方法是运行文件夹 databuilder/databuilder/extractor/postgres_metadata_extractor.py 中的脚本，它执行时没有任何错误

但是当我尝试从 databuilder/example/scripts/sample_postgres_loader.py 脚本当我尝试执行脚本时，我收到了来自

的错误

def run_postgres_job():
    where_clause_suffix = textwrap.dedent("""
        where table_schema = 'public'
    """)

    tmp_folder = '/var/tmp/amundsen/table_metadata'
    node_files_folder = '{tmp_folder}/nodes/'.format(tmp_folder=tmp_folder)
    relationship_files_folder = '{tmp_folder}/relationships/'.format(tmp_folder=tmp_folder)

    job_config = ConfigFactory.from_dict({
        'extractor.postgres_metadata.{}'.format(PostgresMetadataExtractor.WHERE_CLAUSE_SUFFIX_KEY):
            where_clause_suffix,
        'extractor.postgres_metadata.{}'.format(PostgresMetadataExtractor.USE_CATALOG_AS_CLUSTER_NAME):
            True,
        'extractor.postgres_metadata.extractor.sqlalchemy.{}'.format(SQLAlchemyExtractor.CONN_STRING):
            connection_string(),
        'loader.filesystem_csv_neo4j.{}'.format(FsNeo4jCSVLoader.NODE_DIR_PATH):
            node_files_folder,
        'loader.filesystem_csv_neo4j.{}'.format(FsNeo4jCSVLoader.RELATION_DIR_PATH):
            relationship_files_folder,
        'publisher.neo4j.{}'.format(neo4j_csv_publisher.NODE_FILES_DIR):
            node_files_folder,
        'publisher.neo4j.{}'.format(neo4j_csv_publisher.RELATION_FILES_DIR):
            relationship_files_folder,
        'publisher.neo4j.{}'.format(neo4j_csv_publisher.NEO4J_END_POINT_KEY):
            neo4j_endpoint,
        'publisher.neo4j.{}'.format(neo4j_csv_publisher.NEO4J_USER):
            neo4j_user,
        'publisher.neo4j.{}'.format(neo4j_csv_publisher.NEO4J_PASSWORD):
            neo4j_password,
        'publisher.neo4j.{}'.format(neo4j_csv_publisher.JOB_PUBLISH_TAG):
            'unique_tag',  # should use unique tag here like {ds}})```

getting the error such as 

**/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
    cursor.execute(statement, parameters)
psycopg2.errors.SyntaxError: syntax error at or near "where"
LINE 24:             WHERE att.attnum >=0 and where table_schema = 'p...**

**sqlalchemy.exc.ProgrammingError: (psycopg2.errors.SyntaxError) syntax error at or near "where"
LINE 24:             WHERE att.attnum >=0 and where table_schema = 'p...**

and showing this SQL statement after the error

    

**[SQL: 
    SELECT
        current_database() as cluster,
        st.schemaname as schema,
        st.relname as name,
        pgtd.description as description,
        att.attname as col_name,
        pgtyp.typname as col_type,
        pgcd.description as col_description,
        att.attnum as col_sort_order
    FROM pg_catalog.pg_attribute att
    INNER JOIN
        pg_catalog.pg_statio_all_tables as st
        on att.attrelid=st.relid
    LEFT JOIN
        pg_catalog.pg_type pgtyp
        on pgtyp.oid=att.atttypid
    LEFT JOIN
        pg_catalog.pg_description pgtd
        on pgtd.objoid=st.relid and pgtd.objsubid=0
    LEFT JOIN
        pg_catalog.pg_description pgcd
        on pgcd.objoid=st.relid and pgcd.objsubid=att.attnum
     WHERE att.attnum >=0 and where table_schema = 'public'
     ORDER by cluster, schema, name, col_sort_order;
](Background on this error at: http://sqlalche.me/e/13/f405)**

can any one help me in setting the source data from the postgres to Amundsen or from the airflow

Answer 1

你应该改变

    where_clause_suffix = textwrap.dedent("""
        where table_schema = 'public'
    """)

到

    where_clause_suffix = textwrap.dedent("""
        schemaname = 'public'
    """)

我建议您将问题发布到Amundsen Slack