【问题标题】:What's the best way to store access token generated from third party API in AWS在 AWS 中存储从第三方 API 生成的访问令牌的最佳方式是什么
【发布时间】:2022-02-08 20:26:46
【问题描述】:

我目前正在本地运行脚本以使用 GoogleAdsManager API 生成报告。在运行脚本之前,我已经创建了 json 格式的新服务帐户密钥作为密钥类型以及 ~/googleads.yaml

Here's 开发指南。

但是,我想将此脚本安排在 (AWS Glue) 上。

这是示例脚本,我目前面临的问题是:

如何从 aws 调用此方法 ad_manager.AdManagerClient.LoadFromStorage()?我已将凭证(JSONYAML)存储在 AWS Secrets Manager 中

from googleads import ad_manager, oauth2
import tempfile
import _locale

_locale._getdefaultlocale = (lambda *args: ['en_US', 'UTF-8'])

ad_unit_id = XXXXXXXXXX

def generate_ad_impressions(client):
    # Initialize appropriate service.
    report_service = client.GetService("ReportService", 
    version="v202108")
    # Initialize a DataDownloader.
    report_downloader = client.GetDataDownloader(version="v202108")
    # Create statement object to filter for an order.
    statement = (
        ad_manager.StatementBuilder(version="v202108")
        .Where("PARENT_AD_UNIT_ID = :id")
        .WithBindVariable("id", mbns_aa_vod_ad_unit_id)
        .Limit(None)  # No limit/offset for reports
        .Offset(None)
    )
    report_job = {
        "reportQuery": {
        "dimensions": ["DATE", "HOUR"],
        "columns": [
            "AD_SERVER_IMPRESSIONS",
        ],
        "dateRangeType": "TODAY",
        "startDate": {
            "year": "2022",
            "month": "1",
            "day": "25"
        },
        "endDate": {
            "year": "2022",
            "month": "1",
            "day": "25"
        },
        "statement": statement.ToStatement(),
      }
    }
    try:
        # Run the report and wait for it to finish.
        report_job_id = report_downloader.WaitForReport(report_job)
    except:
        print("Failed to generate report.")
        # Change to your preferred export format.
        export_format = "CSV_DUMP"
        # report_file = 
        tempfile.NamedTemporaryFile(suffix=".csv.gz", 
        delete=False)
    with open('ad_unit_report.csv.gz', mode='wb') as report_file:
        # Download report data.
        report_downloader.DownloadReportToFile(report_job_id, 
        export_format, report_file)
        report_file.close()

    # Download report data.
    downloaded_report = 
    report_downloader.DownloadReportToFile(report_job_id, 
    export_format, report_file)

    report_file.close()
    print('success!')

if __name__ == '__main__':
   ad_manager_client = 
   ad_manager.AdManagerClient.LoadFromStorage('path_to_yaml_file')
   generate_ad_impressions(ad_manager_client)

【问题讨论】:

  • 有很多选择..你考虑过AWS system manager parameter store吗?
  • 嗨,目前我正在使用 AWS Secrets Manager 来存储这两个文件(JSON n YAML 文件)。我尝试使用 AWS Glue 运行脚本。通常,如果它从我的本地运行,我只需调用类方法“LoadFromStorage”并提供 YAML 文件的路径。但在 AWS 我不太确定如何提供 yaml 文件的路径。附上google adsgoogleads.github.io/googleads-python-lib/…提供的可用参数
  • Google 广告管理器 API 还提供了一个名为 LoadFromString('yaml_string') 的方法。您是否考虑过使用它而不是 LoadFromStorage?您也可以通过 boto3 python 库获取字符串。让我知道这是否适合您或需要样品。
  • 是的,请提供样品。谢谢!~
  • 作为解决方案提供。在你身边测试它并告诉我们..

标签: python amazon-web-services aws-glue google-ads-api aws-secrets-manager


【解决方案1】:

解决方案应该是这样的:

import boto3
from botocore.exceptions import ClientError
from googleads import ad_manager, oauth2
import tempfile
import _locale

_locale._getdefaultlocale = (lambda *args: ['en_US', 'UTF-8'])

ad_unit_id = XXXXXXXXXX

def get_secret():
    secret_name = "MySecret"
    region_name = "us-west-2"

    session = boto3.session.Session()
    client = session.client(
        service_name='secretsmanager',
        region_name=region_name,
    )

    try:
        get_secret_value_response = client.get_secret_value(
            SecretId=secret_name
        )
    except ClientError as e:
        if e.response['Error']['Code'] == 'ResourceNotFoundException':
            print("The requested secret " + secret_name + " was not found")
        elif e.response['Error']['Code'] == 'InvalidRequestException':
            print("The request was invalid due to:", e)
        elif e.response['Error']['Code'] == 'InvalidParameterException':
            print("The request had invalid params:", e)
        elif e.response['Error']['Code'] == 'DecryptionFailure':
            print("The requested secret can't be decrypted using the provided KMS key:", e)
        elif e.response['Error']['Code'] == 'InternalServiceError':
            print("An error occurred on service side:", e)
    else:
        # Secrets Manager decrypts the secret value using the associated KMS CMK
        # Depending on whether the secret was a string or binary, only one of these fields will be populated
        if 'SecretString' in get_secret_value_response:
            text_secret_data = get_secret_value_response['SecretString']
            return text_secret_data
        else:
            binary_secret_data = get_secret_value_response['SecretBinary']
            return binary_secret_data

def generate_ad_impressions(client):
    # Initialize appropriate service.
    report_service = client.GetService("ReportService", 
    version="v202108")
    # Initialize a DataDownloader.
    report_downloader = client.GetDataDownloader(version="v202108")
    # Create statement object to filter for an order.
    statement = (
        ad_manager.StatementBuilder(version="v202108")
        .Where("PARENT_AD_UNIT_ID = :id")
        .WithBindVariable("id", mbns_aa_vod_ad_unit_id)
        .Limit(None)  # No limit/offset for reports
        .Offset(None)
    )
    report_job = {
        "reportQuery": {
        "dimensions": ["DATE", "HOUR"],
        "columns": [
            "AD_SERVER_IMPRESSIONS",
        ],
        "dateRangeType": "TODAY",
        "startDate": {
            "year": "2022",
            "month": "1",
            "day": "25"
        },
        "endDate": {
            "year": "2022",
            "month": "1",
            "day": "25"
        },
        "statement": statement.ToStatement(),
      }
    }
    try:
        # Run the report and wait for it to finish.
        report_job_id = report_downloader.WaitForReport(report_job)
    except:
        print("Failed to generate report.")
        # Change to your preferred export format.
        export_format = "CSV_DUMP"
        # report_file = 
        tempfile.NamedTemporaryFile(suffix=".csv.gz", 
        delete=False)
    with open('ad_unit_report.csv.gz', mode='wb') as report_file:
        # Download report data.
        report_downloader.DownloadReportToFile(report_job_id, 
        export_format, report_file)
        report_file.close()

    # Download report data.
    downloaded_report = 
    report_downloader.DownloadReportToFile(report_job_id, 
    export_format, report_file)

    report_file.close()
    print('success!')

if __name__ == '__main__':
   ad_manager_client = 
   ad_manager.AdManagerClient.LoadFromString(get_secret())
   generate_ad_impressions(ad_manager_client)

这里的函数 get_secret 从存储为 MySecret 的 AWS Secrets Manager 中获取 YAML 字符串,该字符串存储在区域 us-west-2 中。随后,该 YAML 字符串将用于使用函数 LoadFromString 创建 Google Ad Manager 客户端。

请注意,我没有对此进行测试,因为我没有 Google API 密钥和 ID。

【讨论】:

  • 感谢分享意见。它很有用,但在我运行脚本后收到一条错误消息。错误日志如下:data = self.stream.read(size) AttributeError: 'NoneType' object has no attribute 'read'
  • 你能分享完整的堆栈跟踪吗?想知道错误出自哪一行。
  • 我决定像上面给出的建议解决方案一样使用参数存储。但是在 yaml 文件中,我们需要提供私钥文件的完整路径。出于测试目的,我将文件存储在我的 s3 中并提供路径。但是在错误日志中它说该文件不存在。感谢您的帮助!
  • 看起来是权限问题。您检查 IAM 政策了吗?
  • 它就像一个魅力。谢谢!
猜你喜欢
  • 1970-01-01
  • 2019-10-04
  • 1970-01-01
  • 2013-01-20
  • 2020-03-20
  • 1970-01-01
  • 1970-01-01
  • 2022-01-21
  • 2014-07-21
相关资源
最近更新 更多