boto3 在 logrotate 中执行时找不到配置文件（myprofile）答案

【问题标题】：boto3 the config profile (myprofile) could not be found when executed in logrotateboto3 在 logrotate 中执行时找不到配置文件（myprofile）
【发布时间】：2021-02-23 02:06:48
【问题描述】：

我想将一些日志从实例上传到 S3。日志轮换配置如下。 Logrotate 调用一个包装器外壳程序，该外壳程序调用 python (2.7) 脚本以使用 boto3 执行上传。我尝试了各种设置 AWS_CONFIG_FILE 的方法

作为操作系统全局变量 /etc/profile.d/my_aws_config.sh
使用python设置全局变量

/var/log/secure
{
    rotate 1
    hourly
    missingok
    compress
    sharedscripts
    copytruncate
    postrotate
        /bin/kill -HUP `cat /var/run/syslogd.pid 2> /dev/null` 2> /dev/null || true
    endscript
    dateext
    dateformat -%Y-%m-%d-%s
    lastaction
        /bin/sh -xv /opt/upload_to_s3.sh 
    endscript
}

它是一个包装脚本，依次将参数传递给python脚本带代码：

import sys
import os
import logging
import boto3
from botocore.exceptions import ClientError

os.environ["AWS_CONFIG_FILE"] = "/root/.aws/config"

archive_session = boto3.session.Session(profile_name='dev')
s3_client = archive_session.client('s3')

def upload_file(file_name, bucket, object_name=None):

    # If S3 object_name was not specified, use file_name
    if object_name is None:
        object_name = file_name

    try:
        response = s3_client.upload_file(file_name, bucket, object_name)
    except ClientError as e:
        logging.error(e)
        return False
    return True

folder_path=sys.argv[5]+"/"+sys.argv[4]+"/{}"
print folder_path
upload_file(sys.argv[1],sys.argv[2], folder_path.format(sys.argv[3]))

错误

Traceback (most recent call last):
  File "/opt/techarch-scripts/python.py", line 29, in <module>
    archive_session = boto3.session.Session(profile_name='dev')
  File "/usr/lib/python2.7/site-packages/boto3/session.py", line 80, in __init__
    self._setup_loader()
  File "/usr/lib/python2.7/site-packages/boto3/session.py", line 120, in _setup_loader
    self._loader = self._session.get_component('data_loader')
  File "/usr/lib/python2.7/site-packages/botocore/session.py", line 685, in get_component
    return self._components.get_component(name)
  File "/usr/lib/python2.7/site-packages/botocore/session.py", line 924, in get_component
    self._components[name] = factory()
  File "/usr/lib/python2.7/site-packages/botocore/session.py", line 158, in <lambda>
    lambda:  create_loader(self.get_config_variable('data_path')))
  File "/usr/lib/python2.7/site-packages/botocore/session.py", line 241, in get_config_variable
    logical_name)
  File "/usr/lib/python2.7/site-packages/botocore/configprovider.py", line 301, in get_config_variable
    return provider.provide()
  File "/usr/lib/python2.7/site-packages/botocore/configprovider.py", line 398, in provide
    value = provider.provide()
  File "/usr/lib/python2.7/site-packages/botocore/configprovider.py", line 459, in provide
    scoped_config = self._session.get_scoped_config()
  File "/usr/lib/python2.7/site-packages/botocore/session.py", line 340, in get_scoped_config
    raise ProfileNotFound(profile=profile_name)
botocore.exceptions.ProfileNotFound: The config profile (dev) could not be found

打印 python 的变量，看起来变量正在设置：

{'MAILTO': 'root', 'LANG': 'en_US.UTF-8', 'SHELL': '/bin/bash', 'XDG_RUNTIME_DIR': '/run/user/0', 'SHLVL': '5', 'PWD': '/root', 'LOGNAME': 'root', 'USER': 'root', 'AWS_CONFIG_FILE': '/root/.aws/config', 'HOME': '/root', 'PATH': '/sbin:/bin:/usr/sbin:/usr/bin', 'XDG_SESSION_ID': '871', '_': '/usr/bin/python'}

我的个人资料肯定在那里：

[profile dev]
role_arn = arn:aws:iam::XXXXXXXX:role/ec2-s3-role-for-dev
credential_source = Ec2InstanceMetadata

【问题讨论】：

如果您将 AWS CLI 与 --profile dev 一起使用，它也会出错吗？

标签： boto3 logrotate

【解决方案1】：

Named profiles 在 aws 中对名为 user1 的配置文件使用以下命名模式

[user1]
aws_access_key_id=AKIAI44QH8DHBEXAMPLE
aws_secret_access_key=je7MtGbClwBF/2Zp9Utk/h3yCo8nvbEXAMPLEKEY

您只需将dev 放在方括号中，而不是profile dev：

[dev]
role_arn = arn:aws:iam::XXXXXXXX:role/ec2-s3-role-for-dev
credential_source = Ec2InstanceMetadata

【讨论】：