您是正确的,因为 JobSchedule 将在指定的时间间隔创建一个新作业。此外,一旦任务完成,您不能每 5 分钟“重新运行”一次任务。你可以这样做:
- 让一个任务循环运行,每 5 分钟执行一次相同的操作。
- 使用作业管理器每 5 分钟添加一个新任务(执行相同的操作)。
我可能会推荐第二个选项,因为它可以更灵活地监控任务和工作的进度并采取相应的措施。
创建作业的示例客户端可能如下所示:
job_manager = models.JobManagerTask(
id='job_manager',
command_line="/bin/bash -c 'python ./job_manager.py'",
environment_settings=[
mdoels.EnvironmentSettings('AZ_BATCH_KEY', AZ_BATCH_KEY)],
resource_files=[
models.ResourceFile(blob_sas="https://url/to/job_manager.py", file_name="job_manager.py")],
authentication_token_settings=models.AuthenticationTokenSettings(
access=[models.AccessScope.job]),
kill_job_on_completion=True, # This will mark the job as complete once the Job Manager has finished.
run_exclusive=False) # Whether the job manager needs a dedicated VM - this will depend on the nature of the other tasks running on the VM.
new_job = models.JobAddParameter(
id='my_job',
job_manager_task=job_manager,
pool_info=models.PoolInformation(pool_id='my_pool'))
batch_client.job.add(new_job)
现在我们需要一个脚本来作为计算节点上的作业管理器运行。在这种情况下,我将使用 Python,因此您需要将 StartTask 添加到您的池中(或将 JobPrepTask 添加到作业中)以安装 azure-batch Python 包。
此外,作业管理器任务将需要能够针对批处理 API 进行身份验证。根据作业管理器将执行的活动范围,有两种方法可以做到这一点。如果您只需要添加任务,那么您可以使用 authentication_token_settings 属性,该属性将向 Job Manager 任务添加一个 AAD 令牌环境变量,该任务具有仅访问当前作业的权限。如果您需要权限来执行其他操作,例如更改池或开始新作业,您可以通过环境变量传递帐户密钥。上面显示了这两个选项。
您在 Job Manager 任务上运行的脚本可能如下所示:
import os
import time
from azure.batch import BatchServiceClient
from azure.batch.batch_auth import SharedKeyCredentials
from azure.batch import models
# Batch account credentials
AZ_BATCH_ACCOUNT = os.environ['AZ_BATCH_ACCOUNT_NAME']
AZ_BATCH_KEY = os.environ['AZ_BATCH_KEY']
AZ_BATCH_ENDPOINT = os.environ['AZ_BATCH_ENDPOINT']
# If you're using the authentication_token_settings for authentication
# you can use the AAD token in the environment variable AZ_BATCH_AUTHENTICATION_TOKEN.
def main():
# Batch Client
creds = SharedKeyCredentials(AZ_BATCH_ACCOUNT, AZ_BATCH_KEY)
batch_client = BatchServiceClient(creds, base_url=AZ_BATCH_ENDPOINT)
# You can set up the conditions under which your Job Manager will continue to add tasks here.
# It could be a timeout, max number of tasks, or you could monitor tasks to act on task status
condition = True
task_id = 0
task_params = {
"command_line": "/bin/bash -c 'echo hello world'",
# Any other task parameters go here.
}
while condition:
new_task = models.TaskAddParameter(id=task_id, **task_params)
batch_client.task.add(AZ_JOB, new_task)
task_id += 1
# Perform any additional log here - for example:
# - Check the status of the tasks, e.g. stdout, exit code etc
# - Process any output files for the tasks
# - Delete any completed tasks
# - Error handling for tasks that have failed
time.sleep(300) # Wait for 5 minutes (300 seconds)
# Job Manager task has completed - it will now exit and the job will be marked as complete.
if __name__ == '__main__':
main()