如何防止 TeamCity 启动 AWS 云代理实例

Question

将 TeamCity 与 AWS 云配置文件一起使用时，它会无缘无故地每 10 分钟启动一次代理。

我正在使用代理构建 Yocto 发行版，这需要大量时间和资源。出于这个原因，代理运行在一个大型（昂贵）实例上，我只想在需要时启动它。

我想要完成的事情：

• 拥有一个运行 TeamCity 服务器的廉价 AWS 实例，始终开启。
• 拥有运行 TeamCity 代理的大型、昂贵的 AWS 实例，但仅在需要时才打开。
• 只有在有构建要运行时才启动代理机器。
• 构建通过时在 git 中放置标签。
• 最好：检查服务器上是否有新的 git 提交，有提交时启动代理。

请注意，我想在代理上进行标记，因此服务器端结帐将不起作用（根据 TeamCity 文档）。

我在 AWS 中创建了一个设置，其中有一个运行 TeamCity 服务器的小型实例，以及一个运行代理的大型（昂贵）实例。在 TeamCity 中，我在 Root 项目中配置了一个云配置文件，以自动启动运行代理的机器，并在构建完成或机器空闲 5 分钟时停止它。启动和停止 AWS 机器正常工作，触发构建也正常工作。

我创建了一个带有构建配置的项目，但没有添加任何触发器。我还尝试将我们的 git 存储库的 VCS 根目录的最小检查间隔增加到 90000000 秒（这样它实际上永远不会启动代理机器）。

出于某种原因，TeamCity 每 10 分钟启动一次运行代理的 AWS 实例，我不清楚它为什么这样做。 TeamCity 日志显示：

[2019-03-19 10:51:15,894]   INFO - ldServer.AGENT.PollingProtocol - Polling Agent session 'f1nqO09...' was created, agent: "yocto-build-agent1" {id=3}
[2019-03-19 10:56:33,250]   INFO - ldServer.AGENT.PollingProtocol - Polling Agent session 'f1nqO09...' was closed, agent: "yocto-build-agent1" {id=3}
[2019-03-19 10:56:33,257]   INFO -  jetbrains.buildServer.CLEANUP - Removing agent Unregistered agent "yocto-build-agent1" {id=3}
[2019-03-19 10:56:33,262]   INFO -    jetbrains.buildServer.AGENT - Agent "yocto-build-agent1" {id=3} has been unregistered. Reason: Agent has unregistered
[2019-03-19 11:02:30,612]   INFO -    jetbrains.buildServer.AGENT - Agent has been registered: "yocto-build-agent1" {id=4, protocol=unidirectional, host=10.55.20.127:9090, agentTypeId=3, pool=Default, registered since 2019-03-19 11:02:30.590}, not running a build
[2019-03-19 11:02:30,615]   INFO - ldServer.AGENT.PollingProtocol - Polling Agent session 'FcY5MVV...' was created, agent: "yocto-build-agent1" {id=4}
[2019-03-19 11:02:30,666]   WARN -    jetbrains.buildServer.AGENT - Agent "yocto-build-agent1" {id=4} is unauthorized on registration
[2019-03-19 11:07:09,821]   INFO - ldServer.AGENT.PollingProtocol - Polling Agent session 'FcY5MVV...' was closed, agent: "yocto-build-agent1" {id=4}
[2019-03-19 11:07:09,823]   INFO -    jetbrains.buildServer.AGENT - Agent "yocto-build-agent1" {id=4} has been unregistered. Reason: Agent has unregistered
[2019-03-19 11:07:22,455]   INFO -    jetbrains.buildServer.AGENT - Agent has been registered: "yocto-build-agent1" {id=4, protocol=unidirectional, host=10.55.20.127:9090, agentTypeId=3, pool=Default

teamcity-clouds.log 显示：

[2019-03-19 10:50:38,910]   INFO [uled executor 1] - .instances.StoppedInstanceTask - Instance has changed status from stopped to Starting: Amazon Instance{instanceId=i-071eff7574b93d12f, imageId=i-071eff7574b93d12f, status: Starting}, profile 'AWS Cloud Profile'{id=amazon-2, projectId=_Root}
[2019-03-19 10:50:43,910]   INFO [uled executor 5] - .server.impl.CloudEventsLogger - Cloud instance entered 'starting' state, profile 'AWS Cloud Profile'{id=amazon-2, projectId=_Root}, Amazon Instance{instanceId=i-071eff7574b93d12f, imageId=i-071eff7574b93d12f, status: Starting}
[2019-03-19 10:51:15,883]   INFO [nio-8111-exec-2] - r.impl.DBCloudStateManagerImpl - Image: AmazonImageInstance{id=i-071eff7574b93d12f, amazonId=i-071eff7574b93d12f}, profile: profile 'AWS Cloud Profile'{id=amazon-2, projectId=_Root} was marked to CONTAIN agent
[2019-03-19 10:51:15,883]   INFO [nio-8111-exec-2] - .server.impl.CloudEventsLogger - Detected cloud agent "yocto-build-agent1" {id=3}, profile 'AWS Cloud Profile'{id=amazon-2, projectId=_Root}, Amazon Instance{instanceId=i-071eff7574b93d12f, imageId=i-071eff7574b93d12f, status: Starting}
[2019-03-19 10:51:43,912]   INFO [uled executor 2] - .server.impl.CloudEventsLogger - Cloud instance entered 'running' state, profile 'AWS Cloud Profile'{id=amazon-2, projectId=_Root}, Amazon Instance{instanceId=i-071eff7574b93d12f, imageId=i-071eff7574b93d12f, status: Running}
[2019-03-19 10:56:32,807]   INFO [uled executor 2] - te.IdleTimeoutTerminateFactory - Will stop instance Amazon Instance{instanceId=i-071eff7574b93d12f, imageId=i-071eff7574b93d12f, status: Running}, profile 'AWS Cloud Profile'{id=amazon-2, projectId=_Root} with idle time 5 minutes (idle time is 5 minutes)
[2019-03-19 10:56:32,808]   INFO [uled executor 2] - l.instances.StopInstanceAction - Terminating instance: Amazon Instance{instanceId=i-071eff7574b93d12f, imageId=i-071eff7574b93d12f, status: Running}, profile 'AWS Cloud Profile'{id=amazon-2, projectId=_Root}, reason: "After 5 minutes idle"
[2019-03-19 10:56:32,808]   INFO [uled executor 2] - r.impl.DBCloudStateManagerImpl - Image: AmazonImageInstance{id=i-071eff7574b93d12f, amazonId=i-071eff7574b93d12f}, profile: profile 'AWS Cloud Profile'{id=amazon-2, projectId=_Root} was marked to CONTAIN agent
[2019-03-19 10:56:33,004]   INFO [2 Stop Instance] - r.impl.DBCloudStateManagerImpl - Image: i-071eff7574b93d12f, Instance: i-071eff7574b93d12f, profile=amazon-2 is marked with state: stopped.
[2019-03-19 10:56:33,253]   INFO [nio-8111-exec-7] - ces.terminate.AgentRemoverImpl - Removing cloud agent "yocto-build-agent1" {id=3} because: Instance has been terminated. Amazon Instance{instanceId=i-071eff7574b93d12f, imageId=i-071eff7574b93d12f, status: Running}, profile 'AWS Cloud Profile'{id=amazon-2, projectId=_Root}
[2019-03-19 10:56:33,260]   INFO [nio-8111-exec-7] - .server.impl.CloudEventsLogger - Cloud agent Unregistered agent "yocto-build-agent1" {id=3} is no longer recognized as cloud agent because instance has gone: profileId=amazon-2, imageId=i-071eff7574b93d12f, instanceId=i-071eff7574b93d12f
[2019-03-19 10:56:33,260]   INFO [nio-8111-exec-7] - ces.terminate.AgentRemoverImpl - Removing cloud agent Unregistered agent "yocto-build-agent1" {id=3} because: Cloud instance (id=i-071eff7574b93d12f, imageId=i-071eff7574b93d12f, profileId=amazon-2) has gone (is not reported by cloud profile)
[2019-03-19 10:56:33,922]   INFO [uled executor 2] - .server.impl.CloudEventsLogger - Cloud instance entered 'scheduled to stop' state, profile 'AWS Cloud Profile'{id=amazon-2, projectId=_Root}, Amazon Instance{instanceId=i-071eff7574b93d12f, imageId=i-071eff7574b93d12f, status: Scheduled to stop}
[2019-03-19 10:56:43,923]   INFO [uled executor 5] - .server.impl.CloudEventsLogger - Cloud instance entered 'stopping' state, profile 'AWS Cloud Profile'{id=amazon-2, projectId=_Root}, Amazon Instance{instanceId=i-071eff7574b93d12f, imageId=i-071eff7574b93d12f, status: Stopping}
[2019-03-19 10:57:43,925]   INFO [uled executor 1] - .server.impl.CloudEventsLogger - Cloud instance entered 'stopped' state, profile 'AWS Cloud Profile'{id=amazon-2, projectId=_Root}, Amazon Instance{instanceId=i-071eff7574b93d12f, imageId=i-071eff7574b93d12f, status: Stopped}

[2019-03-19 11:02:30,613]   WARN [io-8111-exec-11] - .instances.StoppedInstanceTask - Agent "yocto-build-agent1" {id=4} was not matched to stopped instance Amazon Instance{instanceId=i-071eff7574b93d12f, imageId=i-071eff7574b93d12f, status: Stopped}, profile 'AWS Cloud Profile'{id=amazon-2, projectId=_Root}
[2019-03-19 11:02:32,705]   WARN [nio-8111-exec-4] - .instances.StoppedInstanceTask - Agent "yocto-build-agent1" {id=4} was not matched to stopped instance Amazon Instance{instanceId=i-071eff7574b93d12f, imageId=i-071eff7574b93d12f, status: Stopped}, profile 'AWS Cloud Profile'{id=amazon-2, projectId=_Root}
[2019-03-19 11:02:43,939]   INFO [uled executor 4] - .instances.StoppedInstanceTask - Instance has changed status from stopped to Running: Amazon Instance{instanceId=i-071eff7574b93d12f, imageId=i-071eff7574b93d12f, status: Running}, profile 'AWS Cloud Profile'{id=amazon-2, projectId=_Root}
[2019-03-19 11:02:48,939]   INFO [uled executor 3] - .server.impl.CloudEventsLogger - Cloud instance entered 'running' state, profile 'AWS Cloud Profile'{id=amazon-2, projectId=_Root}, Amazon Instance{instanceId=i-071eff7574b93d12f, imageId=i-071eff7574b93d12f, status: Running}
[2019-03-19 11:07:22,457]   INFO [nio-8111-exec-1] - r.impl.DBCloudStateManagerImpl - Image: AmazonImageInstance{id=i-071eff7574b93d12f, amazonId=i-071eff7574b93d12f}, profile: profile 'AWS Cloud Profile'{id=amazon-2, projectId=_Root} was marked to CONTAIN agent
[2019-03-19 11:07:22,458]   INFO [nio-8111-exec-1] - .server.impl.CloudEventsLogger - Detected cloud agent "yocto-build-agent1" {id=4}, profile 'AWS Cloud Profile'{id=amazon-2, projectId=_Root}, Amazon Instance{instanceId=i-071eff7574b93d12f, imageId=i-071eff7574b93d12f, status: Running}
[2019-03-19 11:12:32,818]   INFO [uled executor 5] - te.IdleTimeoutTerminateFactory - Will stop instance Amazon Instance{instanceId=i-071eff7574b93d12f, imageId=i-071eff7574b93d12f, status: Running}, profile 'AWS Cloud Profile'{id=amazon-2, projectId=_Root} with idle time 5 minutes (idle time is 5 minutes)
[2019-03-19 11:12:32,818]   INFO [uled executor 5] - l.instances.StopInstanceAction - Terminating instance: Amazon Instance{instanceId=i-071eff7574b93d12f, imageId=i-071eff7574b93d12f, status: Running}, profile 'AWS Cloud Profile'{id=amazon-2, projectId=_Root}, reason: "After 5 minutes idle"
[2019-03-19 11:12:32,818]   INFO [uled executor 5] - r.impl.DBCloudStateManagerImpl - Image: AmazonImageInstance{id=i-071eff7574b93d12f, amazonId=i-071eff7574b93d12f}, profile: profile 'AWS Cloud Profile'{id=amazon-2, projectId=_Root} was marked to CONTAIN agent
[2019-03-19 11:12:33,050]   INFO [2 Stop Instance] - r.impl.DBCloudStateManagerImpl - Image: i-071eff7574b93d12f, Instance: i-071eff7574b93d12f, profile=amazon-2 is marked with state: stopped.
[2019-03-19 11:12:33,403]   INFO [nio-8111-exec-6] - ces.terminate.AgentRemoverImpl - Removing cloud agent "yocto-build-agent1" {id=4} because: Instance has been terminated. Amazon Instance{instanceId=i-071eff7574b93d12f, imageId=i-071eff7574b93d12f, status: Running}, profile 'AWS Cloud Profile'{id=amazon-2, projectId=_Root}
[2019-03-19 11:12:33,415]   INFO [nio-8111-exec-6] - .server.impl.CloudEventsLogger - Cloud agent Unregistered agent "yocto-build-agent1" {id=4} is no longer recognized as cloud agent because instance has gone: profileId=amazon-2, imageId=i-071eff7574b93d12f, instanceId=i-071eff7574b93d12f
[2019-03-19 11:12:33,415]   INFO [nio-8111-exec-6] - ces.terminate.AgentRemoverImpl - Removing cloud agent Unregistered agent "yocto-build-agent1" {id=4} because: Cloud instance (id=i-071eff7574b93d12f, imageId=i-071eff7574b93d12f, profileId=amazon-2) has gone (is not reported by cloud profile)
[2019-03-19 11:12:33,960]   INFO [uled executor 5] - .server.impl.CloudEventsLogger - Cloud instance entered 'scheduled to stop' state, profile 'AWS Cloud Profile'{id=amazon-2, projectId=_Root}, Amazon Instance{instanceId=i-071eff7574b93d12f, imageId=i-071eff7574b93d12f, status: Scheduled to stop}
[2019-03-19 11:12:43,960]   INFO [uled executor 5] - .server.impl.CloudEventsLogger - Cloud instance entered 'stopping' state, profile 'AWS Cloud Profile'{id=amazon-2, projectId=_Root}, Amazon Instance{instanceId=i-071eff7574b93d12f, imageId=i-071eff7574b93d12f, status: Stopping}
[2019-03-19 11:15:43,967]   INFO [uled executor 3] - .server.impl.CloudEventsLogger - Cloud instance entered 'stopped' state, profile 'AWS Cloud Profile'{id=amazon-2, projectId=_Root}, Amazon Instance{instanceId=i-071eff7574b93d12f, imageId=i-071eff7574b93d12f, status: Stopped}

（实例 ID 和代理名称已更改以保护无辜者）

如何防止 TeamCity Server 启动代理计算机，即使没有配置触发器？ 有 git commit 时启动代理机器的最佳方式是什么？我正在考虑在仅检查提交的较小代理上单独构建，并使较大的构建依赖于较小的构建。

Answer 1

这不是正常行为。逻辑是：如果没有可用的本地代理，Teamcity 将启动一个云构建代理。然后它会在空闲时终止，具体取决于您提供的选项。

我在 Root 项目中创建了几个云配置文件。我使用以下选项：

• 附加终止条件/如果空闲，则在整点前 3 分钟停止
• 最大实例数：x

不确定这是否有帮助，但我在 teamcity-clouds.log 中没有任何“与停止的实例不匹配”条目，我可以在您的日志中看到。

另外 - 您是否尝试过为云配置文件启用调试日志级别？您可以在管理/诊断 (/admin/admin.html?item=diagnostics) 中执行此操作。然后再次检查日志文件。

Answer 2

TeamCity 中的日志显示“状态从已停止变为正在启动”，这让我认为是 TeamCity 启动了此代理。事实证明，运行构建代理的 AWS 实例是由 OpsWorks 启动的。

Amazon OpsWorks 被配置为监控机器的状态并使其保持“正常运行”。因此，当 TeamCity 正确关闭服务器时，Amazon OpsWorks 会检测到机器出现故障，并尝试再次启动它。一段时间后，TeamCity 检测到它正在“启动”和“空闲”并再次停止它。这会无限期地进行。

除了 EC2 和 VMWare 功能外，如果在 TeamCity Cloud 配置中拥有 OpsWorks 功能，那就太好了。这样，TeamCity 可以告诉 Opsworks 机器应该停机了。

目前，我们已通过禁用对代理的 OpsWorks 运行状况监控来解决此问题。