@robinshen
So in our system we have 2 types of configurations, normal configurations and special-purpose configurations.
The main difference is that the special ones run on specified agents, while the normal ones run on server.
For each special configuration we have a specific resource that is consumed at the master step and released when the configuration is complete. That way we prevent other configurations to interact with our agents while they are in use.
The problem arises on a very random time-window that a scheduled normal configuration interacts with an agent and changes its state (for instance shutting it down) and a scheduled special configuration starts while the agent's state is being altered and before the agent is shut down. Then the special configuration would fail when the agent is shutdown.
My ideal scenario would be that when a normal configuration starts, which will do stuff on our agents, we consume the specified resource (or block it from being used) or reserved it somehow. But I understand that I should find another way.
@drdt I believe this won't work because the scheduled special configurations couldn't tell if the resource is being in use and this needs to be added in every special configuration rather than be in a single place like the resources.