Your browser was unable to load all of the resources. They may have been blocked by your firewall, proxy or browser configuration.
Press Ctrl+F5 or Ctrl+Shift+R to have your browser try again.

External Resources #2003

jossy ·
After migration to QB4 I'm wondering if there is a way to use the new resource management for external (not build agent dependent) resources.
For example we are running tests using a remote database. There are multiple configurations (for our different product versions) and multiple QB nodes, which are able to run these tests.
To avoid problems, the number of connections to the database should be limited. Is there a way to realize this with a QB resource?
  • replies 11
  • views 3144
  • stars 0
robinshen ADMIN ·
You may embed the testing step inside a parent step running on server node, and control concurrency of the parent step on server node with resources.
jossy ·
Embedding the testing steps in a parent step running on the server is possible but has some really bad side effects:
All builds waiting for an external resource would be in status "RUNNING_BUILD", which makes the queue rather unclear and would also break the build timeout.

How about using a variable from e.g. the root configuration?
I already use one for a very basic maintenance mode:

groovy: system.getConfiguration("root").getVar("systemEnabled")?.value.equals("true")


(The corresponding resource is used in each master step.)

I think about setting a variable from a resource script (or from the node selection of the step?). Is this an approach that could work or do you see problems with synchronization etc. there?

Thx,
Josef
robinshen ADMIN ·
It is not clear to me how you utilize variable to solve this problem. However for my suggestion, if you acquire the external resource at master step, build will be put into waiting state if the external resource is exhausted.
jossy ·
Hi Robin,

I'm sorry, I think I haven't made the subject clear.
The problem is, that I need node specific and external resources in one build. As we use the "one build - one node" convention, running the master step on the server would break this concept.
It would also mean, that the master step would reserve the external resource, but the node specific resource reservation has to be done by a child step (if the actual build execution cannot be done on the server).
The build will start running, when the external resource is available. It will not check for the node specific resource at first, because it is not referenced by the master step.

Using a variable could be a (indeed very ugly) workaround:

Resource "externalResourceAvailable", Node Selection script:
groovy: system.getConfiguration("root").getVar("externalResourceAvailable")?.value.equals("true")


Configurations:
Pre-Build Scipt:
groovy:
import com.pmease.quickbuild.persistence.SessionManager;
import com.pmease.quickbuild.variable.*;

def configuration = system.getConfiguration("root")
def var = configuration.getVar("externalResourceAvailable")

if (var?.value.equals("true")) {
SessionManager.openSession();

try {
var.setValue("false")
system.configurationManager.save(configuration)

} finally {

SessionManager.closeSession();

}
}

Post-Build Scipt:
groovy:
import com.pmease.quickbuild.persistence.SessionManager;
import com.pmease.quickbuild.variable.*;

def configuration = system.getConfiguration("root")
def var = configuration.getVar("externalResourceAvailable")

if (var?.value.equals("false")) {
SessionManager.openSession();

try {
var.setValue("true")
system.configurationManager.save(configuration)

} finally {

SessionManager.closeSession();

}
}


And adding resource "externalResourceAvailable" to the existing node selection criteria of the master step.

In my test setup this seems to work, but I'm worried about running into synchronization issues etc. under real world conditions.
Do you think this could work or are there problems predictable?
robinshen ADMIN ·
Thanks for the explanation, now I understand your approach. Unfortunately this does have issues, considering two builds running almost at the same time, both may get the resource "externalResourceAvailable", as pre-build script of the first build may not get running yet for the first build when the second build tries to acquire the resource. Instead I propose another approach to accommodate with your current "one build one node" setting:
1. Assume maximum number of allowed database connections is 5. Pick 5 arbitrary agents and define the node user attribute: dbconnection=yes
2. For builds requiring database connections, add another criteria to master step node selection: node.hasAttribute("dbconnection")
3. For builds not requiring database connections, add another criteria to master step node selection: !node.hasAttribute("dbconnection")
jossy ·
Hi Robin!

This approach would need 5 nodes which are only used for builds using dbconnection. Also the problem with this is, that we have many different nodes (which have different OS, compiler versions etc.). We want to run builds (tests) on different platforms with different node configurations and all of those could have the need for an external resource, but these are limited.
And if we can only use them for builds with the need for a database connection this would prevent other builds from running on this node.

We would need to have some sort of a global external resource reservation, as mentioned before with the root configuration variable.

Do you have any suggestions how to achieve such behavior?

Would something like http://wiki.pmease.com/display/QB40/Lim ... bs+on+Node do the trick if I use the resource on the server with grid.getServerNode().getResource(..., ...)?
robinshen ADMIN ·
Calling grid.serverNode.getResource() will not work as expected, as the call will not get synchronized to server node. Instead, I'd suggest specifying a special agent defining a db connection resource with available count, and then set all steps using the db connection resource to be under a special parent step whose node selection setting is configured to acquire that db connection redsource. The side effect is that all your configurations using db connection will goto that special node at some point but child steps can still be configured to run on any desired node as you want.
jossy ·
Hi again!

We want to avoid the build from getting started before all needed resources are available. I think your suggestion would start the build and then wait for the resource in a child step.

I've tried the following:

Master step
Node selection:
return grid.getServerNode().getResource("dbconnection", new java.util.concurrent.Semaphore(1, true)).tryAcquire()


Post-Build Script:
grid.getServerNode().getResource("dbconnection").release()


And it worked on my test environment, but is it guaranteed that the conditions in the node selection are executed in the order of the definition? Because it has to try to acquire the Semaphore resource after all other conditions, so the Semaphore is not modified if the build does not start (The post build script does not get executed -> dead lock).

I see another problem with your suggested approach. How do I know which node is selected on master? I want to run the entire build on one node.
Jonathan ·
Jossy,

I have previously requested such a feature:
http://track.pmease.com/browse/QB-1801

Is this what you are asking for?
jossy ·
That's pretty much it. The license example is very good. It is very hard to achieve what I'm looking for with the current possibilities.

Another thing why I do not want to have the build running when still waiting for an external resource is that we have a build timeout.
robinshen ADMIN ·
Calling grid.getServerNode().getResource will not work as expected if the code runs on different machines. That is, instead of locking actual server resources over network, it only locks local objects, and grid.getServerNode() represents two different objects on different machines. Right now, there is no approach to acquire resources for all steps all at once before build started. If you concerns about build timeout, you may set a very high value for that, and use the step timeout introduced in QB 5.1 instead.