Dynamically add or remove parallel steps #3590

helmut.n · 8 years ago

We are currently creating a new test system for a couple of products with this requirements and characteristics:
* Each of our six test hosts can test all products
* A test host cannot test two products at the same time
* Tests of one product should be distributed to up to four test hosts, if available.
* Test hosts are retrieving test cases dynamically from a central database during the test run. Therefor alle test hosts which are testing the same product are finished nearly at the same time.
* Total testing of different products takes different times

My question is: How can I control test host usage in the following situation?
Product A is currently tested on four test hosts. Now product B should be tested. There are two test host available. But tests of Product B might be finished in 1 minute.

Idea 1:
I start the Parallel Composition with the available two test hosts. Can I then dynamically add additional test steps to a Parallel Composition while its running, to add more test hosts when they become available?

Idea 2:
I add four test hosts to the Parallel Composition even if some of them are currently not available. But used test hosts might not be finished with their tests of product A before the available test hosts are finished with testing product B. Then the Parallel Composition has to wait until product A is tested before the next steps can be performed. Is there a way to remove parallel steps which did not start yet (since the required resource is not available) with a script?

Do you have an idea how I can handle this reqirements with QuickBuild?

replies 4
views 3891
stars 0

robinshen ADMIN · 8 years ago

I'd suggest to add four test steps inside the parallel composition for each of your product. And each of the test step requires a host resource. So if test of product A runs, it will grab four hosts for testing. Before A finishes, if test of product B runs, it will grab two other available hosts and the other two steps will wait until A releases its hosts.

helmut.n · 8 years ago

If I understand you right you are advising to use *Idea 2*. But maybe I didn't explain the problem of this approach exactly enough.

After the parallel composition with the test execution steps, there some other final steps which are informing developers about failed testcases. This information should be send as soon as possible after test execution is finished.


+-- Parallel Composition - Product A ----------------+    +-- Parallel Composition - Product B ----------------+
|                                                    |    |                                                    |
| +- Host 1-+  +- Host 2-+  +- Host 3-+  +- Host 4-+ |    | +- Host 5-+  +- Host 6-+  +- Host ?-+  +- Host ?-+ |
| | running |  | running |  | running |  | running | |    | | running |  | running |  | wait    |  | wait    | |
| +---------+  +---------+  +---------+  +---------+ |    | +---------+  +---------+  +---------+  +---------+ |
|                                                    |    |                                                    |
+----------------------------------------------------+    +----------------------------------------------------+
+-- Final Actions -----------------------------------+    +-- Final Actions -----------------------------------+
| wait                                               |    | wait                                               |
+----------------------------------------------------+    +----------------------------------------------------+

Then Tests of Product B are finished but tests of product A are still executed:


+-- Parallel Composition - Product A ----------------+    +-- Parallel Composition - Product B ----------------+
|                                                    |    |                                                    |
| +- Host 1-+  +- Host 2-+  +- Host 3-+  +- Host 4-+ |    | +- Host 5-+  +- Host 6-+  +- Host ?-+  +- Host ?-+ |
| | running |  | running |  | running |  | running | |    | | success |  | success |  | wait    |  | wait    | |
| +---------+  +---------+  +---------+  +---------+ |    | +---------+  +---------+  +---------+  +---------+ |
|                                                    |    |                                                    |
+----------------------------------------------------+    +----------------------------------------------------+
+-- Final Actions -----------------------------------+    +-- Final Actions -----------------------------------+
| wait                                               |    | wait                                               |
+----------------------------------------------------+    +----------------------------------------------------+

But it still has to wait until ressources for other testhosts are available. Therfore it would be unattractively to wait for some steps even if all tests have been performed on the available hosts before the other hosts are available. Is it possible to cancel or better remove steps, which are waiting for a ressource via script?

robinshen ADMIN · 8 years ago

Unfortunately this is not doable. Dynamically adding/removing steps will break the ongoing build process.

shdror · 7 years ago

Hi helut.n
I would suggest to have a "parent" configuration with all steps needed to run on a generic product, and then inherit them for each of your actual products (A, B, ...)
The parent configuration would use all (4 at most) resources.
When you would run several products simultaneously - they'll share the resources, and in the scenario you described, configA would still run (with 4 hosts) even after the successful end of configB - configB would send the notifications needed - as it is finished, since the child configurations are actually separated.
HTH,
Dror.