Your browser was unable to load all of the resources. They may have been blocked by your firewall, proxy or browser configuration.
Press Ctrl+F5 or Ctrl+Shift+R to have your browser try again.

Step Retry Pattern #4523

MFalkner ·

I've set up a step retry with the following parameters:
If error matches specified pattern:

.*ERROR - Step.*is failed: Failed to delete file.*

Retry Times: 2
Retry Interval: 15

The step output is like this:

13:30:46,653 INFO  - Checking step execute condition...
13:30:46,653 INFO  - Step execute condition satisfied, executing...
13:35:13,201 INFO  - Executing pre-execute action...
13:35:14,104 INFO  - Retry condition not satisified
13:35:14,104 ERROR - Step 'master>Build Necessary>Build VS Version?VSVersion=cmake>Action Parallel>Action Execute Tests>CppUnit Config Parallel>CppUnit Test Config?Config=Release&Plat=Win32>CppUnit Test Run>CppUnit Excute Parallel>CppUnit Excute Targets Parallel?CppTestTarget=CppUnit>CppUnit Execute?Test=PersistenceStorageLib' is failed: Failed to delete file 'D:\Programs\buildagent\workspace\Fusion\SER\SER Branch\CppUnits\v143\Debug\Win32'.

What do I miss, as in my opinion, the pattern matches perfectly?

  • replies 6
  • views 797
  • stars 0
robinshen ADMIN ·

Please test if this works with latest QB 13.0.32 version, which also counts step pre/post execute action when evaluate the retry condition. Also please specify the pattern just as "Failed to delete file" as "Step is failed" is not part of error message, it is derived from step status.

MFalkner ·

Thanks Robin.
I'm already on 13.0.32, so this point is taken care of.

Now, I reduced it to

.*Failed to delete file.*

or ist the regex part '.*' in the front and at the end not needed?

Also, is there any log / flag in the 'Step Status' to indicate whether a step has been repeated or not. As this problem only occurs occasionally, I need something to search for to be able to check if it happened nd worked (so far I only see the case it did not work).
Martin

robinshen ADMIN ·

The step retry information only exists in build log. So this still does not work? The ".*" does not matter as QB does partial match over the error message.

Can you tell me how to set up configuration to reproduce your issue?

MFalkner ·

Thinking about reproducing the problem, I looked at the situation again and noted the following:

  1. The step is being delegated to a different (random) machine
  2. Looks like the problem happens very early in the process
13:30:46,738 INFO  - Checking step execute condition...
13:30:46,738 INFO  - Step execute condition satisfied, executing...
13:35:14,368 INFO  - Executing pre-execute action...
13:35:15,271 INFO  - Retry condition not satisified
13:35:15,272 ERROR - Step 'master>Build Necessary>Build VS Version?VSVersion=cmake>Action Parallel>Action Execute Tests>CppUnit Config Parallel>CppUnit Test Config?Config=Release&Plat=Win32>CppUnit Test Run>CppUnit Excute Parallel>CppUnit Excute Targets Parallel?CppTestTarget=CppUnit>CppUnit Execute?Test=PktFilter' is failed: Failed to delete file 'D:\Programs\buildagent\workspace\Fusion\SER\SER Branch\CppUnits\v143\Debug\Win32'.
  1. Strangely, sometimes the complete step may run through
02:42:29,071 INFO  - Checking step execute condition...
02:42:29,071 INFO  - Step execute condition satisfied, executing...
02:43:19,577 INFO  - Executing pre-execute action...
02:43:19,648 INFO  - Fetching input files...
02:43:41,957 INFO  - Running step...
02:43:42,097 INFO  -
02:43:42,097 INFO  - D:\Programs\buildagent\workspace\Fusion\SER\SER Bullseye\CppUnits\Release\Win32>if exist CppUnit_CellularLib.dll .\..\..\v143\Release\Win32\DllPlugInTester.exe -x CppUnit_CellularLib.xml -s Service.Release.Win32.CellularLib -o CppUnit_CellularLib.dll
02:43:43,968 INFO  - ...........................
02:43:44,079 INFO  - Sending output files...
02:43:44,079 INFO  - Workspace: D:\Programs\buildagent\workspace\Fusion/SER/SER Bullseye
02:43:45,110 INFO  - Executing post-execute action...
02:43:45,124 INFO  - Retry condition not satisified
02:43:45,124 ERROR - Step 'master>Build Bullseye?VSVersion=2022>Action Execute Tests>CppUnit Config Parallel>CppUnit Test Config?Config=Release&Plat=Win32>CppUnit Test Run>CppUnit Excute Parallel>CppUnit Excute Targets Parallel?CppTestTarget=CppUnit>CppUnit Execute?Test=CellularLib' is failed: Failed to delete file 'D:\workspace\Fusion\SER\SER Bullseye\CppUnits\Release\Win32\Coverage_CellularLib.cov'.
  1. The step is of "Type: Shell/Batch Command"
  2. There is no place in the script where a file is being deleted
  3. There is a "Fetch Input Files" and "Send Output Files" defined, but these do not delete files as well
  4. However, there is "Pre-Execute Action: Clean workspace on current node" defined

Could it be that 7 is the cause and this is not being considered in the evaluation?
(Similar to the problem you fixed in version 13.0.31?)

robinshen ADMIN ·

This is odd. I am trying to reproduce at my side and it is working. This is what I am doing:

  1. Create a test configuration, add into the master step a test Command/Batch step simply echoing a string
  2. Configure the test step to run on a build agent, fetching input files from master, and sending back output files
  3. Add a repeat param to test step, for instance db with values "oracle, mysql"
  4. Configure pre-execute action of the test step to clean workspace
  5. Configure step retry error message pattern to be "Failed to delete file" (or ".Failed to delete file.")
  6. Now create a text file under workspace of test configuration on the build agent, and open it with a Notepad.exe (to make workspace cleaning failed)
  7. Run the test configuration, and it fails. Build log shows that the step is retried.

Can you please try the same on your side and compare with your workflow to see if there is any obvious difference?

MFalkner ·

Yes, this is really odd.
I tried the same steps and it worked as well.
I also tried different variants, all worked.
Only one minor strange thing:
As a first step, I added a step that created the file in the workspace and sets the attributes to read only: The file was deleted anyway.
However, this is not part of this problem and might actually be an intentional feature of QB.
I'll try to find out what is the difference.