We had a build fail with this error:

Open database [/share/quickbuild/publish/mainline/builds/162046/reports/scm_activity/DEFAULT] failed. Database status [No active connections].
caused by: Unable to get connection from database url: jdbc:h2:/share/quickbuild/publish/mainline/builds/162046/reports/scm_activity/DEFAULT/scm_activity;ACCESS_MODE_DATA=r
caused by: File corrupted while reading record: "/archive/quickbuild/publish/mainline/builds/162046/reports/scm_activity/DEFAULT/scm_activity.h2.db length: 0". Possible solution: use the recovery tool [90030-175]

The file is zero-length.

The following two entries were in the system log:

2022-06-09 22:40:13,544 [pool-1-thread-137463] ERROR com.pmease.quickbuild.grid.GridTaskFuture - Error finishing job.
    java.lang.RuntimeException: java.io.IOException: No space left on device
        at com.pmease.quickbuild.bootstrap.BootstrapUtils.wrapAsUnchecked(BootstrapUtils.java:56)
        at com.pmease.quickbuild.util.ExceptionUtils.wrapAsUnchecked(ExceptionUtils.java:82)
        at com.pmease.quickbuild.log.BuildLogger.append(BuildLogger.java:82)
        at com.pmease.quickbuild.BuildAwareTask.jobFinished(BuildAwareTask.java:34)
        at com.pmease.quickbuild.grid.GridTaskFuture.jobFinished(GridTaskFuture.java:147)
        at com.pmease.quickbuild.grid.NodeServiceImpl$1.run(NodeServiceImpl.java:149)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:724)
    Caused by: java.io.IOException: No space left on device
        at java.io.FileOutputStream.writeBytes(Native Method)
        at java.io.FileOutputStream.write(FileOutputStream.java:345)
        at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
        at java.io.BufferedOutputStream.write(BufferedOutputStream.java:126)
        at java.io.ObjectOutputStream$BlockDataOutputStream.drain(ObjectOutputStream.java:1870)
        at java.io.ObjectOutputStream$BlockDataOutputStream.setBlockDataMode(ObjectOutputStream.java:1779)
        at java.io.ObjectOutputStream.writeNonProxyDesc(ObjectOutputStream.java:1283)
        at java.io.ObjectOutputStream.writeClassDesc(ObjectOutputStream.java:1228)
        at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1424)
        at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1175)
        at java.io.ObjectOutputStream.writeFatalException(ObjectOutputStream.java:1570)
        at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:350)
        at com.pmease.quickbuild.log.BuildLogger.append(BuildLogger.java:76)


2022-06-09 22:40:13,896 [pool-1-thread-137393] ERROR com.pmease.quickbuild.DefaultBuildEngine - Error processing build request.
    com.pmease.quickbuild.plugin.report.engine.exception.DatabaseException: Open database [/share/quickbuild/publish/mainline/configurations/4182/reports/scm_activity/DEFAULT] failed. Database
status [No active connections].
        at com.pmease.quickbuild.plugin.report.engine.datastore.DbStore.open(DbStore.java:355)
        at com.pmease.quickbuild.plugin.report.changes.generator.ChangesStatsGenerator.generate(ChangesStatsGenerator.java:40)
        at com.pmease.quickbuild.plugin.report.engine.contribution.publisher.StatisticsReportPublisher.doPublish(StatisticsReportPublisher.java:133)
        at com.pmease.quickbuild.plugin.report.engine.contribution.publisher.StatisticsReportPublisher.collectStatistics(StatisticsReportPublisher.java:111)
        at com.pmease.quickbuild.DefaultBuildEngine.process(DefaultBuildEngine.java:416)
        at com.pmease.quickbuild.DefaultBuildEngine.access$000(DefaultBuildEngine.java:139)
        at com.pmease.quickbuild.DefaultBuildEngine$2.run(DefaultBuildEngine.java:1144)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:724)
    Caused by: com.pmease.quickbuild.plugin.report.engine.exception.DatabaseException: Unable to get connection from database url: jdbc:h2:/share/quickbuild/publish/mainline/configurations/4182/reports/scm_activity/DEFAULT/scm_activity
        at com.pmease.quickbuild.plugin.report.engine.datastore.ConnectionManager.getConnection(ConnectionManager.java:25)
        at com.pmease.quickbuild.plugin.report.engine.datastore.DbStore.getConnection(DbStore.java:384)
        at com.pmease.quickbuild.plugin.report.engine.datastore.DbStore.open(DbStore.java:349)
        ... 9 more
    Caused by: org.h2.jdbc.JdbcSQLException: Error opening database: "Could not save properties /archive/quickbuild/publish/mainline/configurations/4182/reports/scm_activity/DEFAULT/scm_activity.lock.db" [8000-175]
        at org.h2.message.DbException.getJdbcSQLException(DbException.java:332)
        at org.h2.message.DbException.get(DbException.java:161)
        at org.h2.store.FileLock.getExceptionFatal(FileLock.java:450)
        at org.h2.store.FileLock.save(FileLock.java:223)
        at org.h2.store.FileLock.lockFile(FileLock.java:357)
        at org.h2.store.FileLock.lock(FileLock.java:133)
        at org.h2.engine.Database.open(Database.java:591)
        at org.h2.engine.Database.openDatabase(Database.java:244)
        at org.h2.engine.Database.<init>(Database.java:239)
        at org.h2.engine.Engine.openSession(Engine.java:58)
        at org.h2.engine.Engine.openSession(Engine.java:162)
        at org.h2.engine.Engine.createSessionAndValidate(Engine.java:141)
        at org.h2.engine.Engine.createSession(Engine.java:124)
        at org.h2.engine.Engine.createSession(Engine.java:30)
        at org.h2.engine.SessionRemote.connectEmbeddedOrServer(SessionRemote.java:323)
        at org.h2.jdbc.JdbcConnection.<init>(JdbcConnection.java:105)
        at org.h2.jdbc.JdbcConnection.<init>(JdbcConnection.java:90)
        at org.h2.Driver.connect(Driver.java:73)
        at java.sql.DriverManager.getConnection(DriverManager.java:571)
        at java.sql.DriverManager.getConnection(DriverManager.java:215)
        at com.pmease.quickbuild.plugin.report.engine.datastore.ConnectionManager.getConnection(ConnectionManager.java:23)
        ... 11 more
    Caused by: java.io.IOException: No space left on device
        at java.io.FileOutputStream.writeBytes(Native Method)
        at java.io.FileOutputStream.write(FileOutputStream.java:345)
        at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
        at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291)
        at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:295)
        at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:141)
        at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229)
        at java.io.BufferedWriter.flush(BufferedWriter.java:254)
        at java.util.Properties.store0(Properties.java:838)
        at java.util.Properties.store(Properties.java:812)
        at org.h2.store.FileLock.save(FileLock.java:213)
        ... 28 more

When I checked, the filesystem had 5G free (out of 1.6T). I did some artifact cleanup and now have 14G free. I have some more cleanup I can do immediately to likely get up to 25-50G free. Next week I should be able to have 200G free.

However, how do I recover from the error?

• The message mentions using the recover tool, but I have no idea what that is.
• Will Quickbuild try the write again when there is sufficient space?
• If not, can I restore the files from the previous night's backup?

Thanks,
Paul