Your browser was unable to load all of the resources. They may have been blocked by your firewall, proxy or browser configuration.
Press Ctrl+F5 or Ctrl+Shift+R to have your browser try again.

Compression on junit.h2.db files? #3275

ronald.brindl ·
Hi, I just tried and found out that our junit.h2.db files shrink to about 8% of their original size when I gzip them. Of course, QB cannot access them anymore.
However, H2 would support database within a zip file: http://www.h2database.com/html/features ... ase_in_zip.
We could store much longer build history if we could use that. It reduce the storage needs per build about 90%, since most of the disk usage is in the junit reports.

Unfortunately, this would require a different url to open the compressed database (e.g. /opt/qb/.../junit.h2.db.gz!junit instead of /opt/qb/.../junit.h2.db).

Since the junit report is generated once and never modified after that, it could apply for this kind of optimization:
After generating the junit report, close database, back it up using org.h2.tools.Backup, remove the original file.
All later accesses could then access the db using above url (e.g. jdbc:h2:/opt/qb/.../junit.h2.db.gz!junit).

This could be made optional via junit plugin parameters.

The same mechanism could apply for other test reports, too.
I don't know if it makes sense for all reports, but test-reports tend to get big (especially when used with system- or integration-tests)


Another option might be the h2 built in compression using "SET COMPRESS_LOB DEFLATE". This would avoid the above complexity with changing URLs.
I don't know how and how well the CLOB compression works in H2, and I don't even know if junit log outputs are stored as CLOBs in junit.h2.db at all.
But if, this could be a quite simple solution, by just adding "SET COMPRESS_LOB DEFLATE" to the <initSql> section of /com.pmease.quickbuild.plugin.report.junit/src/com/pmease/quickbuild/plugin/report/junit/setting.xml.
But as far as I can see, sysout and syserr are not stored as CLOB, but as varchar.
  • replies 1
  • views 2279
  • stars 0
steveluo ADMIN ·
Hi ronald.brindl,

Thank you very much for the information. It is very helpful!

After one day investigation, we think option 2 (use CLOB and compress_lob) would be better, but we still need more time to test this.

I have filed the issue below so you may watch it to get notified when it is fixed:
http://track.pmease.com/browse/QB-2484