Your browser was unable to load all of the resources. They may have been blocked by your firewall, proxy or browser configuration.
Press Ctrl+F5 or Ctrl+Shift+R to have your browser try again.

Upgrade QB8->QB10 with extremely large database. #4492

duclong.ng ·

I am having a problem with upgrade from QB8 to QB10. I using mysql for QB service.
I used ".../qb10/bin/upgrade.sh .../qb8" to upgrade.
Everything was quite easy when I tested with a small amount of data. And it runs for 2 hours with data about 4GB. According to what I observed in the log, the data will be export and import again. so, the process is quite long time.

With the official service, we have more than 100GB of data. This process estimates that it takes too much time and is likely to have an error.
So is there any other way for us to do this process faster and safer?

  • Create more schools that are missing with SQL command?
  • replies 14
  • views 1479
  • stars 0
robinshen ADMIN ·

I guess there are many many build records in the database. You may consider cleaning up obsolete records with build clean up policy to make the database smaller.

duclong.ng ·

Is that the only way? We have scheduled for the old records to be deleted automatically. And here is the remaining data.

duclong.ng ·
2023-05-08 18:19:46,021 INFO  >>> 2023-05-08 18:19:46,021 WARN  OutOfMemory error encountered. Reduce batch size and retry...
2023-05-08 18:19:46,023 INFO  >>> 2023-05-08 18:19:46,022 INFO  Loading table rows (342001->343000) from database...
2023-05-08 18:19:50,562 INFO  >>> 2023-05-08 18:19:50,561 WARN  OutOfMemory error encountered. Reduce batch size and retry...
2023-05-08 18:19:50,562 INFO  >>> 2023-05-08 18:19:50,562 INFO  Loading table rows (342001->342500) from database...
2023-05-08 18:19:51,564 INFO  >>> 2023-05-08 18:19:51,564 WARN  OutOfMemory error encountered. Reduce batch size and retry...
2023-05-08 18:19:51,564 INFO  >>> 2023-05-08 18:19:51,564 INFO  Loading table rows (342001->342250) from database...
2023-05-08 18:19:52,296 INFO  >>> 2023-05-08 18:19:52,296 WARN  OutOfMemory error encountered. Reduce batch size and retry...
2023-05-08 18:19:52,296 INFO  >>> 2023-05-08 18:19:52,296 INFO  Loading table rows (342001->342125) from database...
2023-05-08 18:19:52,349 INFO  >>> 2023-05-08 18:19:52,349 WARN  OutOfMemory error encountered. Reduce batch size and retry...
2023-05-08 18:19:52,349 INFO  >>> 2023-05-08 18:19:52,349 INFO  Loading table rows (342001->342062) from database...
2023-05-08 18:19:52,365 INFO  >>> 2023-05-08 18:19:52,365 INFO  Stopping QuickBuild server...
2023-05-08 18:19:52,385 INFO  >>> 2023-05-08 18:19:52,385 INFO  Stopping plugins...
2023-05-08 18:19:52,389 INFO  >>> 2023-05-08 18:19:52,389 INFO  Stopping task scheduler...
2023-05-08 18:19:52,389 INFO  >>> 2023-05-08 18:19:52,389 INFO  Closing session factory...
2023-05-08 18:19:52,478 INFO  >>> 2023-05-08 18:19:52,476 WARN  FAILED net.bull.javamelody.MonitoringFilter-1140e7a4: java.lang.OutOfMemoryError: GC overhead limit exceeded
2023-05-08 18:19:52,478 INFO  >>>  java.lang.OutOfMemoryError: GC overhead limit exceeded
2023-05-08 18:19:52,482 INFO  >>> 2023-05-08 18:19:52,481 WARN  FAILED org.eclipse.jetty.servlet.ServletHandler@b73433: java.lang.OutOfMemoryError: GC overhead limit exceeded
2023-05-08 18:19:52,482 INFO  >>>  java.lang.OutOfMemoryError: GC overhead limit exceeded
2023-05-08 18:19:52,482 INFO  >>> 2023-05-08 18:19:52,482 WARN  FAILED org.eclipse.jetty.server.session.SessionHandler@1bf14704: java.lang.OutOfMemoryError: GC overhead limit exceeded
2023-05-08 18:19:52,482 INFO  >>>  java.lang.OutOfMemoryError: GC overhead limit exceeded
2023-05-08 18:19:52,482 INFO  >>> 2023-05-08 18:19:52,482 WARN  FAILED o.e.j.s.ServletContextHandler@2be21396{/,null,UNAVAILABLE}: java.lang.OutOfMemoryError: GC overhead limit exceeded
2023-05-08 18:19:52,482 INFO  >>>  java.lang.OutOfMemoryError: GC overhead limit exceeded
2023-05-08 18:19:52,483 INFO  >>> 2023-05-08 18:19:52,483 WARN  FAILED org.eclipse.jetty.server.handler.HandlerCollection@124dac75[o.e.j.s.ServletContextHandler@2be21396{/,null,UNAVAILABLE}]: java.lang.OutOfMemoryError: GC overhead limit exceeded
2023-05-08 18:19:52,483 INFO  >>>  java.lang.OutOfMemoryError: GC overhead limit exceeded
2023-05-08 18:19:52,483 INFO  >>> 2023-05-08 18:19:52,483 WARN  FAILED org.eclipse.jetty.server.Server@315105f: java.lang.OutOfMemoryError: GC overhead limit exceeded
2023-05-08 18:19:52,483 INFO  >>>  java.lang.OutOfMemoryError: GC overhead limit exceeded
2023-05-08 18:19:52,483 ERROR  >>> Exception in thread "main" java.lang.reflect.InvocationTargetException
2023-05-08 18:19:52,484 ERROR  >>>      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2023-05-08 18:19:52,484 ERROR  >>>      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
2023-05-08 18:19:52,484 ERROR  >>>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2023-05-08 18:19:52,484 ERROR  >>>      at java.lang.reflect.Method.invoke(Method.java:498)
2023-05-08 18:19:52,484 ERROR  >>>      at com.pmease.quickbuild.bootstrap.Bootstrap.shutdown(Bootstrap.java:216)
2023-05-08 18:19:52,484 ERROR  >>>      at com.pmease.quickbuild.bootstrap.Bootstrap.boot(Bootstrap.java:659)
2023-05-08 18:19:52,484 ERROR  >>>      at com.pmease.quickbuild.bootstrap.Bootstrap.main(Bootstrap.java:169)
2023-05-08 18:19:52,484 ERROR  >>> Caused by: java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
2023-05-08 18:19:52,484 ERROR  >>>      at com.pmease.quickbuild.bootstrap.BootstrapUtils.wrapAsUnchecked(BootstrapUtils.java:56)
2023-05-08 18:19:52,484 ERROR  >>>      at com.pmease.quickbuild.equinoxadapter.EquinoxAdapter.shutdown(EquinoxAdapter.java:284)
2023-05-08 18:19:52,484 ERROR  >>>      ... 7 more
2023-05-08 18:19:52,484 ERROR  >>> Caused by: java.lang.reflect.InvocationTargetException
2023-05-08 18:19:52,484 ERROR  >>>      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2023-05-08 18:19:52,484 ERROR  >>>      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
2023-05-08 18:19:52,484 ERROR  >>>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2023-05-08 18:19:52,484 ERROR  >>>      at java.lang.reflect.Method.invoke(Method.java:498)
2023-05-08 18:19:52,484 ERROR  >>>      at com.pmease.quickbuild.equinoxadapter.EquinoxAdapter.shutdown(EquinoxAdapter.java:282)
2023-05-08 18:19:52,484 ERROR  >>>      ... 7 more
2023-05-08 18:19:52,484 ERROR  >>> Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded

Hello@robinshen,
I had a problem when Upgrade QB8 -> QB10 with a database only 13GB. Do you have any advice?

robinshen ADMIN ·

How much memory you've assigned to QB server? You may check it at conf/wrapper.conf for property wrapper.java.maxmemory.

duclong.ng ·

The server's memory is quite large, I think it is due to the limit placed in the conf/wrapper.conf file. I will try to change it.
Thanks.

duclong.ng ·

Hello@robinshen,
It seems that the upgrade process has successfully .
But the names of my users written in Korean have been replaced by "????". The same thing happens to configurations using the description in Korean.
Is this normal or abnormal?

robinshen ADMIN ·

This is abnormal. Seems that your database is not using UTF8 encoding. You may check names in the original backup (just unzip and open users.xml) to see if names are displayed correctly.

duclong.ng ·

Our data is stored in UTF8-BIN format. Does this cause errors?

duclong.ng ·

I see that the name displayed in the users.xml file is in accordance with the format.

duclong.ng ·

The problem did not recur when I tried to try again. Thanks!

duclong.ng ·

Hello@robinshen ,
I tried upgrade the QBM and most of them were successful. But one of them had an error.

2023-09-28 19:11:54,183 [pool-2-thread-1] INFO com.pmease.quickbuild.bootstrap.Upgrade - >>> 2023-09-28 19:11:54,182 INFO  Exporting table 'GroupShare'...
2023-09-28 19:11:54,183 [pool-2-thread-1] INFO com.pmease.quickbuild.bootstrap.Upgrade - >>> 2023-09-28 19:11:54,182 INFO  Querying table ids...
2023-09-28 19:11:54,187 [pool-2-thread-1] INFO com.pmease.quickbuild.bootstrap.Upgrade - >>> 2023-09-28 19:11:54,187 INFO  Loading table rows (1->1) from database...
2023-09-28 19:11:54,202 [pool-2-thread-1] INFO com.pmease.quickbuild.bootstrap.Upgrade - >>> 2023-09-28 19:11:54,202 INFO  Converting table rows to XML...
2023-09-28 19:11:54,203 [pool-2-thread-1] INFO com.pmease.quickbuild.bootstrap.Upgrade - >>> 2023-09-28 19:11:54,203 INFO  Writing resulting XML to file 'GroupShares.xml...
2023-09-28 19:11:54,204 [pool-2-thread-1] INFO com.pmease.quickbuild.bootstrap.Upgrade - >>> 2023-09-28 19:11:54,204 INFO
2023-09-28 19:11:54,204 [pool-2-thread-1] INFO com.pmease.quickbuild.bootstrap.Upgrade - >>> 2023-09-28 19:11:54,204 INFO  Exporting table 'Audit'...
2023-09-28 04:27:40,840 INFO  >>> 2023-09-28 04:27:40,840 INFO  Querying table ids...
2023-09-28 04:31:02,476 INFO  >>> 2023-09-28 04:31:02,476 WARN  SQL Error: 0, SQLState: S1000
2023-09-28 04:31:02,478 INFO  >>> 2023-09-28 04:31:02,478 ERROR  Can not read response from server. Expected to read 4 bytes, read 0 bytes before connection was unexpectedly lost.
2023-09-28 04:31:02,481 INFO  >>> 2023-09-28 04:31:02,480 WARN  SQL Error: 0, SQLState: 08007
2023-09-28 04:31:02,481 INFO  >>> 2023-09-28 04:31:02,481 ERROR  Communications link failure during rollback(). Transaction resolution unknown.
2023-09-28 04:31:02,482 INFO  >>> 2023-09-28 04:31:02,481 INFO  Stopping QuickBuild server...
2023-09-28 04:31:02,507 INFO  >>> 2023-09-28 04:31:02,506 INFO  Stopping plugins...
2023-09-28 04:31:02,514 INFO  >>> 2023-09-28 04:31:02,513 INFO  Stopping task scheduler...
2023-09-28 04:31:02,514 INFO  >>> 2023-09-28 04:31:02,514 INFO  Closing session factory...
2023-09-28 04:31:02,707 INFO  >>> 2023-09-28 04:31:02,706 INFO  QuickBuild server stopped.
2023-09-28 04:31:02,707 ERROR  >>> Exception in thread "main" org.hibernate.exception.JDBCConnectionException: Could not close connection
2023-09-28 04:31:02,708 ERROR  >>>      at org.hibernate.exception.internal.SQLExceptionTypeDelegate.convert(SQLExceptionTypeDelegate.java:65)

Perhaps the difference is that QBM has a bigger AUDIT table data size (about 13GB/ 14GB for size of database)
How can I fix the error?

robinshen ADMIN ·

You may delete old records from QB_AUDIT with sql tool and try again.

duclong.ng ·

That works well, but is there any way for me to keep those records?

robinshen ADMIN ·

You may try to backup that table with your db tools, and then restore it after upgrade.