Your browser was unable to load all of the resources. They may have been blocked by your firewall, proxy or browser configuration.
Press Ctrl+F5 or Ctrl+Shift+R to have your browser try again.

QuickBuild

[QB 8.0.11] Agent failed to start after Upgrade: java.net.BindException: Address already in use: bind #3973

bug installation
#1
deefrazier ·

Impact: All nodes are inactive from grid

Experienced: After upgrading our QB test server from 8.0.5 to 8.0.11, all qb agent nodes are shown as inactive on grid page (62 windows agents). Inspection of agent log shows that after the agent updated, it failed to start with the error: java.net.BindException: Address already in use: bind. We looked for a hung java process on the box, but couldnt find one. Restarting the box resolved the issue.

Agent Log:

2018-07-30 10:09:00,938 [Wrapper-Restarter] INFO  com.pmease.quickbuild.Quickbuild - Waiting for finish of existing jobs...
2018-07-30 10:09:00,938 [Wrapper-Restarter] INFO  com.pmease.quickbuild.Quickbuild - Stopping plugins...
2018-07-30 10:09:00,954 [Wrapper-Restarter] INFO  com.pmease.quickbuild.Quickbuild - Stopping task scheduler...
2018-07-30 10:14:02,402 [Wrapper-Restarter] INFO  com.pmease.quickbuild.Quickbuild - QuickBuild agent stopped.
2018-07-30 10:14:19,068 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.Bootstrap - Cleaning cached plugin libraries...
2018-07-30 10:14:21,104 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.Bootstrap - Cleaning temp directory...
2018-07-30 10:14:21,119 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.Bootstrap - Benchmarking...
2018-07-30 10:14:23,925 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.Bootstrap - Connecting to server 'https:/***server address removed***:1443'...
2018-07-30 10:14:24,896 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Updating framework...
2018-07-30 10:14:24,943 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Updating plugins...
2018-07-30 10:14:24,943 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Downloading file 'com.pmease.quickbuild.plugin.scm.github_8.0.7.jar'...
2018-07-30 10:14:24,974 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Downloading file 'com.pmease.quickbuild.plugin.report.jmeter_8.0.5.jar'...
2018-07-30 10:14:24,974 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Downloading file 'com.pmease.quickbuild.plugin.scm.gerrit_8.0.1.jar'...
2018-07-30 10:14:24,974 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Downloading file 'com.pmease.quickbuild.plugin.scm.perforce_8.0.2.jar'...
2018-07-30 10:14:24,974 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Downloading file 'com.pmease.quickbuild_8.0.14.jar'...
2018-07-30 10:14:25,646 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Downloading file 'com.pmease.quickbuild.plugin.notifier.slack_8.0.6.jar'...
2018-07-30 10:14:25,646 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Downloading file 'com.pmease.quickbuild.plugin.scm.bitbucketserver_8.0.1.jar'...
2018-07-30 10:14:25,646 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Downloading file 'com.pmease.quickbuild.plugin.scm.bitbucketcloud_8.0.1.jar'...
2018-07-30 10:14:25,646 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Downloading file 'com.pmease.quickbuild.plugin.scm.git_8.0.9.jar'...
2018-07-30 10:14:25,646 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Downloading file 'com.pmease.quickbuild.plugin.builder.gradle_8.0.7.jar'...
2018-07-30 10:14:25,646 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Downloading file 'com.pmease.quickbuild.plugin.tracker.jira_8.0.10.jar'...
2018-07-30 10:14:25,677 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Downloading file 'com.pmease.quickbuild.plugin.report.jacoco_8.0.11.jar'...
2018-07-30 10:14:25,677 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Downloading file 'com.pmease.quickbuild.plugin.scm.gitlab_8.0.10.jar'...
2018-07-30 10:14:25,677 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Downloading file 'com.pmease.quickbuild.plugin.measurement.core_8.0.10.jar'...
2018-07-30 10:14:25,724 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Downloading file 'com.pmease.quickbuild.plugin.alert.engine_8.0.10.jar'...
2018-07-30 10:14:25,740 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Downloading file 'com.pmease.quickbuild.plugin.report.engine_8.0.5.jar'...
2018-07-30 10:14:25,756 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Downloading file 'com.pmease.quickbuild.plugin.cloud.ec2_8.0.2.jar'...
2018-07-30 10:14:25,912 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Removing file 'com.pmease.quickbuild.plugin.scm.gerrit_8.0.0.jar'...
2018-07-30 10:14:25,927 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Removing file 'com.pmease.quickbuild.plugin.scm.gitlab_8.0.4.jar'...
2018-07-30 10:14:25,927 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Removing file 'com.pmease.quickbuild.plugin.scm.git_8.0.3.jar'...
2018-07-30 10:14:25,927 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Removing file 'com.pmease.quickbuild.plugin.scm.bitbucketserver_8.0.0.jar'...
2018-07-30 10:14:25,927 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Removing file 'com.pmease.quickbuild.plugin.report.engine_8.0.0.jar'...
2018-07-30 10:14:25,927 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Removing file 'com.pmease.quickbuild.plugin.report.jmeter_8.0.0.jar'...
2018-07-30 10:14:25,927 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Removing file 'com.pmease.quickbuild.plugin.scm.github_8.0.1.jar'...
2018-07-30 10:14:25,959 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Removing file 'com.pmease.quickbuild.plugin.tracker.jira_8.0.4.jar'...
2018-07-30 10:14:25,990 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Removing file 'com.pmease.quickbuild.plugin.scm.bitbucketcloud_8.0.0.jar'...
2018-07-30 10:14:25,990 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Removing file 'com.pmease.quickbuild.plugin.builder.gradle_8.0.1.jar'...
2018-07-30 10:14:25,990 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Removing file 'com.pmease.quickbuild.plugin.report.jacoco_8.0.5.jar'...
2018-07-30 10:14:25,990 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Removing file 'com.pmease.quickbuild.plugin.cloud.ec2_8.0.0.jar'...
2018-07-30 10:14:26,084 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Removing file 'com.pmease.quickbuild_8.0.8.jar'...
2018-07-30 10:14:26,099 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Removing file 'com.pmease.quickbuild.plugin.measurement.core_8.0.4.jar'...
2018-07-30 10:14:26,099 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Removing file 'com.pmease.quickbuild.plugin.notifier.slack_8.0.0.jar'...
2018-07-30 10:14:26,099 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Removing file 'com.pmease.quickbuild.plugin.alert.engine_8.0.4.jar'...
2018-07-30 10:14:26,099 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Removing file 'com.pmease.quickbuild.plugin.scm.perforce_8.0.0.jar'...
2018-07-30 10:14:26,115 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Updating plugins/com.pmease.quickbuild.libs...
2018-07-30 10:14:26,115 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Updating plugins/site...
2018-07-30 10:14:26,131 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Updating asset...
2018-07-30 10:14:26,368 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Downloading file 'release'...
2018-07-30 10:14:26,368 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.Bootstrap - Booting QuickBuild from 'C:\QBAgent'...
2018-07-30 10:14:26,383 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.Bootstrap - Starting framework...
2018-07-30 10:14:34,761 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.equinoxadapter.EquinoxAdapter - Caching plugin libraries...
2018-07-30 10:14:43,393 [WrapperStartStopAppMain] WARN  org.eclipse.jetty.util.component.AbstractLifeCycle - FAILED ServerConnector@70cc8492{SSL-HTTP/1.1}{0.0.0.0:443}: java.net.BindException: Address already in use: bind
    java.net.BindException: Address already in use: bind
        at sun.nio.ch.Net.bind0(Native Method)
        at sun.nio.ch.Net.bind(Unknown Source)
        at sun.nio.ch.Net.bind(Unknown Source)
        at sun.nio.ch.ServerSocketChannelImpl.bind(Unknown Source)
        at sun.nio.ch.ServerSocketAdaptor.bind(Unknown Source)
        at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:321)
        at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80)
        at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:236)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
        at org.eclipse.jetty.server.Server.doStart(Server.java:366)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
        at com.pmease.quickbuild.Quickbuild.startJettyServer(Quickbuild.java:388)
        at com.pmease.quickbuild.Quickbuild.start(Quickbuild.java:249)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at com.pmease.quickbuild.equinoxadapter.EquinoxAdapter.startup(EquinoxAdapter.java:59)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at com.pmease.quickbuild.bootstrap.Bootstrap.boot(Bootstrap.java:576)
        at com.pmease.quickbuild.bootstrap.Bootstrap.main(Bootstrap.java:126)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at org.tanukisoftware.wrapper.WrapperStartStopApp.run(WrapperStartStopApp.java:400)
        at java.lang.Thread.run(Unknown Source)
2018-07-30 10:14:43,425 [WrapperStartStopAppMain] WARN  org.eclipse.jetty.util.component.AbstractLifeCycle - FAILED org.eclipse.jetty.server.Server@b137f7d: java.net.BindException: Address already in use: bind
    java.net.BindException: Address already in use: bind
        at sun.nio.ch.Net.bind0(Native Method)
        at sun.nio.ch.Net.bind(Unknown Source)
        at sun.nio.ch.Net.bind(Unknown Source)
        at sun.nio.ch.ServerSocketChannelImpl.bind(Unknown Source)
        at sun.nio.ch.ServerSocketAdaptor.bind(Unknown Source)
        at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:321)
        at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80)
        at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:236)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
        at org.eclipse.jetty.server.Server.doStart(Server.java:366)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
        at com.pmease.quickbuild.Quickbuild.startJettyServer(Quickbuild.java:388)
        at com.pmease.quickbuild.Quickbuild.start(Quickbuild.java:249)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at com.pmease.quickbuild.equinoxadapter.EquinoxAdapter.startup(EquinoxAdapter.java:59)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at com.pmease.quickbuild.bootstrap.Bootstrap.boot(Bootstrap.java:576)
        at com.pmease.quickbuild.bootstrap.Bootstrap.main(Bootstrap.java:126)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at org.tanukisoftware.wrapper.WrapperStartStopApp.run(WrapperStartStopApp.java:400)
        at java.lang.Thread.run(Unknown Source)
2018-07-30 10:14:44,459 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.Quickbuild - Stopping QuickBuild agent...
2018-07-30 10:14:44,709 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.Quickbuild - Disconnecting from server...
2018-07-30 10:14:44,725 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.Quickbuild - Waiting for finish of existing jobs...
2018-07-30 10:14:44,725 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.Quickbuild - Stopping plugins...
2018-07-30 10:14:44,741 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.Quickbuild - Stopping task scheduler...
2018-07-30 10:14:44,788 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.Quickbuild - QuickBuild agent stopped.java.net.BindException: Address already in use: bind
  • solved #5
  • replies 6
  • views 4193
  • stars 1
#2
robinshen ADMIN ·

Sorry for the inconvenience. What is your JDK version running the agents?

1 Reply
1 #3
deefrazier ·

Hey thanks for the quick reply. We are running the following version:

java.runtime.version 	1.8.0_40-b26
java.specification.name 	Java Platform API Specification
java.specification.vendor 	Oracle Corporation
java.specification.version 	1.8
#4
robinshen ADMIN ·

Tried the same setup and agents are upgraded fine. Are you able to set up a sample server to reproduce this issue at your side?

1 Reply
#5
jclx ·
1 Reply
#6
deefrazier ·

This sounds like the issue because nothing was listed in nestat as using the port when i ran the command.

#7
deefrazier ·

Thanks for looking into this. We only saw this issue when we upgraded the test grid and not when we did the development grid (we go dev>test>prod). Checking our schedule, there were windows updates between when we upgraded dev to when we upgraded test. It almost certainly sounds like the issue we are having is the one described by@jclx

jclx ยท 5 hours ago
Maybe related, recent Microsoft updates to Windows are causing a similar issue to this. Can't bind to port yet nothing is listening in netstat.

Log in to reply