Your browser was unable to load all of the resources. They may have been blocked by your firewall, proxy or browser configuration.
Press Ctrl+F5 or Ctrl+Shift+R to have your browser try again.

QuickBuild

Servlethanlder EOFException on /rest/configurations #2635

#1
vanderhu ·
Currently seeing on our master log: Servlethanlder EOFException on /rest/configurations
After several minutes the server becomes unrespsonsive and is reporting out of heap space problem. Is this caused due to fact that someone is traversing whole tree of configurations by RestAPI?

Full error:
2013-12-20 07:54:27,991 [qtp1374847011-1157195] WARN org.eclipse.jetty.servlet.ServletHandler - /rest/configurations
java.lang.RuntimeException: org.eclipse.jetty.io.EofException
at com.pmease.quickbuild.bootstrap.BootstrapUtils.wrapAsUnchecked(BootstrapUtils.java:56)
at com.pmease.quickbuild.util.ExceptionUtils.wrapAsUnchecked(ExceptionUtils.java:82)
at com.pmease.quickbuild.rest.providers.XmlProvider.writeTo(XmlProvider.java:92)
at com.sun.jersey.spi.container.ContainerResponse.write(ContainerResponse.java:289)
at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1029)
at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:941)
at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:932)
at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:384)
at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:451)
at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:632)
at com.pmease.quickbuild.rest.RestServlet.service(RestServlet.java:45)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:847)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:669)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1336)
at com.pmease.quickbuild.Quickbuild$DisableTraceFilter.doFilter(Quickbuild.java:1017)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1307)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:453)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1072)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:382)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1006)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at org.eclipse.jetty.server.Server.handle(Server.java:365)
at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:485)
at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:926)
at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:988)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:635)
at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Unknown Source)
Caused by: org.eclipse.jetty.io.EofException
at org.eclipse.jetty.http.HttpGenerator.flushBuffer(HttpGenerator.java:914)
at org.eclipse.jetty.http.AbstractGenerator.blockForOutput(AbstractGenerator.java:507)
at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:170)
at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:107)
at com.sun.jersey.spi.container.servlet.WebComponent$Writer.write(WebComponent.java:276)
at com.sun.jersey.spi.container.ContainerResponse$CommittingOutputStream.write(ContainerResponse.java:130)
at sun.nio.cs.StreamEncoder.writeBytes(Unknown Source)
at sun.nio.cs.StreamEncoder.implWrite(Unknown Source)
at sun.nio.cs.StreamEncoder.write(Unknown Source)
at java.io.OutputStreamWriter.write(Unknown Source)
at java.io.BufferedWriter.flushBuffer(Unknown Source)
at java.io.BufferedWriter.write(Unknown Source)
at java.io.Writer.write(Unknown Source)
at org.dom4j.io.XMLWriter.indent(XMLWriter.java:1528)
at org.dom4j.io.XMLWriter.writeElement(XMLWriter.java:904)
at org.dom4j.io.XMLWriter.writeNode(XMLWriter.java:1298)
at org.dom4j.io.XMLWriter.writeElementContent(XMLWriter.java:1136)
at org.dom4j.io.XMLWriter.writeElement(XMLWriter.java:954)
at org.dom4j.io.XMLWriter.writeNode(XMLWriter.java:1298)
at org.dom4j.io.XMLWriter.writeElementContent(XMLWriter.java:1136)
at org.dom4j.io.XMLWriter.writeElement(XMLWriter.java:954)
at org.dom4j.io.XMLWriter.writeNode(XMLWriter.java:1298)
at org.dom4j.io.XMLWriter.write(XMLWriter.java:344)
at com.pmease.quickbuild.rest.providers.XmlProvider.writeTo(XmlProvider.java:90)
... 34 more
Caused by: java.net.SocketException: Broken pipe
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(Unknown Source)
at java.net.SocketOutputStream.write(Unknown Source)
at org.eclipse.jetty.io.ByteArrayBuffer.writeTo(ByteArrayBuffer.java:359)
at org.eclipse.jetty.io.bio.StreamEndPoint.flush(StreamEndPoint.java:164)
at org.eclipse.jetty.io.bio.StreamEndPoint.flush(StreamEndPoint.java:194)
at org.eclipse.jetty.http.HttpGenerator.flushBuffer(HttpGenerator.java:841)
... 57 more
  • replies 8
  • views 4420
  • stars 0
#2
robinshen ADMIN ·
Yes that is true. Please avoid traversing the whole configuration tree if there are too many of them.
#3
vanderhu ·
Is this something which can be restricted on user level? Because in large company environment it is hard to control all scripts which are accessing rest API.

It is namely a risk if production CI environment is getting down due to this.
#4
robinshen ADMIN ·
I guess you'd like to only allow certain users from accessing RESTful API? If so, please file an improvement request at track.pmease.com.
#5
vanderhu ·
Yes indeed that is what I woud like to do, see:
http://track.pmease.com/browse/QB-1895

Also I think it is worth to protect the system on certain level against accessing large amount by returning just error code, instead of accepting it and let java run out of heap.
#6
vanderhu ·
Is there a way to switch off completely restful api access to QuickBuild on global level (ie by editing conf file or by Administration in UI)?
#7
robinshen ADMIN ·
Currently no unfortunately. We will get this implemented in next patch release.
#8
vanderhu ·
When do you expect this patch release?

Also good to consider is avoiding heap space problem in general by using different approach then loading everything in memory, for example breaking it down in smaller chunks instead of putting everything in array.
#9
robinshen ADMIN ·
Please upgrade to 5.1.6, and for groups you'd like not to allow restful access, tick out the option "allow restful access". As to not to load everything into memory, this is what QB already does. However to be flexible, QB stores many configuration settings as XML in database, and loading it to frequently (especically via program) cause QB converting a lot of XMLs back/forth, and this might cause GC very busy. Of course we can look into this problem further if you can send our your database backup as well as a demo script being able to reproduce the issue.
Log in to reply