Your browser was unable to load all of the resources. They may have been blocked by your firewall, proxy or browser configuration.
Press Ctrl+F5 or Ctrl+Shift+R to have your browser try again.

Unique global storage for a build #1211

puneetg ·
Hi Robin

We are looking into feasibility of adding Quickbuild remote nodes. When we publish artifacts on remote nodes, the data is transferred to global storage and that expensive in terms of circuit usage and potentially slow. Is it possible to define multiple global storage location inside QB? This way remote sites can publish artifacts inside storage located in there own network.

thanks
-Puneet
  • replies 8
  • views 3663
  • stars 0
robinshen ADMIN ·
Hi Puneet,

You may override the storage directory at advanced settings tab for particular configurations to point to a shared network place seen by QuickBuild server. Then all published artifacts for that configuration will be put into that place.

Regards
Robin
puneetg ·
Thanks Robin, we will give this a try.

Do we also have setup destination folder inside "Publish artifacts" step, or that is derived from configuration storage ?

As configuration storage will be dynamic based on user triggering the build, we will also write a script to return location based on user ipaddress.

thanks
-Puneet
robinshen ADMIN ·
The publish artifacts step does not need to be changed. However, it is not possible to use different storage directory for different users since storage directory is shared by all builds for a particular configuration. Once it is changed during the build to point to a user specific directory, other builds will not be accessible.
To work around the problem, you may consider to write custom plugin with below features:
1. write a custom artifact publish step to copy artifacts to user network based on the info who triggered the build. Location of the build artifacts can be written as a flag file into the normal storage area of QuickBuild.
2. write a custom build tab to read location info from the flag file, and then access artifacts from that location to render the panel.
puneetg ·
Thanks Robin I will look into custom plugins based on existing code.

This looks like a generic feature request for QB as you are supporting DCVS as well. And that in turn has robust support for multi site development.
Addtionally In our case the deployment is accross two data centers at one site due to space constraints/redundancy and such feature can reduce network traffic.

I will add this to jira tracker.

thanks
-Puneet
robinshen ADMIN ·
The difficult part to make this generic is that to avoid round travel when downloading files, file links in the artifacts panel need to be rendered as a direct link to the artifacts located at user network, and the links might be different across projects. For example, you might want to set up a ftp server in user network to serve these artifacts, and these will be ftp links.
Another approach for multi-site is to set up another QB server at remote side, and connect these sites with QuickbuildRepository so that one site can access artifacts of another site.
puneetg ·
Robin

I understand roundtrip travel to serve artifacts, that will not work for multi site development.

You idea with ftp link will work, but for that we need to develop a ftp_artifact_site step that uploads and serves files from relevant ftp server. I will look into this. I hope QB publish artifacts code can be starting point to build this.

I did not understand the second part of connecting two QuickBuild servers ( local and remote ) via QuickbuildRepository. Can you elaborate regarding this setup?

Thanks
-Puneet
robinshen ADMIN ·
The second approach means that the remote site sets up and maintains their own QuickBuild server. If certain projects on remote server depends on some artifacts on local server, a repository of type "QuickBuild" can be set up on the remote server to retrieve desired artifacts from local server. The same mechanism can be used if local server depends on artifacts of remote server.
puneetg ·
Ok understood, I will look into this but our remote sites have circuit limitations. We need is site based artifact servers
Thanks
-Puneet