This post has been motivated by a problem that I recently solved. I had to develop scripts to test a particular scenario. When there was a specific condition, I would print the occurrence of the same in the test build log. Of all the builds, on an average the occurrence rate was 10%. In our TeamCity, we have multiple build agents executing runs queued up and on an average, we have hundreds of build runs in day. So, to prove the occurrence rate of the issue using valid data points, I needed to find all the builds which have the specific string in the build log, and provide the component developers the web console link for the failure.
In my case (which is usually the case with most enterprise projects), there is a main TeamCity project which has multiple sub-projects, which in turn have multiple sub-projects. They follow a naming scheme in the form or <PROJECT-ID>_<SUB-PROJECT-1>_<SUB-PROJECT-2> … and so on. To navigate through all of these to find a specific string is a gargantuan task, and doing it manually would not only have taken me days’ of work (and frustration!), I would have also ended up disliking my job. Plus, who knows, I may have required to do it again in the future when I am logging specific conditions in the build log.
So, I thought automating this is the way to go! I started looking at some already implemented automation modules like PyTeamCity, but most of them did not seem to provide the functionality I was looking for, especially with reading the build log contents. TeamCity REST API documentation does not specifically offer a way to download build log, but you can do so by executing a GET request as described here.
After some trials, I was able to come up with the following flow to implement my automation script.
- Get a list of all build configurations using the REST API command app/rest/buildTypes/. This will return an XML output.
- From the XML output, find all the buildType (build configs) that contain a project specific string in the name, and extract the ‘id’ information of that build configuration.
- Get list of all the finished builds for that build configuration id using the REST API command: app/rest/builds/?locator=state:finished,start:0,count:100000,buildType:<BUILD-CONFIG-ID>. This will give a XML output containing all the builds for that build configuration.
- From the XML output, extract the build id information, and get the raw build log data using the URL http://<TEAMCITY-SERVER>:<TEAMCITY-PORT>/downloadBuildLog.html?buildId=<BUILD-ID>&plain=true.
- You can check if the string exists in the build log, and if it does, log the build web console link as: http://<TEAMCITY-SERVER>:<TEAMCITY-PORT>/viewLog.html?buildTypeId=<BUILD-CONFIG-ID>&buildId=<BUILD-ID>
These links and the log from the script can be provided to the the product team so they can review the entire build information.
Good news, I have uploaded my code on GitHub so you can use it. You can download a copy and customize or consume it the way you want to. :-) :-)