This is a guide for Acumos software developers and testers about the continuous integration/continuous deployment (CI/CD) resources and processes supported by the Linux Foundation.
These resources require a Linux Foundation identity.
|https://identity.linuxfoundation.org||The Linux Foundation identity management web site.|
|https://jira.linuxfoundation.org/servicedesk/customer/portals||The Linux Foundation release engineering team's service desk.|
|https://gerrit.acumos.org||The Gerrit code review server hosts the Git repositories and supports the review, merge and release processes. The review process basically requires all users to follow something like a conventional git pull-request process, but restricts the publication (push) of private branches.|
|https://jenkins.acumos.org||Jenkins is the continuous-integration server aka the build server. All users can see the contents of the Jenkins; no users can modify the configuration nor start a job in this Jenkins.|
|https://jenkins.acumos.org/sandbox||The Jenkins sandbox is a place for testing Jenkins job configurations. After requesting access, users can create jobs, reconfigure jobs, trigger builds etc. After JJB templates have been tested in the sandbox they can be submitted for the main Jenkins server.|
|https://jira.acumos.org||Jira is the web site that tracks issues, epics, stories, etc.|
This Nexus 2 repository holds Maven (i.e., jar) artifacts produced by builds. Snapshot and staging builds are all deployed to this repository. Release artifacts are created by promoting artifacts manually from the staging repository after suitable approval. Publicly accessible to users without credentials. All users should be able to access and browse artifacts through this URL.
This Nexus 3 server publishes external and internal Docker images. Supports the following registries:
These registries are open for anonymous read access. The Jenkins server has credentials to push images to the snapshot and staging registries, and to promote images to the release registry. Manual push of images is not supported.
|https://sonar.acumos.org||Sonar analyzes Java code for quality issues and code coverage achieved by unit tests (JUnit).|
This section guides the developer in submitting code, reviewing code, tracking the status of builds and requesting creation of release versions. These recommended practices come from the Linux Foundation.
All Java project POM files should always use the suffix "-SNAPSHOT" in their project version strings, that's the "V" in the GAV triple defined at the top, at all times. This is slightly counter intuitive. The Jenkins jobs expect this and manage the version strings appropriately.
In contrast, Java project POM files should use release versions of dependencies (not SNAPSHOT versions). It may be necessary to use SNAPSHOT versions of dependencies for development, but that must be regarded as temporary. All releases given to the test team and published for use by the world MUST use only release versions.
A peculiarity of the LF build environment is that every Jenkins job that stages a release candidate strips all "-SNAPSHOT" suffixes within POM files. This affects the project's own version and all named dependencies.
Per the LF, Java project POM files should have neither a "repositories" nor a "pluginRepositories" section. These sections can be used to store URLs of the servers that supply dependencies - jar and plugin files - other than the well-known Maven Central (central.maven.org) server. Those sections are provided on the LF Jenkins build server by a configured "settings.xml" file. Individual developers who build the software on their own machines must create a settings file at path ~/.m2/settings.xml (i.e., in their home directory where maven looks by default) with this information.
When working with a git repository cloned from Gerrit you can create as many local branches as you like. Note that none of the branches will be pushed to the remote Gerrit server, the branches remain forever private. Creating branches for each task will allow you to work on multiple independent tasks in parallel, let you recover gracefully from various situations, and generally save aggravation and time. Also see these instructions on tagging and branching for releases:
Acumos Tagging and Branching Steps Process
The Linux Foundation strongly recommends (and eventually will enforce) Gerrit commit message content. A git commit message must meet these requirements:
An example is shown below:
Null check for ClientResponse NullPointerException might be thrown as cres is nullable here Issue-ID: ACUMOS-999 Change-Id: I14dc792fb67198ebcbabfe80d90c48389af6cc91 Signed-off-by: First Last <firstname.lastname@example.org>
Acumos uses Gerrit to automate the process of reviewing all code changes before they are committed to the Git repository. For a tutorial on git you might start here:
Here is a tutorial and reference on using Gerrit:
Also see this guide published by the LF Release Engineering team about using Gerrit:
Gerrit is designed to host text files – source code. It enforces a size threshold on every commit, the default limit is 5MB. Further the Linux Foundation prohibits committing binary files such as compiled executables, jar files, zip archives and so on. An exception is made for binary picture (image) files in GIF, JPG and PNG formats, but the size limit must still be honored.
The quickstart guides below describe the command-line procedures for performing common tasks on Gerrit. The command-line tool "git review" is the most reliable and can be used on any platform. Windows users should install "Git Bash" to gain support for command-line git.
git checkout -b my-local-branch
git checkout my-local-branch
# Look up the change number shown top-left in Gerrit web, here "999".
git checkout master
Submitted reviews with changes should appear in the Gerrit web interface. The Jenkins job builder will publish test results, giving "Verified +1" if the test succeeds and -1 if the test fails. A contributor or committer can review it with a -1/0/+1. A committer then must approve it with a +2 rating and merge the code to the master branch. All actions are done on the gerrit.acumos.org web site.
The committer may take any of several actions, such as clicking on the "Reply" button, adding reviewers, adding a review comment, and moving the flags to +2 and +1
Once a committer/contributor approves it, the code can be merged to the master branch.
The CLM jobs are still scheduled to run every Saturday, this feature can be useful for debugging on demand. Commenting "run-clm" in a gerrit that is not merged, will not trigger the CLM job based on that revision but will trigger the job based on the tip of the branch. This job is designed to always run on the latest tip of the branch to avoid inconsistencies on the reports.
Once testing of the snapshot version has been completed, prepare a release candidate in the staging area. First ensure that your POM file has only release versions in the dependency sections. Then trigger the appropriate Jenkins job to stage a release candidate as noted above. The job uses magic (ok a shell script) to strip the string "-SNAPSHOT" from every line of all of your project's POM files. This means any lingering dependency version string with -SNAPSHOT will also be modified, not just the project version string. If the modified build succeeds, the build outputs are deployed or pushed to a staging repository.
Jar files in the staging repository are not used by the LF Jenkins server - there is no Maven profile for this. My advice is to treat the staging repository as a brief resting stop on the way to full release status.
If you force a local build job to pull artifacts from a staging repository by fiddling your local settings, be warned that the staging server will give you the OLDEST artifact in the staging repository, not the newest. This is a very real practical limitation on the usefulness of the staging repository.
Once testing against the staging repo version has been completed (see above ) and the project has determined that the artifact in the staged repository is ready for release, the project team can use the new-for-2019 self-release process as follows, once the required jobs are added to the Jenkins server via appropriate entries in the JJB templates. Also see documentation at https://docs.releng.linuxfoundation.org/projects/global-jjb/en/latest/jjb/lf-release-jobs.html
Example release yaml file content:
--- distribution_type: 'maven' version: '1.0.0' project: 'example-project' log_dir: 'example-project-maven-stage-master/17/'
Once complete, the artifacts should appear in the Nexus2 release repository.
Once testing against the staging repo version has been completed (see above ) and the project has determined that the artifact in the staged registry is ready for release, a release can then be performed as follows:
Self service release of docker containers using gerrit and jenkins – you must setup jenkins – more info here: Self Service Docker Release-adoption
distribution_type: container container_release_tag: 0.26.2 container_pull_registry: nexus3.acumos.org:10004 container_push_registry: nexus3.acumos.org:10002 project: license-usage-manager ref: d1b9cd2dd345fbeec0d3e2162e008358b8b663b2 containers: - name: lum-db version: 0.26.2 - name: lum-server version: 0.26.2
Example service request content:
Once complete, the artifacts should appear in the Nexus3 release registry.
Finally, add a tag for the release on the Gerrit web site.
This addresses the need for a project to reproduce from git the EXACT, byte-for-byte state of the project's files as built, which is not a requirement for all projects. The Jenkins build server's stage-release job modifies POM files to strip --SNAPSHOT suffixes. The Jenkins server stores in the local git repository the state of the project after that modification. That state is published via a git bundle file that is pushed to the log server, it is NOT pushed directly back to Gerrit, because Jenkins generally cannot push changes back to Gerrit. To add that item into project history, that commit can be downloaded, merged to the local repository and submitted via a Gerrit review into the global history. Steps: