The Eclipse Common Build Infrastructure (CBI) is an initiative combining infrastructure, services, technologies and best practices for building, testing and delivering software at the Eclipse Foundation.
Goals
Primary goals of the CBI initiative are:
- Make it easy for anyone to contribute to Eclipse projects.
- Make it easy for projects to follow open governance and transparency best practices.
- Provide a set of industry-grade services (or integration with some widely adopted 3rd party services) for building and distributing Eclipse projects.
Asking for Help
- Need help actually building your code: ask your project mentors, or ask on the Common Build mailing list (cbi-dev). There are no dumb questions.
- Subscribe to cbi-dev here: https://dev.eclipse.org/mailman/listinfo/cbi-dev
Service Level Objectives (SLO)
Most CBI services are Tier 2 - Best Effort, which means that we will make a reasonable effort to restore service outside of support hours. Eclipse Strategic Members can contact Webmaster in certain cases of off-hours support.
Please see IT_SLA for more information on the Eclipse Foundation IT Services SLA.
Security
The Eclipse Foundation has authored an Open Source Software Supply Chain Best Practices document. We highly recommend Eclipse OSS projects read, understand and adopt these best practices as part of their role in the Supply Chain.
https://github.com/eclipse-cbi/best-practices/blob/main/software-supply-chain/osssc-best-practices.md
Provided Services
Jenkins CI Environment
We provide dedicated Jenkins instances to projects. See also Jenkins.
Jenkins is a continuous integration (CI) server used on Eclipse Foundation servers for Eclipse projects as part of the Common Build Infrastructure (CBI). Jenkins instances are maintained by the Eclipse Webmasters/Release Engineers.
List of Jenkins Instances Per Project (JIPP):
- https://ci.eclipse.org/
Requesting a JIPP instance
Please file a HelpDesk ticket to request your project's own instance. Please ensure your project lead can +1 the request.
What's provided?
Each Eclipse Project has access to one Jenkins instance (JIPP), including the following:
- (1) Jenkins instance, with (1) Resources Pack (see below)
- Membership-sponsored projects may allocate more resources (see below)
- Digital signing Service: Java JAR, Java Cryptography Extensions, Windows Portable Executable with Microsoft Authenticode, macOS application bundles.
- Packaging service: Apple Disk Image (.dmg), Linux Flatpak
- Disk space: Ephemeral for builds, permanent for release builds.
- (1) 1vCPU/3.75G/30G Linux Virtual Server (if needed) (courtesy of Microsoft Azure). Projects sponsored by Strategic Members can engage with the Foundation to get out of spec Virtual Server.
- Access to worldwide download mirrors
[!NOTE] Windows and macOS agents \ Note that Windows and macOS agents (shared and headless - not suitable to run UI tests) can be added to your Jenkins instance. In this case, each shared headless agent counts for 1 resource pack. These agents have common dependencies like JDK, Maven, etc installed. If your build requires specific/special dependencies, you might need to set up your own external build agent instead.
[!NOTE] RISC-V agents \ Projects that want to build and test on RISC-V architecture can ask for access to one of our self-hosted RISC-V machines. Access is limited. You can find more info here: https://github.com/eclipse-cbi/jiro/wiki/Dedicated-build-agents
Additional Resources
Each Eclipse Project has access to one Resource Pack for building. For some projects, that may not be enough. Projects sponsored by Eclipse Membership (via Project Lead) have additional resources (like resource packs, dedicated agents or GitHub hosted large runners), based on membership level. These additional resources can be allocated to projects. See the Jenkins wiki page to see how packs translate to Jenkins builds.
- Some resources are only available to Enterprise and Strategic members.
- Enterprise and Strategic members can engage with the Foundation to acquire additional Packs.
Resource Pack
Agent type | Linux (x64) (containerized, no root) |
---|---|
vCPU | 2 (burst to 4) |
RAM | 8 GiB |
Disk | 50 GB |
Dedicated Agent
Agent type | Linux (x64) / Windows (x64) / macOS (Apple silicon) (VMs) |
---|---|
vCPU | 4 |
RAM | 16 GB |
Disk | 100 GB |
GitHub hosted large runner
Agent type | Linux (x64) |
---|---|
vCPU | 8 |
RAM | 32 GB |
Disk | 300GB |
Minutes | 2000* |
*GitHub action minutes for large GitHub runners are not covered by free minutes for public projects
Gitlab Runners
We provide dedicated GitLab runners to projects.
Request a runner
Runner can be requested by filling a ticket on the helpdesk. Please ensure your project lead approved with a +1 the request.
First integration
Create your first .gitlab-ci.yml
file in your project.
default:
tags:
- origin:eclipse # allow to target eclipse runner
stages:
- build
my_build_job:
stage: build
script:
- echo "example job 1"
You can find many templates here
Provided Services
- Pipeline template: Basic features
- Pipeline template: Auto Devops
- Pipeline template: Full feature
- Dockerhub publication
- Nexus: repo.eclipse.org
- Signing tool
- Supply Chain Security Best Practices
- Secret management
- Build container image (buildkit)
- Publish to projects-storage (download.eclipse.org)
Request and Allocation Process for Runners (Resource pack)
Allocating Resource Packs
Each Eclipse Project has access to one Resources Pack for building by default.
For some projects, that may not be enough. Projects sponsored by Eclipse Membership (via Project Lead) have additional Packs, based on membership level. These Packs can be allocated to projects.
- Some resources are only available to Enterprise and Strategic members.
- Enterprise and Strategic members can engage with the Foundation to acquire additional Packs.
Resource pack configuration
Considering the microservice aspect in the execution of GitLab CI pipelines, where the aim is to have dedicated jobs for different kind of tasks and thus to have a large number of jobs running in parallel to execute a pipeline.
As a consequence, three types of build containers are proposed with the following specifications:
Small | Medium | Large | |
---|---|---|---|
cpu req | 250m | 1000m | 2000m |
cpu limit | 500m | 2000m | 4000m |
mem | 1024Mi | 4096Mi | 8192Mi |
The distribution of concurrency is based on the resource pack specifications as follows:
# Resource packs | 1 | 2 | 3 | 4 | 5 | 10 |
---|---|---|---|---|---|---|
Concurrent Small | 3 | 5 | 7 | 9 | 11 | 21 |
Concurrent Medium | 1 | 2 | 3 | 4 | 5 | 10 |
Concurrent Large | 0 | 0 | 1 | 1 | 2 | 5 |
max concurrency | 4 | 7 | 11 | 14 | 18 | 36 |
NOTE: * 2 Small per resource pack starting from 3 * 1 Medium per resource pack * 1 Large every 2 resources pack
Dedicated Agent
Β Agent type | Linux/Windows/macOS (VMs) |
---|---|
vCPU | 4 |
Β RAM | Β 8GiB |
Β Disk | Β 100GB |
Resource Packs Included in Membership
Associate / Contributing [β¬0, β¬15k] |
Associate / Contributing [β¬15k, β¬20k] |
Associate / Contributing [β¬25k, β¬50k] |
Strategic [β¬50k, β¬100k]Β |
Strategic [β¬100k, β¬500k] |
|
---|---|---|---|---|---|
Resource packs | Β 1 | Β 2 | Β 3 | Β 5 | Β 10 |
Dedicated Agents | Β 0 | Β 0 | Β 0 | Β 0 | Β 2 |
Assigning Resource Packs to a Project
Resource Packs are assigned by Member organizations of the Eclipse Foundation to Eclipse Projects they sponsor. Packs are assigned as a whole to a single project (i.e., canβt split Packs across multiple projects). A member can assign several packs to a single project.
Important: When asking for packs for your project, please ensure that project leads and your organization representatives are copied to the GitLab ticket. We require approval from project leads but assume immediate approval from organization representatives. We strongly advise you seek authorization internally to your organization before opening such a request, though. Should conflictual requests arise, the organization representatives will be asked to actively arbitrate.
To assign a pack to a project, please file a ticket
By default, resource packs are assigned build agents. In some cases, it may be required to scale up the Jenkins master. In such case, we can allocate resource packs to the master instance. Sponsored Projects
A public API of sponsored projects is accessible. Organizations can check how many Resource Packs they have left for project sponsoring on the membership portal.
Membership Level and additional resources
Additional Resources Included in Membership
Eclipse Foundation Member Organizations have access to additional resources based on their membership level:
Associate / Contributing | Strategic | ||||
---|---|---|---|---|---|
[β¬0, β¬15k) | [β¬15k, β¬20k] | [β¬25k, β¬50k) | [β¬50k, β¬100k) | [β¬100k, β¬500k] | |
Resource packs | 1 | 2 | 3 | 5 | 10 |
Dedicated Agents | 0 | 0 | 0 | 0 | 2 |
GitHub hosted large runners | 0 | 0 | 0 | 0 | 2 |
Assigning Additional Resources to a Project
Additional resources (resource packs/dedicated agents/GitHub hosted large runners) are assigned by Member organizations of the Eclipse Foundation to Eclipse Projects they sponsor. Those resources are assigned as a whole to a single project (i.e., we canβt split resources across multiple projects). A member can assign several resources to a single project.
[!IMPORTANT] When asking for additional resources for your project, please ensure that project leads and your organization representatives are copied to the GitLab ticket. We require approval from project leads but assume immediate approval from organization representatives. We strongly advise you to seek authorization internally to your organization before opening such a request though. Should conflictual requests arise, the organization representatives will be asked to actively arbitrate.
To assign additional resources to a project, please file a ticket.
By default, resource packs are assigned build agents. In some cases, it may be required to scale up the Jenkins master. In such case, we can allocate resource packs to the master instance.
Sponsored Projects
We maintain a public API of sponsored projects. Organizations can check how many resource packs/dedicated agents they have left for project sponsoring on the membership portal.
CI Environment (Third-party)
If you host your Eclipse project Git repository at GitHub, we also support some third-party services:
- CircleCI.
- Azure Pipelines.
- ~~TravisCI~~. We do no longer support Travis CI, see the announcement here: https://www.eclipse.org/lists/cross-project-issues-dev/msg18647.html
If an environment is not listed in the unsupported list, you can ask whether it can be supported by opening a bug.
[!NOTE] Third party build services (e.g., CircleCI) limitations \ The Eclipse Foundation's IT Team is responsible for the credentials (OSSRH, SSH and GPG keys...) it creates for projects. Monitoring where credentials are disseminated is crucial for security reasons. As long as we have no automation in place to set/update/remove credentials on 3rd party build services, we don't put any credential on such services. For instance, it means that you can't use them to publish to Maven Central, or to upload build results to download.eclipse.org.
GitHub and Bot Accounts
The Eclipse Foundation can create a bot user that can be used from your Jenkins (JIPP) instance to build branches and pull requests, push, tag, and comment on the repo.
We don't share the credentials of those bots for external usage (e.g., to use it with a private, behind firewall Jenkins instance, or with third-party services). Those bots have the same set of permissions as a committer and it would be against the Eclipse Development Process to give those permissions to anybody. Likewise, we don't grant any elevated permissions on repositories or organizations to any other bot account.
Webhooks
However, we can set up webhooks for you. We create webhooks that can only receive the following events:
- Branch or tag creation (Branch or tag created)
- Branch or tag deletion (Branch or tag deleted)
- Pull requests (Pull request opened, closed, reopened, edited, assigned, unassigned, review requested, review request removed, labeled, unlabeled, synchronized, ready for review, locked, or unlocked)
- Pushes (Git push to a repository)
See the GitHub developer documentation for more information.
Personal Access Token
We can also create personal access token for the bot account. We create tokens with only the following scopes:
- repo:status
- repo_deployment
- write:discussion
See the GitHub developer documentation about scopes and OAuth for more information.
Getting admin permissions on project's GitHub repositories
Project leads may require admin level access to their project's GitHub repositories. If that is the case, please open a new bug with your request. To be eligible to such permission level, your project must be in the mature phase and the request shall be highly motivated, so please tell us why you need that access (this information will be handy when sorting out the more general solution). Usually, admin permissions will be assigned temporarily.
Docker Hub
The Eclipse Foundation owns the eclipse organization and a couple of other project-specific organizations at https://hub.docker.com. You can ask to get a repository being created on one of these organizations. We will set permissions so that committers have write access to this repo (you will need to share your Docker Hub ID with us).
You can also ask us to create a project-specific organization. The
organization name needs to follow the pattern eclipse<projectname>
(Docker Hub does not support dashes -
in user or org names).
Note that we don't grant admin permissions on any Eclipse Foundation owned organization. We recognize that this means that you will have to go through us for all new repo creation, but we can't grant organizations-wide admin permission assigned to committers for security reasons.
In all cases, requests should be directed to the GitLab HelpDesk.
[!IMPORTANT] DockerHub policy changes (March 2021) \ DockerHub is currently changing its policy regarding the number of free members inside an organization. As such, the rules above will probably need to evolve in the near future. See https://www.eclipse.org/lists/eclipse.org-committers/msg01273.html and https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/issues/473 for details.
Nexus/Maven repository
See Services/Nexus.
Signing tool
Eclipse Platform / plugins specific tooling
CBI license bundle
We offer a P2 repository containing the org.eclipse.license
bundle
which is located at:
- http://download.eclipse.org/cbi/updates/license/
This URL is a composite P2 repo containing the license bundle.
If you use Tycho you can add the p2 repo to the
<repository>
<id>license-feature</id>
<url>http://download.eclipse.org/cbi/updates/license/</url>
<layout>p2</layout>
</repository>
In any particular feature which you need the license you can use the usual feature.xml section:
<?xml version="1.0" encoding="UTF-8"?>
<feature
id="org.eclipse.help"
label="%featureName"
version="2.0.0.qualifier"
provider-name="%providerName"
plugin="org.eclipse.help.base"
license-feature="org.eclipse.license"
license-feature-version="1.0.0.qualifier"/>
[...]
p2 repo checks
A set of "tests" which create reports or can be run as unit tests that check to correctness of p2 repositories. That is partially just "correctness" in general (such as that jars are signed, etc.) but more so that repositories conform to the requirements of the Eclipse Simultaneous release (such as that jars have correct "Provider names", licenses, etc.). For more information, see https://github.com/eclipse-cbi/p2repo-analyzers/blob/main/README.md.
p2 repo aggregator
A tool to combine several p2 repositories. Among other things, it makes sure, they all have consistent constraints (that is, can be "installed together") unlike a raw p2 mirror task. For more information, see CBI/aggregator.