Project needs

Please feel free to add columns if you see any other information that might impact the CI/CD needs of a project.

ProjectTarget Operating SystemsLanguages, Build ToolVCS HostDeveloper Team SizeConcurrencyBuild TasksCurrent solution / usage
OperatorFabricLinux

Java, Angular,

Gradle

GitHub.com

4 full-time developers, 2 occasional contributors (all in CET)

We take advantage of the night to run a CRON job handling the publication of a SNAPSHOT version (docker and documentation) and a bot doing dependencies updates.


Currently we can only run one job at a time but more concurrency would allow us to run our API tests (Karate) and end-to-end (Cypress) tests in parallel.
  • Compile back (Java) and front (Angular)
  • Run unit tests
  • Generate docker images for all services
  • Launch an instance of the app and run:
    • API tests (Karate)
    • end-to-end tests (Cypress)
  • Generate html documentation from Asciidoc files and push it to our opfab.github.io website.
  • Push docker images for all services (5) to DockerHub (1 set of SNAPSHOT images every day, then 1 set of X.X.X.RELEASE images roughly every 3 weeks)

We're currently using a paid 1-concurrent job plan on travis-ci.com.

//TODO Alexandra Guironnet Add average minutes and number of builds once Travis insights are fixed https://app.travis-ci.com/github/opfab?tab=insights

SOGNOLinux, (some services run on Windows and MacOS)mainly C++ (cmake) and Python, (Rust, Java, JavaScript)GitHub.com and mirrors to self-hosted GitLabAbout 10 researchers plus a varying number of students (all in CET).Integration of self-hosted GitLab CI with GitHub plus GitHub actions. GitLab runner can run multiple jobs in parallel.
  • Compile
  • Run tests
  • Generate docker images
  • Generate documentation and push to GitHub pages and other places
  • Push docker images to DockerHub (and GitHub registry in the future) 

We do not pay for any service but we are investing in our server infrastructure for GitLab.

Migrated some jobs to GitHub actions but this is not possible for all jobs.

PowSyBlLinux, MacOS




Free plan on GitHub Actions


Solution/Platform Benchmark

Any feedback or experience with other CI/CD platforms would be welcome.


travis-ci.comGitHub ActionsGitLab (self-hosted)
OperatorFabric

Platform stability has been good so far, and communication on maintenance or incidents is usually quick.

Integration with GitHub was quite straightforward.

The build configuration can sometimes be tricky, especially cache management, but the documentation is pretty good (at least on our use cases).

We only add 2 interactions with the support team and the quality of the response varied greatly.

No experience with GitHub actions so far, we had looked into it in its early days and at the time we found that there was really too little documentation to try and reproduce our Travis build as GitHub actions.
SOGNO

We have already migrated some GitLab CI jobs to GitHub actions. For the moment, it looks like we won't be able to migrate all jobs.

Experience with GitHub actions is ok so far although the majority of our developers still prefers GitLab CI.

We are quite happy with the GitLab CI but since we are running a self-hosted instance, it is not easily accessible for external developers. GitLab.com is not as established for LF projects, which is why we are moving to GitHub,com and actions.
  • No labels

3 Comments

  1. John Mertic

    Notes from 10/5 call:

    • OpFab - wanting to move off of Travis due to costs
    • SOGNO - needing build infra with GPU support, also running into performance issues with Github Actions vs locally hosted Gitlab runners. Avoid triggering builds from external PRs due to resource concerns/abuses.
    • Discussed https://opensource.microsoft.com/azure-credits#credits-overview as an option - not sure if this gives GPU support
    • What's missing?
      • OpFab - more resources to do additional/more thorough integration testing
      • SonarCloud - could be used better; would like to explore Dynamic/Static Code Analysis.
      • Also would have an interest in code coverage analysis. 
      • Concerns with moving to GH Actions - which doesn't have the same breadth of coverage as other tools.
      • Other language-specific tooling
      • Deployed instance that could be used for external validation - public demo ( nice to have vs critical )

  2. Alexandra Guironnet

    On our side it's the paperwork involved more than the cost that's the issue I would say.

    I looked at the Azure link you provided, and the good news is that they don't have a criterion on project funding:

    Any open source project with an OSI-approved license and a formalized Code of Conduct is eligible to apply.

    But then it's up to Azure to grant the credits or not and it doesn't say on what basis they make the decision.

    1. John Mertic

      True. From an LFE perspective we could help this process to reduce the burden on the individual project ( much like we did for a DockerHub org )