Setting up gitlab-ci for rust

We're going to create a multi-stage pipeline based on the one I'm using at work, featuring:

• a build image
• linting
• test & release builds
• docker containers

What we're gonna need first is the build image. This is a docker image we use to compile our code. I'm used to ubuntu, but you can use whatever distribution you're most comfortable with. Note that it has to be the same distro you'll be using in production containers.

Some prerequisities:

You need a docker gitlab executor ([runners.docker] in your gitlab.runner config.toml) with docker socket mapped inside the container and /cache folder mapped to a host directory. Ask your devops guy about this (or if you are the devops guy, consult gitlab docs).

[[runners]]
  name = "rust-ci.whatever"
  url = "https://gitlab.blahblah.com/"
  token = "blahblahblah"
  executor = "docker"
  [runners.docker]
    image = "ubuntu:16.04"
    privileged = false
    disable_cache = true
    volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/srv/ssd/cache:/cache"]
    shm_size = 0

I assume you got your runner up and running, so let's start with a Dockerfile for the build container.

FROM ubuntu:16.04

RUN apt-get update && apt-get -qq -y install curl libssl-dev build-essential pkg-config git
RUN curl https://sh.rustup.rs -sSf | sh -s -- -y --default-toolchain nightly-2018-05-11
RUN /bin/bash -c "source ~/.cargo/env && cargo install clippy --vers 0.0.198"

Here we're installing system components and dependencies we need for our builds. curl is only needed for the second step, build-essential brings all the C compiler infrastructure and linker, pkg-config we may need to find ffi headers, and git is used to retrieve code.

On the next step we install the rust toolchain. As you can see here, I'm using nightly. I recommend sticking to a specific version (not just "nightly").

And on the next line we install clippy, the rust linter. It also has to be a specific version. Clippy is tied to the compiler version, you need some trial & error runs to find the right one.

Now we need to build the image in gitlab. Here's a snippet of the .gitlab-ci.yml (I assume you put the Dockerfile.build in the top-level directory):

variables:
  BUILD_IMAGE: $CI_REGISTRY_IMAGE/build:$CI_COMMIT_REF_SLUG

create_build_image:
  stage: pre_build
  image: docker:17.10
  tags: [rust]
  before_script:
    - docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY
  script:
    - docker build --pull -t $BUILD_IMAGE -f Dockerfile.build .
    - docker push $BUILD_IMAGE
    - echo "Pushed $BUILD_IMAGE"

Now we have our build image built and pushed and are ready to go the next stage - clippy:

clippy:
  stage: clippy
  image: $BUILD_IMAGE
  tags: [rust]
  before_script:
    - rm -fr ~/.cargo/registry && mkdir -p /cache/registry && ln -s /cache/registry ~/.cargo/registry
    - rm -fr ~/.cargo/git && mkdir -p /cache/git && ln -s /cache/git ~/.cargo/git
    - rm -fr target && mkdir -p /cache/$CI_COMMIT_REF_SLUG/target && ln -s /cache/$CI_COMMIT_REF_SLUG/target target
    - source ~/.cargo/env
    - rustc --version
  script:
    - RUSTFLAGS="-D warnings" CARGO_INCREMENTAL=1 RUST_BACKTRACE=1 cargo clippy

Here we're symlinking cargo's cache and git to /cache directory and target directory to /cache/<branch-name> directory, so that you would reuse compilation cache for the branch (and not the whole repository). Note that gitlab can manage cache for you with cache directive but I recommend against it because the cache tends to grow with time (it's several gigabytes on my project) and uploading / downloading it every time will significantly slower your builds.

Same thing for test run:

test:
  stage: test
  image: $BUILD_IMAGE
  tags: [rust]
  before_script:
    - rm -fr ~/.cargo/registry && mkdir -p /cache/registry && ln -s /cache/registry ~/.cargo/registry
    - rm -fr ~/.cargo/git && mkdir -p /cache/git && ln -s /cache/git ~/.cargo/git
    - rm -fr target && mkdir -p /cache/$CI_COMMIT_REF_SLUG/target && ln -s /cache/$CI_COMMIT_REF_SLUG/target target
    - source ~/.cargo/env
    - rustc --version
  script:
    - RUSTFLAGS="-D warnings" CARGO_INCREMENTAL=1 RUST_BACKTRACE=1 cargo test -- --nocapture

And now on to release:

build_binary:
  stage: build
  image: $BUILD_IMAGE
  tags: [rust]
  only:
    - tags
  before_script:
    - source ~/.cargo/env
    - rustc --version
  script:
    - rm -fr ~/.cargo/registry && mkdir -p /cache/registry && ln -s /cache/registry ~/.cargo/registry
    - rm -fr ~/.cargo/git && mkdir -p /cache/git && ln -s /cache/git ~/.cargo/git
    - rm -fr target && mkdir -p /cache/$CI_COMMIT_REF_SLUG/target && ln -s /cache/$CI_COMMIT_REF_SLUG/target target
    - RUST_BACKTRACE=1 cargo build --release
  artifacts:
      paths:
        - target/release/${CI_PROJECT_NAME}

Build and push the final container:

push_images:
  stage: package
  image: docker:17.10
  only:
    - tags
  tags:
    - rust
  before_script:
    - docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN $CI_REGISTRY
  script:
    - docker build --pull -t $DEPLOY_IMAGE .
    - docker push $DEPLOY_IMAGE

That's all! You can find the complete project on github.