11. March 2023

Bitwarden as a Secret Manager

An easy way to keep pipeline secrets secure is using bitwarden. We will create a lightweight container to get secrets in Gitlab CI pipeline and use them to manage resoures.

I will include a version using bitwarden cloud and a version using a self hosted container with an domain name using traefik (This version must have https).

Creating the Container

Required

Docker CLI

To create the docker container we are going to use the bitwarden CLI npm package. We want to keep the image small so the base image will be a Alpine container.

To keep it secure we will also create a non root user to run the container in

 1FROM alpine:latest
 2
 3ARG UID=1000
 4ARG GID=1000
 5ENV USER=vault
 6
 7RUN apk add --update nodejs npm curl jq 
 8RUN npm install -g @bitwarden/cli 
 9
10RUN addgroup -g "${GID}"  ci
11RUN adduser --disabled-password -h /home/vault  -u "${UID}" -G "ci" vault
12
13ENV HOME=/home/$USER
14COPY ["./data/","/home/vault/.config/Bitwarden CLI/"]
15RUN chown -R $UID:$GID $HOME
16
17USER $USER
18WORKDIR $HOME

To crerate this container specify the correct label that points to your gitlab repo address or whichever service you are using to store your containers.

1sudo docker build -t "docker.registry/path/to/repo/ci_vault:v1.0.0" ./relative/that/to/Dockerfile

Then push the image to your repo

1sudo docker push "docker.registry/path/to/repo/ci_vault:v1.0.0"

Pipeline using bitwarden cloud account

Keep in mind this will send an email alert every time you login and get secrets, It would be a good idea to create a email address specifically for your homelab notifications or your bot email.

Base Configuration

I like to create a configuration to extend onto my pipeline jobs to make it easier to mange them

I will set the runer tags to apply to the jobs and also the container image to use.

Using your runner

1.bitwarden_config:
2  tags:
3    - yourRunner
4    -  tags
5  variables:
6    BW_IMAGE: registry.contaer.name/ci_vault:1.0.0

Using shared, would rather use private runner over public shared but if you wanted to you can(don't your container will explode!)

1.bitwarden_config:
2  tags:
3    - shared
4  variables:
5    BW_IMAGE: registry.contaer.name/ci_vault:1.0.0

Next there are two passwords we need to store in the gitlab varialbes, this will replace all the secrets you manage here. The benefit is you can auto genereate complex passwords and also quickly replace them in a single place.

this job will run at the beginning of the pipleine to make sure these values are set.

Just enter the email and password used to login to bitwarden, The api key was unfortunaltely not working as well so i skipped it because it only gave 42.

Flight check

1bitarden_login_check:
2  stage: .pre
3  script:
4    - if [  -z "${BW_PASSWORD}"]; then echo "ERROR - BW_PASSWORD not set"; exit 1; fi
5    - if [  -z "${BW_USERNAME}"]; then echo "ERROR - BW_USERNAME not set"; exit 1; fi```

We will Use this pipeline script to login.

Creates

vault/ folder in root of active pipeline dir
vault/bw_session session token

you can manage this 2 ways, getting the token ince and then using it whenever needed as an artifact but the current cli tool wasnt working correctly so i did it loging in per secret, it will get set to an ENV variable BW_SESSION

1
2.bitwarden_login_username_pass: &bw_login_username_pass
3    - mkdir vault
4    - echo ${BW_PASSWORD} | bw login --raw ${BW_USERNAME} > vault/bw_session
5    - export BW_SESSION=$(cat ./vault/bw_session)

You can try and make use of the failed mission using client id and persisting session token

 1
 2# bitarden_login_check_APIKEYS:
 3#   stage: .pre 
 4#   script:
 5#     - if [  -z "${BW_CLIENTID}"]; then echo "ERROR - BW_CLIENTID not set"; exit 1; fi
 6#     - if [  -z "${BW_CLIENTSECRET}"]; then echo "ERROR - BW_CLIENTSECRET not set"; exit 1; fi
 7
 8# .bitwarden_login_api_key: &bw_login_apikey
 9#     - mkdir vault
10#     - bw login --apikey 
11#     - export BW_SESSION=$(echo ${BW_PASSWORD} | bw unlock --raw )
12#     - if [ -z "${BW_SESSION}"]; then echo "ERROR - BW_SESSION not found"; exit 1; fi
13
14  # artifacts:
15  #   paths:
16  #     - vault/bw_session
17  #   expire_in: 30 mins

BW Get Note

Finally the pipeline job the can be used to get secrets stored in notes, that way its flexible and can store ssh keys and what not. Will store all as strings.

 1
 2.bitwarden_get_notes:
 3  image: ${BW_IMAGE}
 4  extends:
 5    - .bitwarden_config
 6  before_script:
 7    - if [  -z "${BW_FIELD}"]; then echo "ERROR - BW_FIELD not set"; exit 1; fi
 8    - if [  -z "${OUTPUT_FILE_NAME}"]; then echo "ERROR - OUTPUT_FILE_NAME not set"; exit 1; fi
 9    - *bw_login_username_pass
10    - if [ -z "${BW_SESSION}"]; then echo "ERROR - BW_SESSION not found"; exit 1; fi
11  script:
12    - bw get notes ${BW_FIELD} > vault/${OUTPUT_FILE_NAME}
13  after_script:
14    - bw lock
15  artifacts:
16    paths:
17      - vault/${OUTPUT_FILE_NAME}
18    expire_in: 30 mins

How to Use Job

set the variables on the job of the name of the secret and file you want to save it to

For example we will create a job called test and extend our pipleine job, all that we have to do is set the variables and the field will be persisted in vault/${OUTPUT_FILE_NAME} and can be used in other jobs.

 1 
 2
 3test:
 4  stage: pre
 5  extends:
 6    - .bitwarden_get_notes
 7    - .bitwarden_config
 8  variables:
 9    BW_FIELD: docker_token_gitlab
10    OUTPUT_FILE_NAME: token.json
11

Can see the token in the artifacts of the pipleine to confirm

to use the token in another job just set it as an ENV variable like this

export VARIABLE=$(cat vault/$OUTPUT_FILENAME)

you can specify this as a global ENV variable or .config_job on both jobs to easily manage Secrets

Secret VAR Config example

1.super_secret_vars:
2  variables:
3    UI_BUILD_TOKEN: ui-build
4    API_TOKEN: api-token
5    AWS_CLIENT_ID: name-of-secret-in-bitwarden

secret job now looks like this

 1 
 2
 3test:
 4  stage: pre
 5  extends:
 6    - .bitwarden_get_notes
 7    - .bitwarden_config
 8    - .super_secret_vars
 9  variables:
10    BW_FIELD: ${UI_BUILD_TOKEN}
11    OUTPUT_FILE_NAME: ${UI_BUILD_TOKEN}

then the job you wanna use the value in you just extend the vars, and use the file variable

export VARIABLE=$(cat vault/$UI_BUILD_TOKEN)

1test:
2  stage: build
3  extends:
4    - .super_secret_vars
5  script:
6    - export DOCKER_TOKEN=$(cat vault/$UI_BUILD_TOKEN)
7    - echo $DOCKER_TOKEN

Self host a bitwarden container

On a traefik docker network using labels as a method of service discovery, this way you can have a private hosted secret management system for your homelab

this line is needed to add in the bitwarden login script

bw config server https://vault.domain.io

!!! runner must be using private dns to locate server, https required !!!

1
2.bitwarden_login_username_pass: &bw_login_username_pass
3    - mkdir vault
4    - bw config server https://vault.domain.io
5    - echo ${BW_PASSWORD} | bw login --raw ${BW_USERNAME} > vault/bw_session
6    - export BW_SESSION=$(cat ./vault/bw_session)

 1#docker run -d --name vaultwarden -v /vw-data/:/data/ -p 80:80 vaultwarden/server:latest
 2
 3version: '3.2'
 4
 5volumes:
 6  bitwarden:
 7
 8  
 9networks:
10  proxy:
11    external: true
12
13
14services:
15  bitwarden:
16    image: vaultwarden/server:latest
17    container_name: bitwarden
18    volumes:
19      - bitwarden:/data/
20    networks:
21      - proxy
22    ports:
23      - 9800:80
24    labels:
25      - "traefik.enable=true"
26      - "traefik.http.routers.bitwarden.entrypoints=web,websecure"
27      - "traefik.http.routers.bitwarden.rule=Host(`vault.domain.io`)"
28      - "traefik.http.routers.bitwarden.tls=true"
29      - "traefik.http.routers.bitwarden.service=bitwarden@docker"
30      - "traefik.http.services.bitwarden.loadbalancer.server.port=80"
31      - "traefik.http.services.bitwarden.loadbalancer.server.scheme=http"

Bitwarden pipeline example

 1.bitwarden_config:
 2  tags:
 3    - mantisd
 4    -  docker
 5  variables:
 6    BW_IMAGE: registry.gitlab.com/mantisd/core/ci_vault:1.0.0
 7
 8bitarden_login_check:
 9  stage: .pre
10  script:
11    - if [  -z "${BW_PASSWORD}"]; then echo "ERROR - BW_PASSWORD not set"; exit 1; fi
12    - if [  -z "${BW_USERNAME}"]; then echo "ERROR - BW_USERNAME not set"; exit 1; fi
13
14.bitwarden_login_username_pass: &bw_login_username_pass
15    - mkdir vault
16    - echo ${BW_PASSWORD} | bw login --raw ${BW_USERNAME} > vault/bw_session
17    - export BW_SESSION=$(cat ./vault/bw_session)
18
19
20.bitwarden_get_notes:
21  image: ${BW_IMAGE}
22  extends:
23    - .bitwarden_config
24  before_script:
25    - if [  -z "${BW_FIELD}"]; then echo "ERROR - BW_FIELD not set"; exit 1; fi
26    - if [  -z "${OUTPUT_FILE_NAME}"]; then echo "ERROR - OUTPUT_FILE_NAME not set"; exit 1; fi
27    - *bw_login_username_pass
28    - if [ -z "${BW_SESSION}"]; then echo "ERROR - BW_SESSION not found"; exit 1; fi
29  script:
30    - bw get notes ${BW_FIELD} > vault/${OUTPUT_FILE_NAME}
31  after_script:
32    - bw lock
33  artifacts:
34    paths:
35      - vault/${OUTPUT_FILE_NAME}
36    expire_in: 30 mins
37 
38
39test:
40  stage: pre
41  extends:
42    - .bitwarden_get_notes
43    - .bitwarden_config
44  variables:
45    BW_FIELD: mac_docker_token_gitlab
46    OUTPUT_FILE_NAME: test.json
47  
48
49test-2:
50  stage: pre
51  extends:
52    - .bitwarden_get_notes
53    - .bitwarden_config
54  variables:
55    BW_FIELD: docker-token
56    OUTPUT_FILE_NAME: token.json
57
58use-test-2:
59  stage: pre 
60  needs:
61    - test-2 
62  script:
63    - export DOCKER_TOKEN=$(cat ./vault/token.json)
64    - echo ${DOCKER_TOKEN}
65

Here is my example using my custom image and private runners.