21 Dec 2020, 20:26

Navigation

For installation, and basic usage, see getting started.

For an overview of the features of floki, see the feature overview.

Full documentation can be found here.

06 Apr 2019, 20:22

Feature Overview

floki aims to provide reproducible and shareable build tooling. It does this by helping you run docker containers interactively from a declarative yaml file.

The ideal workflow is

  • clone the source repository
  • run floki
  • get to work

floki has a number of features to help achieve this. The following outlines these features.

Container images

floki offers a couple of ways to configure the container image to use.

Prebuilt images

Using a prebuilt image (e.g. one from dockerhub or a docker registry) is as simple as providing its name as a top-level key in floki.yaml:

image: debian:sid

floki will use docker to pull this image if you need it.

Custom registries can be used by configuring docker to use these registries. floki defers to docker to locate and pull images.

Build an image

floki can use an image built from a Dockerfile in source tree. It’s easiest to see an example of floki.yaml to see how to configure this.

image:
  build:
    name: foo                    # Will build the image with name foo:floki
    dockerfile: Dockerfile.foo   # Relative location in source tree; defaults to Dockerfile
    context: .                   # Defaults to .
    target: builder              # Target to use, for multi-stage dockerfiles (optional)

Referencing a key in another yaml file

floki can use an image by reference to another yaml file. This can help keep local development environments synced with a CI environment.

image:
  yaml:
    file: .gitlab-ci.yaml
    key: variables.RUST-IMAGE

Updating an image

floki pull forces a pull of the container specified in image. While it is better to version images properly, this can be used when tracking a latest tag, or similar.

Setting the shell

Different containers require different shells, so floki allows you to configure this. Sometimes you will want a different shell to run the init commands to the shell presented to the user, and so floki also allows you to set an outer (used for init) and inner (used by the user) shell.

The default shell is sh.

Single shell

A shell can be set for a container using the top-level shell key:

image: alpine:latest
shell: sh

Inner and outer shell

A different shell can be used for initialization and the interactive shell provided to the user.

image: alpine:latest
shell:
  inner: bash
  outer: sh
init:
  - apk update && apk install bash

A useful use case here is if you want to run the container with the same user as on the host. floki exposes the user id and user group id in environment variables, so you can add a user to the running container and switch to the new user in the inner shell:

image: foo:latest
shell:
  inner: bash
  outer: switch_user
init:
  - add_new_user $FLOKI_HOST_UID $FLOKI_HOST_GID

The commands to make the above work depend on the container you are running. floki just provides the tools to allow you to make it happen.

Docker-in-docker

Docker-in-docker (dind) can be enabled by setting the top-level dind key to true.

image: foo:bar
dind: true

Note that the docker CLI tools are still required in the container, and the docker host is a linked container, with the working directory mounted in the same place as the interactive container.

The precise dind image can also be set

dind:
  image: docker:stable-dind

This helps properly pin and version the docker-in-docker container.

Floki volumes

floki has the ability to use volumes for caching build artifacts between runs of the container (amongst other things). Volumes can be configured in floki.yaml:

volumes:
  cargo-registry:
    mount: /home/rust/.cargo/registry

The key names the volume (it can be any valid yaml name), while the mount key specifies where the volume will be mounted inside the floki container.

It’s also possible to share volumes across different floki.yamls. For example, you may want to share a cargo registry across all Rust build containers. These shared volumes are identified by the name given to the volume.

volumes:
  cargo-registry:
    shared: true
    mount: /home/rust/.cargo/registry

floki creates directories on the host to back these volumes in ~/.floki/volumes. Non-shared volumes are given names unique to the source directory.

Environment forwarding

User details

floki captures the host user details in environment variables, and forwards these into the running container.

  • FLOKI_HOST_UID is set to the host user’s user id (the output of id -u)
  • FLOKI_HOST_GID is set to the host user’s group id (the output of id -g)

These can be used to configure users in the container dynamically. This can be a little fiddly, especially if the container already uses a non-root user with the same id as the host user.

Host working directory

The host path to the mounted directory is forwarded into the floki container as an environment variable, FLOKI_HOST_MOUNTDIR.

You can set where this directory is mounted in the container using the mount key in floki.yaml.

SSH agent

Sometimes it is useful to be able to pull dependencies from source code management servers for builds. To make this easier to do in an automated fashion, floki can forward and ssh-agent socket into the container, and expose its path through SSH_AUTH_SOCK.

forward_ssh_agent: true

You will need to have an ssh-agent running on the host before launching floki.

Sandboxed commands with floki run

floki also allows single commands to be run, rather than dropping into an interactive shell.

$ floki run ls
floki.yaml

Note that if you have configured an inner shell, the command will run within the inner shell.

Escaping with docker_switches

floki also allows you to pass additional switches to the underlying docker command, for example to forward port 8080 to the host.

image: debian:sid
docker_switches:
  - -p
  - 8080:8080
init:
  - echo "Welcome to your server container!"

Note that use of docker_switches may reduce the reproducibility and shareability of your floki.yaml (for instance it could be used to mount a volume with a specific host path that works on no other machines).

Nonetheless, it is useful to be able to add arbitrary switches in a pinch, just to be able to get something working. If there are things you can add with docker_switches which are reproducible and shareable, please raise a feature request, or go ahead and implement it yourself!

03 Apr 2019, 23:01

Getting Started

Installation

Prerequisites

It’s recommended you add your user to the docker group:

$ sudo usermod -a -G docker USERNAME

and logout and in again to pick up the changes.

Alternatively you can run floki (after installation) with sudo -E floki.

Installation from pre-built binaries

Precompiled binaries for linux and OSX can be downloaded from the releases page.

For example, to obtain the latest binary with curl and extract it, run

$ curl -L https://github.com/Metaswitch/floki/releases/download/0.6.2/floki-0.6.2-linux.tar.gz | tar xzvf -

in a shell. You should now be able to run floki from your working directory:

$ ./floki --version
floki 0.6.1

Copy this into your path to run it without needing to specify the path absolutely. E.g.

$ mv floki /usr/local/bin/

Installation from cargo

floki can also be installed directly from cargo.

$ cargo install floki

Shell completions

Shell completions can be added to your existing shell session with

source <(floki completion <shell>)

See floki completion --help for a list of available <shell>s. Add this command to your shell’s rc file to get completions in all new shell sessions.

Enjoy!

Getting started

floki is configured using a configuration file typically placed in the root of your codebase. As a basic example, write a basic configuration file, and name it floki.yaml.

image: debian:latest
init:
  - echo "Welcome to your first floki container!"

Now, in the same directory, run

floki

A container will launch with the working directory mounted as your working directory. Verify this by running ls:

$ ls
...  floki.yaml  ...

In general, invoking floki in any child directory of this root directory will launch a container with: - The directory containing floki.yaml mounted; - The container shell located in the guest directory corresponding to the child.

Using a different configuration file

You can use a different configuration file with floki by telling it to use a different file from the command line. For example, if you have another configuration in config.yaml, you can run floki with

floki -c config.yaml

Note that, in contrast to invoking floki without the -c flag, this will always mount the current working directory.

Features you may want to look at next

  • Forwarding of ssh-agent (useful for authenticating with remote private git servers to pull private dependencies)
  • Docker-in-docker support
  • Forwarding of host user information (allows non-root users to be added and used).
  • floki volumes for setting up cross-session build caches.