Continous Integration of Android Emulator Docker Image

Continous Integration (CI) is important to reduce time to market while ensuring your app works as expected (through testing). I created this image in support of the One App Three Ways: Weather project.

This article will walk through the Docker file itself, using Gitlab as the continous integration platform, and as the image registry. There will be a sister article about setting up a Digital Ocean VM to run this image.

Android UI testing in a CI environment has proven to take a long time to get correct through a lot of trial and error. Referencing multple sites, I was able to put together my version of a docker image that allows me to run emulator tests in the gilab ci environment. The Gitlab Runner has to have kvm access. I can use gitlab to run CI for the image and host it in the gitlab registry as part of the project.

Dockerfile

The Dockerfile is split into three areas; dependencies, android sdk, creating a non-root user.

Dependencies

The dependencies took several iteration to ensure I was able to set up the emulator properly. This included KVM and QEMU. I also include git-secret for secrets management in the application that will run in the container.

LABEL Maintainer Cavelle Benjamin <cavelle@thecb4.io>
LABEL Name Swift LLVM
LABEL Release Alpha
LABEL Vendor TheCB4
LABEL Version 0.0.1


# https://gist.github.com/illuzor/988385c493d3f7ed7193a6e3ce001a68
ARG ANDROID_PLATFORM
ARG ANDROID_COMPILE_SDK
ARG ANDROID_BUILD_TOOLS
ARG ANDROID_SDK_TOOLS
ARG ANDROID_EMULATOR
ARG ANDROID_NDK
ARG CMAKE
ARG DEBIAN_FRONTEND
ARG DEBCONF_NONINTERACTIVE_SEEN


# locale and timezone
ENV LANG='en_US.UTF-8' \
  LANGUAGE='en_US:en' \
  LC_ALL='en_US.UTF-8' \
  ENV_TIMEZONE="America/New_York"
RUN echo '$ENV_TIMEZONE' > /etc/timezone && ln -fsn /usr/share/zoneinfo/$ENV_TIMEZONE /etc/localtime
RUN apt-get update -qq && \
  apt-get upgrade -qq && \
  DEBIAN_FRONTEND=noninteractive apt-get install -qq -y --no-install-recommends \
  locales tzdata ntp ntpstat ntpdate
RUN locale-gen $LANG && dpkg-reconfigure --frontend noninteractive tzdata


# Install dependencies
RUN apt-get update -qq && apt-get upgrade -qq && DEBIAN_FRONTEND=noninteractive apt-get install -qq -y --no-install-recommends \
  sudo \
  ssl-cert openssl \
  ca-certificates \
  wget \
  curl \
  gpg \
  gpg-agent \
  zip \
  unzip \
  openjdk-11-jdk \
  python3 \
  git \
  git-secret \
  lsb-release \
  libc6 libdbus-1-3 libfontconfig1 libgcc1 \
  libpulse0 libtinfo5 libx11-6 libxcb1 libxdamage1 \
  libnss3 libxcomposite1 libxcursor1 libxi6 \
  libxext6 libxfixes3 zlib1g libgl1 pulseaudio socat \
  qemu-system-x86 libvirt-daemon-system libvirt-clients bridge-utils \
  mesa-vulkan-drivers \
  locales \
  qtbase5-dev qtchooser qt5-qmake qtbase5-dev-tools \
  && apt-get autoremove --purge \
	&& apt-get clean \
	&& rm -rf /var/lib/apt/lists/*

RUN rm -f /etc/ssl/certs/java/cacerts; \
    /var/lib/dpkg/info/ca-certificates-java.postinst configure

Android Command Line Tools

Installing the command line tools also took seeral iterations. Some of it was driven by out of date documentation and certain versions not being compatible with the tools and the APIs. Specifically I needed to download a particular version of the emulator when running sdkmanager install.

RUN wget --quiet --output-document=android-sdk-linux.zip https://dl.google.com/android/repository/commandlinetools-linux-${ANDROID_SDK_TOOLS}_latest.zip
RUN mkdir /usr/share/android-sdk-linux
RUN unzip -d /usr/share/android-sdk-linux/cmdline-tools android-sdk-linux.zip
RUN mv /usr/share/android-sdk-linux/cmdline-tools/cmdline-tools /usr/share/android-sdk-linux/cmdline-tools/latest
RUN rm -rf android-sdk-linux.zip

# Install Gradle
RUN cd /usr/local/ && \
	curl -L -O https://services.gradle.org/distributions/gradle-7.3.3-bin.zip && \
	unzip -qo gradle-7.3.3-bin.zip && \
	rm -rf gradle-7.3.3-bin.zip

RUN echo "$JAVA_HOME"
ENV ANDROID_SDK_ROOT="/usr/share/android-sdk-linux"
ENV ANDROID_HOME="/usr/share/android-sdk-linux"
ENV GRADLE_HOME "/usr/local/gradle-7.3.3"
ENV PATH="$ANDROID_HOME/cmdline-tools/latest/bin:$ANDROID_HOME/emulator:$ANDROID_HOME/platform-tools:$PATH:$GRADLE_HOME/bin"
ENV ADB_INSTALL_TIMEOUT=8


RUN yes "y" | sdkmanager --licenses
RUN echo y | sdkmanager --update --verbose --channel=3

RUN echo y | sdkmanager --install "platform-tools"
RUN echo y | sdkmanager --install "platforms;android-${ANDROID_COMPILE_SDK}"
RUN echo y | sdkmanager --install "build-tools;${ANDROID_BUILD_TOOLS}"
RUN echo y | sdkmanager --install "cmake;${CMAKE}"
RUN echo y | sdkmanager --install "emulator" --verbose --channel=3
RUN echo y | sdkmanager --install "system-images;android-${ANDROID_COMPILE_SDK};google_apis_playstore;${ANDROID_PLATFORM}"

Non-root user

Creating the non-root user was necessary due to the host configuration and ownership of files and certain commands around kvm not working because it was a root user. There

# Create and switch to non root user
RUN useradd -rm -d /home/thecb4 -s /bin/sh -g root -G sudo,kvm,libvirt -u 1001 thecb4
RUN echo '%sudo ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers
USER thecb4
WORKDIR /home/thecb4


ENV GRADLE_USER_HOME=/home/thecb4/.gradle GRADLE_OPTS=-Dorg.gradle.daemon=false
RUN mkdir -p $GRADLE_USER_HOME && echo "org.gradle.daemon=false" >> $GRADLE_USER_HOME/gradle.properties


RUN mkdir .android
COPY sources/advancedFeatures.ini .android/advancedFeatures.ini
COPY sources/emu-update-last-check.ini .android/emu-update-last-check.ini

With the advancedFeatures.ini file

Vulkan = off

GLDirectMem = on

The emu-update-last-check.ini is an empty file

The full docker file

LABEL Maintainer Cavelle Benjamin <cavelle@thecb4.io>
LABEL Name Swift LLVM
LABEL Release Alpha
LABEL Vendor TheCB4
LABEL Version 0.0.1


# https://gist.github.com/illuzor/988385c493d3f7ed7193a6e3ce001a68
ARG ANDROID_PLATFORM
ARG ANDROID_COMPILE_SDK
ARG ANDROID_BUILD_TOOLS
ARG ANDROID_SDK_TOOLS
ARG ANDROID_EMULATOR
ARG ANDROID_NDK
ARG CMAKE
ARG DEBIAN_FRONTEND
ARG DEBCONF_NONINTERACTIVE_SEEN


# locale and timezone
ENV LANG='en_US.UTF-8' \
  LANGUAGE='en_US:en' \
  LC_ALL='en_US.UTF-8' \
  ENV_TIMEZONE="America/New_York"
RUN echo '$ENV_TIMEZONE' > /etc/timezone && ln -fsn /usr/share/zoneinfo/$ENV_TIMEZONE /etc/localtime
RUN apt-get update -qq && \
  apt-get upgrade -qq && \
  DEBIAN_FRONTEND=noninteractive apt-get install -qq -y --no-install-recommends \
  locales tzdata ntp ntpstat ntpdate
RUN locale-gen $LANG && dpkg-reconfigure --frontend noninteractive tzdata


# Install dependencies
RUN apt-get update -qq && apt-get upgrade -qq && DEBIAN_FRONTEND=noninteractive apt-get install -qq -y --no-install-recommends \
  sudo \
  ssl-cert openssl \
  ca-certificates \
  wget \
  curl \
  gpg \
  gpg-agent \
  zip \
  unzip \
  openjdk-11-jdk \
  python3 \
  git \
  git-secret \
  lsb-release \
  libc6 libdbus-1-3 libfontconfig1 libgcc1 \
  libpulse0 libtinfo5 libx11-6 libxcb1 libxdamage1 \
  libnss3 libxcomposite1 libxcursor1 libxi6 \
  libxext6 libxfixes3 zlib1g libgl1 pulseaudio socat \
  qemu-system-x86 libvirt-daemon-system libvirt-clients bridge-utils \
  mesa-vulkan-drivers \
  locales \
  qtbase5-dev qtchooser qt5-qmake qtbase5-dev-tools \
  && apt-get autoremove --purge \
	&& apt-get clean \
	&& rm -rf /var/lib/apt/lists/*

RUN rm -f /etc/ssl/certs/java/cacerts; \
    /var/lib/dpkg/info/ca-certificates-java.postinst configure


# Install Android Command Line Tools
RUN wget --quiet --output-document=android-sdk-linux.zip https://dl.google.com/android/repository/commandlinetools-linux-${ANDROID_SDK_TOOLS}_latest.zip
RUN mkdir /usr/share/android-sdk-linux
RUN unzip -d /usr/share/android-sdk-linux/cmdline-tools android-sdk-linux.zip
RUN mv /usr/share/android-sdk-linux/cmdline-tools/cmdline-tools /usr/share/android-sdk-linux/cmdline-tools/latest
RUN rm -rf android-sdk-linux.zip

# Install Gradle
RUN cd /usr/local/ && \
	curl -L -O https://services.gradle.org/distributions/gradle-7.3.3-bin.zip && \
	unzip -qo gradle-7.3.3-bin.zip && \
	rm -rf gradle-7.3.3-bin.zip

RUN echo "$JAVA_HOME"
ENV ANDROID_SDK_ROOT="/usr/share/android-sdk-linux"
ENV ANDROID_HOME="/usr/share/android-sdk-linux"
ENV GRADLE_HOME "/usr/local/gradle-7.3.3"
ENV PATH="$ANDROID_HOME/cmdline-tools/latest/bin:$ANDROID_HOME/emulator:$ANDROID_HOME/platform-tools:$PATH:$GRADLE_HOME/bin"
ENV ADB_INSTALL_TIMEOUT=8


RUN yes "y" | sdkmanager --licenses
RUN echo y | sdkmanager --update --verbose --channel=3

RUN echo y | sdkmanager --install "platform-tools"
RUN echo y | sdkmanager --install "platforms;android-${ANDROID_COMPILE_SDK}"
RUN echo y | sdkmanager --install "build-tools;${ANDROID_BUILD_TOOLS}"
RUN echo y | sdkmanager --install "cmake;${CMAKE}"
RUN echo y | sdkmanager --install "emulator" --verbose --channel=3
RUN echo y | sdkmanager --install "system-images;android-${ANDROID_COMPILE_SDK};google_apis_playstore;${ANDROID_PLATFORM}"


# Create and switch to non root user
RUN useradd -rm -d /home/thecb4 -s /bin/sh -g root -G sudo,kvm,libvirt -u 1001 thecb4
RUN echo '%sudo ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers
USER thecb4
WORKDIR /home/thecb4


ENV GRADLE_USER_HOME=/home/thecb4/.gradle GRADLE_OPTS=-Dorg.gradle.daemon=false
RUN mkdir -p $GRADLE_USER_HOME && echo "org.gradle.daemon=false" >> $GRADLE_USER_HOME/gradle.properties


RUN mkdir .android
COPY sources/advancedFeatures.ini .android/advancedFeatures.ini
COPY sources/emu-update-last-check.ini .android/emu-update-last-check.ini

The Docker Compose file

I used docker compose to avoid a lot of command line work. This allows me to update any of the android settings from one place. I could also create multiple versions of the image.

version: "3.9"

services:
  android:
    image: registry.gitlab.com/thecb4-universe/private/docker/android-x86_64:latest
    build:
      context: .
      dockerfile: sources/android.dockerfile
      args:
        ANDROID_PLATFORM: "x86_64"
        ANDROID_COMPILE_SDK: "32"
        ANDROID_BUILD_TOOLS: "32.0.0"
        ANDROID_SDK_TOOLS: "8512546"
        ANDROID_EMULATOR: "31.2.10"
        ANDROID_NDK: "24.0.8215888"
        CMAKE: "3.18.1"
        DEBIAN_FRONTEND: noninteractive
        DEBCONF_NONINTERACTIVE_SEEN: true
    privileged: true

The gitlab-ci.yaml file

The gitlab ci file has two steps. The first is building the image. The second is scanning the image for security vulnerabilities. The image is built using docker-in-docker (dind) configuration on a Digital Ocean hosted Gitlab Runner. This is done to ensure KVM is available at the time of build. Before building and pushing to the registry, you have to log in to the gitlab registry using your CI variables.

include:
  - template: Security/Container-Scanning.gitlab-ci.yml

services:
  - docker:dind


stages:
 - build
 - test

build:
  image: docker:latest
  variables:
    REPOSITORY: $CI_REGISTRY/thecb4-universe/private/docker/android-x86_64
    DOCKER_HOST: tcp://docker:2375
    DOCKER_DRIVER: overlay
    DOCKER_TLS_CERTDIR: ""
  stage: build
  before_script:
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
  script:
    - docker compose -f docker-compose-ci.yml build --force-rm --no-cache
    - docker push $REPOSITORY
  only:
    - main
  tags:
    - thecb4-universe
    - docker

container_scanning:
  needs: [build]
  variables:
    DOCKER_IMAGE: "$CI_REGISTRY/thecb4-universe/private/docker/android-x86_64"
    DOCKER_USER: $CI_REGISTRY_USER
    DOCKER_PASSWORD: $CI_REGISTRY_PASSWORD

Find me on Twitter and tell me all about it!