Zero Instrumentation Observability for Android™¹ (AMOS WS 2024)

Product Mission

ZIOFA (Zero Instrumentation Observability for Android) aims to implement observability use cases relevant to performance specified by our industry partner using eBPF. Examples include tracing long-running blocking calls, leaking JNI indirect references or signals like SIGKILL sent to processes, all without instrumenting the observed application itself.
The eBPF programs are loaded and unloaded using a backend daemon running as root that will collect metrics and send them to a client. For displaying these metrics to the user, we are implementing an on-device UI that can display visualizations for these use cases and allow for configuration of the enabled use cases, but using a decoupled Client SDK so that future work may easily make the data accessible the external processing.

Building

Nix

The easiest way to get all dependencies is to use the provided development shell in flake.nix. You can use a tool like direnv to automatically load the environment for this repository.

The shell will setup:

• The rust nightly toolchain (nightly is required currently for ebpf because of the unstable build-std feature)
• The bpf-linker
• The Android SDK and NDK
• The cargo-ndk package for compiling for rust for android
• The protobuf programs for generating grpc server and client code

OCI Container

We use the same input that go into the development shell for building a layered docker image. This image is used for the ci, so it supports building the project as well, however you graphical applications like the android emulator require custom setups depending on your OS.

The container image is built via nix and can be automatically uploaded to the registry via nix run .#dockerBuilder.copyToRegistry. It is currently hosted in the github registry under ghcr.io/fhilgers/ziofa-builder.

For a working emulator inside the docker image, you will need some form of X11 forwarding. Installing nix and using the development shell is the recommended approach for development.

Emulator Setup

As we need a modified version of Android, we cannot use the standard system images that come with the default Android SDK. To make development easier, a custom Android SDK is loaded into your environment using the nix development shell.

Note

The Android SDK shipped with the shell is built of the standard parts with everything necessary for building Android Apps. The only "custom' part is the system image, which was build externally and is currently hosted on S3. The image can be also downloaded manually from sdk-repo-linux-system-images.zip with its manifest stored in package.xml. You can also download and unzip the system image to $ANDROID_SDK_ROOT/system-images/android-VanillaIceCream/android-automotive/x86_64 manually and copy the package.xml to that directory as well.

This sdk includes the automotive system image, which is built with a custom kernel having CONFIG_FTRACE_SYSCALLS=y set. To create an emulator using that image, you can use the avdmanager tool also provided with the SDK:

avdmanager create avd -n YOUR_AVD_NAME  -k 'system-images;android-VanillaIceCream;android-automotive;x86_64' --device automotive_1080p_landscape

This can be started with the emulator tool like this:

emulator @YOUR_AVD_NAME

Documentation

To generate the html documentation run the following:

asciidoctor -r asciidoctor-diagram Documentation/asciidoc/main.adoc -o Documentation/build/doc/index.html

License

This project adheres to the reuse Software recommendations.

When modifying a file please add yourself to the list of Copyright Holders. You can do that with reuse annotate --copyright="YOUR NAME <YOUR EMAIL>" FILE.

When adding a new file you have to add yourself to the list of Copyright holders and set the license. You should prefer the MIT license if possible. The easiest way to set the license and copyright is to execute reuse annotate --copyright="YOUR NAME <YOUR EMAIL>" --license "MIT" FILE.

To check whether you have done everything correctly, execute reuse lint in the project root directory.

Footnotes

1. Android is a trademark of Google LLC. The Android robot is reproduced or modified from work created and shared by Google and used according to terms described in the Creative Commons 3.0 Attribution License. ↩