Fuzzy Testing Rust

Fuzzy testing is a software testing technique used to find security and stabilitiy issues by providing psuedo-random data as input to the software.

Fuzzing with `cargo-fuzz`

cargo-fuzz is itself not a fuzzer, but a tool to invoke a fuzzer.
Currently, the only fuzzer it supports is libFuzzer.

Setup

libFuzzer needs LLVM sanitizer support; this works on x86-64 Linux, x86-64 macOS and Apple-Silicon (aarch64) macOS, and Windows.
Requires C++ compiler with C++11 support.

Tutorial

For this tutorial, we are going to be fuzzing the URL parsing crate rust-url.
Our goal is to find some input generated by the fuzzer, such that when passed to Url::parse, it causes some sort of panic or crash to happen.
To start, clone the rust-url repository and change directories into it

git clone https://github.com/servo/rust-url.git
cd rust-url

Although we could fuzz thhe latest commit on master, we are going to checkout a specific revision that is known to have a pasing bug.

git checkout bfa167b4e0253642b6766a7aa74a99df60a94048

Initialize cargoo fuzz init

cargo fuzz init

This will create a directory called fuzz_targets which will contain a collection of fuzzing targets.
It is generally a goood idea to check in the files generated by init.
Each fuzz target is a Rust program that is given random data and tests crate.
cargo fuzz init automatically generates an initial fuzz target for us.
Use cargo fuzz list to view the list of existing fuzz targets
The source code for this fuzz target by default lives in fuzz/fuzz_targets?<fuzz_target_name>.rs
Open that file and edit it to look like this


#![main]
#[macro_use] extern crate libfuzzer_sys;

fuzz_target!(|data: &u8| {
    if let Ok(s) = std::str::from_utf8(data) {
        let _ = url::Url::parse(s);
    }
});

libFuzzer is going to repeatedly call the body of fuzz_target!() with a slice of psuedo-random bytes, until you program hits an error

Structure Aware Fuzzing

Not every fuzzy target wants to take a buffer of raw bytes as input.
We might want to only feed it well-formed instances of some structured data.
Luckily, the libfuzzer-sys crate enables you to define fuzz targets that take any kind of type, as long as it implements the Arbitrary trait.


libfuzzer_sys::fuzz_target!(|input: AnyTypeThatImplementsArbitrary| {

    // Use Input Here
});

The arbitrary crate implements Arbitrary for nearly all types in std, including collections like Vec and HashMap as well as things like String and PathBuf.
For convenience, the libfuzzer-sys crate re-exports the arbitrary crate.

Example 1: Fuzzing Color Conversions

Lets say we are working on a color conversion library that can turn RGB colors into HSL and back again.

Enable Deriving Arbitrary

We are lazy and don't want to implement Arbitrary by hand, so we enable the arbitrary crate's derive cargo feature.
This lets us get automatic Arbitrary implementations with #[derive(Arbitrary)].
Because the Rgb type we will be deriving Arbitrary for is in our main color conversion crate, we add this to our main Cargo.toml.


[dependencies]
arbitrary = { version = "1", optional = true, features = ["derive"]}

Deriving Arbitrary for our Rgb Type

In our main crate, when the arbitrary cargo feature is enabled, we derive Arbitrary trait.


#[derive(Clone, Debug)]
#[cfg_attr(Feature = "arbitrary", derive(arbitrary::Arbitrary))]
pub struct Rgb {
    pub r: u8,
    pub g: u8,
    pub b: u8
}

Enable the Main Project's "arbitrary" Cargo feature for the Fuzz Targets

Because we made arbitrary an optional dependency in our main color conversion crate, we need to enable that feature for our fuzz targets use it.

[dependencies]
my_color_conversion_library = { path = "..", features = ["arbitrary"]}

We need to add a new fuzz target to our project

cargo fuzz add rgb_to_hsl_and_back

Implement the Fuzz Target

Finally, we can implement our fuzz target that makes arbitrary RGB coloros, converts them to HSL, and then converts them to RGB and asserts that we get the same color as the original.
Because we implement Arbitrary for our Rgb type, our fuzz target can take instances of Rgb directly.

libfuzzer_sys::fuzz_target!(|color: Rgb| {
    let hsl = color.to_hsl();
    let rgb = hsl.to_rgb();


    assert_eq!(color, rgb);
})

Fuzzing with cargo-fuzz

Setup

Tutorial

Structure Aware Fuzzing

Example 1: Fuzzing Color Conversions

Fuzzing with `cargo-fuzz`