AutoDiff

Tip of the Iceberg

Automatic Differentiation with Clang

by

Violeta Ilieva

Undergraduate Computer Science Student

at

Princeton University

Supervised by

Vassil Vassilev and Lorenzo Moneta

Special Thanks to Alexander Penev

Automatic Differentiation

clad is a C++ plugin for clang that implements automatic differentiation on user-defined functions by employing the chain rule in its forward mode, coupled with source code transformation and constant folding.

Future of the project

- Enable code generation;

- Decide whether intermediate derivatives should be stored;

- Verify the tool implementation with real-life values and functions;

- Expand the library of predefined derivatives of common functions;

- Embed the tool in cling and ROOT;

- ...

Goals

The goal of the project is to extend the cling functionality in order to make it possible for the tool to differentiate non-trivial functions and find partial derivatives for trivial cases.

The final product will consist of C++ files, documentation, and test cases.

Automatic Differentiation

Differentiation - the process of finding a derivative, which in turn measures how a function changes as its input changes.

Automatic differentiation - a method for evaluating the derivative with machine precision accuracy by employing the chain rule of calculus.

Compared to other methods:

+ yields exact derivatives

- hard to implement

Realization in Code

Operator Overloading - consists of modifying the code semantics via replacing the types of floating-point variables with a new type that contains additional differentiation-relevant information. The arithmetic operations for this type are overloaded with the aim to process and propagate the information.

Source Code Transformation - consists of explicitly building a new source code through a compiler-like process which includes parsing the original program, constructing an internal representation, performing global analysis, and applying transformations where appropriate.

How to use it?

1. Define your function in a C/C++ file.

2. Call our special method diff(...).

3. Invoke clad.

4. Yay, derivative!

Define your function

Call diff(...) and Invoke clad

What is the answer?

Derivative Expression

Under the surface

Automatic Differentiation with clang

1. Collect diff invocations through clang's RecursiveASTVisitor.

2. Parse their arguments:

- functions to derive - example_fn, example_fn, example_fn;

- independent variables - x, y, z (respectively).

Under the surface

3. Transform the user function's declaration.

4. Visit the different components of the function definition, then clone and transform them through the compiler according to the chain rule.

Under the surface

Basic Arithmetic

Control Flow

Overloading

Rosenbrock Function