Higher order derivatives, performance, and optimisations

[athas]

I am using Enzyme to do second-order differentiation of a cost function (first reverse, then forwards, for Newton's Method), but I am getting unexpectedly poor performance. I have a hunch that this is because no optimisation passes are run after the first derivative.

The cost function is for kmeans clustering, and is here: https://github.com/gradbench/gradbench/blob/57ce3ea9cd053a352b1c229afe6db06400ac9f2c/cpp/gradbench/kmeans.h#L18-L35

The pertinent Enzyme parts: https://github.com/gradbench/gradbench/blob/57ce3ea9cd053a352b1c229afe6db06400ac9f2c/tools/enzyme/run_kmeans.cpp#L17-L47

I use Enzyme by passing --load-pass-plugin to lld. My Makefile is here: https://github.com/gradbench/gradbench/blob/57ce3ea9cd053a352b1c229afe6db06400ac9f2c/tools/enzyme/Makefile

The pertinent compilation commands end up being the following:

clang++ run_kmeans.cpp -c -o run_kmeans.o -std=c++17 -O3 -fno-math-errno -Wall -flto -I../../cpp
clang++ -o run_kmeans run_kmeans.o -fuse-ld=lld -O3 -fno-math-errno -flto -Wl,--load-pass-plugin=/nix/store/3j5sqzlw9hki71vhcja40n83dz9n7q55-enzyme-0.0.172/lib/LLDEnzyme-19.so -lm