GitHub

Using automatic differentiation

This tutorial shows how to compute the power spectrum $P(k; \theta)$ and its (logarithmic) derivatives

\[\frac{\partial \lg P}{\partial \lg \theta_i}\]

using automatic differentiation with ForwardDiff.jl. This technique can also differentiate any other quantity.

1. Wrap the evaluation

First, we must decide which parameters $\theta$ the power spectrum $P(k; \theta)$ should be considered a function of. To do so, let us write a small wrapper function that calculates the power spectrum as a function of the parameters $(T_{\gamma 0}, \Omega_{c0}, \Omega_{b0}, N_\textrm{eff}, h, Y_p)$, following the Getting started tutorial:

using SymBoltz
M = ΛCDM(K = nothing)
pars = [M.γ.T₀, M.c.Ω₀, M.b.Ω₀, M.ν.Neff, M.g.h, M.b.YHe, M.h.m_eV, M.I.ln_As1e10, M.I.ns]
prob0 = CosmologyProblem(M, Dict(pars .=> NaN))

probgen = parameter_updater(prob0, pars)
P(k, θ) = spectrum_matter(probgen(θ), k; verbose = true, ptopts = (reltol = 1e-3,))
P (generic function with 1 method)

It is now easy to evaluate the power spectrum:

using Unitful, UnitfulAstro
θ = [2.7, 0.27, 0.05, 3.0, 0.7, 0.25, 0.02, 3.0, 0.95]
ks = 10 .^ range(-3, 0, length=100) / u"Mpc"
Ps = P(ks, θ)
100-element Vector{Unitful.Quantity{Float64, 𝐋^3, Unitful.FreeUnits{(Mpc^3,), 𝐋^3, nothing}}}:
 15091.286397715265 Mpc^3
 16046.453082760385 Mpc^3
 17053.396213856788 Mpc^3
 18113.46280521223 Mpc^3
 19228.35290147974 Mpc^3
 20398.740354841022 Mpc^3
 21625.696976118008 Mpc^3
 22909.558130443726 Mpc^3
 24250.6142779927 Mpc^3
 25648.743224363072 Mpc^3
     ⋮
   234.8451657111912 Mpc^3
   200.77790110972597 Mpc^3
   171.15645909873427 Mpc^3
   145.98778165382728 Mpc^3
   124.34227899932044 Mpc^3
   105.57252376845412 Mpc^3
    89.81638869100148 Mpc^3
    76.3336687211135 Mpc^3
    64.76070616679414 Mpc^3

This can be plotted with

using Plots
plot(log10.(ks/u"1/Mpc"), log10.(Ps/u"Mpc^3"); xlabel = "lg(k/Mpc⁻¹)", ylabel = "lg(P/Mpc³)", label = nothing)
Example block output

2. Calculate the derivatives

To get $\partial \lg P / \partial \lg \theta$, we can simply pass the wrapper function P(k, θ) through ForwardDiff.jacobian:

using ForwardDiff
lgP(lgθ) = log10.(P(ks, 10 .^ lgθ) / u"Mpc^3") # in log-space
dlgP_dlgθs = ForwardDiff.jacobian(lgP, log10.(θ))
100×9 Matrix{Float64}:
 -0.148129  -1.23995  -0.229643  -0.00985249  …  -0.0045606   3.0  -3.71642
 -0.16216   -1.23398  -0.228542  -0.0108759      -0.0045308   3.0  -3.65014
 -0.177471  -1.22759  -0.227354  -0.0119988      -0.00452774  3.0  -3.58385
 -0.194118  -1.2206   -0.226059  -0.0132194      -0.00455568  3.0  -3.51756
 -0.212186  -1.21307  -0.224661  -0.0145463      -0.00461834  3.0  -3.45128
 -0.231784  -1.20507  -0.223178  -0.0159908   …  -0.00472004  3.0  -3.38499
 -0.252907  -1.19627  -0.221541  -0.0175545      -0.00486063  3.0  -3.3187
 -0.275637  -1.18659  -0.219748  -0.0192396      -0.00503862  3.0  -3.25242
 -0.300228  -1.17647  -0.217887  -0.0210669      -0.00525512  3.0  -3.18613
 -0.326558  -1.16547  -0.215847  -0.0230376      -0.00550646  3.0  -3.11984
  ⋮                                           ⋱                    
 -5.9802     1.88732  -0.368798  -0.383752       -0.0253149   3.0   2.31565
 -6.02133    1.90445  -0.360193  -0.383383       -0.0253166   3.0   2.38194
 -6.0466     1.92119  -0.366931  -0.385121       -0.0253174   3.0   2.44823
 -6.08018    1.93611  -0.367469  -0.386857       -0.0253198   3.0   2.51451
 -6.11169    1.94946  -0.365861  -0.387967    …  -0.0253204   3.0   2.5808
 -6.14017    1.9635   -0.36696   -0.389302       -0.0253215   3.0   2.64709
 -6.1655     1.97766  -0.367343  -0.389805       -0.0253213   3.0   2.71337
 -6.18939    1.99097  -0.368437  -0.390854       -0.0253216   3.0   2.77966
 -6.21662    2.00424  -0.36816   -0.39222        -0.0253226   3.0   2.84595

The matrix element dlgP_dlgθs[i, j] now contains $\partial \lg P(k_i) / \partial \lg \theta_j$. We can plot them all at once:

plot(
    log10.(ks/u"1/Mpc"), dlgP_dlgθs;
    xlabel = "lg(k/Mpc⁻¹)", ylabel = "∂ lg(P) / ∂ lg(θᵢ)",
    labels = "θᵢ=" .* ["Tγ0" "Ωc0" "Ωb0" "Neff" "h" "YHe" "mh" "ln(10¹⁰As)" "ns"]
)
Example block output

Get values and derivatives together

The above example showed how to calculate the power spectrum values and their derivatives through two separate calls. If you need both, it is faster to calculate them simultaneously with the package DiffResults.jl:

using DiffResults

# Following DiffResults documentation:
Pres = DiffResults.JacobianResult(ks/u"1/Mpc", θ) # allocate buffer for values+derivatives for a function with θ-sized input and ks-sized output
Pres = ForwardDiff.jacobian!(Pres, lgP, log10.(θ)) # evaluate values+derivatives of lgP(log10.(θ)) and store the results in Pres
lgPs = DiffResults.value(Pres) # extract values
dlgP_dlgθs = DiffResults.jacobian(Pres) # extract derivatives

p1 = plot(
   log10.(ks/u"1/Mpc"), lgPs;
   ylabel = "lg(P/Mpc³)", label = nothing
)
p2 = plot(
   log10.(ks/u"1/Mpc"), dlgP_dlgθs;
   xlabel = "lg(k/Mpc⁻¹)", ylabel = "∂ lg(P) / ∂ lg(θᵢ)",
   labels = "θᵢ=" .* ["Tγ0" "Ωc0" "Ωb0" "Neff" "h" "YHe" "mh" "ln(10¹⁰As)" "ns"]
)
plot(p1, p2, layout=(2, 1), size = (600, 600))
Example block output

General approach

The technique shown here can be used to calculate the derivative of any SymBoltz.jl output quantity:

  1. Write a wrapper function output(input) that calculates the desired output quantities from the desired input quantities.
  2. Use ForwardDiff.derivative(output, input) (scalar-to-scalar), ForwardDiff.gradient(output, input) (vector-to-scalar) or ForwardDiff.jacobian(output, input) (vector-to-vector) to evaluate the derivative of output at the values input. Or use the similar functions in DiffResults to calculate the value and derivatives simultaneously.