GitHub

Functions

This page contains documentation for all exported functions.

Training

Call the main function bolfi! to run the BOLFI procedure, which sequentially queries the expensive blackbox simulator to learn the parameter posterior efficiently.

BOLFI.bolfi! — Function
bolfi!(::BolfiProblem; kwargs...)

Run the BOLFI method on the given BolfiProblem.

The bolfi! function is a wrapper for BOSS.bo!, which implements the underlying Bayesian optimization procedure.

Arguments

  • problem::BolfiProblem: Defines the inference problem, together with all model hyperparameters.

Keywords

  • model_fitter::BOSS.ModelFitter: Defines the algorithm used to estimate the model hyperparameters.
  • acq_maximizer::BOSS.AcquisitionMaximizer: Defines the algorithm used to maximize the acquisition function in order to select the next evaluation point in each iteration.
  • term_cond::Union{<:BOSS.TermCond, <:BolfiTermCond}: Defines the termination condition of the whole procedure.
  • options::BolfiOptions: Can be used to specify additional miscellaneous options.

References

BOSS.bo!, BolfiProblem, BolfiAcquisition, BOSS.ModelFitter, BOSS.AcquisitionMaximizer, BOSS.TermCond, BolfiTermCond, BolfiOptions

Examples

See 'https://soldasim.github.io/BOLFI.jl/stable/example_lfi' for example usage.

source

Call the function estimate_parameters! to fit the model hyperparameters according to the current dataset. One can also call bolfi! with term_cond = IterLimit(0) to fit the hyperparameters without running any simulations. This will additionally only refit the model if the dataset changed since the last parameter estimation. In contrast, calling estimate_parameters! will always re-run the parameter estimation.

BOSS.estimate_parameters! — Function
estimate_parameters!(::BossProblem, ::ModelFitter)

Estimate the model parameters & hyperparameters using the given model_fitter algorithm.

Keywords

  • options::BossOptions: Defines miscellaneous settings.
source
estimate_parameters!(::BolfiProblem, ::ModelFitter)

Estimate the hyperparameters of the model. Uses the provided ModelFitter to fit the hyperparameters of the model according to the data stored in the BolfiProblem.

Keywords

  • options::BolfiOptions: Defines miscellaneous settings.
source

Call the function maximize_acquisition to obtain a promising candidate for the next simulation.

BOSS.maximize_acquisition — Function
x = maximize_acquisition(::BossProblem, ::AcquisitionMaximizer)

Maximize the given acquisition function via the given acq_maximizer algorithm to find the optimal next evaluation point(s).

Keywords

  • options::BossOptions: Defines miscellaneous settings.
source
x = maximize_acquisition(::BolfiProblem, ::BolfiAcquisition, ::AcquisitionMaximizer)

Select parameters for the next simulation. Uses the provided AcquisitionMaximizer to maximize the BolfiAcquisition function and find the optimal candidate parameters.

Keywords

  • options::BolfiOptions: Defines miscellaneous settings.
source

Call the function eval_objective! to start a simulation run.

BOSS.eval_objective! — Function
eval_objective!(::BossProblem, x::AbstractVector{<:Real})

Evaluate the objective function and update the data.

Keywords

  • options::BossOptions: Defines miscellaneous settings.
source
eval_objective!(::BolfiProblem, x::AbstractVector{<:Real})

Evaluate the blackbox simulation for the given parameters x.

Keywords

  • options::BolfiOptions: Defines miscellaneous settings.
source

Parameter Posterior

This section contains function used to obtain the trained parameter posterior/likelihood approximations.

The approx_posterior function can be used to obtain the (un)normalized approximate posterior $p(\theta|y_o) \propto p(y_o|\theta) p(\theta)$ obtained by substituting the predictive means of the GPs directly as the discrepancies from the true observation.

BOLFI.approx_posterior — Function
approx_posterior(::BolfiProblem; kwargs...)

Return the MAP estimation of the unnormalized approximate posterior $\hat{p}(y_o|x) p(x)$ as a function of $x$.

If normalize=true, the resulting posterior is approximately normalized.

The posterior is approximated by directly substituting the predictive means of the GPs as the discrepancies from the true observation and ignoring both the uncertainty of the GPs due to a lack of data and due to the simulator evaluation noise.

By using approx_posterior or posterior_mean one controls, whether to integrate over the uncertainty in the discrepancy estimate. In addition to that, by providing a ModelFitter{MAP} or a ModelFitter{BI} to bolfi! one controls, whether to integrate over the uncertainty in the GP hyperparameters.

Keywords

  • normalize::Bool: If normalize is set to true, the evidence $\hat{p}(y_o)$is estimated by sampling and the normalized approximate posterior\hat{p}(y_o|x) p(x) / \hat{p}(y_o)` is returned instead of the unnormalized one.
  • xs::Union{Nothing, <:AbstractMatrix{<:Real}}: Can be used to provide a pre-sampled set of samples from the parameter prior $p(x)$ as a column-wise matrix. Only has an effect if normalize == true.
  • samples::Int: Controls the number of samples used to estimate the evidence. Only has an effect if normalize == true and isnothing(xs).

See Also

posterior_mean, posterior_variance, approx_likelihood

source

The posterior_mean function can be used to obtain the expected value of the (un)normalized posterior $\mathbb{E}\left[p(\theta|y_o)\right] \propto \mathbb{E}\left[p(y_o|\theta)p(\theta)\right]$ obtained by analytically integrating over the uncertainty of the GPs and the simulator.

BOLFI.posterior_mean — Function
posterior_mean(::BolfiProblem; kwargs...)

Return the expectation of the unnormalized posterior $\mathbb{E}[\hat{p}(y_o|x) p(x)]$ as a function of $x$.

If normalize=true, the resulting expected posterior is approximately normalized.

The returned function maps parameters x to the expected posterior probability density value integrated over the uncertainty of the GPs due to a lack of data and due to the simulator evaluation noise.

By using approx_posterior or posterior_mean one controls, whether to integrate over the uncertainty in the discrepancy estimate. In addition to that, by providing a ModelFitter{MAP} or a ModelFitter{BI} to bolfi! one controls, whether to integrate over the uncertainty in the GP hyperparameters.

Keywords

  • normalize::Bool: If normalize is set to true, the evidence $\hat{p}(y_o)$is estimated by sampling and the normalized expected posterior\mathbb{E}[\hat{p}(y_o|x) p(x)]` is returned instead of the unnormalized one.
  • xs::Union{Nothing, <:AbstractMatrix{<:Real}}: Can be used to provide a pre-sampled set of samples from the parameter prior $p(x)$ as a column-wise matrix. Only has an effect if normalize == true.
  • samples::Int: Controls the number of samples used to estimate the evidence. Only has an effect if normalize == true and isnothing(xs).

See Also

approx_posterior, posterior_variance, likelihood_mean

source

The posterior_variance function can be used to obtain the variance of the (un)normalized posterior $\mathbb{V}\left[p(\theta|y_o)\right] \propto \mathbb{V}\left[p(y_o|\theta)p(\theta)\right]$ obtained by analytically integrating over the uncertainty of the GPs and the simulator.

BOLFI.posterior_variance — Function
posterior_variance(::BolfiProblem; kwargs...)

Return the variance of the unnormalized posterior $\mathbb{V}[\hat{p}(y_o|x) p(x)]$ as a function of $x$.

If normalize=true, the resulting posterior variance is approximately normalized.

The returned function maps parameters x to the variance of the posterior probability density value estimate caused by the uncertainty of the GPs due to a lack of data and due to the simulator evaluation noise.

By providing a ModelFitter{MAP} or a ModelFitter{BI} to bolfi! one controls, whether to compute the variance over the uncertainty in the GP hyperparameters as well.

Keywords

  • normalize::Bool: If normalize is set to true, the evidence $\hat{p}(y_o)$is estimated by sampling and the normalized posterior variance\mathbb{V}[\hat{p}(y_o|x) p(x) / \hat{p}(y_o)]` is returned instead of the unnormalized one.
  • xs::Union{Nothing, <:AbstractMatrix{<:Real}}: Can be used to provide a pre-sampled set of samples from the parameter prior $p(x)$ as a column-wise matrix. Only has an effect if normalize == true.
  • samples::Int: Controls the number of samples used to estimate the evidence. Only has an effect if normalize == true and isnothing(xs).

See Also

approx_posterior, posterior_mean, likelihood_variance

source

The approx_likelihood function can be used to obtain the approximate likelihood $p(y_o|\theta)$ obtained by substituting the predictive means of the GPs directly as the discrepancies from the true observation.

The likelihood_mean function can be used to obtain the expected value of the likelihood $\mathbb{E}\left[p(y_o|\theta)\right]$ obtained by analytically integrating over the uncertainty of the GPs and the simulator.

BOLFI.likelihood_mean — Function
likelihood_mean(::Likelihood, bolfi, gp_post)

Returns a function mapping $x$ to $\mathbb{E}[ \hat{p}(y_o|x) | GP ]$.

source

The likelihood_variance function can be used to obtain the variance of the likelihood $\mathbb{V}\left[p(y_o|\theta)\right]$ obtained by analytically integrating over the uncertainty of the GPs and the simulator.

BOLFI.likelihood_variance — Function
likelihood_variance(::BolfiProblem)

Return the variance of the likelihood approximation $\mathbb{V}[\hat{p}(y_o|x)]$ as a function of $x$.

The returned function maps parameters x to the variance of the likelihood probability density value estimate caused by the uncertainty of the GPs due to a lack of data and the uncertainty of the simulator due to the evaluation noise.

By providing a ModelFitter{MAP} or a ModelFitter{BI} to bolfi! one controls, whether to compute the variance over the uncertainty in the GP hyperparameters as well.

See Also

approx_likelihood, likelihood_mean, posterior_variance

source

The evidence function can be used to approximate the evidence $p(y_o)$ of a given posterior function by sampling. It is advisable to use this estimate only in low parameter dimensions, as it will require many samples to achieve reasonable precision on high-dimensional domains.

The evidence is the normalization constant needed to obtain the normalized posterior. The evidence function is used to normalize the posterior if one calls approx_posterior, posterior_mean, or posterior_variance with normalize=true.

BOLFI.evidence — Function
evidence(post, x_prior; kwargs...)

Return the estimated evidence $\hat{p}(y_o)$.

Arguments

  • post: A function ::AbstractVector{<:Real} -> ::Real representing the posterior $p(x|y_o)$.
  • x_prior: A multivariate distribution representing the prior $p(x)$.

Keywords

  • xs::Union{Nothing, <:AbstractMatrix{<:Real}}: Can be used to provide a pre-sampled set of samples from the x_prior as a column-wise matrix.
  • samples::Int: Controls the number of samples used to estimate the evidence. Only has an effect if isnothing(xs).
source

Sampling from the Posterior

The sample_approx_posterior, sample_posterior_mean, and sample_posterior functions can be used to obtain approximate samples from the trained parameter posterior.

BOLFI.sample_approx_posterior — Function
using Turing
xs = sample_approx_posterior(::BolfiProblem)
xs = sample_approx_posterior(::BolfiProblem, ::TuringOptions)

Sample from the approximate posterior (see approx_posterior).

The TuringOptions argument controls the hyperparameters of the sampling. It is an optional argument and defaults to TuringOptions() if not specified.

See Also

sample_posterior_mean, sample_posterior

source
BOLFI.sample_posterior_mean — Function
using Turing
xs = sample_posterior_mean(::BolfiProblem)
xs = sample_posterior_mean(::BolfiProblem, ::TuringOptions)

Sample from the expected posterior (see posterior_mean).

The TuringOptions argument controls the hyperparameters of the sampling. It is an optional argument and defaults to TuringOptions() if not specified.

See Also

sample_approx_posterior, sample_posterior

source
BOLFI.sample_posterior — Function
using Turing
xs = sample_posterior(logpost, bounds::AbstractBounds, options::TuringOptions)
xs = sample_posterior(loglike, prior::MultivariateDistribution, options::TuringOptions)

Sample from the learned posterior stored in problem.

Either provide the log-posterior (as a function) and the domain bounds. Or provide the log-likelihood (as a function) and the prior distribution.

The last options argument controls the hyperparameters of the sampling. It is an optional argument and defaults to TuringOptions() if not specified.

See Also

sample_approx_posterior, sample_posterior_mean

source

The sampling is performed via the Turing.jl package. The Turing.jl package is a quite heavy dependency, so it is not loaded by default. To sample from the posterior, one has to first load Turing.jl as using Turing, which will also compile the sample_posterior function.

Confidence Sets

This section contains function used to extract approximate confidence sets from the posterior. It is advised to use these approximations only with low-dimensional parameter domains, as they will require many samples to reach reasonable precision in high-dimensional domains.

The find_cutoff function can be used to estimate some confidence set of a given posterior function.

BOLFI.find_cutoff — Function
c = find_cutoff(target_pdf, xs, q)
c = find_cutoff(target_pdf, xs, ws, q)

Estimate the cutoff value c such that the set {x | post(x) >= c} contains q of the total probability mass.

The value c is estimated based on the provided samples xs sampled according to the target_pdf.

Alternatively, one can provide samples xs sampled according to some proposal_pdf with corresponding importance weights ws = target_pdf.(eachcol(xs)) ./ proposal_pdf.(eachcol(xs)).

See Also

approx_cutoff_area set_iou

source

The approx_cutoff_area function can be used to estimate the ratio of the area of a confidence set given by sum cutoff constant (perhaps found by find_cutoff) and the whole domain.

BOLFI.approx_cutoff_area — Function
V = approx_cutoff_area(target_pdf, xs, c)
V = approx_cutoff_area(target_pdf, xs, ws, c)

Approximate the ratio of the area where target_pdf(x) >= c relative to the whole support of target_pdf.

The are is estimated based on the provided samples xs sampled uniformly from the whole support of target_pdf.

Alternatively, one can provide samples xs sampled according to some proposal_pdf with corresponding importance weights ws = 1 ./ proposal_pdf.(eachcol(xs)).

See Also

find_cutoff set_iou

source

The set_iou function can be used to estimate the intersection-over-union (IoU) value between two sets.

BOLFI.set_iou — Function
iou = set_iou(in_A, in_B, x_prior, xs)

Approximate the intersection-over-union of two sets A and B.

The parameters in_A, in_B are binary arrays declaring which samples from xs fall into the sets A and B. The column-wise matrix xs contains the parameter samples. The samples have to be drawn from the common prior x_prior.

See Also

find_cutoff approx_cutoff_area

source

Plotting Posterior Marginals

The functions plot_marginals_int and plot_marginals_kde are provided to visualize the trained posterior. Both functions create a matrix of figures containing the approximate marginal posteriors of each pair of parameters and the individual marginals on the diagonal.

The function plot_marginals_int approximates the marginals by numerical integration whereas the function plot_marginals_kde approximates the marginals by kernel density estimation.

BOLFI.plot_marginals_int — Function
using CairoMakie
plot_marginals_int(::BolfiProblem; kwargs...)

Create a matrix of plots displaying the marginal posterior distribution of each pair of parameters with the individual marginals of each parameter on the diagonal.

Approximates the marginals by numerically integrating the marginal integrals over a generated latin hypercube grid of parameter samples. The plots are normalized according to the plotting grid.

Also provides an option to plot "marginals" of different functions by using the func and normalize keywords.

Kwargs

  • func::Function: Defines the function which is plotted. The plotted function f is defined as f = func(::BolfiProblem). Reasonable options for func include approx_posterior, posterior_mean, posterior_variance etc.
  • normalize::Bool: Specifies whether the plotted marginals are normalized. If normalize=false, the plotted values are simply averages over the random LHC grid. If normalize=true, the plotted values are additionally normalized sum to 1. Defaults to true.
  • lhc_grid_size::Int: The number of samples in the generate LHC grid. The higher the number, the more precise marginal plots.
  • plot_settings::PlotSettings: Settings for the plotting.
  • info::Bool: Set to false to disable prints.
  • display::Bool: Set to false to not display the figure. It is still returned.
source
BOLFI.plot_marginals_kde — Function
using CairoMakie, Turing
plot_marginals_kde(::BolfiProblem; kwargs...)

Create a matrix of plots displaying the marginal posterior distribution of each pair of parameters with the individual marginals of each parameter on the diagonal.

Approximates the marginals by kernel density estimation over parameter samples drawn by MCMC methods from the Turing.jl package. The plots are normalized according to the plotting grid.

One should experiment with different kernel length-scales to obtain a good approximation of the marginals. The kernel and length-scales are provided via the kernel and lengthscale keyword arguments.

Kwargs

  • turing_options::TuringOptions: Settings for the MCMC sampling.
  • kernel::Kernel: The kernel used in the KDE.
  • lengthscale::Union{<:Real, <:AbstractVector{<:Real}}: The lengthscale for the kernel used in the KDE. Either provide a single length-scale used for all parameter dimensions as a real number, or provide individual length-scales for each parameter dimension as a vector of real numbers.
  • plot_settings::PlotSettings: Settings for the plotting.
  • info::Bool: Set to false to disable prints.
  • display::Bool: Set to false to not display the figure. It is still returned.
source