BossProblem(; kwargs...)

Defines the whole optimization problem for the BOSS algorithm.

Problem Definition

There is some (noisy) blackbox function y = f(x) = f_true(x) + ϵ where ϵ ~ Normal.

We wish to find x ∈ domain such that fitness(f(x)) is maximized while satisfying the constraints f(x) <= y_max.

Keywords

The following keywords correspond to all fields of the BossProblem type.

The keywords marked by "(*)" are required. Note that at least a single initial data point must be provided to initialize the BossProblem.

• (*) f::Union{Function, Missing}: The objective blackbox function.
• (*) domain::Domain: The Domain of the input x.
• y_max::AbstractVector{<:Real}: The constraints on the output y.
• (*) acquisition::AcquisitionFunction: The acquisition function used to select the next evaluation point in each iteration. Usually contains the fitness function.
• (*) model::SurrogateModel: The SurrogateModel.
• params::Union{FittedParams, Nothing}: The fitted model parameters. Defaults to nothing.
• (*) data::ExperimentData: The data obtained by evaluating the objective function.
• consistent::Bool: True iff the model_params have been fitted using the current data. Is set to consistent = false after updating the dataset, and to consistent = true after re-fitting the parameters. Defaults to constistent = false.

See Also

bo!

Domain(; kwargs...)

Describes the optimization domain.

Keywords

The following keywords correspond to all fields of the Domain type.

The keywords marked by "(*)" are required.

• (*) bounds::AbstractBounds: The basic box-constraints on x. This field is mandatory.
• discrete::AbstractVector{Bool}: Can be used to designate some dimensions of the domain as discrete.
• cons::Union{Nothing, Function}: Used to define arbitrary nonlinear constraints on x. Feasible points x must satisfy all(cons(x) .> 0.). An appropriate acquisition maximizer which can handle nonlinear constraints must be used if cons is provided. (See AcquisitionMaximizer.)

const AbstractBounds = Tuple{<:AbstractVector{<:Real}, <:AbstractVector{<:Real}}

Defines box constraints.

Example: ([0, 0], [1, 1]) isa AbstractBounds

AcquisitionFunction

Specifies the acquisition function describing the "quality" of a potential next evaluation point.

Defining custom acquisition function

To define a custom acquisition function, define a new subtype of AcquisitionFunction.

• struct CustomAcq <: AcquisitionFunction ... end

All acquisition functions should implement: construct_acquisition(::CustomAcq, ::BossProblem, ::BossOptions) -> (x -> ::Real)

Acquisition functions may implement:

• get_fitness(::CustomAcq) -> (y -> ::Real): Usually will return a callable instance of Fitness.

See the docs of the individual functions for more information.

See Also

construct_acquisition, ExpectedImprovement

ExpectedImprovement(; kwargs...)

The expected improvement (EI) acquisition function.

Measures the quality of a potential evaluation point x as the expected improvement in best-so-far achieved fitness by evaluating the objective function at y = f(x).

In case of constrained problems, the expected improvement is additionally weighted by the probability of feasibility of y. I.e. the probability that all(cons(y) .> 0.).

If the problem is constrained on y and no feasible point has been observed yet, then the probability of feasibility alone is returned as the acquisition function.

Rather than using the actual evaluations (xᵢ,yᵢ) from the dataset, the best-so-far achieved fitness is calculated as the maximum fitness among the means ŷᵢ of the posterior predictive distribution of the model evaluated at xᵢ. This is a simple way to handle evaluation noise which may not be suitable for problems with substantial noise.

In case Bayesian Inference of model parameters is used, the expectation of the expected improvement over the model parameter samples is calculated.

Keywords

• fitness::Fitness: The fitness function mapping the output y to the real-valued score.
• ϵ_samples::Int: Controls how many samples are used to approximate EI. The ϵ_samples keyword is ignored unless MAP model fitter and NonlinFitness are used! In case of BI model fitter, the number of samples is instead set equal to the number of posterior samples. In case of LinearFitness, the expected improvement can be calculated analytically.
• cons_safe::Bool: If set to true, the acquisition function acq(x) is made 'constraint-safe' by checking the bounds and constraints during each evaluation. Set cons_safe to true if the evaluation of the model at exterior points may cause errors or nonsensical values. You may set cons_safe to false if the evaluation of the model at exterior points can provide useful information to the acquisition maximizer and does not cause errors. Defaults to true.

Fitness

An abstract type for a fitness function measuring the quality of an output y of the objective function.

Fitness is used by (most) AcquisitionFunctions to determine promising points for future evaluations.

All fitness functions should implement:

• (::CustomFitness)(y::AbstractVector{<:Real}) -> fitness::Real

See also: LinFitness, NonlinFitness, AcquisitionFunction

LinFitness(coefs::AbstractVector{<:Real})

Used to define a linear fitness function measuring the quality of an output y of the objective function.

May provide better performance than the more general NonlinFitness as some acquisition functions can be calculated analytically with linear fitness functions whereas this may not be possible with a nonlinear fitness function.

See also: NonlinFitness

Example

A fitness function f(y) = y[1] + a * y[2] + b * y[3] can be defined as:

julia> LinFitness([1., a, b])

NonlinFitness(fitness::Function)

Used to define a general nonlinear fitness function measuring the quality of an output y of the objective function.

If your fitness function is linear, use LinFitness which may provide better performance.

See also: LinFitness

Example

julia> NonlinFitness(y -> cos(y[1]) + sin(y[2]))

SurrogateModel

An abstract type for a surrogate model approximating the objective function.

Defining Custom Surrogate Model

To define a custom surrogate model, define a new subtype struct CustomModel <: SurrogateModel ... end as well as the other structures and methods described below.

The inputs in square brackets [...] are optional and can be used to provide additional data. It is prefferable to define the methods without the optional inputs if possible.

See the docstrings of the individual functions for more information.

Model Posterior Methods

Each model should define at least one of the following posterior constructors:

• model_posterior(::SurrogateModel, ::ModelParams, ::ExperimentData) -> ::ModelPosterior
• model_posterior_slice(::SurrogateModel, ::ModelParams, ::ExperimentData, slice::Int) -> ::ModelPosteriorSlice

and will usually implement the corresponding posterior type(s):

• struct CustomPosterior <: ModelPosterior{CustomModel} ... end
• struct CustomPosteriorSlice <: ModelPosteriorSlice{CustomModel} ... end

However, the model may reuse a posterior type defined for a different model.

Defining both ModelPosterior and ModelPosteriorSlice is also possible, and can be used to provide a more efficient implementation of the posterior.

Additionally, the API described in the docstring(s) of the ModelPosterior or/and ModelPosteriorSlice type(s) must be implemented for new posterior types.

Model Parameters Methods

Each model should define a new type:

• struct CustomParams <: ModelParams{CustomModel} ... end

Each model should implement the following methods used for parameter estimation:

• data_loglike(::SurrogateModel, ::ExperimentData) -> (::ModelParams -> ::Real)
• params_loglike(::SurrogateModel, [::ExperimentData]) -> (::ModelParams -> ::Real)
• _params_sampler(::SurrogateModel, [::ExperimentData]) -> (::AbstractRNG -> ::ModelParams)
• vectorizer(::SurrogateModel, [::ExperimentData]) -> (vectorize, devectorize) where vectorize(::ModelParams) -> ::AbstractVector{<:Real} and devectorize(::ModelParams, ::AbstractVector{<:Real}) -> ::ModelParams
• bijector(::SurrogateModel, [::ExperimentData]) -> ::Bijectors.Transform

Additionally, the following methods are provided and need not be implemented:

• model_loglike(::SurrogateModel, ::ExperimentData) -> (::ModelParams -> ::Real)
• params_sampler(::SurrogateModel, ::ExperimentData) -> ([::AbstractRNG] -> ::ModelParams)

Utility Methods

Models may implement:

• make_discrete(model::SurrogateModel, discrete::AbstractVector{Bool}) -> discrete_model::SurrogateModel
• sliceable(::SurrogateModel) = true (defaults to false)

If sliceable(::SurrogateModel) == true, then the model should additionally implement:

• slice(model::SurrogateModel, slice::Int) -> model_slice::SurrogateModel
• slice(params::ModelParams, slice::Int) -> params_slice::ModelParams
• join_slices(slices::AbstractVector{ModelParams}) -> params::ModelParams

Defining the SurrogateModel as sliceable allows for significantly more efficient parameter estimation, but is generally not possible for all models.

SurrogateModels implementing model_posterior_slice will usually be sliceable, whereas models implementing model_posterior will not, but the API does not require this.

See Also

LinearModel, NonlinearModel, GaussianProcess, Semiparametric

Parametric{N}

An abstract type for parametric surrogate models.

The model function can be reconstructed using the following functions:

• (::Parametric)() -> ((x, θ) -> y)
• (::Parametric)(θ::AbstractVector{<:Real}) -> (x -> y)
• (::Parametric)(x::AbstractVector{<:Real}, θ::AbstractVector{<:Real}) -> y

The parametric type N <: Union{Nothing, NoiseStdPriors} determines whether the model is deterministic or probabilistic.

A deterministic version of a Parametric model has N = nothing, does not implement the SurrogateModel API and cannot be used as a standalone model. It is mainly used as a part of the Semiparametric model.

A probabilistic version of a Parametric model has defined noise_std_priors, implements the whole SurrogateModel API, and can be used as a standalone model.

See also: LinearModel, NonlinearModel

LinearModel(; kwargs...)

A parametric surrogate model linear in its parameters.

This model definition will provide better performance than the more general 'NonlinearModel' in the future. This feature is not implemented yet so it is equivalent to using NonlinearModel for now.

The linear model is defined as

     ϕs = lift(x)
     y = [θs[i]' * ϕs[i] for i in 1:m]

where

     x = [x₁, ..., xₙ]
     y = [y₁, ..., yₘ]
     θs = [θ₁, ..., θₘ], θᵢ = [θᵢ₁, ..., θᵢₚ]
     ϕs = [ϕ₁, ..., ϕₘ], ϕᵢ = [ϕᵢ₁, ..., ϕᵢₚ]

and $n, m, p ∈ R$.

Keywords

• lift::Function: Defines the lift function (::Vector{<:Real}) -> (::Vector{Vector{<:Real}}) according to the definition above.
• theta_priors::ThetaPriors: The prior distributions for the parameters [θ₁₁, ..., θ₁ₚ, ..., θₘ₁, ..., θₘₚ] according to the definition above.
• discrete::Union{Nothing, AbstractVector{Bool}}: A vector of booleans indicating which dimensions of x are discrete. If discrete = nothing, all dimensions are continuous. Defaults to nothing.
• noise_std_priors::Union{Nothing, NoiseStdPriors}: The prior distributions of the noise standard deviations of each y dimension. If the model is used by itself, the noise_std_priors must be defined. If the model is used as a part of the Semiparametric model, the noise_std_priors must be left undefined, as the evaluation noise is modeled by the GaussianProcess in that case.

NonlinearModel(; kwargs...)

A parametric surrogate model.

If your model is linear, you can use LinearModel which will provide better performance in the future. (Not yet implemented.)

Define the model as y = predict(x, θ) where θ are the model parameters.

Keywords

• predict::Function: The predict function according to the definition above.
• theta_priors::ThetaPriors: The prior distributions for the model parameters. function during optimization. Defaults to nothing meaning all parameters are real-valued.
• discrete::Union{Nothing, AbstractVector{Bool}}: A vector of booleans indicating which dimensions of x are discrete. If discrete = nothing, all dimensions are continuous. Defaults to nothing.
• noise_std_priors::Union{Nothing, NoiseStdPriors}: The prior distributions of the noise standard deviations of each y dimension. If the model is used by itself, the noise_std_priors must be defined. If the model is used as a part of the Semiparametric model, the noise_std_priors must be left undefined, as the evaluation noise is modeled by the GaussianProcess in that case.

ParametricParams(θ, σ)

The parameters of the Parametric model.

Parameters

• θ::AbstractVector{<:Real}: The parameters of the Parametric model.
• σ::AbstractVector{<:Real}: The noise standard deviations.

Nonparametric

An alias for GaussianProcess.

GaussianProcess(; kwargs...)

A Gaussian Process surrogate model. Each output dimension is modeled by a separate independent process.

Keywords

• mean::Union{Nothing, AbstractVector{<:Real}, Function}: Used as the mean function for the GP. Defaults to nothing equivalent to x -> zeros(y_dim).
• kernel::Kernel: The kernel used in the GP. Defaults to the Matern32Kernel().
• lengthscale_priors::LengthscalePriors: The prior distributions for the length scales of the GP. The lengthscale_priors should be a vector of y_dim x_dim-variate distributions where x_dim and y_dim are the dimensions of the input and output of the model respectively.
• amplitude_priors::AmplitudePriors: The prior distributions for the amplitude hyperparameters of the GP. The amplitude_priors should be a vector of y_dim univariate distributions.
• noise_std_priors::NoiseStdPriors: The prior distributions of the noise standard deviations of each y dimension.

GaussianProcessParams(λ, α, σ)

The parameters of the GaussianProcess model.

Parameters

• λ::AbstractMatrix{<:Real}: The length scales of the GP.
• α::AbstractVector{<:Real}: The amplitudes of the GP.
• σ::AbstractVector{<:Real}: The noise standard deviations.

Semiparametric(; kwargs...)

A semiparametric surrogate model (a combination of a Parametric model and a GaussianProcess).

The parametric model is used as the mean of the Gaussian Process and the evaluation noise is modeled by the Gaussian Process. All parameters of the models are estimated simultaneously.

Keywords

• parametric::Parametric: The parametric model used as the GP mean function.
• nonparametric::Nonparametric{Nothing}: The outer GP model without mean.

Note that the parametric model should be defined without noise priors, and the nonparametric model should be defined without mean function.

SemiparametricParams(θ, λ, α, σ)

The parameters of the [Semiparametric]@ref model.

Parameters

• θ::AbstractVector{<:Real}: The parameters of the parametric model.
• λ::AbstractMatrix{<:Real}: The length scales of the GP.
• α::AbstractVector{<:Real}: The amplitudes of the GP.
• σ::AbstractVector{<:Real}: The noise standard deviations.

AbstractModelPosterior{<:SurrogateModel}

An abstract model posterior. The subtypes include ModelPosterior and ModelPosteriorSlice.

ModelPosterior{M<:SurrogateModel}

Contains precomputed quantities for the evaluation of the predictive posterior of the SurrogateModel M.

Each subtype of ModelPosterior should implement:

• mean(::ModelPosterior, ::AbstractVector{<:Real}) -> ::AbstractVector{<:Real}
• mean(::ModelPosterior, ::AbstractMatrix{<:Real}) -> ::AbstractMatrix{<:Real}
• var(::ModelPosterior, ::AbstractVector{<:Real}) -> ::AbstractVector{<:Real}
• var(::ModelPosterior, ::AbstractMatrix{<:Real}) -> ::AbstractMatrix{<:Real}
• cov(::ModelPosterior, ::AbstractMatrix{<:Real}) -> ::AbstractArray{<:Real, 3}

and may implement corresponding methods:

• mean_and_var(::ModelPosterior, ::AbstractVector{<:Real}) -> ::Tuple{...}
• mean_and_var(::ModelPosterior, ::AbstractMatrix{<:Real}) -> ::Tuple{...}
• mean_and_cov(::ModelPosterior, ::AbstractMatrix{<:Real}) -> ::Tuple{...}

Additionally, the following methods are provided and need not be implemented:

• std(::ModelPosterior, ::AbstractVector{<:Real}) -> ::AbstractVector{<:Real}
• std(::ModelPosterior, ::AbstractMatrix{<:Real}) -> ::AbstractMatrix{<:Real}
• mean_and_std(::ModelPosterior, ::AbstractVector{<:Real}) -> ::Tuple{...}
• mean_and_std(::ModelPosterior, ::AbstractMatrix{<:Real}) -> ::Tuple{...}
• average_mean(::AbstractVector{<:ModelPosterior}, ::AbstractVector{<:Real})
• average_mean(::AbstractVector{<:ModelPosterior}, ::AbstractMatrix{<:Real})

See SurrogateModel for more information.

See also: ModelPosteriorSlice

ModelPosteriorSlice{M<:SurrogateModel}

Contains precomputed quantities for the evaluation of the predictive posterior of a single output dimension of the SurrogateModel M.

Each subtype of ModelPosteriorSlice should implement:

• mean(::ModelPosteriorSlice, ::AbstractVector{<:Real}) -> ::Real
• mean(::ModelPosteriorSlice, ::AbstractMatrix{<:Real}) -> ::AbstractVector{<:Real}
• var(::ModelPosteriorSlice, ::AbstractVector{<:Real}) -> ::Real
• var(::ModelPosteriorSlice, ::AbstractMatrix{<:Real}) -> ::AbstractVector{<:Real}
• cov(::ModelPosteriorSlice, ::AbstractMatrix{<:Real}) -> ::AbstractMatrix{<:Real}

and may implement corresponding methods:

• mean_and_var(::ModelPosteriorSlice, ::AbstractVector{<:Real}) -> ::Tuple{...}
• mean_and_var(::ModelPosteriorSlice, ::AbstractMatrix{<:Real}) -> ::Tuple{...}
• mean_and_cov(::ModelPosteriorSlice, ::AbstractMatrix{<:Real}) -> ::Tuple{...}

Additionally, the following methods are provided and need not be implemented:

• std(::ModelPosteriorSlice, ::AbstractVector{<:Real}) -> ::Real
• std(::ModelPosteriorSlice, ::AbstractMatrix{<:Real}) -> ::AbstractVector{<:Real}
• mean_and_std(::ModelPosteriorSlice, ::AbstractVector{<:Real}) -> ::Tuple{...}
• mean_and_std(::ModelPosteriorSlice, ::AbstractMatrix{<:Real}) -> ::Tuple{...}
• average_mean(::AbstractVector{<:ModelPosteriorSlice}, ::AbstractVector{<:Real})
• average_mean(::AbstractVector{<:ModelPosteriorSlice}, ::AbstractMatrix{<:Real})

See SurrogateModel for more information.

See also: ModelPosterior

ModelParams{M<:SurrogateModel}

Contains all parameters of the SurrogateModel M.

See SurrogateModel for more information.

FittedParams{M<:SurrogateModel}

The subtypes of FittedParams contain ModelParams fitted to the data via different methods.

There are two abstract subtypes of FittedParams:

• UniFittedParams: Contains a single ModelParams instance fitted to the data.
• MultiFittedParams: Contains multiple ModelParams samples sampled according to the data.

The contained ModelParams can be obtained via the get_params(::FittedParams) function, which return either a single ModelParams object or a vector of ModelParams objects.

All subtypes of UniFittedParams implement:

• get_params(::FittedParams) -> ::ModelParams
• slice(::FittedParams, idx::Int) -> ::FittedParams

All subtypes of MultiFittedParams implement:

• get_params(::FittedParams) -> ::Vector{<:ModelParams}
• slice(::FittedParams, idx::Int) -> ::FittedParams

See Also

MAPParams, BIParams, RandomParams.

MAPParams{M<:SurrogateModel}

ModelParams estimated via MAP estimation.

Keywords

• params::ModelParams{M}: The fitted model parameters.
• loglike::Float64: The log likelihood of the fitted parameters.

BIParams{M<:SurrogateModel, P<:ModelParams{M}}

Contains ModelParams samples obtained via (approximate) Bayesian inference.

The individual ModelParams samples can be obtained by iterating over the BIParams object.

Keywords

• samples::Vector{P}: A vector of the individual model parameter samples.

RandomParams{M<:SurrogateModel}

A single random ModelParams sample from the prior.

Keywords

• params::ModelParams{M}: The random model parameters.

FixedParams{M<:SurrogateModel}

Fixed ModelParams values for a given SurrogateModel.

Keywords

• params::ModelParams{M}: The parameter values.

ExperimentData(X, Y)

Stores all the data collected during the optimization.

At least one initial datapoint has to be provided (purely for implementation reasons). One can for example use LatinHypercubeSampling.jl to obtain a small intial grid, or provide a single random initial datapoint.

Keywords

• X::AbstractMatrix{<:Real}: Contains the objective function inputs as columns.
• Y::AbstractMatrix{<:Real}: Contains the objective function outputs as columns.

ModelFitter{T<:FittedParams}

Specifies the library/algorithm used for model parameter estimation. The parametric type T specifies the subtype of FittedParams returned by the model fitter.

Defining Custom Model Fitter

Define a custom model fitter algorithm by defining a new subtype of ModelFitter.

Example: struct CustomFitter <: ModelFitter{MAPParams} ... end

All model fitters should implement: `estimateparameters(modelfitter::CustomFitter, problem::BossProblem, options::BossOptions; return_all::Bool) -> ::FittedParams

See Also

OptimizationMAP, TuringBI

OptimizationMAP(; kwargs...)

Finds the MAP estimate of the model parameters and hyperparameters using the Optimization.jl package.

To use this model fitter, first add one of the Optimization.jl packages (e.g. OptimizationPRIMA) to load some optimization algorithms which are passed to the OptimizationMAP constructor.

Keywords

• algorithm::Any: Defines the optimization algorithm.
• multistart::Union{Int, AbstractVector{<:ModelParams}}: The number of optimization restarts, or a vector of ModelParams containing initial (hyper)parameter values for the optimization runs.
• parallel::Bool: If parallel=true then the individual restarts are run in parallel.
• static_schedule::Bool: If static_schedule=true then the :static schedule is used for parallelization. This is makes the parallel tasks sticky (non-migrating), but can decrease performance.
• autodiff::Union{SciMLBase.AbstractADType, Nothing}:: The automatic differentiation module passed to Optimization.OptimizationFunction.
• kwargs::Base.Pairs{Symbol, <:Any}: Other kwargs are passed to the optimization algorithm.

TuringBI(; kwargs...)

Samples the model parameters and hyperparameters using the Turing.jl package.

To use this model fitter, first add the Turing.jl package.

Keywords

• sampler::Any: The sampling algorithm used to draw the samples.
• warmup::Int: The amount of initial unused 'warmup' samples in each chain.
• samples_in_chain::Int: The amount of samples used from each chain.
• chain_count::Int: The amount of independent chains sampled.
• leap_size: Every leap_size-th sample is used from each chain. (To avoid correlated samples.)
• parallel: If parallel=true then the chains are sampled in parallel.

Sampling Process

In each sampled chain;

• The first warmup samples are discarded.
• From the following leap_size * samples_in_chain samples each leap_size-th is kept.

Then the samples from all chains are concatenated and returned.

Total drawn samples: 'chaincount * (warmup + leapsize * samplesinchain)' Total returned samples: 'chaincount * samplesin_chain'

SamplingMAP()

Optimizes the model parameters by sampling them from their prior distributions and selecting the best sample in sense of MAP.

Keywords

• samples::Int: The number of drawn samples.
• parallel::Bool: The sampling is performed in parallel if parallel=true.

RandomFitter()

Returns random model parameters sampled from their respective priors.

Can be useful with RandomSelectAM to avoid unnecessary model parameter estimations.

SampleOptMAP(; kwargs...)
SampleOptMAP(::SamplingMAP, ::OptimizationMAP)

Combines SamplingMAP and OptimizationMAP to first sample many parameter samples from the prior, and subsequently start multiple optimization runs initialized from the best samples.

Keywords

• samples::Int: The number of drawn samples.
• algorithm::Any: Defines the optimization algorithm.
• multistart::Int: The number of optimization restarts.
• parallel::Bool: If parallel=true, then both the sampling and the optimization are performed in parallel.

AcquisitionMaximizer

Specifies the library/algorithm used for acquisition function optimization.

Defining Custom Acquisition Maximizer

To define a custom acquisition maximizer, define a new subtype of AcquisitionMaximizer.

• struct CustomAlg <: AcquisitionMaximizer ... end

All acquisition maximizers should implement: maximize_acquisition(acq_maximizer::CustomAlg, problem::BossProblem, options::BossOptions) -> (x, val).

This method should return a tuple (x, val). The returned vector x is the point of the input domain which maximizes the given acquisition function acq (as a vector), or a batch of points (as a column-wise matrix). The returned val is the acquisition value acq(x), or the values acq.(eachcol(x)) for each point of the batch, or nothing (depending on the acquisition maximizer implementation).

See also: OptimizationAM

OptimizationAM(; kwargs...)

Maximizes the acquisition function using the Optimization.jl library.

Can handle constraints on x if according optimization algorithm is selected.

Keywords

• algorithm::Any: Defines the optimization algorithm.
• multistart::Union{Int, AbstractMatrix{<:Real}}: The number of optimization restarts, or a matrix of optimization intial points as columns.
• parallel::Bool: If parallel=true then the individual restarts are run in parallel.
• static_schedule::Bool: If static_schedule=true then the :static schedule is used for parallelization. This is makes the parallel tasks sticky (non-migrating), but can decrease performance.
• autodiff:SciMLBase.AbstractADType:: The automatic differentiation module passed to Optimization.OptimizationFunction.
• kwargs...: Other kwargs are passed to the optimization algorithm.

GridAM(kwargs...)

Maximizes the acquisition function by checking a fine grid of points from the domain.

Extremely simple optimizer which can be used for simple problems or for debugging. Not suitable for problems with high dimensional domain.

Can be used with constraints on x.

Keywords

• problem::BossProblem: Provide your defined optimization problem.
• steps::Vector{Float64}: Defines the size of the grid gaps in each x dimension.
• parallel::Bool: If parallel=true, the optimization is parallelized. Defaults to true.
• shuffle::Bool: If shuffle=true, the grid points are shuffled before each optimization. Defaults to true.

SamplingAM(; kwargs...)

Optimizes the acquisition function by sampling candidates from the user-provided prior, and returning the sample with the highest acquisition value.

Keywords

• x_prior::MultivariateDistribution: The prior over the input domain used to sample candidates.
• samples::Int: The number of samples to be drawn and evaluated.
• parallel::Bool: If parallel=true then the sampling is parallelized. Defaults to true.

RandomAM()

Selects a random interior point instead of maximizing the acquisition function. Can be used for method comparison.

Can handle constraints on x, but does so by generating random points in the box domain until a point satisfying the constraints is found. Therefore it can take a long time or even get stuck if the constraints are very tight.

GivenPointAM(x::Vector{...})

A dummy acquisition maximizer that always returns predefined point x.

See Also

GivenSequenceAM,

GivenSequenceAM(X::Matrix{...})
GivenSequenceAM(X::Vector{Vector{...}})

A dummy acquisition maximizer that returns the predefined sequence of points in the given order. The maximizer throws an error if it runs out of points in the sequence.

See Also

GivenPointAM,

SampleOptAM(; kwargs...)

Optimizes the acquisition function by first sampling candidates from the user-provided prior, and then running multiple optimization runs initiated from the samples with the highest acquisition values.

Keywords

• x_prior::MultivariateDistribution: The prior over the input domain used to sample candidates.
• samples::Int: The number of samples to be drawn and evaluated.
• algorithm::Any: Defines the optimization algorithm.
• multistart::Int: The number of optimization restarts.
• parallel::Bool: If parallel=true, both the sampling and individual optimization runs are performed in parallel.
• autodiff:SciMLBase.AbstractADType:: The automatic differentiation module passed to Optimization.OptimizationFunction.
• kwargs...: Other kwargs are passed to the optimization algorithm.

SequentialBatchAM(::AcquisitionMaximizer, ::Int)
SequentialBatchAM(; am, batch_size)

Provides multiple candidates for batched objective function evaluation.

Selects the candidates sequentially by iterating the following steps:

• 1. Use the 'inner' acquisition maximizer to select a candidate x.
• 1. Extend the dataset with a 'speculative' new data point
  created by taking the candidate x and the posterior predictive mean of the surrogate ŷ.
• 1. If batch_size candidates have been selected, return them.
  Otherwise, goto step 1).

Keywords

• am::AcquisitionMaximizer: The inner acquisition maximizer.
• batch_size::Int: The number of candidates to be selected.

TermCond

Specifies the termination condition of the whole BOSS algorithm. Inherit this type to define a custom termination condition.

Example: struct CustomCond <: TermCond ... end

All termination conditions should implement: (cond::CustomCond)(problem::BossProblem)

This method should return true to keep the optimization running and return false once the optimization is to be terminated.

See also: IterLimit

IterLimit(iter_max::Int)

Terminates the BOSS algorithm after predefined number of iterations.

See also: bo!

NoLimit()

Never terminates.

BossOptions(; kwargs...)

Stores miscellaneous settings of the BOSS algorithm.

Keywords

• info::Bool: Setting info=false silences the BOSS algorithm.
• debug::Bool: Set debug=true to print stactraces of caught optimization errors.
• parallel_evals::Symbol: Possible values: :serial, :parallel, :distributed. Defaults to :parallel. Determines whether to run multiple objective function evaluations within one batch in serial, parallel, or distributed fashion. (Only has an effect if batching AM is used.)
• callback::BossCallback: If provided, callback(::BossProblem; kwargs...) will be called before the BO procedure starts and after every iteration.

See also: bo!

BossCallback

If a BossCallback is provided to BossOptions, the callback is called once before the BO procedure starts, and after each iteration.

All callbacks should implement:

• (::CustomCallback)(::BossProblem; ::ModelFitter, ::AcquisitionMaximizer, ::TermCond, ::BossOptions, first::Bool, )

The kwarg first is true only on the first callback before the BO procedure starts.

See PlotCallback for an example usage of a callback for plotting.

NoCallback()

Does nothing.

PlotOptions(Plots; kwargs...)

If PlotOptions is passed to BossOptions as callback, the state of the optimization problem is plotted in each iteration. Only works with one-dimensional x domains but supports multi-dimensional y.

Arguments

• Plots::Module: Evaluate using Plots and pass the Plots module to PlotsOptions.

Keywords

• f_true::Union{Nothing, Function}: The true objective function to be plotted.
• points::Int: The number of points in each plotted function.
• xaxis::Symbol: Used to change the x axis scale (:identity, :log).
• yaxis::Symbol: Used to change the y axis scale (:identity, :log).
• title::String: The plot title.