Can it support automatic differentiation like Ceres-Solver?

[1879615351]

Can it support automatic differentiation like Ceres-Solver?

[stefan-k]

Unfortunately no. You may be able to utilize other AD crates, but I have no experience with those and therefore cannot provide any details.

[g-bauer]

You can use num-dual to do just that. I just took a quick look at Ceres but it seems like they also use dual numbers. Take the lbfgs_nalgebra example and replace the test function with a dual number implementation:

// additional imports - I'm using a static vector instead of a dynamic vector but both work
use nalgebra::SVector;
use num_dual::{gradient,DualNum};

struct Rosenbrock {}

/// Rosenbrock function with a = 1 and b = 100, that uses dual numbers as inputs.
/// Allows for arbitrary order of derivatives.
///
/// DualNum<f64> means that the inner type is a f64. 
/// You can perform the most common arithmetic operations with DualNum.
/// When you use functions like gradient (see below), you don't have to know how these numbers work. 
fn rosenbrock_dual<D: DualNum<f64>>(v: &SVector<D, 2>) -> D {
    // note that the lefthand site of a binary arithmetic operation has to have
    // the resulting type - i.e. we have to put v[0] on the lhs instead of writing
    // 1.0 - v[0].clone()
    (-v[0].clone() + 1.0).powi(2) + (v[1].clone() - v[0].powi(2)).powi(2) * 100.0
}

impl CostFunction for Rosenbrock {
    type Param = SVector<f64, 2>;
    type Output = f64;

    fn cost(&self, p: &Self::Param) -> Result<Self::Output, Error> {
        Ok(rosenbrock_dual(p))
    }
}

impl Gradient for Rosenbrock {
    type Param = SVector<f64, 2>;
    type Gradient = SVector<f64, 2>;

    fn gradient(&self, p: &Self::Param) -> Result<Self::Gradient, Error> {
        // gradient returns value and gradient
        // internally, num-dual will generate the correct dual number for the specific case.
        let (_, g) = gradient(|v| rosenbrock_dual(&v), *p);
        Ok(g)
    }
}

fn run() -> Result<(), Error> {
    // Define cost function
    let cost = Rosenbrock {};

    // Define initial parameter vector
    let init_param: SVector<f64, 2> = SVector::from([-1.2, 1.0]);
    
    // rest is the same as in the example

num-dual also provides a hessian function that can be used for solvers that need second derivatives. Hope that helps.