Relax Element Trait constraints

[skewballfox]

Feature description

Somehow Relax the constraints for the Element trait to make it possible to define elements outside of burn-tensor

Feature motivation

This came up primarily in #3608 in the process of trying to move complex element definitions to the crate burn-complex. The current major blocker is ElementConversion, which has explicit function calls per primitive type which relys on ToElement which has explict calls per type. This made it impossible to define an Element outside of burn-tensor. ElementComparison I believe depends on ElementConversion.

This currently blocks implementing basicOps (and thus quite a few other traits) on anything outside of burn tensor if the underlying dtype is not one of the already defined and supported primitives.

(Optional) Suggest a Solution

completely spitballing:

• remove the ElementConversion, ElementComparision constraint from element. Make the basic ops that require members of a new trait (maybe InteropOps? lol). Really just shuffles the problem around though.
• find a way to rework the functions in element conversion to more like the signature of mem::transmute (i.e. convert<E1,E2>(E1)-> E2 where both are Elements, but elements are not required to cover every implementor. The core primitives will cover all the other primitives in burn-tensor. You can have 2 variants, convert which is fast but unsafe in that it will panic if the required conversion method doesn't exists, and then try_convert<E1,E2>(E1)-> Result<E2> that will throw an error. This might be tricky converting external elements from burn-tensor elements since that would probably violate the orphan rule
• get rid of ElementConvert, and make the ops that currently require it require a From, Into specific to the 2 elements in question (the current element and product element).

I'm open to other possibilities, and if there are downsides or tradeoffs to the above approaches I'm not seeing, please let me know.

[skewballfox]

from the discord, crutcher mentioned a few situations where relaxing the element constraints could allow us to represent useful primitives: unsigned variants, log-space, value plus complex confidence. frequency is apparently useful for spiking nets. He also mentioned "complex but values stored as dims", but I'm not how that might be representable as an individual element

[laggui]

Cross-posting my comment here as well:

The problem you were having with ToElement is simply because you were trying to make everything go through another trait for complex conversion. But the from_elem<E: ToElement> should be used to help convert to your type, given an element that implements ToElement. In other words, extended element types (i.e., anything beyond the built-in primitives) need to define how they’re built from a primitive-ish thing that implements ToElement.

So if we take Complex32 as an example, you only need to implement ToElement for that type and provide a method that creates a Complex32 from a primitive element.

impl Complex32 {
    pub fn from_elem<E: ToElement>(elem: &E) -> Self {
        Self::from_real(elem.to_f32())
    }
}

impl ToElement for Complex32 {
    // ...
}

You can even use the make_element! macro (adapted from your PR):

make_element!(
    ty Complex32,
    convert Complex32::from_elem,
    random |distribution: Distribution, rng: &mut R| {
        let real: f32 = distribution.sampler(rng).sample();
        let imag: f32 = distribution.sampler(rng).sample();
        Complex32::new(real, imag)
    },
    cmp |a: &Complex32, b: &Complex32| {
        // Compare by magnitude, then by real part if magnitudes are equal
        let mag_cmp = a.abs().total_cmp(&b.abs());
        if mag_cmp == Ordering::Equal {
            a.real.total_cmp(&b.real)
        } else {
            mag_cmp
        }
    },
    dtype DType::F32, // TODO: DType extensions
    min Complex32::new(f32::MIN, f32::MIN),
    max Complex32::new(f32::MAX, f32::MAX)
);

ToElement and Element traits are not closed, they should actually help to construct your type from any type that implements them.

The real limitation I believe is that DType doesn't support extensions.