Off-Axis Camera Projection suggestion

[DBraun]

I think it would be nice to have an off-axis camera projection method for PerspectiveCamera.

I've been trying to adapt code from headtrackr and HoloToy's main function

My headtrackr approach (which still doesn't work) is index.html in this repo, and my holotoy approach (through manually updating the camera's projectionMatrix) is index2.html

I'm so eager to get the 3d illusions possible through off-axis projections.

update: my StackOverflow question

[mrdoob]

/ping @WestLangley

[WestLangley]

Yes, that would be a nice feature, and could yield some interesting effects.

@DBraun Do you have a reference to the literature on this?

[DBraun]

@WestLangley I put up a conceptual answer to my SO question: http://stackoverflow.com/questions/26070386/head-coupled-perspective-in-three-js

Robert Kooima seems to be the leading academic expert on the subject: http://csc.lsu.edu/~kooima/articles/genperspective/index.html

These may be other helpful resources:
http://en.wikibooks.org/wiki/Cg_Programming/Unity/Projection_for_Virtual_Reality
http://www.ixagon.se/surfacemapper/
http://kode80.com/2012/04/09/holotoy-perspective-in-webgl/ (the main.js)
http://blogs.bl0rg.net/netzstaub/2008/08/24/wiimote-headtracking-in-processing/

I've noticed that the headtrackr code locks the camera onto the XY plane. The camera's perspective can change, but there's no simple way to move and rotate the camera before the perspective adjustment. This is typically done with a projectionMatrix, but headtrackr uses setCameraOffset. I don't think setOffAxisProjection should use setCameraOffset. It probably needs a camera XYZ, a viewer XYZ in relation to the surface, and a conversion between the two kinds of units (Three.js units and real-life centimeters for example).

[zalo]

I just ported Kooima's Generalized Projection Matrix formulation for a project of my own. Hopefully this helps an intrepid visitor in the future:

/** Set the PerspectiveCamera's projectionMatrix to match the corners of an arbitrary rectangle
 * @param {THREE.Camera} camera
 * @param {THREE.Vector3} bottomLeftCorner
 * @param {THREE.Vector3} bottomRightCorner
 * @param {THREE.Vector3} topLeftCorner */
fromCorners(camera, bottomLeftCorner, bottomRightCorner, topLeftCorner, estimateViewFrustum = false) {
    let pa = bottomLeftCorner, pb = bottomRightCorner, pc = topLeftCorner;
    let pe = camera.position;     // eye position
    let n  = camera.near;         // distance of near clipping plane
    let f  = camera.far;          // distance of far clipping plane

    _vr.copy(pb).sub(pa)      .normalize();
    _vu.copy(pc).sub(pa)      .normalize();
    _vn.crossVectors(_vr, _vu).normalize();

    _va.copy(pa).sub(pe);          // from pe to pa
    _vb.copy(pb).sub(pe);          // from pe to pb
    _vc.copy(pc).sub(pe);          // from pe to pc

    let d = -_va.dot(_vn);         // distance from eye to screen
    let l =  _vr.dot(_va) * n / d; // distance to left screen edge
    let r =  _vr.dot(_vb) * n / d; // distance to right screen edge
    let b =  _vu.dot(_va) * n / d; // distance to bottom screen edge
    let t =  _vu.dot(_vc) * n / d; // distance to top screen edge

    // Set the camera rotation to match the focal plane to the corners' plane
    _quat.setFromUnitVectors(_vec.set(0, 1, 0), _vu);
    camera.quaternion.setFromUnitVectors(_vec.set(0, 0, 1).applyQuaternion(_quat), _vn).multiply(_quat);

    // Set the off-axis projection matrix to match the corners
    camera.projectionMatrix.set(2.0 * n / (r - l), 0.0,
                                (r + l) / (r - l), 0.0, 0.0,
                                2.0 * n / (t - b),
                                (t + b) / (t - b), 0.0, 0.0, 0.0,
                                     (f + n) / (n - f),
                                2.0 * f * n  / (n - f), 0.0, 0.0, -1.0, 0.0);
    camera.projectionMatrixInverse.copy(camera.projectionMatrix).invert();

    // Untested FoV estimation to fix frustum culling
    if (estimateViewFrustum) {
        // set fieldOfView to a conservative estimate
        // to make frustum tall/wide enough to encompass it
        camera.fov =
            57.2958 / Math.min(1.0, camera.aspect) *
            Math.atan(((pb - pa).length() + (pc - pa).length()) / _va.length());
    }
}

const _va   = /*@__PURE__*/ new THREE.Vector3(), // from pe to pa
      _vb   = /*@__PURE__*/ new THREE.Vector3(), // from pe to pb
      _vc   = /*@__PURE__*/ new THREE.Vector3(), // from pe to pc
      _vr   = /*@__PURE__*/ new THREE.Vector3(), // right axis of screen
      _vu   = /*@__PURE__*/ new THREE.Vector3(), // up axis of screen
      _vn   = /*@__PURE__*/ new THREE.Vector3(), // normal vector of screen
      _vec  = /*@__PURE__*/ new THREE.Vector3(), // temporary vector
      _quat = /*@__PURE__*/ new THREE.Quaternion(); // temporary quaternion