Possumwood (named after Sandbox Tree) is a graph-based procedural sandbox, implementing concepts of graph-based visual programming in a simple interface. It is intended to provide a user-friendly and accessible UI for experimenting with common computer graphics algorithms and libraries (e.g., OpenGL, CGAL, WildMagic, OpenCV).
Possumwood is built on top of a simple graph-evaluation engine and a Qt-based node graph editor, with an OpenGL viewport. Its main strength is its extensibility - it is trivial to implement new plugins supporting new libraries and data types, which allows free and user-friendly experimentation with parameters of each algorithm.
Possumwood is a sandbox, not a production tool. As a sandbox, it is quite open to radical changes and redesign, and for the time being it does not guarantee any form of backwards compatibility.
Possumwood comes with a number of example setups that can serve as a starting point for experimentation.
|
A simple demo showing how to load an Named vertex attributes p and n are loaded from an |
|
|
When an mesh file doesn't include explicit normals, it is relatively easy to use a fragment shader to "autogenerate" normals using screen-space derivatives |
|
|
A simple shader setup passing a time value (i.e., the timeline value) as a uniform into the shader (together with viewport parameters). This value is then converted in the |
|
|
Wireframe mode is one of the rendering modes of OpenGL. A similar but more controllable effect can be achieved by processing a model using a geometry shader, which allows to convert primitives to a different number of primitives of the same or different type. This setup shows how to emit a line for each polygon edge of the input geometry using a program in |
|
|
While modern GPUs contain a bespoke functionality for subdivision, this simple demo shows how a similar (if limited) effect can be achieved using a simple geometry shader. This demo implements a simple interpolative sudvision scheme with normal-based displacement. All subdivision computation is done in a geometry shader, with additional vertices and topology emitted by calling a recusive function. This approach is just a toy example, as it is strictly limited by the capabilities of individual GPUs - in practical applications, bespoke tesselation shaders perform much better. |
|
|
Curved PN Triangles is a simple geometry subdivision scheme, replacing all triangles with a bezier patch directly on the GPU. The shape interpolation uses a cubic spline, and normals are interpolated using a quadratic spline to maintain continuity. This implementation uses a geometry shader for emitting additional polygons. Vlachos, Alex, et al. "Curved PN triangles." Proceedings of the 2001 symposium on Interactive 3D graphics. ACM, 2001. |
|
|
A simple way of generating procedural graphics using GLSL employs rendering a single polygon covering the whole frame, and a per-pixel fragment shader (with additional inputs) to compute the final colour. A number of examples of this approach can be found at ShaderToy. This demo is a simple example of such approach - based on a ray generated from a near and far plane point corresponding to a screen pixel, it computes the position of ray's intersection with the ground plane ( |
|
|
ShaderToy is a community of artists and programmers centered around generating interesting graphics content in GLSL using a single fragment shader. One of the common techniques to generate procedural 3D content in a fragment shader is Raymarching - a simple method of tracing rays against shapes represented as Signed Distance Functions, and generating 3D content based on them. This demo is a port of a ShaderToy demo by Indigo Quilezles, showing different SDFs and a few additional effects that can be achieved by raymarching. A comprehensive raymarching tutorial can be found at the RaymarchingWorkshop. |
|
|
A classical way of generating sky and background in games is to render a cube or spherical map on the far plane (behind all other objects). This method is called CubeMap, SkyBox or SkyDome. This simple demo shows this technique on a spherical texture from HDRI Haven. |
|
|
Reflection mapping is a simple image-based lighting technique for simulating purely reflective materials. It only behaves correcty for convex shapes, but even with this strict limitation, it has been long established as a simple method for reflective surfaces in computer graphics. This demo combines a skybox with a "purely reflective" material. |
|
|
Gourand shading is one of the simplest method of polygonal mesh shading. It computes colour by linearly interpolating vertex colours in screen space, making it cheap but less physically accurate than Phong or physics-based shading methods. This demo combines Gourand shading with Phong reflection model, reproducing an old "fixed-pipeline" shading model of OpenGL. |
|
|
Phong shading computes lighting of a surface by interpolating per-pixel surface normal. Compared to Gourand shading, this method leads to smoother and more realistic surface looks better describing the shape of the object, and significantly more accurate high-frequency components computation (i.e., specular component in the Phong lighting model). This demo implements Phong shading with a Phong reflection model and a single point light source. The setup is wrapped in a subnetwork - just double-click the blue node to "enter" it. |
|
|
MatCap or LitSphere is a simple technique for representing the properties of both the material and the environment light in a single texture. By "baking in" the two dimensions of BRDF representing incoming light, MatCap determines the final colour solely by using the difference vector between the view ray and the surface normal. This simple approach can yield surprisingly realistic results, and has been used in games, scultping software and for capturing and representing artistic styles. Sloan, Peter-Pike J., et al. "The lit sphere: A model for capturing NPR shading from art." Graphics interface. Vol. 2001. 2001. |
|
|
Normal mapping is a simple method for adding texture-based geometry details to a coarse model. It uses a texture to represent normal perturbation that would result from a significantly more detail in the surface mesh, without having to represent that detail explicitly in the underlying geometry. In games, this technique is often used to represent a complex model with all its detail, without having to explicitly store a heavy geometry. This simple demo demonstrates a combination of normal mapping implemented in a fragment shader, together with displacement mapping implemented as a geometry shader. The lighting is implemented as a simple single light-source Phong reflection model. Blinn, James F. "Simulation of wrinkled surfaces." ACM SIGGRAPH computer graphics. Vol. 12. No. 3. ACM, 1978. Mikkelsen, Morten. "Simulation of wrinkled surfaces revisited." (2008). |
|
|
Rendering text in OpenGL is usually implemented using a simple texture. However, such text "pixelates" when viewed up-close. A simple method to improve on its appearance uses a Signed distance function to represent letter boundaries - by exploiting the interpolation abilities of GPU hardware, it is possible to represent a text boundary with significantly more precision by using the directional information represented in SDF texture differential. This demo shows a simple implementation of this technique. Green, Chris. "Improved alpha-tested magnification for vector textures and special effects." ACM SIGGRAPH 2007 courses. ACM, 2007. |
|
|
This simple demo shows how to load an object from The display code is contained in a subnetwork (double click the blue node to "enter" it), and is based on a trivial implementation of a vertex and fragment OpenGL shader. |
|
|
To display a polygonal mesh on a GPU, the model needs to contain surface-normal information for shading. While this information can be auto-generated, an explicitly represented normal can improve surface properties like smoothness or shape details. This simple demo shows two methods of computing normals using the CGAL library. |
|
|
Mesh simplification (decimation) is a common operation in mesh processing - it aims to reduce the mesh complexity without removing its details, reducing the memory footprint without compromising the quality of the resulting model. This demo shows how to use CGAL's implementation of mesh decimation in Possumwood, allowing to experiment with its various parameters. |
|
|
This simple demo shows how to load an image using OpenCV's In following demos, this OpenGL setup is wrapped in a subnetwork. |
|
|
A simple demo showing how to load an image from a file, and display it using an OpenGL setup (wrapped in a subnetwork - enter it by double-clicking the blue node). |
|
|
While Possumwood's viewport is primarily intended for viewing 3D content, using a set of shaders it is possible to alter it to display 2D textures as well. This demo uses shaders that ignore the perspective projection matrix and draw 2D image in a 2D front-facing view in the viewport. |
|
|
Color conversion in OpenCV is implemented in the |
|
|
This simple demo shows how to create a subnetwork performing simple addition (and exposing input parameters) with a simple Lua script. Double click the node too see its content. |
|
|
This simple demo shows how to create a subnetwork performing simple multiplication (and exposing input parameters) with a simple Lua script. Double click the node too see its content. |
|
|
A simple demo showing how to use a Lua script to create animated data, which are then fed as an input to a vector input of the |
|
|
Possumwood contains a simple integration of the Lua scripting language, allowing to manipulate in-scene objects using code contained in nodes. This demo shows how to synthesize an image programatically. It "injects" the |
|
|
This demo builds on the Lua Grid setup, extending it by additional parameters, and wrapping it in a subnetwork (double click any blue nodes to "enter" them). |
|
|
Lua expressions in Possumwood can be also used to manipulate HDR images. This demo shows how to implement simple Gamma compression tonemapping operator in Lua, and how to wrap it into a subnetwork with exposed parameters. Possumwood allows arbitrary nesting of nodes in this way, allowing to abstract-away any unnecessary complexity into simple nodes with clean interfaces. |
|
|
This demo shows how to load animation data from a The loaded animation data are then processed (scaled, translated), and displayed in OpenGL using a simple skeleton render node. |
|
|
This demo uses GLSL (geometry shader) to generate a shaded polygonal stick figure from a set of line segments. |
|
|
A motion capture data source captures reality, and due to both capturing noise and imperfections, a locomotion or another periodic motion will never be perfectly periodic. This demo uses a simple hierarchical linear blend to convert motion capture data to a periodic animation. To allow user to understand the input data well, the user interface uses a simple display of per-frame comparison error metric, similar to Kovar et al.'s Motion Graphs. Kovar, Lucas, Michael Gleicher, and Frédéric Pighin. "Motion graphs." ACM SIGGRAPH 2008 classes. ACM, 2008. |
|
|
A linear blend is the simplest way to transition between two animations. This demo shows how a hand-picked linear blend can be used in Possumwood to transition between two animations. To assist the user with picking the blending point, the editor window displays a visualisation of frame similarity metric, similar to Kovar et al.'s Motion Graphs. Kovar, Lucas, Michael Gleicher, and Frédéric Pighin. "Motion graphs." ACM SIGGRAPH 2008 classes. ACM, 2008. |
|
|
A motion graph is a simple directional graph data structure describing possible transition points between a number of motion clips. Each transition is determined by comparing kinematic and dynamic properties of corresponding frames, which should lead to a transition that does not introduce any salient discontinuities or artifacts. A random walk through such a graph would generate a variation of the original motion(s) by adding random transitions. This demo shows a random walk through a motion graph. In its current state, it does not attempt to synthesize smooth continuous transitions - it just switches between different clips. (todo: introduce smooth transitions) Kovar, Lucas, Michael Gleicher, and Frédéric Pighin. "Motion graphs." ACM Transactions on Graphics (TOG). Vol. 21. No. 3. ACM, 2002. Kovar, Lucas, Michael Gleicher, and Frédéric Pighin. "Motion graphs." ACM SIGGRAPH 2008 classes. ACM, 2008. |
|
|
Animations represented as sequences of poses of hierarchical skeletons do not naturally represent information about character interaction with the environment. This information is necessary for motion editing to maintain desirable properties, such as realistic stationary footsteps. A number of methods has been used to perform this task, but a simple velocity thresholding has proven sufficient for most practical applications. This demo shows how this simple technique can be used in Possumwood. The resulting constraint information can be further processed via median filtering, to remove any noise introduced in the detection process. Kovar, Lucas, John Schreiner, and Michael Gleicher. "Footskate cleanup for motion capture editing." Proceedings of the 2002 ACM SIGGRAPH/Eurographics symposium on Computer animation. ACM, 2002. Ikemoto, Leslie, Okan Arikan, and David Forsyth. "Knowing when to put your foot down." Proceedings of the 2006 symposium on Interactive 3D graphics and games. ACM, 2006. |
|
|
This demo follows the OpenCV's Haar Cascade tutorial - it allows to load a Haar Cascade Lienhart, Rainer, and Jochen Maydt. "An extended set of haar-like features for rapid object detection." Proceedings. international conference on image processing. Vol. 1. IEEE, 2002. |
|
|
This tutorial closely follows the OpenCV's denoising tutorial for grayscale images, removing the noise using the non-local means algorithm. |
|
|
This tutorial closely follows the OpenCV's denoising tutorial for color images, removing the noise using the non-local means algorithm. |
|
|
This tutorial follows the OpenCV's denoising tutorial for grayscale images, but at the end of the processing it subtracts the denoised image from the original (leaving only extracted noise), and equalises the result using |
|
|
This tutorial follows the OpenCV's denoising tutorial for color images, but at the end of the processing it subtracts the denoised image from the original (leaving only extracted noise), and equalises the result using |
|
|
The Discrete Fourier Transform (implemented in OpenCV) converts a signal (image) into its complex frequency spectrum. By manipulating the spectrum, it is possible to implement simple frequency filters. This demo shows how to perform a DFT on an image, and implements a simple frequency cutoff low-pass filter, showing both the image before and after applying this filter. |
|
|
OpenCV's VideoCapture object can be used to extract frames from a video file. This demo shows how to use a |
|
|
OpenCV provides a method to compute dense optical flow in the This demo shows how to set up this type of optical flow in Possumwood, and how to display its results as colours on top of a grayscale animated image. Farnebäck, Gunnar. "Two-frame motion estimation based on polynomial expansion." Scandinavian conference on Image analysis. Springer, Berlin, Heidelberg, 2003. |
|
|
Pattern detection is the first step of camera calibration for many computer vision algorithms. This demo shows how a pattern build of circles can be detected using Hough transformation, and assembled into a grid for the use of camera intrinsic and extrinsic calibration. |
|
|
Pattern detection is the first step of camera calibration for many computer vision algorithms. This demo shows how a chessboard pattern can be detected using corners feature detection, and assembled into a grid for the use of camera intrinsic and extrinsic calibration. |
|
|
The calibration of a camera's intrinsic parameters is an important step for most algorithms in machine vision. This demo shows how to perform multi-image intrinsic camera calibration using a chessboard pattern, and how to use the solved parameters to undistort an image. This functionality is fully built using OpenCV's functions, wrapped in a set of Possumwood nodes. The left side of the graph detects a pattern in a set of 4 images, accumulating the results in a |
|
|
Image inpainting is an image reconstruction technique, allowing to fill-in missing parts of an image based on existing pixels. Telea's algorithm shown in this demo uses a Fast Marching method to inpaint pixels from a boundary based on surrounding pixels and boundary's normal direction. Telea, Alexandru. "An image inpainting technique based on the fast marching method." Journal of graphics tools 9.1 (2004): 23-34. |
|
|
Image inpainting is an image reconstruction technique, allowing to fill-in missing parts of an image based on existing pixels. The Navier-Stokes-based algorithm shown in this demo uses a heuristic based on fluid dynamics to inpaint a large region of dark pixels in an image, obtained by image thresholding. Bertalmio, Marcelo, Andrea L. Bertozzi, and Guillermo Sapiro. "Navier-stokes, fluid dynamics, and image and video inpainting." Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001. Vol. 1. IEEE, 2001. |
|
|
Image inpainting can be also used to attempt to restore an image where a large portion of the original data is missing. This setup shows how an image can be partially restored using inpainting from a very sparse set of sample points. |
|
|
A slightly different take on image inpainting can use the smoothness prior to try to reconstruct missing pixels in a single linear global solve step. Cohen-Or, Daniel, et al. A sampler of useful computational tools for applied geometry, computer graphics, and image processing. AK Peters/CRC Press, 2015. |
|
|
The smoothness prior can be also used to try to reconstruct missing pixels in a sparse set of samples using a single linear global solve step. Cohen-Or, Daniel, et al. A sampler of useful computational tools for applied geometry, computer graphics, and image processing. AK Peters/CRC Press, 2015. |
|
|
A simple implementation of the Anisotropic Diffusion filter, based on the legendary Perona-Malik paper. Implementation based on discretisation by Mackiewich (1995). Perona, Pietro, and Jitendra Malik. "Scale-space and edge detection using anisotropic diffusion." IEEE Transactions on pattern analysis and machine intelligence 12.7 (1990): 629-639. Mackiewich, Blair. Intracranial boundary detection and radio frequency correction in magnetic resonance images. Diss. Theses (School of Computing Science)/Simon Fraser University, 1995. |
|
|
Superpixel algorithms derive meaningful atomic primitives from dense grid images. Many computer vision algorithms can work with such regions, reducing both computational complexity and ambiguity. SLIC (Simple Linear Iterative Clustering) is a simple adaptation of k-means clustering that generates superpixels via iterative linear averaging and simple non-linear grouping. Achanta, Radhakrishna, et al. "SLIC superpixels compared to state-of-the-art superpixel methods." IEEE transactions on pattern analysis and machine intelligence 34.11 (2012): 2274-2282. |
|
|
Superpixel algorithms derive meaningful atomic primitives from dense grid images, which can then be used as inputs to other computer vision algorithms. One of the simplest such algorithm is mean, averaging the values in a superpixel, leading to a "mosaic-like" look. Achanta, Radhakrishna, et al. "SLIC superpixels compared to state-of-the-art superpixel methods." IEEE transactions on pattern analysis and machine intelligence 34.11 (2012): 2274-2282. |
|
|
Multi-exposure merge is an procedure in high dynamic range imaging. This setup merges 5 hand-held exposures of a scene to an HDR image using Mertens algorithm. As the inputs are not perfectly aligned, the output shows "ghosting" artifacts, as expected in this type of setup. Mertens, Tom, Jan Kautz, and Frank Van Reeth. "Exposure fusion." 15th Pacific Conference on Computer Graphics and Applications (PG'07). IEEE, 2007. |
|
|
For a HDR merge algorithm to correctly process its inputs, the input images need to be precisely aligned. This can be either achived by careful capture, via a calibration and alignment step, or by a simpler image-alignment algorithm. This demo shows how a simple alignment of hand-held multiple exposures can be achieved using the Mertens, Tom, Jan Kautz, and Frank Van Reeth. "Exposure fusion." 15th Pacific Conference on Computer Graphics and Applications (PG'07). IEEE, 2007. Ward, Greg. "Fast, robust image registration for compositing high dynamic range photographs from hand-held exposures." Journal of graphics tools 8.2 (2003): 17-30. |
|
|
When merging multiple exposures for HDR imaging, it is often beneficial to use raw image data, avoiding any processing and gamma correction a camera might do to produce a perceptually good looking result. In some cases, when using cheaper cameras or a mobile phone, it might not be possible to use raw data, which means that an additional calibration step is required. This setup shows how to use EXIF data from a set of JPG images to estimate the camera response curve using Debevec's algorithm, and then use the resulting curve for a more accurate HDR merge using Debevec's HDR merge algorithm. Compared to previous demos using Merten's algorithm, the dynamic range of the resulting image is much wider, with a more accurate representation of luminance information present in the scene. Debevec, Paul E., and Jitendra Malik. "Recovering high dynamic range radiance maps from photographs." ACM SIGGRAPH 2008 classes. ACM, 2008. |
|
|
Tonemapping is the process of converting High Dynamic Range content to a standard display. The simplest operators are global (such as Gamma correction, while others act adaptively, mimicking the behaviour of human visual system. This demo shows how to set up Drago's tonemapping algorithm in Possumwood. Drago, Frédéric, et al. "Adaptive logarithmic mapping for displaying high contrast scenes." Computer graphics forum. Vol. 22. No. 3. Oxford, UK: Blackwell Publishing, Inc, 2003. |
|
|
Tonemapping is the process of converting High Dynamic Range content to a standard display. The simplest operators are global (such as Gamma correction, while others act adaptively, mimicking the behaviour of human visual system. This demo shows how to set up Reinhards's tonemapping algorithm in Possumwood. Reinhard, Erik, and Kate Devlin. "Dynamic range reduction inspired by photoreceptor physiology." IEEE transactions on visualization and computer graphics 11.1 (2005): 13-24. |
|
|
Tonemapping is the process of converting High Dynamic Range content to a standard display. The simplest operators are global (such as Gamma correction, while others act adaptively, mimicking the behaviour of human visual system. This demo shows how to set up Mantiuk's tonemapping algorithm in Possumwood. Mantiuk, Rafal, Karol Myszkowski, and Hans-Peter Seidel. "A perceptual framework for contrast processing of high dynamic range images." ACM Transactions on Applied Perception (TAP) 3.3 (2006): 286-308. |
|
|
Lytro cameras output a proprietary raw image format, which has never been officially documented. However, it has been reverse-engineered, and the results were documented in detail by Jan Kučera. This demo implements a reader for Lytro RAW files, extracting both image data and metadata of each image. Kučera, Jan. Lytro Meltdown - the file format. Kučera, Jan. "Výpočetní fotografie ve světelném poli a aplikace na panoramatické snímky." (2014). Georgiev, Todor, et al. "Lytro camera technology: theory, algorithms, performance analysis." Multimedia Content and Mobile Devices. Vol. 8667. International Society for Optics and Photonics, 2013. |
|
|
Calibration of lightfield cameras is a complex process. One of the main issue facing microlens-based cameras is vignetting, caused both by the main lens, and by each of the microlenses. A naive approach to address vignetting is by simply dividing the image pixels by the values of a white diffuse image captured with the same settings. While this approach addresses vignetting sufficiently, it doesn't take into account the camera response curve, leading to some artifacts still being present in the final image. Georgiev, Todor, et al. "Lytro camera technology: theory, algorithms, performance analysis." Multimedia Content and Mobile Devices. Vol. 8667. International Society for Optics and Photonics, 2013. |
|
|
The sensor of Lytro cameras are conventional Bayer-pattern color filters. In standard photography, one of the first steps of image processing involves the interpolation of values of neighbouring pixels to remove the Bayer pattern. This demo shows how to obtain a debayered normalised lightfield in Possumwood. Kucera, J. "Computational photography of light-field camera and application to panoramic photography." Institute of Information Theory and Automation (2014). |
|
|
Reconstruction of the microlens pattern is essential for recovering the lightfield data from a Lytro image. This demo shows how the information embedded in the metadata of a Lytro raw image can be used to reconstruct the microlens array, and visualise the result. Georgiev, Todor, et al. "Lytro camera technology: theory, algorithms, performance analysis." Multimedia Content and Mobile Devices. Vol. 8667. International Society for Optics and Photonics, 2013. |
|
|
Lytro files come with metadata that descibe the parameters of the lenslet pattern used to capture the image. Unfortunately, this lenslet pattern was detected using a simple peak-detection algorithm, with together with the main-lens vignetting causes the lenslet centers to be slightly biased towards the centre of the image. This demo attempt to detect the lenslets using a different method - by inverting the image, locally normalising it and convoluting it with an inverted lenslet pattern (a "hexagonal" shape representing the darker spaces between lenslets), we can achieve a better match with the original lenslet placement. This demo only shows the image-processing part - the pattern fitting is demonstrated in the next demo. |
|
|
This demo implements fitting of lenslet data detected using a convolution-based setup described in the previous demo. Display allows to preview both the original pattern, and the new detected pattern - just change the "blend" parameter. |
|
|
Epipolar images show slices of the 4D lightfield along different axis. The x-y image (center) shows a slice corresponding to a single dept-of-field image. By changing the UV offset or interval, it is possible to create images with different depth of field or focal plane. The u-x and v-y images (left, right, top and bottom) show slices with "line direction" (based on incoming direction of rays) corresponding to individual depth planes. Ng, Ren, et al. "Light field photography with a hand-held plenoptic camera." Computer Science Technical Report CSTR 2.11 (2005): 1-11. |
|
|
The epipolar images accumulated in UV space show images representing averages of lenslets in different part of the image. This provides a good measure for the quality of Lytro lenslet pattern detection and main lens calibration - if correct, all lenslets should look very similar. Ng, Ren, et al. "Light field photography with a hand-held plenoptic camera." Computer Science Technical Report CSTR 2.11 (2005): 1-11. |
|
|
Lightfield data contain enough information to allow for reconstruction of multiple narrow-base sub-apreture images, each with slightly different "point of view". This demo shows how to use demultiplexing to reconstruct multiple views from raw Lytro data. Sabater, Neus, et al. "Accurate disparity estimation for plenoptic images." European Conference on Computer Vision. Springer, Cham, 2014. |
|
|
Lightfield data contain enough information to allow for reconstruction of multiple narrow-base sub-apreture images, each with slightly different "point of view". This demo shows how to use demultiplexing to reconstruct multiple views from normalized raw Lytro data. Normalization is performed by simple division of the image data by white diffuse image values. Sabater, Neus, et al. "Accurate disparity estimation for plenoptic images." European Conference on Computer Vision. Springer, Cham, 2014. |
|
|
Demultiplexing a Bayern image leads to a significantly larger number of samples for green pixels, compared to red and blue values. This demo addempts to visualise this difference explicitly, by showing both each channel of the demultiplexed image, and the number of samples used to reconstruct each of its pixels. |
|
|
To reconstruct missing data after obtaining sub-apreture images, this demo uses a simple inpainting technique from OpenCV to reconstruct missing data in each image separately. Sabater, Neus, et al. "Accurate disparity estimation for plenoptic images." European Conference on Computer Vision. Springer, Cham, 2014. Bertalmio, Marcelo, Andrea L. Bertozzi, and Guillermo Sapiro. "Navier-stokes, fluid dynamics, and image and video inpainting." Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001. Vol. 1. IEEE, 2001. |
|
|
To reconstruct missing data after obtaining sub-apreture images, this demo uses a simple Laplacian-based inpainting technique to reconstruct missing data in each image separately. Sabater, Neus, et al. "Accurate disparity estimation for plenoptic images." European Conference on Computer Vision. Springer, Cham, 2014. Cohen-Or, Daniel, et al. A sampler of useful computational tools for applied geometry, computer graphics, and image processing. AK Peters/CRC Press, 2015. |
|
|
Reconstructed mosaic data can be used to reconstruct a high-resolution image using a simple multiplexing algorithm. This demo shows the result of this algorithm on an inpainted mosaic from previous demos. |
|
|
The simplest method of lightfields digital refocusing is to directly use the lightfield's equation to determine the target pixel position for each sample, given its position in the lightfield image and its corresponding While very simple and fast, the number of samples for each pixel can vary depending on the bayern pattern and focal distance, often leading to noisy and/or low-resolution results with artifacts. Ng, Ren, et al. "Light field photography with a hand-held plenoptic camera." Computer Science Technical Report CSTR 2.11 (2005): 1-11. |
|
|
This demo shows the detail of the sampling and integration pattern of previous demo. Ng, Ren, et al. "Light field photography with a hand-held plenoptic camera." Computer Science Technical Report CSTR 2.11 (2005): 1-11. |
|
|
Applying a de-Bayer filter to raw lightfield data reduces some artifacts seen in previous two demos. Ng, Ren, et al. "Light field photography with a hand-held plenoptic camera." Computer Science Technical Report CSTR 2.11 (2005): 1-11. |
|
|
To reduce the gaps in refocused images caused by missing samples, and to exploit the information in samples that would otherwise be averaged-out into a single pixel, it is possible to use a radial-basis function regression superresolution framework. This demo shows how a simple gaussian-based accumulation, similar to one used for mult-frame supersampling on modern phones, can be used to improve the resolution and quality of a super-sampled refocused image. Ng, Ren, et al. "Light field photography with a hand-held plenoptic camera." Computer Science Technical Report CSTR 2.11 (2005): 1-11. Bartlomiej Wronski, Ignacio Garcia-Dorado, Manfred Ernst, Damien Kelly, Michael Krainin, Chia-Kai Liang, Marc Levoy, and Peyman Milanfar. 2019. Handheld multi-frame super-resolution. ACM Trans. Graph. 38, 4, Article 28 (July 2019), 18 pages. Fiss, Juliet, Brian Curless, and Richard Szeliski. "Refocusing plenoptic images using depth-adaptive splatting." 2014 IEEE international conference on computational photography (ICCP). IEEE, 2014. |
|
|
This demo shows the detail of the sampling and integration pattern of previous demo. |
|
|
Applying a de-Bayer filter to raw lightfield data reduces some artifacts seen in previous two demos, while introducing a level of additional colour ambiguity (i.e., a less sharp result). |
|
|
Briand, Thibaud, and Pascal Monasse. "Theory and Practice of Image B-Spline Interpolation." Image Processing On Line 8 (2018): 99-141. |
|
|
Briand, Thibaud, and Pascal Monasse. "Theory and Practice of Image B-Spline Interpolation." Image Processing On Line 8 (2018): 99-141. |
|
|
This demo shows the difference in integration patterns between different methods shown in previous demos. |
|
|
This demo shows raw depth reconstruction using the correspondence metric of depth values - i.e., for each pixel, the algorithm picks the depth value where the variance of integrated samples was lowest. Tao, Michael W., et al. "Depth from combining defocus and correspondence using light-field cameras." Proceedings of the IEEE International Conference on Computer Vision. 2013. |
|
|
This demo shows how to reconstruct raw depth using the correspondence metric of depth values. In the bottom-right image, the correspondence value's brightness is altered using a confidence metric value, based on the shape of the correspondence curve for each pixel (i.e., pixels with lower confidence are darker). Tao, Michael W., et al. "Depth from combining defocus and correspondence using light-field cameras." Proceedings of the IEEE International Conference on Computer Vision. 2013. Hu, Xiaoyan, and Philippos Mordohai. "A quantitative evaluation of confidence measures for stereo vision." IEEE transactions on pattern analysis and machine intelligence 34.11 (2012): 2121-2133. |
|
|
This demo shows reconstructs the lightfield depth using the correspondence metric of depth values, and propagates it based on the confidence value for each reconstructed pixel. Tao, Michael W., et al. "Depth from combining defocus and correspondence using light-field cameras." Proceedings of the IEEE International Conference on Computer Vision. 2013. Hu, Xiaoyan, and Philippos Mordohai. "A quantitative evaluation of confidence measures for stereo vision." IEEE transactions on pattern analysis and machine intelligence 34.11 (2012): 2121-2133. |
|
|
This demo shows raw depth reconstruction using the defocus (Laplacian) metric of depth values - i.e., for each pixel, the algorithm picks the depth value where contrast value was highest. Tao, Michael W., et al. "Depth from combining defocus and correspondence using light-field cameras." Proceedings of the IEEE International Conference on Computer Vision. 2013. |
|
|
This demo shows how to reconstruct raw depth using the defocus (Laplacian) metric of depth values. In the bottom-right image, the defocus value's brightness is altered using a confidence metric value, based on the shape of the defocus curve for each pixel (i.e., pixels with lower confidence are darker). Tao, Michael W., et al. "Depth from combining defocus and correspondence using light-field cameras." Proceedings of the IEEE International Conference on Computer Vision. 2013. Hu, Xiaoyan, and Philippos Mordohai. "A quantitative evaluation of confidence measures for stereo vision." IEEE transactions on pattern analysis and machine intelligence 34.11 (2012): 2121-2133. |
|
|
This demo shows reconstructs the lightfield depth using the defocus metric of reconstructed images, and propagates it based on the confidence value for each reconstructed pixel. Tao, Michael W., et al. "Depth from combining defocus and correspondence using light-field cameras." Proceedings of the IEEE International Conference on Computer Vision. 2013. Hu, Xiaoyan, and Philippos Mordohai. "A quantitative evaluation of confidence measures for stereo vision." IEEE transactions on pattern analysis and machine intelligence 34.11 (2012): 2121-2133. |
|
|
This demo shows raw depth reconstruction using the narrow-base comparison metric of pixel values - i.e., each pixel's error value is based on its difference to a narrow-base (i.e., sharp) version of the image. Tao, Michael W., et al. "Depth from combining defocus and correspondence using light-field cameras." Proceedings of the IEEE International Conference on Computer Vision. 2013. |
|
|
This demo shows raw depth reconstruction using the narrow-base comparison metric of pixel values - i.e., each pixel's error value is based on its difference to a narrow-base (i.e., sharp) version of the image. The per-pixel metric is evaluated for a range of depth reconstruction values, with the lowest difference value picked as the appropriate depth value candidate. Tao, Michael W., et al. "Depth from combining defocus and correspondence using light-field cameras." Proceedings of the IEEE International Conference on Computer Vision. 2013. |
|
|
This demo shows raw depth reconstruction using the narrow-base comparison metric of pixel values - i.e., each pixel's error value is based on its difference to a narrow-base (i.e., sharp) version of the image. The per-pixel metric is evaluated for a range of depth reconstruction values, with the lowest difference value picked as the appropriate depth value candidate. In the bottom-right image, the depth candidate brightness is altered using a confidence metric value, based on the shape of the defocus curve for each pixel (i.e., pixels with lower confidence are darker). Tao, Michael W., et al. "Depth from combining defocus and correspondence using light-field cameras." Proceedings of the IEEE International Conference on Computer Vision. 2013. |
|
|
This demo shows reconstructs the lightfield depth using the narrow-base metric of reconstructed images, and propagates it based on the confidence value for each reconstructed pixel. Tao, Michael W., et al. "Depth from combining defocus and correspondence using light-field cameras." Proceedings of the IEEE International Conference on Computer Vision. 2013. Hu, Xiaoyan, and Philippos Mordohai. "A quantitative evaluation of confidence measures for stereo vision." IEEE transactions on pattern analysis and machine intelligence 34.11 (2012): 2121-2133. |
|
|
Tao, Michael W., et al. "Depth from combining defocus and correspondence using light-field cameras." Proceedings of the IEEE International Conference on Computer Vision. 2013. |
|
|
The correspondence metric corresponds to the variance of samples integrated into one pixel. The higher the variance, the less likely it is that the particular integration depth value corresponds to the data. The defocus metric is based on the laplacian smoothness of the neighbourhood of the pixel. The higher the smoothness, the more "blur" is present in that particular part of the image, and the less likely it is that the particular depth value corresponds to the data. The narrow-base metric compares the integrated pixel values to a narrow-base image (i.e., an image with much less depth blur present). The closer the values are, the more likely it is that the depth value is correct. Tao, Michael W., et al. "Depth from combining defocus and correspondence using light-field cameras." Proceedings of the IEEE International Conference on Computer Vision. 2013. |
|
|
A simple ST Graph Cut demo, implementing a 2-label version of the Graph Cut classification algorithm. Specifically, it implements a single-threaded version of the Edmonds-Karp algorithm. (to be improved in later pull requests) |
|
|
A simple ST Graph Cut demo, implementing a 2-label version of the Graph Cut classification algorithm, following the graph structure and optimisatino approach described by Ishikawa (2000). (to be improved in later pull requests) Ishikawa, Hiroshi, and Davi Geiger. Global optimization using embedded graphs. Diss. New York University, Graduate School of Arts and Science, 2000. |
|
|
The simplest use-case for a graph-cut algorithm, splitting the image into two distinct parts based on the brightness of its pixels. The graph-cut algorithm guarantees that the continuity of the regions is maintained even in the presence of a significant amount of noise. |
|
|
Depth reconstruction resolved using a graph cut on a multi-level grid graph, with vertical axis defined via each level's correspondence values (modified via confidence values), and horizontal axis by a contrast measure. |
|
|
Previous demos in this category demonstrated several options to reconstruct depth values from input lightfields data, with a confidence value for each recovered depth pixel. This demo tries to use a data-driven anisotropic diffusion to propagate these values from pixels with high confidence to pixels with low confidence, reconstructing the whole field. Mackiewich, Blair. Intracranial boundary detection and radio frequency correction in magnetic resonance images. Diss. Theses (School of Computing Science)/Simon Fraser University, 1995. Perona, Pietro, and Jitendra Malik. "Scale-space and edge detection using anisotropic diffusion." IEEE Transactions on pattern analysis and machine intelligence 12.7 (1990): 629-639. |
|
|
Superpixel algorithms derive meaningful atomic primitives from dense grid images, which can them be used in a variety of computer vision algorithms. By averaging the depth values derived from the correspondence metric for each superpixel, we can create a piecewise-constant set of depth values that respect the continuous regions in the colours of the lightfield image. Achanta, Radhakrishna, et al. "SLIC superpixels compared to state-of-the-art superpixel methods." IEEE transactions on pattern analysis and machine intelligence 34.11 (2012): 2274-2282. Tao, Michael W., et al. "Depth from combining defocus and correspondence using light-field cameras." Proceedings of the IEEE International Conference on Computer Vision. 2013. Chuchvara, Aleksandra, Attila Barsi, and Atanas Gotchev. "Fast and Accurate Depth Estimation from Sparse Light Fields." IEEE Transactions on Image Processing (2019). |
|
Tutorials are intended to introduce the concepts in Possumwood, and help a user navigate its UIs. They are still work-in-progress, and more will be added over time.
This tutorial introduces the Lua integration of Possumwood, using a simple example of an addition node.
Introduces basic concepts of GLSL shaders in Possumwood, via a simple setup displaying a rotating car. Its model is loaded from an .obj
file, converted to an OpenGL Vertex Buffer Object and displayed using a combination of a vertex and a fragment shader.
A simple GLSL skybox setup, using fragment shader and the background
node (based on gluUnProject) to render a background quad with spherically-mapped texture.
Environment map is a simple technique to approximate reflective surfaces in GLSL, without the need of actual raytracing. This tutorial builds on the previous Skybox tutorial, and extends it to make a teapot's material tool like polished metal.
Most implementation of OpenGL 4 remove the ability to influence the width of lines using the classical glLineWidth
call. This tutorial describes in detail how to achieve a similar effect (and much more) using a geometry shader, by rendering each line as a camera-facing polygon strip.
Implementation of Curved PN Triangles tessellation technique, presented by Alex Vlachos on GDC 2011. This method tessellates input triangles using solely their vertex positions and normals (independently of the mesh topology) in a geometry shader.
Infinite ground plane, implemented using a single polygon and simple fragment shader "raytracing" against a plane with Y=0.
The Phong reflectance model, and Phong shading are two basic models of light behaviour in computer graphics. This tutorial explains their function, and provides a step-by-step implementation in GLSL, using a single orbiting light as the lightsource.
This tutorial introduces basic concepts of image manipulation in Possumwood, and a simple shader-based OpenGL setup to display a texture in the scene.
Apart from rendering existing images loaded from a file, Possumwood can also generate and edit images using Lua scripting. In this tutorial, we explore a very simple setup which uses Lua and per-pixel expressions to generate an image.
Possumwood has been tested only on Linux (several distributions). While it should work on Windows, it has not been compiled or tested there. No support for MacOS is planned for the time being due to heavy dependency on OpenGL.
On Ubuntu, the easiest way to install Possumwood is to use the snapshots PPA, which is built against daily builds of common CGI libraries in the cg-daily PPA:
sudo add-apt-repository ppa:martin-prazak/possumwood
sudo add-apt-repository ppa:cgi-daily-devs/cgi-daily
sudo apt-get update
sudo apt-get install possumwood
This will install Possumwood to your system, enabling to simply run possumwood
command from any terminal.
Daily build of Possumwood is built against a number of packages that are newer than most distros provide out of the box. These are contained in the ppa:martin-prazak/cgi-daily
PPA. Some packages might conflict with the system versions (OpenImageIO, OpenCV), requiring the original system packages to be uninstalled first.
Currently, Possumwood is released in Snap as a development/testing package only. The latest build and its status can be accessed here.
To install a testing version, please run:
sudo snap install --edge possumwood --devmode
This will download and install the latest successful build of Possumwood with its dependencies. To start the application, run possumwood
from the command line. As a dev build, snap will not automatically update this installation. Moreover, snap skin support is currently rather rudimentary, making Possumwood not inherit the system look correctly.
The project is structured as a standard CMake-built project. To build, just run these in the directory of the repository on any Linux distro:
mkdir build
cd build
cmake .. -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=./install && make -j
This will build Possumwood and all its plugins, and install the result inside ./install
directory. Installation is necessary to make sure Possumwood is able to find its plugins and its dependencies. After that, you can run the build via:
./install/bin/possumwood
Possumwood is designed to be easily extensible. A simple addition node, using float attributes, can be implemented in a few lines of code:
#include <possumwood_sdk/node_implementation.h>
namespace {
// strongly-typed attributes
dependency_graph::InAttr<float> input1, input2;
dependency_graph::OutAttr<float> output;
// main compute function
dependency_graph::State compute(dependency_graph::Values& data) {
// maintains attribute types
const float a = data.get(input1);
const float b = data.get(input2);
data.set(output, a + b);
// empty status = no error (both dependency_graph::State and exceptions are supported)
return dependency_graph::State();
}
void init(possumwood::Metadata& meta) {
// attribute registration
meta.addAttribute(input1, "a");
meta.addAttribute(input2, "b");
meta.addAttribute(output, "out");
// attribute dependencies
meta.addInfluence(input1, output);
meta.addInfluence(input2, output);
// and setting up a compute method
meta.setCompute(compute);
}
// static registration of a new node type
possumwood::NodeImplementation s_impl("maths/add", init);
}
An auto-generated Doxygen reference can be found here
Please feel free to contribute! I would welcome any form of feedback, opinions, bugs, code, or description of projects you are planning to use my code in!
At the moment, the project is in its prototype stage, and any feedback or help is welcome. In particular, it is in dire need of a MS Windows build, preferably using AppVeyor. Any help in that area would be greatly appreciated!
The code is released under the MIT license. Included example assets come with their own licenses, included in the directory of each asset.