Skip to content

Add bitmap convolution operations #94

@noelwelsh

Description

@noelwelsh

Overview

Convolution is useful for image processing tasks, such as smoothing and edge detection, as well as being a creative tool. An example of what can be achieved using convolutions, and other image processing effects, is given at https://www.smashingmagazine.com/2015/05/why-the-svg-filter-is-awesome/

The goal of this task is to add convolutions to Doodle. Both Java2D and SVG support convolutions, but in different ways. Thus this task requires some API design work. For example, it might make more sense to have different algebras for the different backends.

Details

Convolutions typically apply to rectangular bitmaps, while pictures in Doodle can have arbitrary boundaries and are usually represented a vector shapes, not bitmaps. This means there are several implementation options:

  1. Convolutions apply to an entire Canvas and cannot be applied to individual elements. A convolution is only applied once a Picture is rendered to screen and applies to everything that is rendered.
  2. Convolutions apply to individual elements (a Picture). On the Java2D backend this will require rendering an element to a bitmap buffer, performing the convolution, and then rendering the bitmap ("compositing") back into the main image. In SVG this is directly supported.
  3. There could be an explicit representation of a bitmap (see Refactor bitmap support #85, Convert Picture to BufferedImage #87, and Add "immediate mode" algebra ("Doodle Canvas") #93 for related issues) on which convolutions are applied, and then methods to transform this bitmap to and from a Picture. This achieves the same effect as 2 but is more explicit and will be difficult to support on SVG.

A convolution algebra may look something like the below. This example corresponds to option 3 above, with an explicit representation of a Bitmap.

final case class Kernel(/* Some representation here */)

trait Convolve[Bitmap]{
  def convolve(bitmap: Bitmap, kernel: Kernel): Bitmap
}

See backend specific details:

  • Java convolutions involve much ceremony, as is typical for Java libraries.
  • SVG has a fairly complex DSL for specifying image filters, of which convolution is one part. It might be interesting to model all of this in Doodle.
  • the HTML canvas doesn't directly support convolution as far as I know, but it is straightforward to implement convolutions in software.

A few notes:

  • Convolution operators compose and can be multiplied by a scalar.

See #85 for a related issue.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    Status

    Done

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions