Add :class:~.OpenGLImageMobject

[hydrobeam]

Changelog / Overview

Motivation

Make ImageMobject use opengl shaders.

Explanation for Changes

Ported over some code from manimgl with some adjustment to make it manimce-compatible.

Also brought over a bugfix for transparent background from manimgl to the image shader: commit here

Doesn't work, only half of the png is rendered, and when zooming in some more weird behaviour with the movement of the image is observed:

becomes:

via

class a(Scene):
    def construct(self):
        a = ImageMobject("day_texture.png")
        self.add(a)
        self.interactive_embed()

Documentation Reference

N/A

Testing Status

Further Comments

Checklist

• I have read the Contributing Guidelines
• I have written a descriptive PR title (see top of PR template for examples)
• I have written a changelog entry for the PR or deem it unnecessary
• My new functions/classes either have a docstring or are private
• My new functions/classes have tests added and (optional) examples in the docs
• My new documentation builds, looks correctly formatted, and adds no additional build warnings

Reviewer Checklist

• The PR title is descriptive enough
• The PR is labeled correctly
• The changelog entry is completed if necessary
• Newly added functions/classes either have a docstring or are private
• Newly added functions/classes have tests added and (optional) examples in the docs
• Newly added documentation builds, looks correctly formatted, and adds no additional build warnings

[jsonvillanueva]

This will fix it, the reason for this is because we're using a VAO and not an EBO. So when it's rendering triangles with the VBO, it uses coordinates 0,1,2 and then 3,4,5 to do two separate triangles. We could use an EBO and not have redudant vertices by specifying only 4 vertices (as you have now) and then selecting vertices, 0,1,2 and 1,2,3 -- but that would involve changing more lines of code.

[hydrobeam]

With the most recent commit, a full square is rendered, although there are multiple issues I noticed (seems to be at the border of the original half-rendered triangle).

1. There's some weird interaction going on in the center of the image when moved.

a.mp4

2. When writing to a movie using --write_to_movie it produces an error:

[mov,mp4,m4a,3gp,3g2,mj2 @ 0000022f4f3272c0] moov atom not found
[concat @ 0000022f4f31de80] Impossible to open 'file:C:/Users/laith/Documents/GitHub/hydro_manim/media/videos/mkt/1080p60/partial_movie_files/a/1491410813_1875215358_2043878721.mp4'
C:\Users\laith\Documents\GitHub\hydro_manim\media\videos\mkt\1080p60\partial_movie_files\a\partial_movie_file_list.txt: Invalid data found when processing input 

2.1. However, when opacity is set to not be the default value of 1, (i.e. 0.99), it renders properly with --write_to_movie

2.2. Also, the scene is written to movie normally regardless of opacity if --disable_caching is passed.

3. Currently not possible to produce the correct dimensions of the image with this setup 🤔. Here is the image with ImageMobject:

[jsonvillanueva]

3. Currently not possible to produce the correct dimensions of the image with this setup 🤔. Here is the image with ImageMobject:

This is because the width/height setters have a rescale_to_fit(..., scale=False) function that makes the width/height match the height/width respectively resulting in a 1:1 aspect ratio. Setting it to True instead will allow you to set the ImageMobject's width/height and have a rectangular texture. This needs to be refactored in some manner.

1. There's some weird interaction going on in the center of the image when moved.

I'm not exactly sure why this is happening, but my intial guesses are that it's an issue with the vertex shader and the calculations from the line gl_Position = get_gl_Position(position_point_into_frame(point)); -- Actually, this is the case. The perspective_scale_factor stuff in that shaders/include/get_glPosition.glsl is buggy and probably shouldn't be incorporated into the shader logic in the first place (it should be a part of the Camera's View/Projection matrices).

[jsonvillanueva]

So I would not recommend merging this because this commit modifies a shader that is used by pretty much every Mobject (manim/renderer/shaders/include/get_gl_Position.glsl) but I commit this now to demonstrate the problem code I referred to in the previous comment:

The perspective_scale_factor stuff in that shaders/include/get_glPosition.glsl is buggy and probably shouldn't be incorporated into the shader logic in the first place (it should be a part of the Camera's View/Projection matrices).

This code should not have the weird warping issue -- BUT it will have other weird issues as a side-effect.

We should be using model/view/projection matrices as uniforms within the shaders instead of this perspective_scale_factor magic. In reality, this may mean doing some other refactors first in order to properly have ImageMobjects in ManimCE. I'm not sure if ManimGL has working ImageMobjects; if so, their shader pipeline is probably different... but I would recommend fixing this after ManimCE properly uses MVP matrices.

[MrDiver]

I think this is already merged with #2534 closing this PR. Feel free to reopen if this is not the case