Skip to content

[EXPERIMENTAL] Remove ThreeDScene and rewrite Camera internals #4059

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
Dec 17, 2024

Conversation

chopan050
Copy link
Contributor

@chopan050 chopan050 commented Dec 15, 2024

  • Deleted ThreeDScene and moved many of its methods into Camera. For example, ThreeDScene.begin_ambient_camera_rotation() becomes Camera.begin_ambient_rotation() and ThreeDScene.begin_3dillusion_camera_rotation() becomes Camera.begin_precession().
  • All the references to ThreeDScene were changed to Scene.
  • Camera was rewritten to work directly with ._theta, ._phi and ._gamma angles rather than using an .orientation quaternion which is constantly converted to a SciPy Rotation and then to a matrix, Euler angles or another quaternion.
  • Now the starting ._theta angle for Camera is -TAU/4 instead of 0, which is more representative of its initial orientation. When increasing ._phi a bit, the Camera is positioned over the negative Y axis, which corresponds to theta=-TAU/4, not theta=0.
  • Camera now directly uses .focal_distance instead of .focal_dist_to_height. This was needed in order to correctly implement zooming.
  • Now Camera can be properly zoomed in and out. Previously, it couldn't. Scaling the Camera used to only cause a difference in the focal distance.

Reviewer Checklist

  • The PR title is descriptive enough for the changelog, and the PR is labeled correctly
  • If applicable: newly added non-private functions and classes have a docstring including a short summary and a PARAMETERS section
  • If applicable: newly added functions and classes are tested

@chopan050 chopan050 changed the base branch from main to experimental December 15, 2024 22:47
@chopan050 chopan050 changed the title [EXPERIMENTAL] Implement ThreeDScene and rewrite Camera internals [EXPERIMENTAL] Remove ThreeDScene and rewrite Camera internals Dec 16, 2024
Copy link
Member

@JasonGrace2282 JasonGrace2282 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Just one small change/discussion

@@ -28,6 +29,7 @@ def __init__(self):
self.capabilities = [
(OpenGLVMobject, self.render_vmobject),
(ImageMobject, self.render_image),
(OpenGLMobject, lambda mob: None),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not a big fan of this in experimental, I would prefer a hard error (XYZ cannot be rendered)

Maybe a marker class is neccessary? Something like

class InvisibleMobject:
    pass

class Camera(OpenGLMobject, InvisibleMobject):
    ...

Copy link
Contributor Author

@chopan050 chopan050 Dec 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While using lambda mob: None was admittedly a bad idea, and the InvisibleMobject marker class is interesting (see below), it would be great if we could render the submobjects of any OpenGLMobject such as a Group.

I was thinking of something like:

	def __init__(self):
        self.capabilities = [
            (OpenGLVMobject, self.render_vmobject),
            (ImageMobject, self.render_image),
        ]

    def render(self, state: SceneState) -> None:
        self.pre_render(state.camera)
        for mob in state.mobjects:
            self.render_mobject(mob)
        self.post_render()

    def render_mobject(self, mob: OpenGLMobject) -> None:
        for MobClass, render_func in self.capabilities:
            if isinstance(mob, MobClass):
                render_func(mob)
                break
        else:
			if not isinstance(mob, InvisibleMobject):
				logger.warn(
                    f"{type(mob).__name__} cannot be rendered by {type(self).__name__}. "
					"Attempting to render its submobjects..."
                )
            for submob in mob.submobjects:
                self.render_mobject(submob)

The idea is that, if the OpenGLMobject is neither an OpenGLVMobject nor an ImageMobject, then it is not directly rendered. Instead, this will attempt to render any of its submobjects, in case the renderer is capable of rendering them. This skips the Camera and attempts to render the contents of a Group.

Plus, because everything in SceneState.mobjects is an OpenGLMobject, we wouldn't really be raising any error. Instead, maybe we could only log a warning if the OpenGLMobject is not an InvisibleMobject such as a Group or a Camera.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My concern is that it wouldn't raise any error/message for something we don't support (yet).
Maybe the following changes would work:

  • logger.debug instead of logger.warn (so users don't get bombarded when using Group)
  • if not mob.submobjects: logger.warn(f"Cannot render {mob}")

Copy link
Contributor Author

@chopan050 chopan050 Dec 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made an implementation, and I would like you to take a look, please.

I'm still using logger.warning, but I implemented an ._unsupported_mob_warnings attribute to store warning messages and avoid spamming the console on every frame.

I marked Group as InvisibleMobject, so no warnings would be raised for it unless any of its submobjects is not renderable and is not an InvisibleMobject itself. Same with Camera. Any other OpenGLMobject will have a warning raised.

IDK about raising a hard error, though. If we want to go down that route, maybe we should enforce that Mobjects can't have submobjects unless they're explicitly Groups, but that might be controversial and should be done in another PR.

Copy link
Member

@JasonGrace2282 JasonGrace2282 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Out of curiosity, have you tried running the tests to see how many pass (with -n 0 to prevent crashing the computer)

)
# This prevents spamming the console.
if warning_message not in self._unsupported_mob_warnings:
logger.warning(warning_message)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not specific to this PR/holding it back, but I wonder if we should switch to using the official method of warnings in python (warnings.warn) 🤔

chopan050 and others added 4 commits December 16, 2024 20:19
Co-authored-by: Aarush Deshpande <110117391+JasonGrace2282@users.noreply.github.com>
Co-authored-by: Aarush Deshpande <110117391+JasonGrace2282@users.noreply.github.com>
@chopan050
Copy link
Contributor Author

chopan050 commented Dec 17, 2024

Out of curiosity, have you tried running the tests to see how many pass

332 failed, 226 passed, 29 skipped, 15 warnings, 3 errors

None of them seem related to the changes in this PR. I made sure to remove all references to ThreeDScene.

Many of them fail with the following error:

manim/renderer/opengl_renderer.py:442: in get_pixels
    raw = self.output_fbo.read(components=4, dtype="f1", clamp=True)  # RGBA, floats
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <moderngl.Framebuffer object at 0x7d462446e0f0>, viewport = (0, 0, 1920, 1080), components = 4, attachment = 0, alignment = 1, dtype = 'f1', clamp = True

    def read(self, viewport=None, components=3, attachment=0, alignment=1, dtype="f1", clamp=False):
        if viewport is None:
            viewport = (0, 0, self.width, self.height)
        if len(viewport) == 2:
            viewport = (0, 0, *viewport)
        res, mem = mgl.writable_bytes(mgl.expected_size(viewport[2], viewport[3], 1, components, alignment, dtype))
>       self.mglo.read_into(mem, viewport, components, attachment, alignment, clamp, dtype, 0)
E       AttributeError: 'InvalidObject' object has no attribute 'read_into'

so we have to figure out why moderngl.Framebuffer.mglo is becoming an InvalidObject.

@chopan050 chopan050 merged commit 19980e3 into ManimCommunity:experimental Dec 17, 2024
2 of 4 checks passed
@chopan050 chopan050 deleted the exp-threedscene branch December 17, 2024 01:18
@chopan050 chopan050 mentioned this pull request Dec 17, 2024
19 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants