Refactor the structure of the package and add configuration guidelines by TonyCrane · Pull Request #1366 · 3b1b/manim

TonyCrane · 2021-02-07T14:07:52Z

Motivation

The current manimlib is relatively inconsistent with the package specification, so some slight refactoring has been carried out.
And added an automation guide for setting custom_default.yml

Proposed changes

Move manimlib/__init__.py into manimlib/__main__.py as an entry point of manim and remove all non-manim imports.
Move manimlib/imports.py into manimlib/__init__.py. So we can use from manimlib import * to import this package.
Add empty __init__.py for each folder
Delete old entry point manim.py, Use manimgl (or manim-render) instead of python manim.py.
Change file names of yaml files: custom_defaults.yml -> custom_config.yml, defaults.yml -> default_config.yml
Add configuration guide, which will be run when there is no configuration file (custom_config.yml or manimlib/default_config.yml)
Add a warning if there is no custom_config.yml and using the default_config.yml
Add a CLI flag --config. Use manimgl --config to create a config file according to the guide for a second time.
Update README and docs

Update

3b1b · 2021-02-07T18:27:32Z

Most of this is certainly in the right direction, thank you.

Move manimlib/imports.py into manimlib/init.py. So we can use from manimlib import * to import this package.

Given that the current use of manimlib/imports.py is not the best practice, I'd be hesitant to make it something that's called by default when you import the package.

At the very least, it would be best to separate out the non-manim libraries that it imports at the bottom. In fact, arguably most of those should be removed, and we should encourage more explicit imports of numpy, math, etc. If for someone's personal use-case they want to bring all those in with a single highly-implicit import call, they can always do that, but I don't think it should be systematized into the structure of manim.

Add configuration guide, which will be run when there is no configuration file (custom_config.yml or manimlib/default_config.yml)

I don't think the first time someone runs manim they should have to fill all this out. It's a nicer user experience if it falls back on some defaults so they can at least jump in and start running scenes. It might be better to print a warning that certain defaults are used, which points the user to where they can go to set their own defaults for where files are output, etc.

3b1b · 2021-02-07T23:58:53Z

After asking the manim community a little about their decisions, I suppose it seems fine to put all the "from import *" commands in the __init__.py file, but we should remove the non-manim imports from there.

TonyCrane · 2021-02-08T02:43:53Z

After asking the manim community a little about their decisions, I suppose it seems fine to put all the "from import *" commands in the init.py file, but we should remove the non-manim imports from there.

I have deleted the non-manim imports in __init__.py.

I don't think the first time someone runs manim they should have to fill all this out. It's a nicer user experience if it falls back on some defaults so they can at least jump in and start running scenes. It might be better to print a warning that certain defaults are used, which points the user to where they can go to set their own defaults for where files are output, etc.

I re-added the global default configuration manimlib/default_config.yml.
Now it will only warn that the default configuration is being used when it is run for the first time, and give the path to change it.
At the same time, if one accidentally delete the default configuration in manimlib/, he will enter the configuration guide during runtime.

But there is another issue here. In Windows, if temporary_storage is not set and the system default temporary folder is used, OSError will be thrown:

OSError: C:\Users\...\AppData\Local\Temp\Tex\cf5d7f9f2e57398a.svg not Found

3b1b · 2021-02-08T04:19:06Z

When I run it right now with the manimgl command, there seems to be an issue in initializing the Window. I can try to look into it, but do you have any sense of what might have affected Window?

It works fine for file writing.

Traceback (most recent call last):
  File "/opt/anaconda3/envs/py39/bin/manimgl", line 10, in <module>
    sys.exit(main())
  File "/Users/grant/cs/manim/manimlib/__main__.py", line 14, in main
    scenes = manimlib.extract_scene.main(config)
  File "/Users/grant/cs/manim/manimlib/extract_scene.py", line 112, in main
    return [BlankScene(**scene_config)]
  File "/Users/grant/cs/manim/manimlib/scene/scene.py", line 47, in __init__
    self.window = Window(self, **self.window_config)
  File "/Users/grant/cs/manim/manimlib/window.py", line 18, in __init__
    super().__init__(**kwargs)
  File "/opt/anaconda3/envs/py39/lib/python3.9/site-packages/moderngl_window/context/pyglet/window.py", line 51, in __init__
    self._window = PygletWrapper(
  File "/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pyglet/window/__init__.py", line 648, in __init__
    self._create()
  File "/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pyglet/window/cocoa/__init__.py", line 197, in _create
    self.context.attach(self.canvas)
  File "/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pyglet/gl/cocoa.py", line 299, in attach
    self._nscontext.setView_(canvas.nsview)
AttributeError: 'NoneType' object has no attribute 'setView_'

TonyCrane · 2021-02-08T04:30:16Z

But there is no problem running manimgl on my computer. And I probably didn’t make any changes related to Window.

envs

OS: Windows 10
Python: Python 3.7.9 (anaconda env)
pyglet: 1.5.14

Maybe it's a problem with Python 3.9 ? I am trying.

TonyCrane · 2021-02-08T04:44:02Z

I used anaconda to create a clean Python 3.9 environment, and executed with the following commands, successfully running without error.

conda create -n py39 python=3.9
conda activate py39
pip install cmake -i https://pypi.tuna.tsinghua.edu.cn/simple/ # for mapbox_earcut
pip install -e . -i https://pypi.tuna.tsinghua.edu.cn/simple/
manimgl

pip freeze

backcall==0.2.0
certifi==2020.12.5
cmake==3.18.4.post1
colorama==0.4.4
colour==0.1.5
cycler==0.10.0
decorator==4.4.2
glcontext==2.3
ipython==7.20.0
ipython-genutils==0.2.0
jedi==0.18.0
kiwisolver==1.3.1
-e git+https://github.com/TonyCrane/manim.git@7c683c8992b5829b1de42320f51bcff2e9bce2ea#egg=manimgl
mapbox-earcut==0.12.10
matplotlib==3.3.4
moderngl==5.6.3
moderngl-window==2.3.0
mpmath==1.1.0
multipledispatch==0.6.0
numpy==1.20.1
parso==0.8.1
pickleshare==0.7.5
Pillow==8.1.0
progressbar==2.5
prompt-toolkit==3.0.14
pycairo==1.20.0
pydub==0.24.1
pyglet==1.5.14
Pygments==2.7.4
PyOpenGL==3.1.5
pyparsing==2.4.7
pyreadline==2.1
pyrr==0.10.3
python-dateutil==2.8.1
PyYAML==5.4.1
scipy==1.6.0
screeninfo==0.6.7
six==1.15.0
sympy==1.7.1
tqdm==4.56.0
traitlets==5.0.5
validators==0.18.2
wcwidth==0.2.5
wincertstore==0.2

3b1b · 2021-02-08T18:20:30Z

Creating a similar clean environment with all the same commands, I still run into some OpenGL related issue. (I'm on OSX 11.1)

Traceback (most recent call last):
  File "/opt/anaconda3/envs/tonytest/lib/python3.9/site-packages/OpenGL/platform/darwin.py", line 35, in GL
    return ctypesloader.loadLibrary(
  File "/opt/anaconda3/envs/tonytest/lib/python3.9/site-packages/OpenGL/platform/ctypesloader.py", line 36, in loadLibrary
    return _loadLibraryWindows(dllType, name, mode)
  File "/opt/anaconda3/envs/tonytest/lib/python3.9/site-packages/OpenGL/platform/ctypesloader.py", line 89, in _loadLibraryWindows
    return dllType( name, mode )
  File "/opt/anaconda3/envs/tonytest/lib/python3.9/ctypes/__init__.py", line 382, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: ('dlopen(OpenGL, 10): image not found', 'OpenGL', None)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/anaconda3/envs/tonytest/bin/manimgl", line 6, in <module>
    from manimlib.__main__ import main
  File "/Users/grant/cs/manim/manimlib/__init__.py", line 17, in <module>
    from manimlib.camera.camera import *
  File "/Users/grant/cs/manim/manimlib/camera/camera.py", line 3, in <module>
    import OpenGL.GL as gl
  File "/opt/anaconda3/envs/tonytest/lib/python3.9/site-packages/OpenGL/GL/__init__.py", line 3, in <module>
    from OpenGL import error as _error
  File "/opt/anaconda3/envs/tonytest/lib/python3.9/site-packages/OpenGL/error.py", line 12, in <module>
    from OpenGL import platform, _configflags
  File "/opt/anaconda3/envs/tonytest/lib/python3.9/site-packages/OpenGL/platform/__init__.py", line 36, in <module>
    _load()
  File "/opt/anaconda3/envs/tonytest/lib/python3.9/site-packages/OpenGL/platform/__init__.py", line 33, in _load
    plugin.install(globals())
  File "/opt/anaconda3/envs/tonytest/lib/python3.9/site-packages/OpenGL/platform/baseplatform.py", line 97, in install
    namespace[ name ] = getattr(self,name,None)
  File "/opt/anaconda3/envs/tonytest/lib/python3.9/site-packages/OpenGL/platform/baseplatform.py", line 15, in __get__
    value = self.fget( obj )
  File "/opt/anaconda3/envs/tonytest/lib/python3.9/site-packages/OpenGL/platform/darwin.py", line 62, in GetCurrentContext
    return self.CGL.CGLGetCurrentContext 
  File "/opt/anaconda3/envs/tonytest/lib/python3.9/site-packages/OpenGL/platform/baseplatform.py", line 15, in __get__
    value = self.fget( obj )
  File "/opt/anaconda3/envs/tonytest/lib/python3.9/site-packages/OpenGL/platform/darwin.py", line 45, in CGL
    def CGL(self): return self.GL
  File "/opt/anaconda3/envs/tonytest/lib/python3.9/site-packages/OpenGL/platform/baseplatform.py", line 15, in __get__
    value = self.fget( obj )
  File "/opt/anaconda3/envs/tonytest/lib/python3.9/site-packages/OpenGL/platform/darwin.py", line 41, in GL
    raise ImportError("Unable to load OpenGL library", *err.args)
ImportError: ('Unable to load OpenGL library', 'dlopen(OpenGL, 10): image not found', 'OpenGL', None)

Similar pip freeze

appnope==0.1.2
backcall==0.2.0
certifi==2020.12.5
cmake==3.18.4.post1
colour==0.1.5
cycler==0.10.0
Cython==0.29.21
decorator==4.4.2
glcontext==2.3
ipython==7.20.0
ipython-genutils==0.2.0
jedi==0.18.0
kiwisolver==1.3.1
-e git+git@github.com:3b1b/manim.git@cb4b67655bf9a385344e44dc0b9331c155b921b7#egg=manimgl
mapbox-earcut==0.12.10
matplotlib==3.3.4
moderngl==5.6.3
moderngl-window==2.3.0
mpmath==1.1.0
multipledispatch==0.6.0
numpy==1.20.1
parso==0.8.1
pexpect==4.8.0
pickleshare==0.7.5
Pillow==8.1.0
progressbar==2.5
prompt-toolkit==3.0.14
ptyprocess==0.7.0
pycairo==1.20.0
pydub==0.24.1
pyglet==1.5.14
Pygments==2.7.4
pyobjc-core==7.1
pyobjc-framework-Cocoa==7.1
PyOpenGL==3.1.5
pyparsing==2.4.7
pyrr==0.10.3
python-dateutil==2.8.1
PyYAML==5.4.1
scipy==1.6.0
screeninfo==0.6.7
six==1.15.0
sympy==1.7.1
tqdm==4.56.0
traitlets==5.0.5
validators==0.18.2
wcwidth==0.2.5

TonyCrane · 2021-02-09T02:45:03Z

I merged your commits after this pr and created a new environment, and no errors appeared. This shows that there is no problem on Windows.

I asked a friend of mine who also uses MacOS, and he has similar problems when using the new version of manim. He sent me the following document on how to fix it, I hope it can help you.

In the latest macOS Big Sur, the default location of OpenGL has been changed from the library to the frameworks under the library, so in the current version PyOpenGL cannot find the original location, which will cause an error

ImportError: ('Unable to load OpenGL library','dlopen(OpenGL, 10): image not found','OpenGL', None)

For this, the long-term solution is to wait for the next Python update.
The short-term solution is to change the reading position of PyOpenGL, the steps are as follows:

Find the PyOpenGL file OpenGL/platform/ctypesloader.py under Python folder, and make a complete backup before making any changes.
Find line: fullName = util.find_library( name )
Change it to: fullName = '/System/Library/Frameworks/OpenGL.framework/OpenGL'

3b1b · 2021-02-10T22:13:15Z

Ah, my apologies, I should have been able to recognize that. It's an issue that shows up in other places too but is fixed in Python 3.9, which is why that's what I've usually been using. In setting up the clean environment above, I guess I had forgotten this fact.

Still, when I set it up in a python 3.9 environment, I get that same separate issue, which seems to be related to the Window not having a proper OpenGL context.

Traceback (most recent call last):
  File "/opt/anaconda3/envs/py39/bin/manimgl", line 10, in <module>
    sys.exit(main())
  File "/Users/grant/cs/manim/manimlib/__main__.py", line 14, in main
    scenes = manimlib.extract_scene.main(config)
  File "/Users/grant/cs/manim/manimlib/extract_scene.py", line 115, in main
    scenes = get_scenes_to_render(all_scene_classes, scene_config, config)
  File "/Users/grant/cs/manim/manimlib/extract_scene.py", line 91, in get_scenes_to_render
    return [scene_class(**scene_config) for scene_class in result]
  File "/Users/grant/cs/manim/manimlib/extract_scene.py", line 91, in <listcomp>
    return [scene_class(**scene_config) for scene_class in result]
  File "/Users/grant/cs/manim/manimlib/scene/scene.py", line 47, in __init__
    self.window = Window(self, **self.window_config)
  File "/Users/grant/cs/manim/manimlib/window.py", line 18, in __init__
    super().__init__(**kwargs)
  File "/opt/anaconda3/envs/py39/lib/python3.9/site-packages/moderngl_window/context/pyglet/window.py", line 51, in __init__
    self._window = PygletWrapper(
  File "/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pyglet/window/__init__.py", line 648, in __init__
    self._create()
  File "/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pyglet/window/cocoa/__init__.py", line 197, in _create
    self.context.attach(self.canvas)
  File "/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pyglet/gl/cocoa.py", line 299, in attach
    self._nscontext.setView_(canvas.nsview)
AttributeError: 'NoneType' object has no attribute 'setView_'

I'm still very confused by why anything from this PR (which seems unrelated to the OpenGL context) would have changed this. Mostly I remain confused by what the issue is, exactly.

3b1b · 2021-02-10T22:27:01Z

I suspect it has something to do with how the PygletWindow is being used, and that I probably didn't know what I was doing as I wrote that. Let me look into that a bit more.

TonyCrane · 2021-02-11T02:21:29Z

I think this may be a bug on Big Sur.
But have you tried creating a new environment and then running the current master branch?
I did not make any changes to the window.

TonyCrane · 2021-02-11T03:40:24Z

Or maybe you can try to create a new Python 3.8 environment and run it?

Auxence289 · 2021-02-11T09:35:52Z

Hey Grant. I am the “friend” up there, and I’m currently using the macOS on my MacBook, so I also had to face this problem, though I found a way to solve it. And here’s what I did:
The brand new macOS 11(Big Sur) no longer has the OpenGL library nor other system libraries in standard locations in the file system and instead uses a cache.
As we’re talking, there’s a pull request in Python to fix the ctypes:
python/cpython#21241
But for currently use, we can perform some change in the PyOpenGL file:
Go to ‘/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/OpenGL/platform/ctypesloader.py ’
And in line79, you should find
fullName = util.find_library( name )
And change it into :
fullName = ‘/System/Library/Frameworks/OpenGL.framework/OpenGL’
After the change, the new version of Manim works just fine.
And if there’s any other problems you can just write’em down there, and even though I cannot solve it when troubles getting too difficult, I’ll also try to help;)

3b1b · 2021-02-11T17:39:11Z

I think there are two separate issues. One has to do with Big Sur and find_library, which it seems can be fixed either the way you're describing, by changing the OpenGL code, or by updating to a sufficiently new version of Python, where 3.9 seems to work for me.

But it looks like something separate is going on too, associated with the moderngl window. I suspect it has to do with something wrong I did when it comes to initializing that window and the OpenGL context. In the same environment where master branch runs fine on my system, something about the changes on this branch cause this window initializing issue to happen. This happens in environments with both 3.8 and 3.9, and looks unrelated to the question of loading libraries.

I don't think it's anything wrong with what @TonyCrane wrote, but I'm still hesitant to merge while knowing it will break on some systems, at least without knowing why.

3b1b · 2021-02-11T18:14:17Z

Okay, this is incredibly bizarre. I could fix the issue by removing the line

from manimlib.mobject.interactive import *

From __init__.py. Something about the import line

from pyglet.window import key as PygletWindowKeys

being called before the window creation seemed to affect things.

sahilmakhijani · 2021-02-11T18:55:17Z

Okay, this is incredibly bizarre. I could fix the issue by removing the line
from manimlib.mobject.interactive import *
From init.py. Something about the import line
from pyglet.window import key as PygletWindowKeys
being called before the window creation seemed to affect things.

Ohh! If that's the problem then move the import statement in TextBox on_key_press method since, the PygletWindowKeys is only used there.

def on_key_press(self, mob, event_data):
        from pyglet.window import key as PygletWindowKeys
        symbol = event_data["symbol"]
        modifiers = event_data["modifiers"]
        char = chr(symbol)
        ...

3b1b · 2021-02-11T19:19:48Z

It's also solved by calling "from manimlib.window import *" above that line.

This seems like it's exposing a deeper problem, though, which I'll look into.

TonyCrane · 2021-02-12T01:24:50Z

But this will cause ControlsExample in example_scenes.py can't run, because Textbox, etc. haven't been imported.

kevin-rowney · 2021-06-14T00:24:05Z

Am also struggling with the OpenGL problem. No apparent resolution in sight.

Running Mac OS Big Sur 11.2.3 and I see that:
(i) Mac OS deprecated OpenGL some time ago and the Big Sur upgrade appears to have broken several packages (including manim)
(ii) This StackExchange thread appears to be a great resolution for many people but the many great suggestions provided by contributors but hasn't worked for me.

I gather that not only is OpenGL a deprecated library, but also the Python-native routines to find libraries are also undergoing revisions in an effort to do better at finding binaries to prevent these kinds of problems.

So... I've been >>stuck in a ditch<< for days trying to do what should be a simple install. Sigh....

Help I think I could use:
(A) How do I run a linux "find" command that will precisely identify the specific location of the the OpenGL library? I have, for instance, clearly eliminated the possibility that /System/Library/Frameworks/OpenGL.framework/OpenGL is the path I need; since that file (after many attempts at installation) just does not exist on my system. Could be that I have it....but so far all I can find is >>directories<< named OpenGL (not libraries.)

(B) What command can I run that will guarantee the download, installation, and report-of-location of the OpenGL library? It'd be great if there was some easy way to easily install a fresh OpenGL lib based on Apple's new "Metal" API; but I have failed to find an easy way to do this.

Your help would be appreciated! :)

BTW -- me and my son just LOVE the vids and graphics from the 3b1b team! Stay strong you fine people!

klintan · 2022-03-19T18:57:16Z

I had this problem on Python 3.8 but changing to 3.9 solved the issue. Worth a try for anyone who is on 3.8 and get OpenGL missing issue on Mac.

TonyCrane and others added 7 commits February 7, 2021 10:53

Merge pull request #3 from 3b1b/master

634c3d6

Update

refactor __init__.py and __main__.py

2394bdc

add auto config guide

e3d09d1

add config file to .gitignore

3fa68ad

delete old entry point file and add __init__ for folders

cbe0163

rename custom_defaults -> custom_config defaults -> default_config

00fe339

update README

36107c2

TonyCrane requested a review from 3b1b February 7, 2021 14:08

TonyCrane marked this pull request as draft February 7, 2021 14:08

TonyCrane added 2 commits February 8, 2021 09:55

remove non-manim lib in __init__.py

69235f0

readd default_config and improve config guide

47d4b02

TonyCrane added 3 commits February 8, 2021 10:46

delete comment

608b2d9

Update README

f0cbc05

update docs

24d9d9e

TonyCrane marked this pull request as ready for review February 8, 2021 04:06

adjust the order of arguments in config

944128d

TonyCrane added 2 commits February 8, 2021 12:22

import numpy in example_scenes

6331f76

updte default universal_import_line

7c683c8

update

ddda43e

3b1b merged commit 0cd0a56 into 3b1b:master Feb 11, 2021

TonyCrane deleted the package-improve branch February 12, 2021 01:21

TonyCrane mentioned this pull request Feb 12, 2021

Incorrect window position #1382

Closed

TonyCrane mentioned this pull request Nov 15, 2021

I have been trying to install and run Manim on my Mac and the test scene hasn't been running if anyone could help I would really appreciate it #1681

Closed

Comments

Conversation

TonyCrane commented Feb 7, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Proposed changes

Uh oh!

3b1b commented Feb 7, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

3b1b commented Feb 7, 2021

Uh oh!

TonyCrane commented Feb 8, 2021

Uh oh!

3b1b commented Feb 8, 2021 • edited by TonyCrane Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

TonyCrane commented Feb 8, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

TonyCrane commented Feb 8, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

3b1b commented Feb 8, 2021

Uh oh!

TonyCrane commented Feb 9, 2021

Uh oh!

3b1b commented Feb 10, 2021

Uh oh!

3b1b commented Feb 10, 2021

Uh oh!

TonyCrane commented Feb 11, 2021

Uh oh!

TonyCrane commented Feb 11, 2021

Uh oh!

Auxence289 commented Feb 11, 2021

Uh oh!

3b1b commented Feb 11, 2021

Uh oh!

3b1b commented Feb 11, 2021

Uh oh!

sahilmakhijani commented Feb 11, 2021

Uh oh!

3b1b commented Feb 11, 2021

Uh oh!

TonyCrane commented Feb 12, 2021

Uh oh!

kevin-rowney commented Jun 14, 2021

Uh oh!

klintan commented Mar 19, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

TonyCrane commented Feb 7, 2021 •

edited

Loading

3b1b commented Feb 7, 2021 •

edited

Loading

3b1b commented Feb 8, 2021 •

edited by TonyCrane

Loading

TonyCrane commented Feb 8, 2021 •

edited

Loading

TonyCrane commented Feb 8, 2021 •

edited

Loading