Error on compiling video: partial_movie_file_list.txt: No such file or directory

[cobordism]

If this is a support request:

I encountered this error mulitple times today

$ rm -rf /home/cobordism/Documents/renderings/
$ manim --media_dir /home/cobordism/Documents/renderings/ ./examples.py ycombinator -pl --disable_caching -n 30
---
---
[concat @ 0x560c5416b7c0] Impossible to open 'file:/home/cobordism/Documents/renderings/videos/examples/480p15/partial_movie_files/ycombinator/uncached_00031.mp4'
/home/cobordism/Documents/renderings/videos/examples/480p15/partial_movie_files/ycombinator/partial_movie_file_list.txt: No such file or directory
                    INFO                                                                                scene_file_writer.py:635
                             File ready at /home/cobordism/Documents/renderings/videos/examples/480p15/                         
                             ycombinator.mp4                                                                                    
                                                                                                                                



Traceback (most recent call last):
  File "/home/cobordism/.local/lib/python3.8/site-packages/manim/__main__.py", line 170, in main
    scene = SceneClass()
  File "/home/cobordism/.local/lib/python3.8/site-packages/manim/scene/scene.py", line 93, in __init__
    self.file_writer.finish()
  File "/home/cobordism/.local/lib/python3.8/site-packages/manim/scene/scene_file_writer.py", line 387, in finish
    self.combine_movie_files()
  File "/home/cobordism/.local/lib/python3.8/site-packages/manim/scene/scene_file_writer.py", line 592, in combine_movie_files
    modify_atime(file_path)
  File "/home/cobordism/.local/lib/python3.8/site-packages/manim/utils/file_ops.py", line 53, in modify_atime
    os.utime(file_path, times=(time.time(), os.path.getmtime(file_path)))
  File "/usr/lib/python3.8/genericpath.py", line 55, in getmtime
    return os.stat(filename).st_mtime
FileNotFoundError: [Errno 2] No such file or directory: '/home/cobordism/Documents/renderings/videos/examples/480p15/partial_movie_files/ycombinator/uncached_00031.mp4'

I don't know exactly what is triggering it, but running without -n (and with an empty media dir and no cache ..). that seems to work. Will post more info if I find it.

[huguesdevimeux]

Thanks for the report :D
On second thought I don't think it is related to scene-caching.
Could you try to run the scene, but without passing --media_dirflag and with -v DEBUG ? (and post the full logs).

For me, the issue comes from the directories that are wrongly initialized. The first error message is thrown by FFMPEG and basically says "I can't find the partial movie files there !", and the Expection basically says the same thing, except that it's manim and not FFMEPG.
It's like partial movie files have been created somewhere else.

[cobordism]

Status:
$ manim --media_dir /home/cobordism/Documents/renderings/ ./examples.py ycombinator -pl -v DEBUG
produces the video, no problem.

$ manim --media_dir /home/cobordism/Documents/renderings/ ./examples.py ycombinator -pl -v DEBUG -n 30
produces a strange staccato video up to animation 30 and then proceeds normally.
For example, instead of starting at animation 30, the text that was being written in previous animations just appears. It's as if all the write animations were replaced by scene.add() but all the timings - scene.wait() - remain.
This odd behavious might lead to the following:

$ rm -rf ~/Documents/renderings/*
$ manim --media_dir /home/cobordism/Documents/renderings/ ./examples.py ycombinator -pl -v DEBUG -n 30

This causes the error.

[concat @ 0x55e90a8d27c0] Impossible to open 'file:/home/cobordism/Documents/renderings/videos/examples/480p15/partial_movie_files/ycombinator/582350547_2_2328097757.mp4'
/home/cobordism/Documents/renderings/videos/examples/480p15/partial_movie_files/ycombinator/partial_movie_file_list.txt: No such file or directory
                    INFO                                                                                                          scene_file_writer.py:635
                             File ready at /home/cobordism/Documents/renderings/videos/examples/480p15/ycombinator.mp4                                    
                                                                                                                                                          



Traceback (most recent call last):
  File "/home/cobordism/.local/lib/python3.8/site-packages/manim/__main__.py", line 170, in main
    scene = SceneClass()
  File "/home/cobordism/.local/lib/python3.8/site-packages/manim/scene/scene.py", line 93, in __init__
    self.file_writer.finish()
  File "/home/cobordism/.local/lib/python3.8/site-packages/manim/scene/scene_file_writer.py", line 387, in finish
    self.combine_movie_files()
  File "/home/cobordism/.local/lib/python3.8/site-packages/manim/scene/scene_file_writer.py", line 592, in combine_movie_files
    modify_atime(file_path)
  File "/home/cobordism/.local/lib/python3.8/site-packages/manim/utils/file_ops.py", line 53, in modify_atime
    os.utime(file_path, times=(time.time(), os.path.getmtime(file_path)))
  File "/usr/lib/python3.8/genericpath.py", line 55, in getmtime
    return os.stat(filename).st_mtime
FileNotFoundError: [Errno 2] No such file or directory: '/home/cobordism/Documents/renderings/videos/examples/480p15/partial_movie_files/ycombinator/582350547_2_2328097757.mp4'

and for good measure:

$ manim ./examples.py ycombinator -pl -v DEBUG -n 30

causes the same error - so it is not a media-dir question. It's a problem of starting at -n 30

PS the extra debug output is not helpful at all. All it has is stuff like:

[09/21/20 19:15:54] DEBUG    Hashing ...                                                                                                    hashing.py:234
[09/21/20 19:15:55] DEBUG    Hashing done in 1.595105 s.                                                                                    hashing.py:249
[09/21/20 19:15:57] INFO     Animation 44 : Partial movie file written in {'/home/cobordism/Documents/manimstuff/mylectures/lambd scene_file_writer.py:463
                             a-manim-ce/media/videos/examples/480p15/partial_movie_files/ycombinator/582350547_2_2387872213.mp4'}                         
                    DEBUG    Hashing ...                                                                                                    hashing.py:280
[09/21/20 19:15:58] DEBUG    Hashing done in 1.582779 s.                                                                                    hashing.py:305
[09/21/20 19:15:59] INFO     Animation 45 : Partial movie file written in {'/home/cobordism/Documents/manimstuff/mylectures/lambd scene_file_writer.py:463
                             a-manim-ce/media/videos/examples/480p15/partial_movie_files/ycombinator/582350547_373238890_42411689                         
                             09.mp4'}                                                                                                                     
                    WARNING  TextMobject has been deprecated (due to its confusing name)in favour of Tex. Please use Tex instead!       tex_mobject.py:349
                    INFO     Writing "\raggedright So now we can define factorial as:\\factorial $\equiv$ (Y factorial')." to       tex_file_writing.py:42
                             media/Tex/be15788f046f08a9.tex                                                                                               
[09/21/20 19:16:00] DEBUG    Hashing ...                                                                                                    hashing.py:234
[09/21/20 19:16:01] DEBUG    Hashing done in 1.660851 s.                                                                                    hashing.py:249
[09/21/20 19:16:02] INFO     Animation 46 : Partial movie file written in {'/home/cobordism/Documents/manimstuff/mylectures/lambd scene_file_writer.py:463
                             a-manim-ce/media/videos/examples/480p15/partial_movie_files/ycombinator/582350547_2_2595929461.mp4'}                         
                    DEBUG    Hashing ...                                                                                                    hashing.py:280
[09/21/20 19:16:04] DEBUG    Hashing done in 1.654192 s.                                                                                    hashing.py:305
[09/21/20 19:16:05] INFO     Animation 47 : Partial movie file written in {'/home/cobordism/Documents/manimstuff/mylectures/lambd scene_file_writer.py:463
                             a-manim-ce/media/videos/examples/480p15/partial_movie_files/ycombinator/582350547_7561646_674770672.                         
                             mp4'} 

PPS
I might add that media/videos/examples/480p15/partial_movie_files/ycombinator/partial_movie_file_list.txt contains 34 entries, but
media/videos/examples/480p15/partial_movie_files/ycombinator/ contains only 18 videos one the txt file.

[huguesdevimeux]

Intersting. If the bug is reproducible, could you please share (a reduced version, if possible ^^) the code causing the error?

produces a strange staccato video up to animation 30 and then proceeds normally.

And, what do you mean by this ? could you share the video ?

And, maybe you should take a look at the directory where partial movie files are stored; there might be some clues down there. My guess is that it's comes from the scene caching mechanism, which is tricked into thinking that a partial movie file exists but in fact it does not exist. The question is now, why does it think that.

And last thing, do you think it has something to do with tex generation or/and caching ?

(And, just to let you know, there is --flush_cache flag that can remove all the partial movie files. Can be useful ;)

[cobordism]

so. here is a mimimal version that produces the error message:

class test(Scene):
    def write(self, txt, **kwargs):
        self.play(ShowCreation(txt, **kwargs))    
    def construct(self):
        self.write(TextMobject("this is a test"))
        self.wait(1)
        self.write(TextMobject("this is a test 2 ").shift(DOWN))
        self.wait(1)
        self.write(TextMobject("this is a test 3 ").shift(2* DOWN))
        self.wait(1)
        self.write(TextMobject("this is a fourth test").shift(3*DOWN))
        self.wait(1)    

run the command $ manim example.py test -n 3 -pl

[cobordism]

And, what do you mean by this ? could you share the video ?

I am currently not able to reproduce this. What I meant was that I had added a lot of text with self.play(ShowCreation(txt)) but I was getting text appearing as if I had been using self.add(txt) instead... i.e. the animations were gone but the timings were still there.
I'll let you know if I can recreate it, until then let's regard it as a fluke.

[cobordism]

Update. I have been able to recreate it.

Try this:
Step 1: empty the data dir. rm -rf ./media
Step 2: $ manim example.py test -pl (should work fine)
Step 3: $ manim example.py test -pl -n 3 (staccato effect)
Step 4: empty the data dir. rm -rf ./media
Step 5: $ manim example.py test -pl -n 3 (error message, but video produced ok)
Step 6: $ manim example.py test -pl (error message, but video produced ok)

[huguesdevimeux]

It should be fixed by #468 . Could you check this out ?

[behackl]

I am running into a similar issue when doing make html && make clean && make html from docs:

                    INFO     Animation 0 : Using cached data (hash : 2818502609_1974117047_3316235269)                                                                                                                                                                                                 scene.py:825
[09/23/20 09:44:33] INFO     Animation 1 : Partial movie file written in {'/home/behackl/code/manim/docs/source/media/videos/720p30/partial_movie_files/TestZoom1-1/2818502609_1974117047_3316235269.mp4'}                                                                                 scene_file_writer.py:465
[concat @ 0x55e6003f2560] Impossible to open 'file:/home/behackl/code/manim/docs/source/media/videos/720p30/partial_movie_files/TestZoom1-1/2796914319_881011468_1267139589.mp4'
/home/behackl/code/manim/docs/source/media/videos/720p30/partial_movie_files/TestZoom1-1/partial_movie_file_list.txt: No such file or directory
                    INFO                                                                                                                                                                                                                                                                   scene_file_writer.py:635
                             File ready at /home/behackl/code/manim/docs/source/media/videos/720p30/TestZoom1-1.mp4

The produced file is not complete, however.

Log from the first pass of rendering:

[09/23/20 09:41:54] INFO     Animation 0 : Partial movie file written in {'/home/behackl/code/manim/docs/source/media/videos/720p30/partial_movie_files/TestZoom1-1/2818502609_1974117047_3316235269.mp4'}                                                                                 scene_file_writer.py:465
[09/23/20 09:41:55] INFO     Animation 1 : Partial movie file written in {'/home/behackl/code/manim/docs/source/media/videos/720p30/partial_movie_files/TestZoom1-1/1160423895_881011468_1267139589.mp4'}                                                                                  scene_file_writer.py:465
                    INFO                                                                                                                                                                                                                                                                   scene_file_writer.py:635
                             File ready at /home/behackl/code/manim/docs/source/media/videos/720p30/TestZoom1-1.mp4                                                                                                                                                                                                
                                                                                                                                                                                                                                                                                                                   
                    INFO     Rendered TestZoom1                                                                                                                                                                                                                                                        scene.py:129
                             Played 2 animations

partial_movie_file_list.txt after the first pass:

# This file is used internally by FFMPEG.
file 'file:/home/behackl/code/manim/docs/source/media/videos/720p30/partial_movie_files/TestZoom1-1/2818502609_1974117047_3316235269.mp4'
file 'file:/home/behackl/code/manim/docs/source/media/videos/720p30/partial_movie_files/TestZoom1-1/1160423895_881011468_1267139589.mp4'

partial_movie_file_list.txt after the second pass:

# This file is used internally by FFMPEG.
file 'file:/home/behackl/code/manim/docs/source/media/videos/720p30/partial_movie_files/TestZoom1-1/2818502609_1974117047_3316235269.mp4'
file 'file:/home/behackl/code/manim/docs/source/media/videos/720p30/partial_movie_files/TestZoom1-1/2796914319_881011468_1267139589.mp4'

Actually, the same thing happens when running manim on the corresponding code

from manim import *


class TestZoom1(ZoomedScene):
        CONFIG = {
            "zoomed_camera_frame_starting_position": [0, 0, 0],
            "zoomed_display_corner": [0, 0, 0],
            "zoomed_display_height": config['frame_height'],
            "zoomed_display_width": config['frame_width'],
            "zoom_factor": 0.1,
        }
        def construct(self):
            self.activate_zooming(animate=True)
            d = Dot()
            self.add(d)

twice. Unfortunately, #468 does not resolve this.

[cobordism]

It should be fixed by #468 . Could you check this out ?

I can confirm that this seems to fix my issue.

@behackl Your TestZoom1 scene compiles just fine over here. I believe you are encountering a different problem than the one in this github issue.

[behackl]

It should be fixed by #468 . Could you check this out ?

I can confirm that this seems to fix my issue.

Good to hear! Then I'll move this to a separate issue.

@behackl Your TestZoom1 scene compiles just fine over here. I believe you are encountering a different problem than the one in this github issue.

Yes, I thought so as well. But I am still surprised: compiling the scene twice (without changing anything in the media directory in between) works for you? I get the same error on the second try consistently on multiple machines.

[cobordism]

Your right!

I also get an error on second compiling!

$ rm -rf media/

$ manim example2.py TestZoom1 -pl 
[09/23/20 14:17:44] INFO     Animation 0 : Partial movie file written in {'/home/cobordism/[...]/media/videos/example2/480p15/partial_m                         
                             ovie_files/TestZoom1/734027784_1974117047_3316235269.mp4'}                                         
[09/23/20 14:17:45] INFO     Animation 1 : Partial movie file written in {'/home/cobordism/[...]/media/videos/example2/480p15/partial_m                         
                             ovie_files/TestZoom1/1724874366_881011468_1267139589.mp4'}                                         
                    INFO                                                                                scene_file_writer.py:635
                             File ready at /home/cobordism/[...]/media/videos/example2/480p15/TestZoom1.mp4                                                     
                                                                                                                                
                    INFO     Rendered TestZoom1                                                                     scene.py:129
                             Played 2 animations       

                                                                         
$ manim example2.py TestZoom1 -pl 
[09/23/20 14:17:51] INFO     Animation 0 : Using cached data (hash : 734027784_1974117047_3316235269)               scene.py:823
                    INFO     Animation 1 : Partial movie file written in {'/home/cobordism/[...]/media/videos/example2/480p15/partial_m                         
                             ovie_files/TestZoom1/734027784_1974117047_3316235269.mp4'}                                         
[concat @ 0x55fd9bb9b7c0] Impossible to open 'file:/home/cobordism/[...]/media/videos/example2/480p15/partial_movie_files/TestZoom1/2803391946_881011468_1267139589.mp4'
/home/cobordism/[...]/media/videos/example2/480p15/partial_movie_files/TestZoom1/partial_movie_file_list.txt: No such file or directory
                    INFO                                                                                scene_file_writer.py:635
                             File ready at /home/cobordism/[...]/media/videos/example2/480p15/TestZoom1.mp4                                                     
                                                                                                                                



Traceback (most recent call last):
  File "/home/cobordism/.local/lib/python3.8/site-packages/manim/__main__.py", line 170, in main
    scene = SceneClass()
  File "/home/cobordism/.local/lib/python3.8/site-packages/manim/scene/scene.py", line 93, in __init__
    self.file_writer.finish()
  File "/home/cobordism/.local/lib/python3.8/site-packages/manim/scene/scene_file_writer.py", line 387, in finish
    self.combine_movie_files()
  File "/home/cobordism/.local/lib/python3.8/site-packages/manim/scene/scene_file_writer.py", line 592, in combine_movie_files
    modify_atime(file_path)
  File "/home/cobordism/.local/lib/python3.8/site-packages/manim/utils/file_ops.py", line 53, in modify_atime
    os.utime(file_path, times=(time.time(), os.path.getmtime(file_path)))
  File "/usr/lib/python3.8/genericpath.py", line 55, in getmtime
    return os.stat(filename).st_mtime
FileNotFoundError: [Errno 2] No such file or directory: '/home/cobordism/[...]/media/videos/example2/480p15/partial_movie_files/TestZoom1/2803391946_881011468_1267139589.mp4'