Problems with Video/Audio if media source is v.redd.it

[rom-1]

So, maybe there is no solution to this, but I just want to give it a try and ask you this.
(Side Note: In the RedditImage-Extension I did not enable the option to mute videos, so every video should play with audio.)

This is example Reddit Post A: https://www.reddit.com/r/interestingasfuck/comments/mnjke0/the_bionic_woman/
It embeds a video from gfycat.com. The source code in FreshRSS of the post is:

<div class="reddit-image figure">
    <video controls="" preload="metadata" class="reddit-image">
        <source src="https://giant.gfycat.com/HeartySillyCockatiel.webm" type="video/webm">
        <source src="https://giant.gfycat.com/HeartySillyCockatiel.mp4" type="video/mp4">
    </video>
</div>

This post works flawless in any scenario:

• I can play the video within FreshRSS, and audio plays without any problems
• I can visit both source urls manually, and the video plays in my browser
• my smartphone RSS Reader App (FeedMe) plays the video without any problem

This is example Reddit Post B: https://www.reddit.com/r/nextfuckinglevel/comments/mnouq4/learning_to_sing/
It embeds a video from v.redd.it. The source code in FreshRSS of the post is:

<div class="reddit-image figure">
    <video controls="" preload="metadata" class="reddit-image">
        <source src="https://v.redd.it/g5xu53hv97s61/DASH_720.webm" type="video/webm">
        <source src="https://v.redd.it/g5xu53hv97s61/DASH_720.mp4" type="video/mp4">
    </video>
</div>

Posts who embed media from v.redd.it are almost never working. I experience the following problems:

• I can play the video within FreshRSS, but only without audio. It seems the audio portion of the video is missing (speaker button is greyed out), but when visiting the original reddit post on reddit.com, it turns out there is an audio part!
• The first source url (webm type) does not work at all if visiting manually -> "Access denied"
• The second source url (mp4 type) shows the video, but again the audio part is missing
• my smartphone RSS Reader App (FeedMe) tells me the link to the video is broken and cannot be loaded. I guess the App only tries accessing the first webm url which results in an "access denied" http response.

So somehow, on reddit.com, the video does contain audio. I have no idea why it is missing audio in FreshRSS or using the direct mp4 link. Also, if the webm url never works with v.redd.it, it should not be listed as an available source in html, as this is confusing for 3rd party software.

Hope this makes sense ;)

[rom-1]

ok, according to this post and many more angry posts, it is only possible to find a direct link using the fallback url in the json file. and even then, this file always does not contain any audio. it is possible to get the audio part by changing the filename from DASH_xxx.mp4 to DASH_audio.mp4 (example: https://v.redd.it/g5xu53hv97s61/DASH_audio.mp4), but this only gives you the audio part. There seem to be no direct why to retrieve a mp4 for both parts. :(

nevertheless, stripping the inaccessible .webm source url from v.redd.it media would be an improvement.

[rom-1]

sorry for spamming here. I was thinking of a possible solution for this audio problem. This is some kind of a dirty hack, but actually working: if we are referencing the video and audio urls, it is possible to use javascript to play both at the same time. I have no idea if this is a good idea... we would need to randomly generate id's for every video/audio pair on that page (multiple posts) so the javascript is referencing the correct files, but we could maybe use the FreshRSS Flux id of the post?

<video id="myvideo" controls muted>
    <source src="https://v.redd.it/g5xu53hv97s61/DASH_720.mp4" type="video/mp4" />
    <audio id="myaudio" controls>
        <source src="https://v.redd.it/g5xu53hv97s61/DASH_audio.mp4" type="audio/mp4"/>
    </audio>
</video>

<script>
var myvideo = document.getElementById("myvideo");
var myaudio = document.getElementById("myaudio");
myvideo.onplay  = function() { myaudio.play();  }
myvideo.onpause = function() { myaudio.pause(); }
myvideo.onseeking = function() { myaudio.currentTime = myvideo.currentTime; }
</script>

edit: added onseeking function

[aledeg]

@rom-1 thank you for digging into that. I am not inclined to add such hack in the extension. I've noticed that there was some discrepancies on some videos but I never searched how to fix that. The video support in this extension is a bonus. When I started it, I wanted to have image support hence the name.
I have other things to improve before maybe working on that.
If you find some way of having something not too hacky and you're willing to contribute, I might add it.

[rom-1]

Sure, that's fine with me. I agree that this is solution would be a really dirty hack. Because of the way v.redd.it delivers it's media files, currently I see no good way to enhance the handling of the audio part with video files for now.

BUT this still leaves one "bug" I mentioned in this issue: could you leave out the faulty .webm source url? that would be quite helpful for 3rd party applications. v.redd.it has no .webm file, and linking to it as a source file results in some errors on my side.

[aledeg]

BUT this still leaves one "bug" I mentioned in this issue: could you leave out the faulty .webm source url? that would be quite helpful for 3rd party applications. v.redd.it has no .webm file, and linking to it as a source file results in some errors on my side.

You're right! Could you open a separate issue for that to keep track of it? Thank you!

[aledeg]

@rom-1 Do you know if the hack you've mentioned is working? I am working on that at the moment and I am trying to figure how to include correctly the audio tag. I do not want to spend some time for nothing. Thank you for your feedback.

[rom-1]

yep, this actually seems to work for me, I tried it by testing it with an empty html file. example:
this reddit post currently results in this html code using your extension:

<div class="reddit-image figure">
<video controls="1" preload="metadata" class="reddit-image" muted=""><source src="https://v.redd.it/5xaihidtjmt61/DASH_720.mp4" type="video/mp4"></video>
</div>

the video does not have audio when pressing "play".

if we change the source code to this, it still shows no audio (speaker icon is greyed out), but actually we have sound when pressing play. please note that I used the flux id of the particular post for every id used in this part.

<div class="reddit-image figure">
<video id="flux_1618666827618273_video" controls muted>
    <source src="https://v.redd.it/5xaihidtjmt61/DASH_720.mp4" type="video/mp4" />
    <audio id="flux_1618666827618273_audio" controls>
        <source src="https://v.redd.it/5xaihidtjmt61/DASH_audio.mp4" type="audio/mp4"/>
    </audio>
</video>

<script>
var flux_1618666827618273_video = document.getElementById("flux_1618666827618273_video");
var flux_1618666827618273_audio = document.getElementById("flux_1618666827618273_audio");
flux_1618666827618273_video.onplay  = function() { flux_1618666827618273_audio.play();  }
flux_1618666827618273_video.onpause = function() { flux_1618666827618273_audio.pause(); }
flux_1618666827618273_video.onseeking = function() { flux_1618666827618273_audio.currentTime = flux_1618666827618273_video.currentTime; }
</script>
</div>

also, please note that the audio part has not to be within the video tag. you could place the audio part below the video part, but this would mean that the audio part would be rendered as a separate control below the video.

[aledeg]

Thank you for those tests. I will add the audio tag but not the javascript.
I will add a note in the README with a snippet to add in the customJS extension.
I don't know when this will be available though!

[rom-1]

Sure thing, thank you! I will give it a try once it's available. 🙂

[aledeg]

@rom-1 I've added the audio tag in the video tag for v.redd.it videos. It's not released yet but you can still try it. Let me know what you think. There is a section in the readme for that.
I've completely rewrote the extension to make it more maintainable. There is still room for improvement but it's getting better and less hackish.

[rom-1]

Thanks! I just gave it a try, but unfortunately it looks like the audio tag is not embedded in html. I double checked Media/Video.php on my webspace, and it contains the new code you added.
Test post: https://www.reddit.com/r/aww/comments/kzzt3f/turns_out_cows_like_music/

Results in this html post in FreshRSS:

<div class="reddit-image figure">
<!--xExtension-RedditImage | DisplayTransformer | Video link-->
<video controls="1" preload="metadata" class="reddit-image" muted="1">
<source src="https://v.redd.it/q4rfn6ioq4c61/DASH_720.mp4" type="video/mp4">
</video>
</div>

[aledeg]

There is 2 transformers. One for display (executed each time you display an article), one for insertion (executed once when the feed is read). I've added it in the insertion transformer.
Your example comes from the display transformer hence the comment

[aledeg]

I rework the extension to do most of the work during insertion since it's pretty heavy to process api.
One of my goals is to have a maintenance mode that will rework articles that are already in the database.
But for now, you'll have to wait for new articles to see the change. The old one wont have the audio track added.

[rom-1]

ok, looks I have no clue what I am talking about. I don't understand it, sorry. I thought the "audio" tag has to be within the html somewhere, so my browser sees which audio file has to be played/loaded. otherwise, the browser does not have any clue about the source path to the audio file.

so, the current state with the new extension code is: it does not work for me. I inserted the javascript into the CustomJS and the audio file does not play

[rom-1]

I rework the extension to do most of the work during insertion since it's pretty heavy to process api.

ahh!! this is the reason for my problems. ok then, lets wait a bit until there is a new media post coming from reddit 😂

[aledeg]

You can make some tests with https://reddit.com/user/Palifaith/submitted/.rss

Here is what I have in my DOM for that kind of content:

<div class="reddit-image figure">
    <!--xExtension-RedditImage | InsertTransformer | Reddit video-->
    <video controls="1" preload="metadata" class="reddit-image">
        <audio controls="1">
            <source src="https://v.redd.it/ciild7rrogv61/DASH_audio.mp4">
        </audio>
        <source src="https://v.redd.it/ciild7rrogv61/DASH_480.mp4" type="video/mp4">
    </video>
</div>

[rom-1]

Thanks, this helped. In the feed you mentioned, there is this post: https://www.reddit.com/r/nextfuckinglevel/comments/mhirwn/people_buy_out_entire_stores_doughnuts_so_the/

This results in the following html code:

<div class="reddit-image figure">
<!--xExtension-RedditImage | InsertTransformer | Reddit video-->
<video controls="1" preload="metadata" class="reddit-image">
<audio controls="1"><source src="https://v.redd.it/vqwj6ayv9gq61/DASH_audio.mp4"></audio>
<source src="https://v.redd.it/vqwj6ayv9gq61/DASH_360.mp4" type="video/mp4"></video>
</div>

so far, so good.

When pressing the play button on one of the videos, I am now receiving a Javascript Console Error:

ext.php?f=xExtension-CustomJS%2Fstatic%2Fscript.firstname.js&t=js&1619440995:3 Uncaught TypeError: Cannot read property 'play' of null
    at HTMLVideoElement.<anonymous> (ext.php?f=xExtension-CustomJS%2Fstatic%2Fscript.firstname.js&t=js&1619440995:3)
(anonymous) @ ext.php?f=xExtension-CustomJS%2Fstatic%2Fscript.firstname.js&t=js&1619440995:3

maybe this matters because my default view in FreshRSS is, to "unfold articles by default" so the JS does not find a valid target?

[rom-1]

so... turns out, the JS error only appears if the media file does not have audio, or if the file is hosted at gifycat instead of v.redd.it (but should have audio, though).

So the JS error can be ignored for now. Nevertheless: It unfortunately it does not work for me. Pressing "play" does not have any effect on the audio part, and I have no clue why not. I guess the JS code does not add a listener as it should do?

do you have any tips how I could debug this any further?

[aledeg]

Have you tried the snippet in the readme directly in your console?
It should work even if the video is collapsed.

The snippet is really simple, it does not check if there is audio, it does not stop or pause the video.
You'll have to implement it yourself.

[rom-1]

ok, after a lot of tries, I finally figured my problem out. and it's embarassing, I hope you have a good laugh:
sometime in the past, I have "muted" the chrome tab of my FreshRSS Instance. The browser was blocking audio the whole time. Because I told him to do so in the past. omg I am dumb.

So, this works flawless now. I am using the following javascript for a better experience, if you want to update your readme file:

document.querySelectorAll('video.reddit-image').forEach(element => {
  element.addEventListener('play', event => { event.target.querySelector('audio').play(); })
  element.addEventListener('pause', event => { event.target.querySelector('audio').pause(); })
  element.addEventListener('seeking', event => { event.target.querySelector('audio').currentTime = event.target.currentTime })
});

of course, this does not work in the FeedMe Android App. I guess the app is stripping all kind of javascript or the extension CustomJS is not even used when displaying posts within the app. I think that is as far as you can get to enable v.redd.it videos with your extension to play flawless, thanks a lot for your help!

[aledeg]

ok, after a lot of tries, I finally figured my problem out. and it's embarassing, I hope you have a good laugh:
sometime in the past, I have "muted" the chrome tab of my FreshRSS Instance. The browser was blocking audio the whole time. Because I told him to do so in the past. omg I am dumb.

We've all made similar mistake, don't beat yourself up.

I am guessing that the snippet you provided does not work on articles that are loaded while scrolling. Could you confirme?

I will update the readme file with your content. Thank you!

[rom-1]

I am guessing that the snippet you provided does not work on articles that are loaded while scrolling. Could you confirme?

mh.. no? I have not seen any bad side effects while scrolling, all videos have audio and seeking works as expected. I am using lazy loading if that helps?

[aledeg]

OK, good. I guess that the CustomJS extension has some kind of handler to do that. Which makes sense.

[aledeg]

I'll add your example in the readme and I'll close that issue afterward. Except if you have something that is not addressed in what I have done.
I'll publish a new version later.

[rom-1]

Not that I am aware of. Go ahead 🙂

[aledeg]

Fixed in https://github.com/aledeg/xExtension-RedditImage/tree/v0.12.0