Display link preview

[alanpoon]

Fixes #384
Closes #81

Display link previews beneath a message containing a link.
https://github.com/user-attachments/assets/30f36549-f1d5-47e8-aa08-683eac1d1224

[kevinaboos]

Very nice work!

It looks like we've migrated away from Alex's link preview crate, in favor of using the matrix homeserver's own link preview feature? I didn't know the homeserver offered that, so that's pretty cool to see.

I haven't yet reviewed the link_preview.rs widget code yet, but I wanted to give you some quick feedback here to avoid making you wait longer.

Two top-level questions:

1. Can you test what happens when a message has multiple link previews? Based on the populate_link_preview_below_message() function, I assume they will each be shown underneath each other, but I haven't tried it myself. Generally, we want to limit the number of link previews to a small number, maybe 3 maximum. We could also make that a user setting later, if desired.
2. Most importantly, we want to ensure that the timeline view doesn't jump around unexpectedly while the user is looking at it. In order to do that, we need to always show a placeholder view for the link preview even when the preview content itself is not yet available yet, or if the preview failed to load. That way, the height of the entire message view is always the same. I haven't checked if you've already done this, but please confirm that this behavior is followed or change the code to do so.

Kindly capture a screen recording showing what happens in the two above cases: multiple link previews in one message, and a consistent message view including a placeholder for a link preview that hasn't loaded yet or fails to load.

[alanpoon]

nsistent message view including a placeholder for a link preview that hasn't loaded yet or fails to load.

Here is the screen recording.
https://github.com/user-attachments/assets/d0cbbcf9-eec8-4d74-a5f6-cc893630db65
The multiple links in a message is at 0:08.

[kevinaboos]

Here is the screen recording. user-attachments/assets/d0cbbcf9-eec8-4d74-a5f6-cc893630db65 The multiple links in a message is at 0:08.

Thanks for sharing the recording. So, the behavior around 0:05 - 0:06 is exactly what I was referring to when I wrote this:

we want to ensure that the timeline view doesn't jump around unexpectedly while the user is looking at it. In order to do that, we need to always show a placeholder view for the link preview even when the preview content itself is not yet available yet, or if the preview failed to load. That way, the height of the entire message view is always the same.

You can see that the timeline jumps around a lot from 0:05 to 0:07 when the link previews actually load. What we need to do instead is to immediately (and always) show the link preview view if there are any links found in the message, rather than waiting for the preview to load before showing it. This will ensure that the timeline height is always the same for any message containing links.

It also looks like there's too much vertical space beneath the link preview image/text:

[alanpoon]

Fixed the height by 80. show the link preview view if there are any links, rather than waiting for preview.

[kevinaboos]

Looks pretty good, just a few small comments.

There's a to-do item in the link_preview widget about using a TextOrImage widget, as well as something about the text not being displayed if the image is too small. Let me know what's going on there.

[kevinaboos]

i left some comments on the existing conversations.

[alanpoon]

Added default image

[kevinaboos]

I noticed some UI jumping that appears to be caused when there is an error fetching the link preview. It looks like the link preview disappears (perhaps it's just hidden upon an error)... but let's not do that. Once we show the link preview, we must never hide it in order to keep the timeline UI stable.

Here's a message with a link that failed to load the link preview:

src/sliding_sync.rs:1307:25 - URL preview response status for https://github.com/element-hq/element-x-android/issues/5430: 200 OK
src/sliding_sync.rs:1319:25 - URL preview response body length for https://github.com/element-hq/element-x-android/issues/5430: 444 bytes
src/sliding_sync.rs:1323:29 - URL preview response body for https://github.com/element-hq/element-x-android/issues/5430: {"og:image:alt":"Please see and discuss the details in the meta issue.","og:image:width":"1200","og:image:height":"600","og:site_name":"GitHub","og:type":"object","og:title":"[Task] Introduce a Labs section in EX \u00b7 Issue #5430 \u00b7 element-hq/element-x-android","og:url":"https://github.com/element-hq/element-x-android/issues/5430","og:description":"Please see and discuss the details in the meta issue.","og:author:username":"manuroe"}
src/sliding_sync.rs:1328:33 - Failed to parse JSON response for URL preview https://github.com/element-hq/element-x-android/issues/5430: invalid type: string "1200", expected u64 at line 1 column 95
src/sliding_sync.rs:1329:33 - Response body that failed to parse: {"og:image:alt":"Please see and discuss the details in the meta issue.","og:image:width":"1200","og:image:height":"600","og:site_name":"GitHub","og:type":"object","og:title":"[Task] Introduce a Labs section in EX \u00b7 Issue #5430 \u00b7 element-hq/element-x-android","og:url":"https://github.com/element-hq/element-x-android/issues/5430","og:description":"Please see and discuss the details in the meta issue.","og:author:username":"manuroe"}
src/sliding_sync.rs:1340:29 - URL preview fetch failed for https://github.com/element-hq/element-x-android/issues/5430: JSON parsing failed: invalid type: string "1200", expected u64 at line 1 column 95
src/home/link_preview.rs:500:13 - Failed to fetch link preview data for https://github.com/element-hq/element-x-android/issues/5430: Json(Error("invalid type: string \"1200\", expected u64", line: 1, column: 95))

It looks like the response body wasn't parsed properly, i.e., you didn't expect that the og:image:width field might be a string or just a raw integer.

EDIT: here's another example:

src/sliding_sync.rs:1279:25 - Getting Matrix client for URL preview: https://github.com/element-hq/element-web/issues/30969
src/sliding_sync.rs:1291:25 - Fetching URL preview from endpoint: https://matrix-client.matrix.org/_matrix/media/v3/preview_url for URL: https://github.com/element-hq/element-web/issues/30969
src/sliding_sync.rs:1307:25 - URL preview response status for https://github.com/element-hq/element-web/issues/30969: 200 OK
src/sliding_sync.rs:1319:25 - URL preview response body length for https://github.com/element-hq/element-web/issues/30969: 740 bytes
src/sliding_sync.rs:1323:29 - URL preview response body for https://github.com/element-hq/element-web/issues/30969: {"og:image:alt":"Steps to reproduce Where are you starting? What can you see? When a notification message sound is played through a RDP session the audio clips and is jaring to the ears, the old sound did not do th...","og:image:width":"1200","og:image:height":"600","og:site_name":"GitHub","og:type":"object","og:title":"New message notification sound clips over RDP \u00b7 Issue #30969 \u00b7 element-hq/element-web","og:url":"https://github.com/element-hq/element-web/issues/30969","og:description":"Steps to reproduce Where are you starting? What can you see? When a notification message sound is played through a RDP session the audio clips and is jaring to the ears, the old sound did not do th...","og:author:username":"edwardsdean"}
src/sliding_sync.rs:1328:33 - Failed to parse JSON response for URL preview https://github.com/element-hq/element-web/issues/30969: invalid type: string "1200", expected u64 at line 1 column 242
src/sliding_sync.rs:1329:33 - Response body that failed to parse: {"og:image:alt":"Steps to reproduce Where are you starting? What can you see? When a notification message sound is played through a RDP session the audio clips and is jaring to the ears, the old sound did not do th...","og:image:width":"1200","og:image:height":"600","og:site_name":"GitHub","og:type":"object","og:title":"New message notification sound clips over RDP \u00b7 Issue #30969 \u00b7 element-hq/element-web","og:url":"https://github.com/element-hq/element-web/issues/30969","og:description":"Steps to reproduce Where are you starting? What can you see? When a notification message sound is played through a RDP session the audio clips and is jaring to the ears, the old sound did not do th...","og:author:username":"edwardsdean"}
src/sliding_sync.rs:1340:29 - URL preview fetch failed for https://github.com/element-hq/element-web/issues/30969: JSON parsing failed: invalid type: string "1200", expected u64 at line 1 column 242
src/home/link_preview.rs:500:13 - Failed to fetch link preview data for https://github.com/element-hq/element-web/issues/30969: Json(Error("invalid type: string \"1200\", expected u64", line: 1, column: 242))

[kevinaboos]

In addition to my above comment about not hiding link previews upon failure to fetch, here are some other minor issues/nitpicks that we should address now before merging this in.

1. I think we should remove the vertical bar on the left of the link preview, because that makes it look like an inline reply preview or a quote, which might confuse the user.
2. We need to ensure that the link preview's image is vertically centered with respect to the gray-background view. For example, this doesn't look very good because it's not centered:

4. Add a bit of vertical margin to the top of the link preview, maybe just 3-5 pixels, such that it looks a bit cleaner. There's not much separation right now and it can be a bit hard to read. For example, the link preview almost collides/overlaps with the underlined link in this message:

5. Please horizontally align the link preview title with the body text. In the above screenshot, "Introduction" and "Keyboard" should be left-aligned together.
6. We could also consider using a different background color for links, instead of a light gray. Something like a light blue, light green, or light purple. You can experiment with a few options and see which one looks best.
7. The "default image" i provided was just a suggestion. You can use it, but you probably need to crop the actual image file to remove the surrounding grayspace such that you don't need to pan/mask it when displaying it within an Image widget.
   • I found a bunch of versions of that image here that you can choose from: https://www.aputf.org/wp-content/uploads/2015/06/

[alanpoon]

Added collapsible button
https://github.com/user-attachments/assets/2bbee5dc-a7ef-4d76-a5ad-060db8f3818b

1. Remove vertical bar
2. Centered the image.
3. Added vertical margin of 3 to top margin
4. Left Align the title and description
5. Added Purple color for link for hovering
6. Cropped the default image, to make it look bigger

[kevinaboos]

Great work Alan, thanks so much!