jdev - 2026-03-12

As I was wondering what size I should use for in-band link metadata image preview, I ran some quick tests with pillow and numpy, maybe some of you will find that interesting: ``` median size of 50 base64-encoded JPEG random RGB image 128x128 pixels: 14kb ± 0.03 256x256 pixels: 53kb ± 0.06 384x384 pixels: 119kb ± 0.10 512x512 pixels: 211kb ± 0.15 640x640 pixels: 329kb ± 0.20 768x768 pixels: 473kb ± 0.21 896x896 pixels: 644kb ± 0.20 1024x1024 pixels: 841kb ± 0.24 ``` I was initially going for 128x128 but that seems way too conservative as prosody defaults to 256kb stanza size for s2s and and 10Mb for c2s. I'll go with 256x256 I think.

(also, non-random images probably JPEG-compress much more efficiently)

matrix uses an amorhan blurhash, which seems to work decently as a preview with very very minimal data

cheogram and slidge implements blurhash, but that's more for a placeholder before downloading the actual data, isn't it?

sure, though im not sure about the use of downloading a small version of the real thing

here I was wondering about https://xmpp.org/extensions/xep-0511.html

> here I was wondering about https://xmpp.org/extensions/xep-0511.html ahhhhh ↺

my bad, cheogram and slidge actually use https://evanw.github.io/thumbhash/

also, `data:image/jpeg;base64,...` is standard, but what we do: `data:image/thumbhash;base64,...` isn't. at least that what's larma replied to me when I said that I did it 😬

nicoco_, now I wonder which legacy protocol has link metadata image inline rather than as an additional http download...

Do avatars allow for unlimited size images?

Without downscaling

Cynthia, avatars can be hosted on http, then there's no file size limit

I could upload an avatar to my profile up to 512x512 (using a modified build of profanity), any bigger and it kept getting rejected due to stanza-too-big ✏

I'm assuming the avatar is uploaded directly as base64

nicoco_, Please consider that stanza size limits are not the only thing that matters, it's also that it increases the amount of data being transferred, which may be problematic on low bandwidth connections. Avatars are always retrieved on request only, so it's up to the client to fetch them or not, but link metadata is directly in the message, so putting significant data there means that clients have no way to opt out of things they might not even be able to display.

> I'm assuming the avatar is uploaded directly as base64 most clients do this, yes. Conversations afair actually scales it down for the base64 upload and puts the full scale version on http upload. ↺

> nicoco_, now I wonder which legacy protocol has link metadata image inline rather than as an additional http download... it's not legacy, it's experimental :) but yes, it could be uploaded but I think inline is fine for this use case, isn't it? that's how cheogram and gajim implemented it anyway… ↺

> nicoco_, Please consider that stanza size limits are not the only thing that matters, it's also that it increases the amount of data being transferred, which may be problematic on low bandwidth connections. Avatars are always retrieved on request only, so it's up to the client to fetch them or not, but link metadata is directly in the message, so putting significant data there means that clients have no way to opt out of things they might not even be able to display. fair point. maybe 128x128 is a saner choice then. ↺

aren't those link preview images typically not a square?

>> I'm assuming the avatar is uploaded directly as base64 > most clients do this, yes. Conversations afair actually scales it down for the base64 upload and puts the full scale version on http upload. So which ones do clients display? ↺

I guess its implementation-defined? ✏

> So which ones do clients display? Conversations displays the full scale version when you go into a contacts' details. I guess it's the only one doing so. For most usecases, the base64 version is already sufficient anyway, as most clients don't display avatars with high resolution anywhere ↺

I’m not entirely sure there’s much value of including a link preview inside the data at all, rather than pointing to the out of band resources. ✏

I was thinking the same as Kev

When we talk about link preview images, those most likely are actual single image files already, with the URL ready for pickup from the meta section in the html of the link target.

don't you have non-nerdy friends that send you youtube, instagram and facebook links?

Anything we put as data in the stanza is either likely to be too small an image to be generally useful, or too large a payload to be reasonable to include (or even potentially both), to my mind.

Kev: strong disagree

> don't you have non-nerdy friends that send you youtube, instagram and facebook links? Yes, the question isn’t whether it’s useful to include the metadata, it’s whether having the image data inband is useful.

I've actually experimented with the whatsapp gateway and instead of seeing https://youtube/xDDEFDSFSD, seeing the actual thumbnail of the video is valuable

I’m keen to add the link metadata to Swift sooner than later - but we can just fetch the preview data when we want it,having it in the stanza as raw data is the issue.

> I've actually experimented with the whatsapp gateway and instead of seeing https://youtube/xDDEFDSFSD, seeing the actual thumbnail of the video is valuable Again, not debating that. Only including the image data within the stanza rather than including a link to the resource

ok my bad I misunderstood you

> Anything we put as data in the stanza is either likely to be too small an image to be generally useful, or too large a payload to be reasonable to include (or even potentially both), to my mind. What about BoB? ↺

> What about BoB? I’m very much not part of the ‘everything must be HTTP’ brigade, but in this instance I’m not sure why we *wouldn’t* want it to be HTTP.

For private groups, fetching the data is probably OK. For public MUCs, it's nice to have the preview in-band IMHO.

I think we’re decades on from the “I don’t want an HTTP library in my XMPP client” arguments being reasonable.

> For private groups, fetching the data is probably OK. For public MUCs, it's nice to have the preview in-band IMHO. I think if we want IP privacy like that (I think that’s what you’re referring to), we probably need something other than inband data - e.g. the service proxying or such.

FWIW, even whatsapp settled on all link metadata in-band, including the image, in their protocol

(and it's a small JPEG thumbnail)

Whatsapp’s somewhat a special case, because of it only targeting mobile screens, though, right?

there are web and desktop clients too

If you use WhatsApp on desktop you’re going to have unhelpfully small link previews, because they’re build for mobile.

For Whatsapp, there is "native" app, on computer

The web is still availlable?

> If you use WhatsApp on desktop you’re going to have unhelpfully small link previews, because they’re build for mobile. Hmmmm never heard any complaint about that, and tests with whatsapp web on my 27'' thinkgs were OK IMHO. ↺

> For Whatsapp, there is "native" app, on computer Yes, that wasn’t what I was saying, I was saying they only target the mobile screens with the link previews.

nicoco_, you made an incorrect statement about Prosody stanza limits earlier. It's actually 256KB c2s, 512KB s2s

Just to avoid that being used for any protocol decisions :)

c.f. Discord or Slack etc., which do preview sizing suitable for mobile or desktop screens.

And our default limits match ejabberd's

MattJ: oh. https://prosody.im/doc/modules/mod_c2s says 10MiB

https://prosody.im/doc/s2s#stanza_restrictions says 512kb

Thanks for the report :D

hehe you're welcome :)

> c.f. Discord or Slack etc., which do preview sizing suitable for mobile or desktop screens. anyway, I'll go with bridging whatever I receive from whatsapp, unless it's > 128 in any dimension, in which case I'll resize. I don't feel it'll be useless and I think it'll give me context about whatever instagram posts my friends share without needing to open them ;) ↺

Site updated

> anyway, I'll go with bridging whatever I receive from whatsapp, unless it's > 128 in any dimension, in which case I'll resize. I don't feel it'll be useless and I think it'll give me context about whatever instagram posts my friends share without needing to open them ;) For your bridging usecase, I think it's totally fair to bridge everything over exactly as you receive it, even without resizing. ↺

well I'd like to avoid the stream being terminated because of too-large stanza :)

meaning: if the image is inline, bring it inline, if the image is an http url, provide either the original url or a url that proxies it.

in the latter case, it even has the advantage that your server doesn't even need to fetch the preview image at all when there's no client retrieving it.

The considerations when bridging something inband are different from the general considerations for all of native XMPP, yes.

By the way, the reason why WhatsApp puts them inline is probably privacy: as messages are encrypted, even if the server provided a proxy for retrieving those images from the source, that would result in leaking clear text from the encrypted message, which is undesirable. Ironically, this privacy considerations don't apply equally in the slidge case, because end-to-end encryption is broken on the slidge server, so the server learns about those links anyway and could thus safely act as an image link download proxy.

Someone post the link preview impl pros/cons link

There is no perfect solution

But putting the data inline is the most privacy respecting, and least dangerous ✏

https://mysk.blog/2020/10/25/link-previews/

I got the preview per Link down to ~20kb, we allow 3 previews

And we don't even use the best codecs for this usecase

We use 256x256

lovetox, there's two things: - the metadata itself - the preview image It's very obvious to inline the metadata, as those are small and doing it will improve UX significantly at little cost. For the preview image, that's less obvious IMO. preview images can be NSFW or legally problematic to distribute. I also see some issues when we start to have 20kb messages regularly, just because there are links in them. Does it mean that if I fetch the last day of backlog from my MUC MAM, I might end up downloading several megabytes of link previews, many of which I might never even display?

Every solution has cons, this is the one that focus on privacy

One of the things XMPP gets praised for is that it works extremely well on low-bandwidth connections (and I'm not talking about military usecases here, but rather throttled mobile data connections or f*king german train wifi). If we make downloading larger amounts of data effectively mandatory to use XMPP, we will loose that.

You can of course choose a different one if MAM size is important to you

Well, we're a federated network and I can't control what you send in this room 😉

feel free to provide a better solution, querying all links someone sends in a MUC on the receiver side sounds not convincing to me

server admins are reluctant to do this as well

At least if it's a link, the receiving client is able to choose to fetch it or not (depending on how much they trust the sender, what kind of UI they have, etc.) ✏

Meant "sender", wrote "server"

delegating this decision on every link preview to a user is not good UX, especially if the rational why they have to do is, because of MAM archive size.

have the text metadata inline, generated by the sender + have the preview image as the original URL (as announced in the og meta) + optionally provide the preview image also via http file upload, bob, jingle or whatever other mechanism you consider fitting. Allow the recipient (or its client) to decide when and from where to actually retrieve the data ✏

What users want is the picture, this is the main thing. If i dont put it inline, the receiving client can fetch everything itself, no need to include metadata. P2P mechanisms are hard to even consider in 1:1 (because your device must be online) in MUC its even worse

putting something on my own http server does not help because the receiving client cannot see the difference

lovetox, I didn't mean to imply anything about delegating to the user per request, I said client

> putting something on my own http server does not help because the receiving client cannot see the difference The receiving client can use a privacy preserving proxy to retrieve it from some http url (no matter if your server or the other). ↺

Like, I don't particularly want to receive these in my poezio, which can't display JPEGs

larma, but server admins dont want to provide the privacy respecting proxy, and normal users have no idea what a proxy even is

lovetox, wait, don't most servers provide TURN? That's exactly a privacy preserving proxy.

Most TURN servers are configured in a way that would disallow using them to connect to arbitrary HTTPS sites

If I'm on a throttled mobile data connection (typically 64 kbit/s in Germany) it will take 2.5s per message to retrieve the messages with 20kB of preview in it. It means you add 2.5s delay to every message with a link in it, removing the real time from my messaging. It also means that if I want to join a room on this connection and my client fetches the last 10 mesages on join, it may take me like half a minute to even join a room.

> Most TURN servers are configured in a way that would disallow using them to connect to arbitrary HTTPS sites Might be, but my point stands that server admins generally are open to provide privacy preserving proxies. ↺

> Most TURN servers are configured in a way that would disallow using them to connect to arbitrary HTTPS sites After a short check with several public XMPP servers, I found this statement to be untrue. ↺

😬

>> Most TURN servers are configured in a way that would disallow using them to connect to arbitrary HTTPS sites > After a short check with several public XMPP servers, I found this statement to be untrue. Its funny how you can assume the most basic amount of security to exist, but in reality, its far below that ✏ ↺

> meaning: if the image is inline, bring it inline, if the image is an http url, provide either the original url or a url that proxies it. Why not make the sender download the image, and upload it to their own server ↺

You don't have to make server admins host a proxy for this, and also the sender can encrypt the file (like OMEMO) before attaching it to the preview

I don't really understand this talk about privacy preserving proxies

They would protect it from some people, but obviously not from the server which weakens again E2E

You can have both if the sender just uploads the file into the server

Which allows them to do encryption (as long as the recipient can handle OMEMO media in preview)