jdev - 2023-03-06

https://github.com/mastodon/mastodon/pull/23913 Mastodon implementing rich text formatting with an ok wire format. hint hint

Transform h1 through h6 tags to <p><strong>contents</strong></p>

m(

Wasn't it already HTML embedded in JSON?

Well they're not respecting the semantics of html, but at least they're not just stuffing markdown in there

Zash, yeah but a very much restricted set

I'm curious, why would markdown be a bad thing? too limited?

Not limited enough!

nicoco, we abolished XHTML-IM because web developers can't be trusted not to pass trough <script> and stuff, and markdown is a HTML superset with the same problems

Even with XEP-0393, there was at least one case of someone blindly passing it trough a markdown library with html pass-trough enabled

Thus we can't have nice ~things~ styling, it's not safe!

we can't have nice ~things~ people

thanks for the context. html pass-through sounds terrible indeed.

Ge0rG, we can't fix people tho, only take away their nice things

that's a good start.

What was the reason not just to add many big warnings what not to do in the xep? And if someone does it anyways, it is on them?

🤷 good question

> I'm curious, why would markdown be a bad thing? too limited? Xhtml helps with accesibility https://github.com/mastodon/mastodon/pull/23913#issuecomment-1455180785

Html*

Plus its easier on the client to use the format. Instead of having to guess with whatever we currently have

it would help if they didn't translate headers to <p><strong/></p> ✏

nicoco, to me it's not about the input format. It doesn't matter at all. It's all about wire format ✏

Yeah. Stanadard format to parse instead of arbitary characters

jonas’, this is probably at display time, not in the wire.

sure

Zash: The solution is not to ban the entire (and useful) technology, but to recognize the problem and make sure to highlight/require the script tag not be permitted (i.e. pass HTML or Markdown through a preprocessor that weeds out illegal/unwanted stuff.). To this list, I would add the style attribute and the style element, and any element defined for the HTML header.

Basically what we had in xhtml-im

correct

It might be included in compliancy requirements, that such security vulnerabilities are addressed (i.e. compliancy requirements for clients)

Sounds like a nice step to me

> Xhtml helps with accesibility > https://github.com/mastodon/mastodon/pull/23913#issuecomment-1455180785 Except it doesn't because markdown is HTML so... (No one that I'm aware of is advocating for a markdown wire format)

XHTML-IM is "abolished" just the spec says deprecated on it. It's still the best thing we've got and Implemented by several clients and even by memberbot :)

xhtml-im is XMPP's "malloc/free is fine, just don't make mistakes" :)

Someone motuvated just need to un-deprecate it with a bigget security warning 🤷

Honestly I would remove all the "restricted subset" stuff from the spec, that's up to the implemetation. Lots and lots of apps have to sanitize HTML inputs already, including feed readers etc, it's not like this problem is unique to us or hard to solve

Just use whatever obvious sanitize your framework/context already has that every non xmpp app is using

PS: Markdown is not HTML, but is a content format (and is often used to generate HTML, but can be used to generate other formats as well). Most Markdown dialects support the embedding of HTML elements (even though it restricts its usefulness when it comes to generate other forms of presentation formats). Still, it is not HTML, and it is parsed (and often pre-processed) before HTML is generated. There are very restricted Markdown dialects also, just as XHTML-IM was a very restricted version of XHTML. Benefits of Markdown include (apart from being easy to write) being extensible with a loosely coupled structure. It might make interoperability a challenge, unless interoperability is structured in a similar way as is done with XMPP (with extensions for different types of constructs).

Peter Waher: a valid html document is a valid markdown document

Peter Waher, Markdown is a HTML superset, in the sense that every valid HTML document is also valid Markdown. Markdown just extends HTML with other formatting, like "*foo*" in CDATA becoming synonymous to <em>foo</em>.

Some Markdown implementations also reject HTML elements present in the text.

It is usually configurable.

No its not

from the creator of the original Markdown:

https://daringfireball.net/projects/markdown/ ✏

No one is proposing markdown for xmpp anyway :P

yes

but it was not added to the list of proposed extensions

A content-extension where multiple content formats could be embedded in a message (a more abstract and generic version than the existing)

multipart/alternative? feels like it may be overkill while at the same time not too unlike what we already have with plain ol' <body> and xhtml

Multiple copies of the body in different places is also an antifeature: see: lang

Or email-style <body>Click here to see the newsletter</body>

> Multiple copies of the body in different places is also an antifeature: see: lang xml:lang is actually a proper accessibility feature :)

Maybe someday we'll stop dismissing those

"we"?

Do we actually have a way for me to hint what language i am sending?

I'd like to be excluded from that particular "we"

In the message

MSavoritias (fae,ve), yes, xml:lang on your stream is by default transferred down to the <body/> of your message, unless your client overrides it for the stanza or the body.

This is also a privacy leak! Such fun!

Why? Cant you set it always to english?

you can, but what's the point then :)

Then its not a leak ;)

(and also breaks accessibility)

For the privacy nuts

sure

well, ideally, you'd set it to whatever (klingon or so) on the stream, and then explicitly for each message/body

privacy-wise

jonas’: individually maybe, but as a community I think we largely fail

MSavoritias (fae,ve), you can even have multiple <body/> elements with different xml:lang to send translations of your message!

Sounds cool though. More clients should hint like that imo and have a settinp for it ✏

and then clients can pick based on local preference what to display

😱 where is this stuff? It sounds cooler than stickers almost

MSavoritias (fae,ve), aioxmpp has a complete implementation of that, it's in RFC 6120 ✏

or RFC 6121 maybe ✏

(plus the corresponding dependencies)

(such as the xml:lang inheritance rules in XML 1.0)

Heh. Lots of rfcs to cover

indeed

flipside: you can have multiple <body/> elements which say completely different things in different languages

<body xml:lang="en">We want peace!</body><body xml:lang="de">Hafen um 9 angreifen!</body>

nice covert channel.

Same with xhtml-im

same with email and multipart messages

indeed

and OMEMO, in fact :)

instead of <body>YoUR ClIENt DoeS NOt SUpPOrT OMEmO</body>, you could also send something completely different <3

Yeah, in fact most OMEMO messages already have covert messages!!

but this being one of those things where you have to trade absolute security and solving social issues with technical means vs. nice things, I'm ok with err-ing on the side of nice things

OTR in the <body> of OMEMO messages?

Zash, door's there, hush, out

FINE

most cursed message

Zash, and then add an OX element, for good measure!

jonas’, yeah, that's also a trade-off I'm happy to make if it means including more people in

YESSsS

For some reason I don't get, some people are completly imune to this

I'm beyond the ~popcorn~ chocolated-peanut stage

pep., immune to what?

popcorn?

hah

To understanding that their solution to these "Security issues" are to shut some of our users out ✏

Balance and all that

it's ~turtles~ trade-offs all the way down

Question though: what is bad with 394?

Yeah, except when that balance affects more than just you, but also every other client around you

Xep 394

MSavoritias (fae,ve), it's a solution to a problem which shouldn't exist in the first place

it introduces all kinds of nuances, and we should just go back to properly sanitized XHTML-IM

with nice test vectors

Heh you are one of *these* people /s

server-injected exploits in every message you mean

Testing suites could actually be a useful addition to the XSF's roles.

MSavoritias (fae,ve), *these*?

pep., *subtly points at '392*

That want xhtml :P

MSavoritias (fae,ve), yeah sure

nothing else makes sense

it's semantic, it's well-established, it's well-documented, it embeds perfectly in XMPP, which is already XML-based

all other formats also come with their own non-trivial injection security issues.

we should "just" solve that by providing test vectors which implementations need to pass for security checks

'394 is a huge overly complex mess, '392 is a huge underly complex mess

the mess is constant, you can only move it around

(note who is the author of '394 ...)

s/'392/'393/ obviously, '392 is the best™

jonas’, yeah 392, but again, that's not "The XSF", that's just you :)

Council could enforce this on accepting specs

oh, I'm not listed as author of '394 anymore

I think you removed yourself

yeah

Yeah. Accesibility, privacu and tests

MSavoritias (fae,ve), then maybe for context, I wrote the initial version of '394 before I realized that it's not a good way forward

Yeah. Its not a good idea sending stuff out of band imo

The text should be structured itself

ack

Like markdown or xhtml

markdown is not structuring of the text itself

(like '393 is not)

there is no clear separation of text and structure ✏

but I'm not going down *that* rabbithole tonight

In the sense that the text tells you how to. 394 Seems like it adds it in the metadata

Which is an odd place

pep. can explain all that to you

jonas’, yeah we've been going over it again in other channels..

For the past few days

:)

good thing I was too distracted to pay attention

I guess I'll distract myself for anothe few days until this storm blows over again

You're not in there!

Someday it'll be big enough for someone(tm) to resubmit it, and then it'll be chaos again for some time

I'm tring to gather some thoughts around why conveying intent is important (and actually the whole point of a protocol to me) ✏