XSF Discussion - 2017-10-13

What if we replace xhtml-im with bbcode!!!

Think of all the php implementations at our disposal

moparisthebest, loool

I do wonder if we should revive the <font> tag as replacement for the style attr

I don’t believe in font styling, but people seem to think differently

it’s the number one reason I block inbound formatting

Way easier to sanitize <font color="name | #hexhex">

if we start to go down the route to revive parts of deprecated HTML, we can go down Sams route of inventing our own markup all the way

jonasw: People want stickers!!

people don’t know what’s good for them!

People want the silliest things!

Do we give it to them or do they go elsewhere?

Nice things, none shall have them!

Only shiny, silly things

stickers are possible with XMPP

SIMS + a repository of stickers yes

Sure they are, since literally forever

non, with Bits Of Binary as well

Peoples reasons for XMPP suckage don't need match reality

edhelas, do stickers fit into BOB?

Make them fit

Zash, they sure match the reality of actual implementations :)

well 20kb-30kb is enough for most of them

30kB * 4 / 3 = 40kiB stanza

That's well within the 10M stanza limit, even if base64'd

also BOB use some hash, so they are transfered only once

there’s a 10MiB stanza limit? :)

In Prosody, IIRC inspired by some recommendation for minimal limit somewhere

Movim implement stickers using BOB :)

https://github.com/movim/movim/tree/master/app/widgets/Stickers/stickers

7.5 MiB data limit -- that’s well sufficient for avatars

jonasw at one moment people will ask for high-def-animated-with-sound stickers

I recall that people said that PEP avatars are not feasible due to (among others) stanza size limit?

Here we go. Why can't I have my browser plugin that lets me link directly to sections?

https://xmpp.org/rfcs/rfc6120.html#security-dos > A deployed server's maximum stanza size MUST NOT be smaller than 10000 bytes

Wait that's ~10k

:D

and there is no way to check what limit is applied on the server :)

we need a XEP

Path MTU discovery!

Now with eXtensibility! :D

MattJ: How did we end up with 10M then?

I suppose you could survey peoples vCards and multiply the maximum size by some number out of a hat and call it a day.

Zash, it’s not impossible that I complained a few years ago when my fiancees avatar wouldn’t upload and that made pidgin fail to connect or something. I remotely recall there to be such a problem.

jonasw: Not the thing where data goes into a buffer that the network stack isn't paying attention to?

back in the days of google code

not sure, I think it was some limit thing

Hm

Hey, how do browsers deal with namespaces?

Like, if I stupidly insert some random XML tree into my DOM, and there's like a <p xmlns="not xhtml" onclick="evil();"> in there, what happens?

Zash, horribly

they don’t care a lot

and as always, it depends

So a thing that sanitizes the xhtml-im namespace but lets anything else thorugh would be useless?

I can imagine that this kind of stuff works because it is hared between XML and SVG handling

maybe

I wouldn’t rely on it

there are funny bugs in browsers with prefixes, too

Kev, +1 to your XHTML-IM mail

you brought to the point what I tried to convey in a few paragraphs of prose elsewhere.

I don't have the attention span to write TL;DR mails :)

Ha, I tried to make the same point yesterday.

Let's see if different framings are going to convince the public

itym the council.

guys, what is the agreement on pubsub in push?

I don't think the move towards something markdownish is actually stupid, FWIW, and I think it's much much easier to sanitise something that you can write your own parser/serialiser for, than XHTML-IM. So I don't think this is a bad direction. It is much easier for diligent devs to get it right. I'm just not sure I buy the argument that it's going to suddenly make anyone who wants to dump things into a DOM unsanitised safe.

I like markdown, but it's impossible to write a parser for that.

I agree with Ge0rG

writing your own markdown parser is a mess, and will yield 1000 different implementations

(this holds for all text-based markups, I’m afraid)

It's even less possible to sanitize markdown.

Well, we can go with a binary markup if you want, and then base64 it to get it into the stream, but I don't think it helps :p

Kev: it's already impossible to write correct markdown _text_, which is rendered in the same way everywhere. How are you going to write a parser?

Kev, I sense buffer overflows there.

This is the appropriate reaction: https://pics.zash.se/4c840479.jpeg

using an XML-based markup makes much more sense to me.

Ge0rG: That isn't the hard part. Tedious, but not hard, we just need to spec it. And by 'we', I mean that dwd has volunteered, so yay.

where to vote for asn.1? :)

BER or DER?

hasn’t BER failed?

Is server-side cleaning of xhtml-im sane, or would that just let broken clients be broken and then get hacked by evil servers?

XER

Zash: yes.

Abstract Syntax Notation 71.

from what I’ve heard, the general sentiment is that clients need to trust the server anyways to some extent...

Yes, trust the server, the server is good.

but I guess it’d break e2ee

E2EE is just marketing fluff, did't we establish that?

jonasw: +1, I can't understand what's the point in building client-server architecture where there is no trust to server?

Zash, I won’t get into that argument.

better off using p2p directly

I’m torn on the e2ee thing. I don’t want certain stuff I discuss with people plaintext on my server in some MAM.

zinid, p2p doesn’t scale

p2p doesn't scale *on mobile*

jonasw: I wouldn't say that, there are some research going on

I really don't get why people hate ASN.1

Ge0rG: because it generates lots of boilerplate code for mainstream languages such as C++ or Java

for example, in Erlang it's great

not rather because ~every ASN.1 parser is broken somehow?

jonasw: but we have PKIX working?

sure, take a look how secure the PKIX libraries are

Every browser contains an ASN.1 decoder. We could leverage that and replace JSON, once and for all.

Ge0rG: Call it "binary JSON with schemas and namespaces"

Make a fancy website on whatever is the hip TLD today.

Zash: yeah, marketing is crucial.

.io?

jonasw: I think that's the problem of the language you're using

zinid, excellent, let’s ignore issues because they only occur in a single language.

then we can stay with XHTML-IM, because javascript/DOM is the actual issue.

jonasw: well, we ignore? and what do we have as a result?

ASN.1 was a standard and now there are tons of implementation with 0 interop

the XHTML-IM problem can be extended to Pubsub as well, with the usage of Atom

Kev, I volunteered to document a "snippets" design, which - I think - covers most of the use cases outside of *bold* /italic/ _underline_ and `preformat`.

maybe we need a subset of ASN.1 that is easy to understand and to implement. Let's call it ASN.0. Or maybe ASNdown.

Ge0rG: I would rather use protobuff, frankly

but it's not a standard

zinid, FWIW, I've not come across any of these incompatible ASN.1 implementations, and I've used many of them.

dwd: I had completely misunderstood, I thought you'd volunteered to write the thing that's like markdown spec, sorry.

zinid, After all, when I worked at Isode, M-Link had about 6 in the code.

but I'd say, maybe we can just add modules on servers to checkup the content of messages when they detect the xhtml-im namespace

it's basically applying a schema and check if it fits

Kev, I can probably do that too. But I have a pair of XEPs and a I-D in my pipeline, and adding another one is pushing things enough. (Oh, and I'm trying to shepherd Ash through updateing '314 *and* writing another).

dwd: well I'm not talking about incompatible implementations, I think jonasw said that :)

edhelas, Really? So I can strip tables then?

zinid, nope, I did not mention incompatibilities.

zinid, Then I misunderstood: [09:47:59] ‎zinid‎: ASN.1 was a standard and now there are tons of implementation with 0 interop

dwd: I mean there are now: thrift, protobuff, several json serializers, etc

but those are all not ASN.1?

yes, but they do the same

encode/decode lang structures into/from wire format

It's funny how SQL injection is still a thing in 2017. And XSS.

When I keep talking about markdownish, to be clear, I'm talking about something that is clearly and comprehensively specced, presumably by us. I am not suggesting it's ok to say "use markdown", and then just have everyone pick their own, subtly incompatible, library.

Kev: I'm not sure that makes a difference. If it is sufficiently similar to markdown, people will just plug their own markdown library in and end up wth full HTML support.

Ge0rG: I'd really like the discussion to move away from "People will do what's easy, not what the spec says", and instead try to work out how to have a spec that a reasonable person is likely to implement without significant issue. XHTML-IM, as it stands right now, is not the latter thing.

Kev, I’d like your opinion on a (possibly audited) JS reference implementation of a sanitiser.

I think it's a different question to whether XHTML-IM is sane. I don't think "It's near impossible to get right, but we've already done it, so use our implementation (or port it)" is the right thing to do.

If we get to the stage that we *do* have a sane spec, then a reference implementation could be helpful, but at that point is also probably not as necessary.

I think that XHTML-IM is easy to get right, once we drop @style.

(you’re gonna have the problem of sanitizing URLs in any markup which supports URLs)

Kev: I'm not sure that it's actually possible for a reasonable person to implement a secure web application.

I'm lost, what is required to be sanitized? URLs?

Kev: and I think that XHTML-IM is mostly a strawman here.

xhtml-im doesn't define scripting, so there should no be XSS, no? (I'm not a web developer)

zinid: the problem is that it's insanely hard to sanitize user-controlled strings in a web application. And even more so if those strings contain HTML markup.

zinid: change your nickname to `<script src=.../>` and there is a good chance some client will actually load and execute that.

I know about web application, but this is mostly due to script execution

yes, your example is also about scripting :)

zinid: yes, but there are so many ways to include scripts.

and developers need to know and filter them all.

but xmpp client should not execute scripts?

zinid: should not, no. but if the client is running in the browser, the browser well might do

ah

zinid, the fact that XHTML-IM doesn’t define scripting does not prevent (a) maliciuos entities to include <script>...</script> and (b) stupid clients to embed the XHTML from the message unsanitised into the browsers DOM

jonasw: yes, got it

but don't we have the same problem with markdown? how is markdown processed? web clients can pretty much insert <script/> from markdown into DOM as well

and the same for raw messages (i.e. <body/>)

zinid: that's a very valid point, made multiple times by now.

ah, ok

sorry, TL;DR :D

yeah, that is a common problem :>

Ge0rG> Kev: I'm not sure that it's actually possible for a reasonable person to implement a secure web application. < +1

"Markdown is not HTML so people will have to write their own parser" what? (*me just read the ml thread*)

Or rather, people can't drop that into innerHTML. Don't worry they'll find ways

I mean: I'm a professional IT security specialist, and I don't know all the ways that you can inject code into a web app from user input.

I'm not even sure how to contribute to that thread. People hear "We need to deprecate XHTML-IM" and already it's "I know $NEW_FANCY_MARKUP that we can use". We're just going in circles.

Kev> Ge0rG: I'd really like the discussion to move away from "People will do what's easy, not what the spec says" < That's exactly what's happening at the moment with XHTML-IM.

And that's what sam is ranting about

Can you deprecate a XEP, modify it and then make it draft again?

If it their implementation was compliant there would be no such issue. (Or we could fix the XEP to answer these issues) ✏

Under a different name/number? dunno. That would be silly anyway

Might as well reinstate the original XEP

No under same name and number

Syndace, https://xmpp.org/extensions/xep-0001.html#approval-std

Okay thanks

Anybody can post to standards@ right?

pep., must be subscribed, but subscription is open

I get the emails, so I did that iirc

your mail got through

the list has some delay, in the order of a few minutes

Yeah I saw it/them go through :) ✏

Why do my first emails have to be hate mails :(

it’s not hate, it’s constructive discussion (for now)

Uhm.

https://www.mail-archive.com/standards@xmpp.org/msg17919.html I like goffi's position here

Wasn't the first of those mails "We have to consider people who ignore the spec" and the second mail "Changing the spec will fix it"?

I think as long as your position involves the "stupid people" argument, there is no way to find a solution to injecting markup, but there is certainly no way for xhtml-im markup.

I'm not of the opinion that changing the spec would fix the issue. It might help a bit

Jonas said we should change the spec, you said you strongly agree...

sure, because @style is hard to validate right. everything else can be done.

Well, that would help yes.

(even @style can be done, but I don’t think we should be asking people to do that)

Kev, also look at my second sentence. "the issue is not here"

jonasw: Yes, and that's fine if you agree with my premise that we should be catering to people who want to do the right thing, not to people who ignore the spec and inject HTML directly. If you want to go along with my premise there, then changing the sanitisation rules can help. But if you're on the premise that people will ignore the spec, no amount of changing the spec will help with that.

Kev, I still believe that a working reference implementation will help against people ignoring the spec.

(and a reference implementation would benefit from getting rid of @style, which is my argument there)

I don't even think in a open source environment you have to care about stupid people. If you stumple across an implementation that allows for XSS you open an issue and point them to the reference sanitizer and everything is fine.

If someone writes new client code for fun that only he and his friends use then whatever

I prefer to try to prevent security issues before they happen.

Yeah sure

jonasw: I think you're agreeing with my argument, and disagreeing with pep's, that we should be making this easy for people who want to do the right thing.

yes, we definitely agree on that.

I don’t agree that inventing our own markup will help with that.

Kev, but the issue at hand is people not doing things right, am I wrong?

it will just open another can of worms, be it security issues or interop issues.

(or probably a mix of both)

server plugins could strip out java script and other malicious tags

Alex, that has been proposed, and Zash even drafted an implementation of that for prosody.

I don’t feel that clients should rely on that in any way.

Alex: I'm not convinced by that argument, really.

(but it’d be an interesting tool to detect malicious parties)

I have done this before, the XHTML XEP lists the allowed tags, any other tag I ignored in the parser

Kev, if it's to make things more straightforward for people who follow the spec, I'm all in, and I agree with jonasw. But that's not how I read SamWhited's email.

I also firmly believe that obsoleting XHTML-IM without a replacement which has deployment will achieve nothing, except closing a door for us on fixing things there. See Private XML support.

jonasw: agreed on that

we really need modern markup to compete with Slack and others these days

I will implement XHTML-IM in my client over the next weeks, no matter the XEP status.

simply to gain interoperability

othereise its a step backwards

and I assume that others will do the same

and if they don’t then we’ll end up with a bunch of incompatible tacked-on markups

e.g. clients (not people!) putting markdown in <body/>

(we already have that to some extent with clients interpreting *foo* and /bar/ and such)

I see *foo*, _bar_ etc. as historical, coming from IRC. Not sure where IRC got that first though

pep., agreed

even though there are clients which interpret ">", which I haven’t seen used much in IRC except in the last few years.

Right, they're all implementing their own markup, using <body>, which is meh

much meh

I often rage against Conversations converting ">" at the beginning of a message. It breaks stuff like "><", and messages with a single ">" don't make sense anymore (got this case on prosody@ the other day) ✏

>< works though.

Alex: > we really need modern markup to compete with Slack and others these days Do we really want to compete? I just doubt that this is the goal of XSF

Ah it works now indeed. I haven'T checked in a while

zinid, I think we should be competitive with things like Slack, yes.

zinid: I see many people switching to Slack these days. There is no reason why you can do the same with XMPP. But we lack of modern client and features these days

Alex, I think we have most tools to start implementing. I would rather wait and see if UAs ask for more

Maybe push them to implement such things if you really want

dwd: only like Slack? really, what userbase do XSF target? because from what I see currently it's producing specs for nerds (like e2ee)

zinid, Well, not only like Slack. And I don't entirely disagree with your assertions there.

zinid, ALthough not all nerds focus on the threat model that OMEMO does.

zinid, I think most nerds have their own servers, and e2ee is possibly of less important there

But I don't disagree with your assertions either :)

I just think maybe it's better to define the target userbase?

I mean if this is nerds, then I'm fine and we should go in this direction

I don't think we can produce a protocol for every person on the planet

zinid, Scalability would be an issue, for sure.

the strenght of XMPP was the we always covered a huge variety of use cases an users

zinid, protocol maybe, but for the end-users I don't think the protocol really matters anyway.

dwd: yes, but if we try to resolve scalability issues we will end up with SIP :)

Or rather, for *most end-users, non-tech that is

there is sip p2p rfc already

zinid, Oh, I do hope not.

dwd: well, if you strip shit from sip, it's usable and can be used as building block

zinid: the one that also works for xmpp?

hehe, ok, I didn't want to go into SIP vs XMPP debate :)

zinid, That's not a debate, it's a bloodbath. ;-)

zinid, But anyway, I think we have a set of existing target communities, and I agree that defining these would be of benefit to us as protocol designers - however you implied there can only be one, which I think I take issue with.

the target communities also changed over the last 10+ years. I like this discussion of defining it

dwd: well, their can be several, but I don't think "regular user" class intersects a lot with "nerds" class (intersection is a very basic functionality)

I mean, take a look: 1) a girl spending most of the time in instagram and so on 2) a nerd who lives with console

how can we target both?

zinid, I think the amount of intersection between the sets is actually something we'd need to find out. I suspect it's not nearly as small as you might think.

maybe, would be great to understand though at least for making the priorities

extensions, give teh nerd a console client without XHTML, and the girl a client where she can like messages, have hundreds of emoticons and share images and glyphys

Alex: sounds great in theory, but does it work in practice?

Jack of all trades

yes it does, I am doing XMPP stuff for close to 15 years now, and have customers which do all that with XMPP with great sucess

It does show that a bunch of us are more or less doing stuff for ourselves. Nothing wrong with that.

Zash: Nothing wrong with targetting ourselves if we clarify that. Then I can give up trying to sell XMPP to others and then having to explain why stuff breaks.

Optimally we'd get instagramming teens into software and standards development

nothing wrong of course, the problem I have is that we constantly speak about how to fight with Slack without even defining the target

And if the target is everything then it's a lot of work

yes, 300+ XEPs for sure :)

yet another 300 I mean :)

I think case studies and small tutorials on our webpages would help

Only? Gotta break that initial zero ;)

I think it's a vicious circle. XMPP's niche is mostly nerds, so there's few other users complaining about our stuff, so we concentrate on the nerd stuff because that's more fun to implement.

Holger: Typical in most FOSS circles

you wanna build something liek Slack, hey this iw what you need, extensions A+B+C+D you wanna build a military grade client, then this is what you need, extensions X+Y+Z

Alex: Sounds good

Holger: +1

Zash: Yes but elsewhere there target audience is more obvious.

s/there/the

Alex, In my experience, the Military and similar markets actually want something Slack-like anyway.

Zash: If you build a tiling window manager, you won't break stuff for non-geeks.

you wanna build something like whats app do that you wanna build machine 2 amchien communications do this you need build a system for realtime stick exchange do this you wanna build the next Uber (geoLoc) to that

Alex, I mean, they want labelling, sure, but don't think they don't care about emoji support.

And a large part of FOSS projects is building server/infrastructure stuff ...

Alex: that sounds like a compliance suite.

Alex: ok, let me rephrase maybe: who will write the XEPs for Slack audience? Hell, does anyone even know what Slack audience want? :)

I suck at typing today, so need a client which supports XEP-0308 :-)

zinid, I've used Slack, and I can tell you that Slack themselves don't have a clue.

there is a commercial market for corporate (or government) slack-like things self-hosted with data retention, archival, LDAP integration and other enterprisey requirements.

we can start doing this, but we will eventually end up working on JET and OMEMO :D

dwd: yes, maybe :)

zinid: we don't need to target the nerd audience. They will come to us voluntarily.

Ge0rG: but we do this

JET and OMEMO for whom?

a girl from instagram never heard about e2e

Ge0rG: yes, we had a compliance suite before at the XSF for servers and client, but somehow died

IMO, there are two target groups we should focus on, five years ago: - Slack-like on-premise / hosted services with a good web and mobile UX - slightly nerdy normal users (see Conversations' success)

Alex: the current one is defining "mobile" and "IM" use cases. I'm pretty sure that "Slack" would make a good "Multimedia Chat" one or somesuch

Alex: actually, there is not much specification missing to pull off a Slack-like thing, it's only a lack of dev power to make a proper web client and a one-button easy-deployment server.

okay, web client _and_ mobile client

Give me a dozen competent developers, a year and a time machine so I can start in 2012, and I can pull it off.

Ge0rG, Actually, Openfire has the latter, for sure. You can install webchat clients just by push-button after Openfire itself is installed.

dwd: will Openfire come with AD/LDAP integration and all the Enterprisey checklist features?

Ge0rG, It has had AD integration for a decade or something.

And will it scale to (tens) thousands of users?

Ge0rG, It doesn't scale as well as some servers. Chokes around 20k users or so, I think, until you do clustering.

what about ejabberd ?

edhelas, I think that has LDAP integration, I don't know about Active Directory SSO (ie, GSSAPI).

edhelas: https://blog.process-one.net/ejabberd-massive-scalability-1node-2-million-concurrent-users/

dwd: is it possible to buy an Openfire appliance / virtual machine with around-the-clock tech support?

:)

dwd: but we are getting off-topic here.

There is a German startup, selling enterprise mobile messaging <https://www.teamwire.eu/> - they provide cloud-hosted and on-premise solutions, and ask something like 3€/month/user for a WhatsApp-like experience. And they don't even have federation.

zinid: there is hosted.im if you need that

But they have a sustainable business for something like five years now.

Ge0rG, Sure. We do have a dearth of clients running on Apple-favloured systems.

The problem is: nobody will buy an XMPP-based IM solution if the UI isn't Slack-like

Alex: me? :) now, thanks, I have my own domain

dwd: business clients running openfire on Apple? Or XMPP clients running on macOS?

or iOS?

Ge0rG what is "slack like" for you ?

damn, last time I checked slack I didn't understand what its buzz is about

Ge0rG, Yes.

edhelas: easy to use private messages, channels, markup, some integration with websites / web services

zinid, It introduces millenials to IRC.

edhelas: synchronization across all devices. proper push notifications

zinid, That is, pretty much, it. It's IRC with emojis and working file sharing.

I actually like Slack. It's easy to use.

dwd: then what is a problem to build slack-like using existing XEPs? am I missing something?

zinid: lack of developers

zinid, No, we're not missing much to build something Slack-like, but federating and with security.

I already have a couple of Movim users that deployed it in their business on top of a XMPP server

you add some bots to integrate with github/jira and you're good no ?

I do think that our absolute neutrality does handicap us from showcasing good clients, which in turn reduces the appeal of the platform as a whole.

Ge0rG: that's true

dwd: so the problem is nobody wants to use XMPP for their apps, ok, we go in circles now :)

zinid, Well, sorta. I think people look at the first client or two they see and assume that the UX they encounter is due to restrictions in XMPP.

GitHubs Gitter is another example, also getting quite popular these days

dwd [13:46]: > I do think that our absolute neutrality does handicap us from showcasing good clients, which in turn reduces the appeal of the platform as a whole. Very sad, and very true.

I was thinking of creating a page that showcase those "good clients" actually

if it's a project that is not officially pushed by the XSF, it could work no ?

what good clients? I still use Tkabber...

conversations maybe

zinid, Nerd. :-P

zinid Dino looks very promissing

dwd: but what to use? swift lacks of features, dino is crashing, gajim is buggy (tracebacks are the friends)

zinid, I wonder if - and I know this sounds silly - the XSF should have an awards thing for Best XMPP Client for XYZ Platform, etc? Would that help highlight and motivate?

dwd: no, Durov puts 1M$/month in telegram for instance, I think other major IM players do the same

so if the price will be 1M$ then maybe :)

s/price/award

You don't need that much for developing a client or two

Ge0rG: yes, I understand the most of the money goes into adv, but still there is a lot of guys working on a client

dwd, awards are not so cool

the issue with awards is that only one wins

or let me put it this way: it would require well thought out terms for people to participate

also I’m afraid it might just re-enforce already well-funded (in which way ever) clients

dwd: we really need to revive the Jabber Software Foundation

putting good software projects under an umbrella like Apache does could help

Ge0rG: same idea :-)

edhelas, arguably there is no good clients, it's all shapes of bad :)

Pandion was this client for windows in the early days of Jabber/XMPP

Ge0rG: what will JSF do?

marketing? promotion of clients? and the XSF do standards? (wild guess)

marketing = $$$

$$$ could be raised easily, when there is good software

Alex: wrong, when there is good marketing

Alex: no

Zash: both

OK. I have a strong position on XSF Neutrality, because I don't see a way to do things fairly. So perhaps it'd help to persuade people like me that the XSF can do this stuff if someone came up with a concrete proposal of how the XSF could do recommended clients/servers etc., and do it fairly.

Alex: tons of Conversations user don't even pay for it

zinid: its not easy, but doable

Catching up… fwiw, I don't have a "position" that we should do markdown that likes like plain text in <body>. I have a position that we should deprecate XHTML-IM and then we should take the time top write a good replacement considering the pros and cons of both approaches. Tentatively I suspect that doing it in the body makes sense, but I'm not pushing for that.

Similarly, this started because I've tried an XHTML-IM impl that was broken … again. So it's hardly a strawman for "other xss'". If I thought other xss' were caused by a poor spec, I would probably try to obsolete that too.

Alex: we have a council vote to advance them next week! https://xmpp.org/extensions/xep-0387.html

Or already did and it's waiting on list votes, I forget.

SamWhited, good luck trying to obsolete .innerHTML and others :)

I think Council just approved an LC on 387, and so vote to advance should be the week after, shouldn't it?

jonasw: exactly, that's why I'm focusing on xhtml-im instead :)

I think the LC needs to be announced properly on the ML first

some editor should do that *cough*

Ah yah, that sounds tight

Right, even

I can see if I can get around to do that this weekend

Oh, right. The two weeks don't start until that's announced.

SamWhited: How is "The Web" handling that? Because it's not just XMPP clients having that problem, right?

Zash: by not sending raw html around and trusting that developers can implement a white list properly or will do so at all.

Zash, have you looked at the web? it’s injections everywhere.

I looked at the web recently. It said "It is inherently impossible to make a secure web thing."

I assumed we were talking about transmitting formatting over instant messages, but yes, generally it's injections everywhere. I just still fail to see what other problems that aren't xhtml-im have to do with xhtml-im.

I'm thinking, what are other standards orgs or such doing about equivalent injection issues?

I'm assuming here that there's some overlap between xhtml-im and web apps

Ah, yah… good idea, someone else must do this.

More like "has someone done something already that we can steal?"

SamWhited, for me deprecating xhtml-im is just delaying the same kind of issues with another more or less similar XEP. You're complaining about a range of UAs that are either 1. not following the XEP, 2. Not doing their job correctly. I don't think you can't fix 1., and if you see 2. please report it. If you do and and still get no reply, then I don't think we can't do anything either.

Replacing XHTML-IM won't fix this

The XMPP thing to do is to turn it into XML on the wire. But people will fail at that, whatever we do.

pep.: it won't fix that, you're right. I am not arguing that a new thing will make developers never have any security issues ever again.

Right

I am arguing that I have never seen an xhtmlim impl that worked the first time and that it is especially easy to get wrong. We *can* make something where the default naive implementation does not commonly lead to badness.

This is the best you can do in any security issue.

SamWhited, would bringing up the poezio xhtmlim impl be trolling? ✏

goffi also has an implementation iirc

doesn’t movim have XHTML-IM support?

no

mathieui: sorry, I didn't include it here because phone, but yes, on the emails I think I specifically said "web based"

:)

I thought they talked about that (at least partial support, I think they strip @style, but maybe I’m confusing things)

that makes sense, np

SamWhited, libervia is a web client.

and everything that brings in a webview in a client application too

jonasw, no, edhelas doesn't want xhtml-im for some reason

Right, "things that actually have a javascript engine"

Getting of the bus… may be slow to respond.

SamWhited, also please don't merge markup in <body> :(

Off… I hate phones.

who doesn’t

I give up. Why does everyone think I want to push an alternative format? I will probably volunteer to write one if xhtml-im gets deprecated, but I am not pushing for markdown or whatever in body. I haven't done any research yet.

I personally do not think that, but we are going to need an alternative, and the alternatives all look bad.

We won't improve our situation by deprecating XHTML-IM without an alternative, and I still don't believe we will be able to come up with an idiot-proof alternative.

And if we're not striving for idiot-proof, we should assess whether XHTML-IM can be suitably improved.

If you do, the universe will spawn a better idiot.

Having an official safe JS reference implementation does sound somewhat promising tho.

And and a bigger better security considerations section

And blacklisting style.

What about the teends that crave colorful messages?

Bunch of predefined classes?

If we allow style, I'm not at all sure how even a reasonably diligent implementor avoids issues.

Instead of <span style="color:red">, there would be <span class="fg-red">

classes seem like not a good idea either

hm

it needs sanitization at least

to avoid that some attacker abusse your clients classes

Do they?

Hrm

but they’re much easier to sanitise than @style

(split by " ", make a set, check that only things starting with xhtml-im- are in there or so)

alternatively (but Sam will shoot me for that), xhtml-im:color="..."

howaboutno

why not?

(hm, still needs sanitisation to prevent injection attacks)

so using @class and defining a set of classes seems most safe for now

we should only allow one value for color, which is an integer between `0` and `359` on the XEP-0392 color wheel.

Amusingly, I just sent a short reply to one of Kev's messages that included the example: *&lt;script&gt;alert(123)&lt;script;&gt;* and FastMail rendered that as bold.

Why not hold a public poll where people can suggest names for them

SamWhited: /nick <script>alert("everyhing is broken");</script>

Especially my typing

what happens if you add unquoted HTML to the <body> text?

What happens when you encrypt unquoted HTML in OTR?

Ge0rG, you mean, like, <body><b>foo</b></body>?

Interesting how <b> isn't in xhtml-im

s/b/strong/

Zash, jonasw, reference implementation, and tests also maybe

Although I'd like to have more than just JS, because it's not just a web issue and it's not just an xhtml-im issue

Basically we need a reference client? :)

16:20:06 Zash> What happens when you encrypt unquoted HTML in OTR? → let’s not talk about OTR

jonasw: IIRC <b> was added back into html because nobody cares about semantics

And nobody is going to fix all the web that uses it

jonasw: yeah

Ge0rG: depends; web clients might joyfully let themselves being XSSed

but they do even without xhtml-im

> but they do even without xhtml-im I still can't figure out what the argument is there. Other XSS's aren't related to XHTML-IM… what point are people trying to make when they say that "there are other XSS's too"? If it's obvious to everyone else I apologize, but I'm honestly asking. There is some implied ending to that statement that I don't understand.

That XHTML-IM is not the issue, there's a deeper issue

Yes, XSS is a deeper issue, but you can't fix XSS on the web. You can not recommend a spec that actively encourages them though.

Is that what people are suggesting though? XSS is the underlying issue so there's no point in trying to prevent a subset of them?

You can try and improve XEPs all you want, that won't prevent clients from having bugs. I agree XHTML-IM could be improved to try and make people aware of it, but in the end it's mostly reporting bugs to clients

You won't prevent it, but you can certainly make it less likely that those bugs are catastrophic security issues.

Do you have specific examples of these issues btw? And I suppose you also reported them

Yes, I reported them. I won't go into the most recent one because it's not fixed yet as far as I know, but literally *every* XHTML-IM impl I've tried that was tied to an environment where JavaScript could be executed was either vulnerable at first, or had been vulnerable in the past (there was a CVE or issue about it or something)

HipChat for instance tried to do things right, but had a handful of bugs in their whitelisting (and I'm sure you could easily find more).

With XHTML-IM almost *any* simple bug leads to a security issue. We can absolutely write a spec where not every single simple logic issue allows a script to be executed.

SamWhited, Quite. The problem with XHTML-IM is that the obvious implementation is the most dangerous one, and insatead you more or less have to add an HTML protocol break in to make it safe.

SamWhited, In fact, edhelas suggested doing so in the server (and I thought of doing so in Metre), but the problem there is that XHTML-IM is often silently extended anyway.

dwd: Indeed. HipChat in theory does things right on the client (whitelist of elements and attributes, don't stick things straight into the DOM), *and* it has server side filtering. Despite that I've found at least two injections in that implementation (both of which are fixed now).

what’s an "HTML protocol break"?

jonasw: <bold> mebbe

Or markdown

jonasw, A protocol break is a security programming technique where you extract out the information into a different, fixed, form and then reconstitute it entirely from scratch. Means that, in our case, a bit of Javascript has no place in the intermediate form so cannot pass through.

dwd: And is that even possible while still being XML?

Zash, No, you wouldn't do it with XML, you'd do it with something else. Real protocol breaks would break up all the XMPP traffic into an intermediate form (JSON, maybe) and ship it across a wire to the other side, which then puts it back together. Obviously that's a little overkill for XHTML-IM.

You’re saying we can’t use XML for markup, at all.

jonasw: Yes. But we should, but we can't. Nice things and unavailability

That’s depressing.

Who ever thought web clients would be a good thing :x

What is the advantage to doing an XML based markup? I'm not convinced that we can't do it, but I do think a non-XML based thing (which I'm assuming means "in the <body>") has a few nice advantages (single source of truth, better fallback behavior, etc.)

jonasw, No, I'm not. Just that you can't copy it direct to output without very havey mangling.

jonasw, And the level of safety we're talking about in the mangling negates the advantage of it being XML in the first place, IMO.

dwd: people will find a way to lazily search and replace that lets trough evil stuff

jonasw, I mean, two XMPP servers talking across a heavily secured boundary have all their traffic mangled out, and then back into, XML. It's not the XML itself that's the problem, it's that you cannot do a direct copy and guarantee safety.

SamWhited, whatever we do, not in <body/>.

I agree with everything SamWhited said. Not a single client that I reviewed has gotten xhtml-im secure.

jonasw: what is the advantage of not doing it in the <body/>

dwd, so, I like your HTML protocol break argument. Indeed, based on that argumentation, I think a something-not-XML-based-but-still-structural representation of the semantics of XHTML-IM would make sense.

<body type='html'>&lt;script....

thanks waqas; you're running for council next term right?

SamWhited, doing it in the body ties us to poor plaintext-ish markups like Markdown, Creole or reStructuredText. Those are not extensible, the implementations often vary or are poor in other ways and such.

As part of my research I'd written a library that implements XHTML-IM protcol break: https://github.com/zeen/xhtml-im.js — it makes an XML DOM to an HTML DOM with a strict whitelist of elements, attributes and possible attribute values.

The xep talks about rtf. Do that

waqas: :-O

I need to npm-ify it, to make it easier to use and add docs.

waqas, that sounds like the reference implementation I wanted to see in XEP-0071 which everybody can use.

Ooh nifty, that's nice to have, thanks

So, let's pay for an audit?

actually I forgot, I think you sent this to me and challenged me to break it a while back and I never did.

never tried, I mean.

waqas, except that I think it also gets the "replace invalid tags with their children thing" wrong

(i.e. doesn’t implement it)

jonasw: let's kill that plz

It discards anything invalid first and foremost

i.e., it avoids all positives, but may have false-positives for certain attribute values

And the unknown element case

Zash, I think it’s a nice-to-have for extensibility.

(but I see how it’s hard to implement in anything which isn’t XSLT)

jonasw: you can extend the <message>, it's probably safer

Zash, how’d you extend <message/> with HTML-<video/> elements?

jonasw: why is a plain text protocol not extensible? If I want to add /italics/ later, what is stopping me?

for that matter, why are "plaintext-ish markups" poor?

SamWhited, It still amuses me that the word italics in your message is rendered in italics for me.

SamWhited, that clients which don’t know that /italics/ is italics won’t escape it

See, there we go, it works already!

Let's talk about /etc/foobar/

jonasw: Right, and we have a lovely fallback behavior, they still see the exact same message it just looks like someone added /emphasis/ in a different way

Extensibility is a secondary concern, that we should be thoughtful about. Security is the primary concern. I'm not sure the 'include the children' approach is always correct in HTML.

Zash, :D

so my /path/to/some/file will render italics on your new client supporting italics even though it shouldn’t

Zash, Not italics.

Zash: Yah, I don't know about the specifics of using /, just an example.

SamWhited, it’ll work for anything

+1

*sigh* so pick a different character, it was an example.

So /this is italics/ but yet /etc/passwd is not.

SamWhited, I challenge you, which one which doesn’t look super odd when seen without support?

I mean, right now, in Gajim, this is the actual case.

dwd: Machine learning!

dwd, what if I want to have only a part of a word in italics?

jonasw: That doesn't look odd to me, _emphasis_, /emphasis/, and *emphasis* all look relatively nice.

we're diving into specifics that don't matter though. The question is "why isn't it extensible" which you argued was a problem. Implementation details are not relavant at this point.

SamWhited, And they all worked in Gajim. They show the markup, mind, as well as the effect. But they work.

SamWhited, but that’s exactly why it isn’t extensible.

dwd: Oh that's interesting, I've noticed a few web things (not IM) that keep showing the markup and the effect recently and liked it, I wanted to try it in an IM client. Didn't realize gajim already did it.

You are re-defining things which previously were normal characters as meta-characters when extending it.

That simply breaks, always.

anyways, I gotta go for now.

jonasw: ahh, I see. That's fair.

I still think it doesn't matter as far as deprecating XHTML-IM, mind. The security concern comes first like waqas said.

But it's a fair argument if we're discussing alternatives.

A fair argument that I would love to consider in parallel to deprecating XHTML-IM ;)

So you're going to deprecate XHTML-IM, to then try and find a replacement, that is possibly just XHTML-IM 2.0.

XHTML-IM 2.0, now 200% more security considerations

:)

pep. yes. I certainly hope we don't come up with something that's just as bad, but we *know* XHTML-IM causes problems, it has been for years, so let's stop recommending new implementations of it. Keep in mind that people will still implement it for compatibility, and all existing implementations won't go away. It just means we don't advertise it as the way to do things.

SamWhited, thanks for seeing my point, that has driven me crazy ;-)

So that doesn't fix things at all, deprecating it. Fixing the XEP might be a better option? ✏

pep. we can't fix it without a rewrite, the basic idea is fundamentally broken.

If there is anything to fix

SamWhited, given the arguments from dwd about protocol breaks and such, I agree that we should do other things. at this point, I’m in favour of something like some simple JSON-based markup or so.

At least, I don't think we can, if you have a proposal I'd love to be proven wrong.

but I’m out of the discussion for now, I’ll write a list to standards@

jonasw: thanks, I look forward to reading it.

SamWhited, no I don't see how what you want to fix can be fixed, and I don't know if we have to worry about this to then come up with something as bad

I don't think we'll come up with something as bad. I'm reasonably sure most of us understand the problem.

Zash, Incidentally, `/etc/passwd` works in most IM-based Markdown variants as an escape pattern that also switches to monospaced "code" layout. But not in Gajim.

dwd, that doesn't change jonasw's point

And I agree with him

If you want to do something else please don't use <body>

dwd: Markdown is a html superset. You are doomed.

pep., Oh, I'm somewhat open either way, there.

I wonder if there's a middle ground, body and a hint about the type of markdown being supported. <body>this is *bold*!</body><formatting version="0.2"/>

Zash, I'm not proposing using *full* markdown. That would indeed be insane - nobody needs headers and things in IM messages.

SamWhited, so you'd include all different versions?

pep. you'd just give a hint about what the version of the spec was, then the client could implement all or nothing depending on if it understands the version attribute. I'm not sure this is a good idea mind, just spit balling.

dwd: markdown libs are going to pass through html by default

Same problem

SamWhited: I have this vague feeling that I've seen that proposal before

Like dwd, I'm rather open to either thing though. I do have a vague sense that doing it in body fixes a lot of little problems, but it has the downside that jonasw pointed out. I'm not sure which is the best tradeoff.

Zash, Well, I'm not actually sure most of them do, anymore. And in any case, there are loads of cut-down ones. But I'm merely hinting at a direction rather than wanting to compare it with XHTML-IM. It is, as SamWhited says, more a matter of agreeing we need to deprecate XHTML-IM and wondering what the functionality might be replaced with.

SamWhited, right. I don't think this fixes the issue. If the client doesn't handle *foo*, it'll leave the markup here, which is meh, and for each new version you'll just be defining more and more new meta-characters that weren't before. I'm not sure that addresses our issue

(As you just said ^)

I'm pretty okay with leaving the markup there; it may not be ideal, but it's a better fallback behavior than XHTML-IM right now (where the message just doesn't show up if you don't also include a plaintext body, eg. most HipChat extensions)

Well then HipChat is not compliant?

They should follow the XEP

pep., Do you seriously think that anyone using any Internet-based communications medium for the past two decades would be surprised to see *bold* things expressed as such?

dwd, not us no, but I'm pretty sure you don't want to target only us nerds with this

I'm pretty sure that everybody would understand *emphasis*.

pep., No, not just us. Any Twitter user, for example. Probably any Facebook user too.

Also, you will never be able to have any breaking change

Should it be about styling or semantics ? ?

I'd prefer it to be semantics

But I suppose Slack or Facebook users don't care

That's an interesting distinction. I think a replacement should just deal with text styling. If you want an image you have sims or something, and the client can decide if and how it wants to display that (maybe with an information XEP about displaying media in clients for guidance).

and the XML-based SIMS or OOB or whatever gives the client the semantic meaning of whatever the non-text resource is.

Someone mentioned push and publish-options being invalid?

https://xmpp.org/extensions/xep-0223.html#example-1 has the same issue?

Zash: yes, #persist_items is not registered within XEP-0060

only #access_model is registered

https://xmpp.org/extensions/xep-0222.html#example-5

ah, right :)

even more

Maybe one should do a `grep publish-options xeps/xep-????.xml`

sure, PRs are welcome, I'm told

but it seems like we just copy attributes from #node_config to #publish-options

what's the point? why not using #node_config data form directly?

It seems like at the time, people thought that was what it was

Like how you can create+configure a node in a single step, but this would be publish+configure (and maybe autocreate)

Seems like it'd be weird if you have many clients disagreeing on the options

Oh look at what fun issue I'm having. My xep-0071.epub had unescaped HTML like <body/> passed through, breaking things.

irony :D

Zash, nice

> The root element for including XHTML content within XMPP stanzas is > > .

Good job pandoc

> The raw HTML is passed through unchanged in HTML, [bunch of formats], EPUB, Markdown, [even more]

Maybe I should just generate JSON. That'll solve all problems.

{"blocks":[{"t":"Plain","c":[{"t":"Str","c":"The"},{"t":"Space"},{"t":"Str","c":"root"},{"t":"Space"},{"t":"Str","c":"element"},{"t":"Space"},{"t":"Str","c":"for"},{"t":"Space"},{"t":"Str","c":"including"},{"t":"Space"},{"t":"Str","c":"XHTML"},{"t":"Space"},{"t":"Str","c":"content"},{"t":"Space"},{"t":"Str","c":"within"},{"t":"Space"},{"t":"Str","c":"XMPP"},{"t":"Space"},{"t":"Str","c":"stanzas"},{"t":"Space"},{"t":"Str","c":"is"}]},{"t":"RawBlock","c":["html","<html/>"]},{"t":"Para","c":[{"t":"Str","c":"."}]}],"pandoc-api-version":[1,17,0,4],"meta":{}}

Zash, Just looks like smiley faces to me.

dwd: Please don't use JSON as a markup language. Altho maybe that's the reason it's the only choice.

Zash, I wasn't going to suggest JSON at all.

Forget about XHTML-IM and markdown, let's do this https://www.oasis-open.org/committees/download.php/60/HM.Primary-Base-Spec-1.0.html

Embedded SVG, but the version of the spec that didn't end up making it through where it had functionality to open sockets.

Done.

I really don't get why all examples of XEP-0030: Service Discovery error responses contain the query element mirrored from the request. To me it is as confusing as it looks useless. If a client does not support disco at all it won't know that it has to mirror the query element anyway, so why is it mirrored in all examples and do I really have to mirror it in my implementation? I did not find a MUST or REQUIRED about this in the specification. Do people mirror it? Do people rely on it to exist in error stanzas?

Syndace: I believe that's technically allowed by the specifications, but rarely used.

It's useful for debugging sometimes

Zash: So, I can ignore it as being historical? Is there any reason it might be helpfull that I'm missing?

zinid, okay I can see that, it's easier than matching ids of outgoing and incoming iqs.

I mean when you dump xml on server: you see what exactly your server is replying to with error

Syndace: but there is no requirement in the RFC

Okay got it, thanks :)