XSF Discussion - 2017-10-12

jonasw, +1 for your XHTML-IM mail on standards@

thanks

SamWhited I just saw your mail, kind of agree with it, but be careful as XHTML-IM is also bound to other XEPs

https://xmpp.org/extensions/xep-0231.html#referencing for example

Movim doesn't handle XHTML-IM except for this use case where a BOB image (actually a "sticker") in Movim is sent

oh, so there’s Yet-Another-XEP where images can be "attached" to posts.

(in addition to OOB and SIMS)

yeah but OOB descript a transfer method

*describe

sorry, BOB describe a transfer method

indeed

OOB and SIMS are just metadata

yeah

actually Movim support OOB, BOB and SIMS :D

i'd like to deprecate OOB

in favourof SIMS?

yup

I agree.

https://github.com/siacs/Conversations/issues/2637

hae

daniel, did you misread that issue? I don’t see edhelas mentioning BOB in the issue, but you’re referencing it. Am I missing something there?

jonasw: I'd like to see solutions like you suggested indeed, instead of deprecating xhtml-im

voice yourself on the ML :)

Will do

and thanks

What/how/who would write that reference impl. though? XSF itself doesn't do that kind of stuff right?

reference implementation of what?

pep., I wouldn’t say that

there are RFCs which contain a reference implementation (e.g. the Opus Codec)

so I don’t see any strict reason against that

ideally someone from the web development part among the XSF members (looking in the general direction of the movim people) who know JS etc. would take a shot. even more ideally, the XSF would fund a proper audit.

zinid: xhtml-im thread on standards@

ah

alternatively we could take a look at existing web clients wich get this right, but it appears that there are none :)

not interested :)

afk for a while

jonasw: right. I would love to see movim implement that at some point

Much better than parsing random /commands and still sending that as text

But I guess that's the extent of what's possible without that xep

I choose to not implement XHTML-IM actually, it's also to keep the UI clean

and it's heavy to cleanup/process XHTML messages on the volume

Xhtml-im* messages

I'm absolutely in love with poezio's /code plugin.

https://xmpp.org/extensions/xep-0223.html -- is it implemented anywhere?

Yep it's really nice. And it doesn't pollute people's screen when they don't do xhtml-im

Ge0rG, /code is annoying in the sense that the sender chooses the colours; they may not fit with the recipients background for examle

zinid used in Movim for the bookmarks and other things

Ge0rG I'm also handling /code in Movim :)

edhelas: and how to deal with clients who don't support it?

edhelas: the same thing we did with avatars: nothing? :)

zinid for bookmarks ? well I don't

Movim is actually not in sync with Conversations regarding bookmarks

and you think it's ok? :)

edhelas: not the same way as poezio does. You leave the /code in your body :/

because I choose to not use the old prive-xml XEP

And it doesn't make sense on any other clieny

Client

pep. poezio put colors on the code using xhtml-im ?

Yes

ouch

Why?

then no, because it will look ugly

Why?

well movim has its own color palette and so

I don't want you to impose your pink and green colors in the Movim UI

Sadness

also I can easily colorize the code myself, a simple lib on Movim side can take care of it

edhelas, allowing XHTML-Im but filtering @style seems like a reasonable thing to do

you need to know that it’s code though. XHTML-IM does not allow for a <code/> tag

edhelas: also, you're still polluting the body with /code

then XHTML-IM is not a solution for it

edhelas, itym then XHTML-IM should be extended ;-)

pep. I agree as well for /code, we also have /me (even if there's a XEP yeah)

why ?

having a proper XML tag would be way better

jonasw: handling of it is fine in implementations, but it come directly from IRC

<self-quote/>

And I agree with edhelas here

yes, it would be better. but it works today, and breaking that would break a lot of clients for little gain

if you don't like /code I understand, but let's be consistent

/code is a client-side feature

the "/code" is never transmitted

edhelas: history is never really consistent though. :/

Also it's not just about /code

but in general I'm against clients that are forcing the rendering of content to another client

It's also about all other possibilities that you're missing without xhtml-im. Or that you (somebody) are trying to reproduce badly

this also applies to Atom publications in Pubsub, actually Movim is heavily filtering the XHTML tags to remove all the style and customization of the articles

jonasw: I think that XHTML-IM 2.0 colors should be limited in the same way that the Colors XEP works. The sender defines the hue, and the displaying client is responsible for brightness and saturatio

+n

edhelas: you should probably detail this statement, I'm sending you messages forcing you to display them. (Unless you ignore them)

I'm ignoring xhtml-im

only use the body tag

XMPP is a transport protocol, like HTTP is transporting HTML

Ge0rG: that sounds nice

the forating rules of HTML/CSS/JS is another playground

*formating

to XMPP can tell me if a message contains code, like HTTP tell me if an answer contains text or images

but doesn't tell me how to format thoses

Ge0rG, +1

Well xhtml-im doesn't, you choose

I thought about that when I wrote my reply, but I think that’s another battle.

edhelas: it's all about semantics

jonasw: yeah. But that should work on most neutral backgrounds, and with non-neutral backgrounds the displying client has the option to tune the curves accordingly.

XHTML is, really

Ge0rG, yap

pep., +1, which is why I feel we should keep XHTML-Im

edhelas: which is why, for example, the /code movim leaves in the body is meaningless to me, I don't know what to do with it if I don't use movim

(I agree that the xep could be extended)

sure

edhelas, re bookmarks and movim, does movim use publish-options or otherwise require that the node is configured correctly or would it publish the bookmarks for everyone to read if the pubsub service doesn’t support the correct access model?

jonasw it set the configuration of the node on the first start

set on whitelist actually

mhm

same for other nodes like avatar, vcard4, geoloc, microblog and subscriptions, with their own config

XEP-0357 says that the app can include arbitrary payload for the app server using <publish-options/>. XEP-0060 says that a "pubsub service advertising support for publishing options MUST reject publications with unknown fields." So I can't use a standard PubSub service to implement an app server, right?

depends

is "unknown fields" qualified in the XEP?

otherwise, one could argue that the app server understands those proprietary XEP—0357 fields

I'm not convinced that pubsub is actually the right mechanism for this anyway.

I'm ranting against the PubSub syntax for push notifications since forever :-)

It should just use a plain message.

jonasw: "Fields and their behaviour MUST be registered with the XMPP Registrar." -- https://xmpp.org/extensions/xep-0060.html#publisher-publish-options

Holger, aww

That's a silly requirement

Like saying you MUST use standards

kind of

MattJ: No it lets clients rely on behavior.

MattJ: Publish options are being used to make sure a node is not world-readable, for example.

Yes, rejecting unknown fields makes sense

But requiring that every field MUST be registered I'm less sure of

If you guys are just saying that a non-standard service may accept the 0357 options then that's what I'm saying.

MattJ: if it's not registered, then how to federate?

You can't use a standard service but must build your own PubSub thing to create an app server.

I think you must do so anyway if you actually need to parse those custom options (such as the secret).

A non-standard service is a non-standard service and doesn't need to follow a MUST in a standard, because it's non-standard

So basically I'm back to "don't use PubSub syntax".

zinid, there are cases where federation is not a concern

MattJ: Right. So we shouldn't change the standard to also standardize non-standard behavior.

MattJ: ok, but in that case you don't give a damn whether there is MUST or not

Exactly.

<MattJ> Like saying you MUST use standards

0060 doesn't do that.

It just says "if you follow 0060 and announce publish-options support, you must behave as defined". This only works if only registered options are supported.

I.e. if you reject options that aren't registered.

> Like saying you MUST use standards I think it's like "you MUST use standards to federate/interop"

Yes, to federate/interop you generally must use standards

But this is a case where you don't, right? :)

If people don't bother to use standards, they won't bother more if the standards write that you MUST use standards.

MattJ: The idea is that you could use a standard PubSub service.

right, so Holger is right: we shouldn't use pubsub for this

MattJ: ... and have the app server subscribe to that.

MattJ: That PubSub service would be a federating/interoperating 0060 service.

MattJ: At least that's how Lance argued back when I argued against PubSub :-)

If the thing doing the publishing knows that the thing accepting the publish supports a certain option in a certain way, it doesn't matter if it's an XSF-blessed standard, is what I'm saying

why do we need so complex pubsub, it's kinda relay, what for?

I mean why would an app server subscribe to it instead of directly receiving stanzas?

MattJ: I'm talking about "the thing accepting the publish".

which has to understand the options, no?

MattJ: Lance said he's using PubSub to make it possible for a standard publish service to be that thing.

MattJ: I'm saying this won't work.

Example option?

MattJ: Yes. The thing has to understand non-standard options.

'secret'

Is this something you made up for your implementation, or is it in the XEP?

MattJ: Example 13 in XEP-0357.

MattJ: It's just an example. 0357 says I may include arbitrary options. 0060 says a standard PubSub service MUST reject them.

Well a simple solution is to register the option ;)

No.

You want to allow for arbitrary options.

ChatSecure uses some other stuff.

I still don't see a problem

MattJ: As you said this is really just communication between the app and the non-standard app server.

So, you're using an off-the-shelf pubsub service

MattJ: So it just makes no sense to use PubSub for this.

and it doesn't understand your custom option, but what is it meant to do with it?

MattJ: That's the next problem, which is why I'm saying this won't work anyway :-)

MattJ: but you need to patch the server in order to accept options in order to be passed them to the app server

The node subscriber won't see the option.

Right

MattJ: In practice the problem is that people try to build an app server on top of the existing PubSub code, because that's what the XEP suggests.

MattJ: Then I tell them that this isn't possible because it just looks similar to PubSub but is slightly non-standard.

Ok, I see better now

You're complaining as an XMPP server developer, not as an app server developer

Well.

what's the difference? :)

Sure my immediate issue is that I want to get rid of the support requests :-) But I think the app server developer will also appreciate not running into the problem.

Sure

As I said I think the proper fix is just using <message/> syntax. But if for some reason (which?) PubSub is preffered, use some other container for the custom stuff, not <publish-options/>.

I also had that thought about "why not messages" when reading Push XEP back then

Lance' initial 0357 draft did just that, FWIW ...

maybe Lance knows why it was changed, then?

https://github.com/legastero/customxeps/blob/gh-pages/extensions/push.md

oh, "push token maintenance" is another thing I forgot on the device-identifier mail.

<Holger> Lance: What's the reason for using PubSub as opposed to plain messages to talk to the app server, BTW? <Lance> Holger: existing protocol, terminology, and semantics

There you go.

great

we can use pubsub for messages and presences btw

because why not? existing protocol!

I would say a <message/> offers all those features as well :-)

Time to ask on standards@?

We can use pubsub syntax for not-default-xep60 services just fine, if we want to (e.g. 369).

And I think there's an argument for doing so. But if we go that route we need to be clear that it's not a standard pubsub service you're using.

zinid, https://xmpp.org/extensions/xep-0207.html

MattJ: I know :)

Kev: Same syntax, slightly different semantics sounds awesome to me.

The road to hell is paved with good intentions?

especially that in fact we're not reusing *existing* technology, where are using a fork

have just seen that many of teh projects are gone on the client/server/library page, because of the renew data. I don't think this is helpful for us as a XMPP community

"the renew data"?

Holger: https://github.com/xsf/xmpp.org/blob/master/data/README.rst

Alex, anything specific you’re missing?

*last_renewed* field

jonasw: of course, I think the libraries page was 4x the size before :-)

Alex, sure, but were those maintained?

unmaintained libraries may be worse than no libraries

I don't agree on that

Alex: Seriously? Someone new to XMPP should wade through an endless list of dead projects?

one of the most used JS libraries for example, Strophe

Alex, if strophe.js missed the fact that they need to renew and they’re actively maintained, I think someone should tell them :)

(Huh there's an AstraChat server?)

Strope is only 1 example, there are many others which are missing

it would be nice to get an e-mail that my project is about to disappear

but other then that, the new system is far better then listing a bunch of dead projects

Sleek is another famous Python XMPP lib which is missing

bear ^

when I am new to XMPP and browse this pages I don't see many options, which is not positive

huh

Alex, many options and trying three which are unmaintained may be worse?

should’ve watched out for sleek

jonasw: I don't think that most of the projects which are gone now were not maintained, but that is just an assumption

How about sorting by last update or something like that?

we can send a mail to standard, jdev and members and ask teh authors to renew their active projects

sorting by a non-obvious thing seems bad

Alex, we did that

at least jdev I think

ok, mnissed that probably :-)

Subject: [jdev] XMPP Software Developers: Action Required

Here is a nice one regarding the brokenness of TCP and ICMP on the public Internet. The main focus is fragmentation, but the problems stated also apply to detection of stale TCP connections: https://blog.cloudflare.com/ip-fragmentation-is-broken/

Ge0rG: nobody cares, they just invent crap like http2 instead of really fixing things

Just like XSF 😂 Because interop

it's even worse.

Can someone not using Gajim write something with *bold* and /italic/ things in, please?

dwd: at your service. Italics doesn't work though.

this is not /italic/ and neither that one is *bold*

Damn <https://dev.louiz.org/issues/2722>. ✏

Ge0rG, Yours came as XHTML-IM, I think.

mathieui: "at" was bold.

Ge0rG, yours, yes

mathieui, Whereas yours was rendered as italic and bold for me, Markdown-like.

jonasw, err, I would like xmpp: links in href :p

dwd: ah, so you were asking for markdown in 'body' and not for XHTML-IM formatted.

Ge0rG, Indeed, sorry.

I like the idea of markdown in XMPP (I'm not entirely sure that I didn't originate *stuff* in XMPP with Psi), but I really dislike not having a standard. So I think we'd have to actually define a MD standard ourselves (which we pretty much had to with XHTML-IM, of course).

so we are reinventing XHTML-IM without XML now?

I don't especially care what a new thing looks like, but I do like the idea of it being some sort of plain text formatting in <body> as opposed to a separate <fancybody> element or whatever.

people will just plug-in their favorite markdown parser and your body texts will end up mutilated and you will still end up being pwned.

It keeps things simple to have one version of the message. One place to encrypt when doing E2E, no risk of something malicious sending a different plain text / non-plaintext version so that two different clients have tow representations of the conversastion, etc.

SamWhited: there is no amount of specification that will prevent developers from doing stupid things.

Ge0rG: I agree. What's your point?

Security issues will always exist, so we shouldn't even try to mitigate any of them?

SamWhited: XHTML-IM is good as it is.

Except for the history of insecure implementations.

SamWhited: I have a dozen of CVEs for people not reading the Security Considerations of 0280.

I want to burn Carbons with fire, but for completely different reasons.

SamWhited: by replacing XHTML-IM with markdown, you will keep the exact same security problems.

most markdown implementations even allow raw html right in the plaintext, so I don’t see how that solves the "developer get it wrong" part

And probably add some more.

what mathieui said.

Ge0rG: That's why I'm not arguing that we should use markdown.

Just that we should obsolete XHTML-IM.

yeah, that’s why SamWhited made a point in the thread to keep it about deprecating xhtml-im ✏

SamWhited: Council just refused to deprecate 136 despite it not being a replacement/alternative to 313.

Ge0rG: I know, it made me sad. This is a security issue though, which I think makes it slightly different and more pressing.

I don't see a particular need to deprecate XHTML-IM. I think *anything* involving markup being injected into a DOM is going to see people doing stupid things.

But I do think that it's woefully inadequate at protecting diligent devs from things they haven't considered.

XHTML-IM is not perfect, it's got some ugly warts. But it (or some other markup XEP) has its place, and anything that you can come up with as an alternative will have the same or worse security problems.

I disagree, it would be quite easy to design a spec with the same level of functionality but where it required active effort on the developers part to introduce the same security problems.

SamWhited: maybe you can write that spec up, then?

It doesn't need to be impossible, it just needs to not be the "default".

and then implement it in a bunch of widely-used XMPP clients.

e.g. if we design a new markup language with a secure reference implementation

Ge0rG: no, I'm not interested. I'm just interested in getting rid of XHTML-IM.

And then ask again for deprecation of XHTML-IM?

I remember embedding javascript within my nickname in jappix that would dump account passwords to a MUC

no need for xhtml-im there

Yah, I've found plenty of those too, they're unrelated though.

Yeah, so many places for XSS.

they have the same root issue: the web

mathieui: If you have a proposal for fixing the root issue, I'm all ears :)

:D

I would love to fix "the web" :)

SamWhited: So what you really wanna do is deprecated teh web! :)

Let's do tha!

use markdown :-)

Yeah, let's approach the W3C with that.

SamWhited: I do think that having even a strawman that demonstrates that it's straightforward to write something better would help with those people who think that replacing XHTML-IM with something else is likely to also be easy to carelessly introduce vulns.

*sigh* yah, maybe you're right.

Kev: even if there was a strawman, we still have the wide deployment problem.

XHTML-IM is supported over a pretty large client base, as opposed to non-markdown-strawman.

Ge0rG: Yes, but one bridge at a time.

<message><markup-hint lang="commonmark:1"/>…</message>

I think you are probably right, that would go a long ways towards convincing some folks, I'm just not sure that I want to put in the work on that. I'm not interested in even having a replacement since I'd probably never implement it myself (not that I think one shouldn't be made, I understand that lots of deployments need basic formatting)

Also related: https://blog.plan99.net/its-time-to-kill-the-web-974a9fe80c89

But I do want to deprecate it, so maybe I need to just waste some time on a thing I'll never use to convince people. I'm torn.

s/deprecate/obsolete/ (I'm trying to be precise about my language, but I still mix those two up constantly)

I want to deprecate GC1.0. And then some.

Ge0rG: I think I proposed that onlist as an alternative to 2 or 3 or whatever teh number was, and I don't remember any pushback.

Kev: it was brought up at the last-but-one council and got vetoed immediately, IIRC

Kev: you also still wanted to provide more alternatives to 1-2-3.

Ge0rG: I don't think I did. I think my alternative was to do 2 or 3, whichever the one I said I liked was, and to get rid of gc1.

Kev: oh, sorry. I must be misremembering then.

And I might misremember, but I don't recall anyone objecting to it in the thread.

Maybe because it was TLDR, once again.

Kev: oh, there it is: "Georg’s started a worthwhile discussion here, but I’m sure these aren’t the only 3 options we can consider."

Kev: you also liked (2), which was the new-iq-type.

but I'm interested in the other options before writing a diff to 0045.

Ge0rG: Did I not follow-up by saying that I thought that if gc1 was the reason that (3) was better than (2) we should kill gc1?

I accept there's a very real possibility that I'm simply going mad.

Kev: you ended up discussing with Zash whether we can reliably obtain statistics on "intended GC1 joins" vs "presence updates misunderstood as GC1 joins"

which we can't.

Why not?

Zash: how?

Kev: I've long believed that every person participating in this chat room and the related organizations is affected by a certain kind of madness.

Ge0rG: Look at presence sent to a MUC. Does it have the <x>? Is the user already in the room? Log that.

Zash: and then compare that with <x>-enabled presence from the same full JID in the last X hours?

Zash: That's not the question.

Zash: That's just listing which presences don't have <x/> it's not telling you if it's a gc1 join or a presence update.

Sorry, got pulled away… RE GC1.0 I agree, let's deprecate it. I don't think it was vetoed, I think we just decided to get more feedback on list first, but I don't have the minutes in front of me.

wow, just got a presence from somebody in the form of "phone." + [68 characters that look like base64, including slashes]

the madness is rising.

SamWhited: yeah, the final decision was to get some list discussion, but there were some proponents of keeping GC1, surprisingly

Madness

http://www.youtube.com/watch?v=QH7fIkV8nsk

SamWhited, you quoted me rather unfairly on the ML. I won’t point it out there, because I don’t believe you did that on purpose.

jonasw: Did I? Sorry about that

I removed the extra, but I didn't think I took you out of context or anything

you removed the part where I said "plus providing a reference implementation"

Yah, I know, I wasn't responding to that part

It's unrelated to further restricting the subset of HTML involved

not entirely, it’ll simplify the reference implementation.

ah, I see, restricting the HTML was about making the reference implementation simpler? Yah, I misunderstood what you meant

Either way, I don't think a reference implementation helps either

all implementations, but also the reference implementation, because I agree that simply changing the spec won’t solve a thing

I strongly think so

I think adding a reference implementation won't solve a thing (although I'm certainly not against it, if we can have an implementation that we can more or less guarantee is secure, maybe at least a few things will use it and not be broken out of the box)

But not everyone will use it, so I don't think it really helps

if we link the reference implementation prominently in the XEP and doesn’t have any non-DOM dependencies, I think new implementations certainly will use it.

Yah, some will and that's nice, but not all will

meh

It doesn't really solve the problem, just puts a bandaid on it for a few implementations (which isn't bad, and I think you should totally do it, but I think we should still obsolete the spec either way)

There are so many places to inject code into the DOM of a JavaScript XMPP client, XHTML-IM is just one of them.

see, my main issue is that I really don’t want to see that we use plain text markup when we have the opportunity to use structured markup (be it XHTML-IM or anything else)

Ge0rG: yes… again I don't understand your point though: security is hard, so let's not fix any of the problems?

Other places where DOM injection is possible aren't related to this discussion

I think that providing a reference implementation is the best we can and should do.

jonasw: I'm not 100% sure but I *think* I agree with you. Deprecating XHTML-IM doesn't mean we're going to use plain text markup in the <body> though.

SamWhited: I'm just saying that security-ignorant developers will keep writing vulnerable XMPP clients, with or without XHTML-IM

SamWhited, what else are we going to do? The alternative will be to reinvent XHTML-IM.

Ge0rG: Yes, I agree. I don't see what that has to do with anything though.

jonasw: Yes, reinvent it, but reinvent it in such a way that it's not HTML.

SamWhited: I think my slightly exaggerated point is: you can't make webchat more secure by removing XHTML-IM.

We shouldn’t sacrifice something that indeed works, can be made secure with reasonable effort (if we abolish @style, which we totally should do), to make in total only a few implementations transition from not-secure to secure (only a few will transition by obsoleting XHTML-IM, because many will have other XSS issues)

SamWhited, people will simply patch tag names and embed it in the DOM.

if it’s more complicated than that, people will not implement it.

(and if people simply embed an XML tree into the dom, havoc will be wreaked)

jonasw: Your'e assuming @style is the only problem. That's not the problem that I've seen, the problem that I've seen won't be solved by further restricting what HTML is allowed.

Maybe we should provide an XSS bot that will join your MUC with an evil name, post evil messages and change the subject to some <script> tag?

People are allowing <script> tags and the like, it doesn't matter how much more stuff you take out, they'll still just allow <script> tags.

SamWhited, I’m keeping to mention @style, because @style is really really hard to fix.

the other parts are more or less a piece of cake.

jonasw: Yah, I agree with you we should take it out. I just don't agree with you that it would solve anything.

SamWhited: the problem you are trying to fix is ignorant developers?

Ge0rG: "fix" is the wrong word, but essentially yes. We can't "solve" all the security problems, we can never stop developers from doing a dumb thing and allowing a <script> tag. We *can* make it harder to do that by default.

SamWhited, with @style, it is entirely unrealistic that any implementation is safe, because nobody will parse CSS.

jonasw: I'm not arguing against that. By all means, remove style.

But that's another orthogonal problem.

only if you’re set on obsoleting XHTML-IM, which I’m not ;-).

It's orthogonal either way. The problem that I've seen, over and over again, is that people say "it's HTML, so take it out of the <html> element and stick it in the dom" or they have some subtle bug in how they do their whitelisting that allows it to be bypassed. Removing style won't solve either of those things (though again, I'm not disagreeing, it's something we should do to help existing implementations)

okay, I see your point.

« 08:39:27 jonasw> you need to know that it’s code though. XHTML-IM does not allow for a <code/> tag », it totally does, fyi.

I should totally improve poezio’s XHTML-IM module to run pygment on incoming code!

I'm almost surprised that you haven't already

since 2003!

we’re way behind

note to self: DoS Link Mauve by pasting perl code in MUCs

Does <code> not have a language attr tho?

mathieui, you know you don’t need anything fancy to DoS my server. :p

right.

Zash, hmm, maybe I’d have to write a XEP extending 0071, maybe with a standardised class attribute or something?

Maybe you already discussed this, but what about all the ways CSS can invoke more resources?

Link Mauve: Standardized classes would sorta make sense I guess

Eg like vcard thing

yeah, a class preset would actually solve all the portability issues of 0071, along with solving the potential for CSS abuse

Way past brain-works-o'clock

microformats!

Zash, don’t say that, I just woke up for the fourth time of the day!

yes but you didn’t wake up yesterday, so it evens out

True.

I haven't woken up today.

MMmm GMT+2 jokes ftw

Uh, I should make an email-to-epub thing