jonasw7.5 MiB data limit -- that’s well sufficient for avatars
edhelasjonasw at one moment people will ask for high-def-animated-with-sound stickers
jonaswI recall that people said that PEP avatars are not feasible due to (among others) stanza size limit?
ZashHere we go. Why can't I have my browser plugin that lets me link directly to sections?
> A deployed server's maximum stanza size MUST NOT be smaller than 10000 bytes
ZashWait that's ~10k
Steve Killehas joined
zinidand there is no way to check what limit is applied on the server :)
zinidwe need a XEP
ZashPath MTU discovery!
ZashNow with eXtensibility! :D
ZashMattJ: How did we end up with 10M then?
ZashI suppose you could survey peoples vCards and multiply the maximum size by some number out of a hat and call it a day.
jonaswZash, it’s not impossible that I complained a few years ago when my fiancees avatar wouldn’t upload and that made pidgin fail to connect or something. I remotely recall there to be such a problem.
Zashjonasw: Not the thing where data goes into a buffer that the network stack isn't paying attention to?
jonaswback in the days of google code
jonaswnot sure, I think it was some limit thing
ZashHey, how do browsers deal with namespaces?
ZashLike, if I stupidly insert some random XML tree into my DOM, and there's like a <p xmlns="not xhtml" onclick="evil();"> in there, what happens?
jonaswthey don’t care a lot
jonaswand as always, it depends
ZashSo a thing that sanitizes the xhtml-im namespace but lets anything else thorugh would be useless?
jonaswI can imagine that this kind of stuff works because it is hared between XML and SVG handling
jonaswI wouldn’t rely on it
jonaswthere are funny bugs in browsers with prefixes, too
Steve Killehas left
Steve Killehas joined
jonaswKev, +1 to your XHTML-IM mail
jonaswyou brought to the point what I tried to convey in a few paragraphs of prose elsewhere.
KevI don't have the attention span to write TL;DR mails :)
Ge0rGHa, I tried to make the same point yesterday.
Ge0rGLet's see if different framings are going to convince the public
jonaswitym the council.
zinidguys, what is the agreement on pubsub in push?
KevI don't think the move towards something markdownish is actually stupid, FWIW, and I think it's much much easier to sanitise something that you can write your own parser/serialiser for, than XHTML-IM. So I don't think this is a bad direction. It is much easier for diligent devs to get it right. I'm just not sure I buy the argument that it's going to suddenly make anyone who wants to dump things into a DOM unsanitised safe.
Ge0rGI like markdown, but it's impossible to write a parser for that.
jonaswI agree with Ge0rG
jonaswwriting your own markdown parser is a mess, and will yield 1000 different implementations
jonasw(this holds for all text-based markups, I’m afraid)
Ge0rGIt's even less possible to sanitize markdown.
KevWell, we can go with a binary markup if you want, and then base64 it to get it into the stream, but I don't think it helps :p
Kevpreempts Dave suggesting ASN.1 encoding of markup.
Ge0rGKev: it's already impossible to write correct markdown _text_, which is rendered in the same way everywhere. How are you going to write a parser?
jonaswKev, I sense buffer overflows there.
ZashThis is the appropriate reaction: https://pics.zash.se/4c840479.jpeg
jonaswusing an XML-based markup makes much more sense to me.
Steve Killehas left
KevGe0rG: That isn't the hard part. Tedious, but not hard, we just need to spec it. And by 'we', I mean that dwd has volunteered, so yay.
zinidwhere to vote for asn.1? :)
Ge0rGBER or DER?
jonaswhasn’t BER failed?
ZashIs server-side cleaning of xhtml-im sane, or would that just let broken clients be broken and then get hacked by evil servers?
KevAbstract Syntax Notation 71.
jonaswfrom what I’ve heard, the general sentiment is that clients need to trust the server anyways to some extent...
ZashYes, trust the server, the server is good.
jonaswbut I guess it’d break e2ee
ZashE2EE is just marketing fluff, did't we establish that?
zinidjonasw: +1, I can't understand what's the point in building client-server architecture where there is no trust to server?
jonaswZash, I won’t get into that argument.
zinidbetter off using p2p directly
jonaswI’m torn on the e2ee thing. I don’t want certain stuff I discuss with people plaintext on my server in some MAM.
jonaswzinid, p2p doesn’t scale
Zashp2p doesn't scale *on mobile*
zinidjonasw: I wouldn't say that, there are some research going on
Ge0rGI really don't get why people hate ASN.1
zinidGe0rG: because it generates lots of boilerplate code for mainstream languages such as C++ or Java
zinidfor example, in Erlang it's great
jonaswnot rather because ~every ASN.1 parser is broken somehow?
Steve Killehas left
zinidjonasw: but we have PKIX working?
Steve Killehas joined
jonaswsure, take a look how secure the PKIX libraries are
Ge0rGEvery browser contains an ASN.1 decoder. We could leverage that and replace JSON, once and for all.
ZashGe0rG: Call it "binary JSON with schemas and namespaces"
ZashMake a fancy website on whatever is the hip TLD today.
Ge0rGZash: yeah, marketing is crucial.
zinidjonasw: I think that's the problem of the language you're using
jonaswzinid, excellent, let’s ignore issues because they only occur in a single language.
zinidjonasw: well, we ignore? and what do we have as a result?
zinidASN.1 was a standard and now there are tons of implementation with 0 interop
edhelasthe XHTML-IM problem can be extended to Pubsub as well, with the usage of Atom
dwdKev, I volunteered to document a "snippets" design, which - I think - covers most of the use cases outside of *bold* /italic/ _underline_ and `preformat`.
Ge0rGmaybe we need a subset of ASN.1 that is easy to understand and to implement. Let's call it ASN.0. Or maybe ASNdown.
zinidGe0rG: I would rather use protobuff, frankly
zinidbut it's not a standard
dwdzinid, FWIW, I've not come across any of these incompatible ASN.1 implementations, and I've used many of them.
Kevdwd: I had completely misunderstood, I thought you'd volunteered to write the thing that's like markdown spec, sorry.
dwdzinid, After all, when I worked at Isode, M-Link had about 6 in the code.
edhelasbut I'd say, maybe we can just add modules on servers to checkup the content of messages when they detect the xhtml-im namespace
edhelasit's basically applying a schema and check if it fits
dwdKev, I can probably do that too. But I have a pair of XEPs and a I-D in my pipeline, and adding another one is pushing things enough. (Oh, and I'm trying to shepherd Ash through updateing '314 *and* writing another).
ziniddwd: well I'm not talking about incompatible implementations, I think jonasw said that :)
dwdedhelas, Really? So I can strip tables then?
jonaswzinid, nope, I did not mention incompatibilities.
dwdzinid, Then I misunderstood: [09:47:59] zinid: ASN.1 was a standard and now there are tons of implementation with 0 interop
ziniddwd: I mean there are now: thrift, protobuff, several json serializers, etc
Ge0rGbut those are all not ASN.1?
zinidyes, but they do the same
zinidencode/decode lang structures into/from wire format
Ge0rGIt's funny how SQL injection is still a thing in 2017. And XSS.
KevWhen I keep talking about markdownish, to be clear, I'm talking about something that is clearly and comprehensively specced, presumably by us. I am not suggesting it's ok to say "use markdown", and then just have everyone pick their own, subtly incompatible, library.
Ge0rGKev: I'm not sure that makes a difference. If it is sufficiently similar to markdown, people will just plug their own markdown library in and end up wth full HTML support.
KevGe0rG: I'd really like the discussion to move away from "People will do what's easy, not what the spec says", and instead try to work out how to have a spec that a reasonable person is likely to implement without significant issue. XHTML-IM, as it stands right now, is not the latter thing.
jonaswKev, I’d like your opinion on a (possibly audited) JS reference implementation of a sanitiser.
KevI think it's a different question to whether XHTML-IM is sane. I don't think "It's near impossible to get right, but we've already done it, so use our implementation (or port it)" is the right thing to do.
KevIf we get to the stage that we *do* have a sane spec, then a reference implementation could be helpful, but at that point is also probably not as necessary.
jonaswI think that XHTML-IM is easy to get right, once we drop @style.
jonasw(you’re gonna have the problem of sanitizing URLs in any markup which supports URLs)
Ge0rGKev: I'm not sure that it's actually possible for a reasonable person to implement a secure web application.
zinidI'm lost, what is required to be sanitized? URLs?
Ge0rGKev: and I think that XHTML-IM is mostly a strawman here.
zinidxhtml-im doesn't define scripting, so there should no be XSS, no?
(I'm not a web developer)
Ge0rGzinid: the problem is that it's insanely hard to sanitize user-controlled strings in a web application. And even more so if those strings contain HTML markup.
Ge0rGzinid: change your nickname to `<script src=.../>` and there is a good chance some client will actually load and execute that.
zinidI know about web application, but this is mostly due to script execution
zinidyes, your example is also about scripting :)
Ge0rGzinid: yes, but there are so many ways to include scripts.
Ge0rGand developers need to know and filter them all.
zinidbut xmpp client should not execute scripts?
Ge0rGzinid: should not, no. but if the client is running in the browser, the browser well might do
jonaswzinid, the fact that XHTML-IM doesn’t define scripting does not prevent (a) maliciuos entities to include <script>...</script> and (b) stupid clients to embed the XHTML from the message unsanitised into the browsers DOM
zinidjonasw: yes, got it
zinidbut don't we have the same problem with markdown? how is markdown processed? web clients can pretty much insert <script/> from markdown into DOM as well
zinidand the same for raw messages (i.e. <body/>)
Ge0rGzinid: that's a very valid point, made multiple times by now.
zinidsorry, TL;DR :D
Ge0rGyeah, that is a common problem :>
pep.Ge0rG> Kev: I'm not sure that it's actually possible for a reasonable person to implement a secure web application. < +1
pep."Markdown is not HTML so people will have to write their own parser" what? (*me just read the ml thread*)
pep.Or rather, people can't drop that into innerHTML. Don't worry they'll find ways
Ge0rGI mean: I'm a professional IT security specialist, and I don't know all the ways that you can inject code into a web app from user input.
pep.I'm not even sure how to contribute to that thread. People hear "We need to deprecate XHTML-IM" and already it's "I know $NEW_FANCY_MARKUP that we can use". We're just going in circles.
pep.Kev> Ge0rG: I'd really like the discussion to move away from "People will do what's easy, not what the spec says" < That's exactly what's happening at the moment with XHTML-IM.
pep.And that's what sam is ranting about
pep.If it their implementation were compliant there would be no such issue. (Or we could fix the XEP to answer these issues)
SyndaceCan you deprecate a XEP, modify it and then make it draft again?
pep.If it their implementation was compliant there would be no such issue. (Or we could fix the XEP to answer these issues)
pep.Under a different name/number? dunno. That would be silly anyway
jonaswpep., must be subscribed, but subscription is open
pep.I get the emails, so I did that iirc
jonaswyour mail got through
jonaswthe list has some delay, in the order of a few minutes
pep.Yeah I saw it/them come back :)
pep.Yeah I saw it/them go through :)
pep.Why do my first emails have to be hate mails :(
jonaswit’s not hate, it’s constructive discussion (for now)
pep.https://firstname.lastname@example.org/msg17919.html I like goffi's position here
KevWasn't the first of those mails "We have to consider people who ignore the spec" and the second mail "Changing the spec will fix it"?
KevI think as long as your position involves the "stupid people" argument, there is no way to find a solution to injecting markup, but there is certainly no way for xhtml-im markup.
pep.I'm not of the opinion that changing the spec would fix the issue. It might help a bit
KevJonas said we should change the spec, you said you strongly agree...
jonaswsure, because @style is hard to validate right. everything else can be done.
pep.Well, that would help yes.
jonasw(even @style can be done, but I don’t think we should be asking people to do that)
pep.Kev, also look at my second sentence. "the issue is not here"
Kevjonasw: Yes, and that's fine if you agree with my premise that we should be catering to people who want to do the right thing, not to people who ignore the spec and inject HTML directly. If you want to go along with my premise there, then changing the sanitisation rules can help. But if you're on the premise that people will ignore the spec, no amount of changing the spec will help with that.
jonaswKev, I still believe that a working reference implementation will help against people ignoring the spec.
jonasw(and a reference implementation would benefit from getting rid of @style, which is my argument there)
SyndaceI don't even think in a open source environment you have to care about stupid people. If you stumple across an implementation that allows for XSS you open an issue and point them to the reference sanitizer and everything is fine.
SyndaceIf someone writes new client code for fun that only he and his friends use then whatever
jonaswI prefer to try to prevent security issues before they happen.
Kevjonasw: I think you're agreeing with my argument, and disagreeing with pep's, that we should be making this easy for people who want to do the right thing.
jonaswyes, we definitely agree on that.
jonaswI don’t agree that inventing our own markup will help with that.
pep.Kev, but the issue at hand is people not doing things right, am I wrong?
jonaswit will just open another can of worms, be it security issues or interop issues.
jonasw(or probably a mix of both)
Alexserver plugins could strip out java script and other malicious tags
jonaswAlex, that has been proposed, and Zash even drafted an implementation of that for prosody.
jonaswI don’t feel that clients should rely on that in any way.
KevAlex: I'm not convinced by that argument, really.
jonasw(but it’d be an interesting tool to detect malicious parties)
AlexI have done this before, the XHTML XEP lists the allowed tags, any other tag I ignored in the parser
pep.Kev, if it's to make things more straightforward for people who follow the spec, I'm all in, and I agree with jonasw. But that's not how I read SamWhited's email.
jonaswI also firmly believe that obsoleting XHTML-IM without a replacement which has deployment will achieve nothing, except closing a door for us on fixing things there. See Private XML support.
Alexjonasw: agreed on that
Alexwe really need modern markup to compete with Slack and others these days
jonaswI will implement XHTML-IM in my client over the next weeks, no matter the XEP status.
jonaswsimply to gain interoperability
Alexothereise its a step backwards
jonaswand I assume that others will do the same
jonaswand if they don’t then we’ll end up with a bunch of incompatible tacked-on markups
jonaswe.g. clients (not people!) putting markdown in <body/>
jonasw(we already have that to some extent with clients interpreting *foo* and /bar/ and such)
pep.I see *foo*, _bar_ etc. as historical, coming from IRC. Not sure where IRC got that first though
jonasweven though there are clients which interpret ">", which I haven’t seen used much in IRC except in the last few years.
pep.Right, they're all implementing their own markup, using <body>, which is meh
pep.I often rage against Conversations translating ">" at the beginning of a message. It breaks stuff like "><", and messages with a single ">" don't make sense anymore (got this case on prosody@ the other day)
pep.I often rage against Conversations converting ">" at the beginning of a message. It breaks stuff like "><", and messages with a single ">" don't make sense anymore (got this case on prosody@ the other day)
Holger>< works though.
> we really need modern markup to compete with Slack and others these days
Do we really want to compete? I just doubt that this is the goal of XSF
pep.Ah it works now indeed. I haven'T checked in a while
dwdzinid, I think we should be competitive with things like Slack, yes.
Alexzinid: I see many people switching to Slack these days. There is no reason why you can do the same with XMPP. But we lack of modern client and features these days
pep.Alex, I think we have most tools to start implementing. I would rather wait and see if UAs ask for more
pep.Maybe push them to implement such things if you really want
ziniddwd: only like Slack? really, what userbase do XSF target? because from what I see currently it's producing specs for nerds (like e2ee)
dwdzinid, Well, not only like Slack. And I don't entirely disagree with your assertions there.
dwdzinid, ALthough not all nerds focus on the threat model that OMEMO does.
pep.zinid, I think most nerds have their own servers, and e2ee is possibly of less important there
pep.But I don't disagree with your assertions either :)
zinidI just think maybe it's better to define the target userbase?
zinidI mean if this is nerds, then I'm fine and we should go in this direction
zinidI don't think we can produce a protocol for every person on the planet
dwdzinid, Scalability would be an issue, for sure.
Alexthe strenght of XMPP was the we always covered a huge variety of use cases an users
pep.zinid, protocol maybe, but for the end-users I don't think the protocol really matters anyway.
ziniddwd: yes, but if we try to resolve scalability issues we will end up with SIP :)
pep.Or rather, for *most end-users, non-tech that is
zinidthere is sip p2p rfc already
dwdzinid, Oh, I do hope not.
ziniddwd: well, if you strip shit from sip, it's usable and can be used as building block
Zashzinid: the one that also works for xmpp?
zinidhehe, ok, I didn't want to go into SIP vs XMPP debate :)
dwdzinid, That's not a debate, it's a bloodbath. ;-)
dwdzinid, But anyway, I think we have a set of existing target communities, and I agree that defining these would be of benefit to us as protocol designers - however you implied there can only be one, which I think I take issue with.
Alexthe target communities also changed over the last 10+ years. I like this discussion of defining it
ziniddwd: well, their can be several, but I don't think "regular user" class intersects a lot with "nerds" class (intersection is a very basic functionality)
zinidI mean, take a look:
1) a girl spending most of the time in instagram and so on
2) a nerd who lives with console
zinidhow can we target both?
dwdzinid, I think the amount of intersection between the sets is actually something we'd need to find out. I suspect it's not nearly as small as you might think.
zinidmaybe, would be great to understand though at least for making the priorities
Alexextensions, give teh nerd a console client without XHTML, and the girl a client where she can like messages, have hundreds of emoticons and share images and glyphys
zinidAlex: sounds great in theory, but does it work in practice?
ZashJack of all trades
Alexyes it does, I am doing XMPP stuff for close to 15 years now, and have customers which do all that with XMPP with great sucess
ZashIt does show that a bunch of us are more or less doing stuff for ourselves. Nothing wrong with that.
HolgerZash: Nothing wrong with targetting ourselves if we clarify that. Then I can give up trying to sell XMPP to others and then having to explain why stuff breaks.
ZashOptimally we'd get instagramming teens into software and standards development
zinidnothing wrong of course, the problem I have is that we constantly speak about how to fight with Slack without even defining the target
ZashAnd if the target is everything then it's a lot of work
zinidyes, 300+ XEPs for sure :)
zinidyet another 300 I mean :)
AlexI think case studies and small tutorials on our webpages would help
ZashOnly? Gotta break that initial zero ;)
HolgerI think it's a vicious circle. XMPP's niche is mostly nerds, so there's few other users complaining about our stuff, so we concentrate on the nerd stuff because that's more fun to implement.
ZashHolger: Typical in most FOSS circles
Alexyou wanna build something liek Slack, hey this iw what you need, extensions A+B+C+D
you wanna build a military grade client, then this is what you need, extensions X+Y+Z
ZashAlex: Sounds good
HolgerZash: Yes but elsewhere there target audience is more obvious.
dwdAlex, In my experience, the Military and similar markets actually want something Slack-like anyway.
HolgerZash: If you build a tiling window manager, you won't break stuff for non-geeks.
Alexyou wanna build something like whats app do that
you wanna build machine 2 amchien communications do this
you need build a system for realtime stick exchange do this
you wanna build the next Uber (geoLoc) to that
dwdAlex, I mean, they want labelling, sure, but don't think they don't care about emoji support.
HolgerAnd a large part of FOSS projects is building server/infrastructure stuff ...
Ge0rGAlex: that sounds like a compliance suite.
zinidAlex: ok, let me rephrase maybe: who will write the XEPs for Slack audience? Hell, does anyone even know what Slack audience want? :)
AlexI suck at typing today, so need a client which supports XEP-0308 :-)
dwdzinid, I've used Slack, and I can tell you that Slack themselves don't have a clue.
Ge0rGthere is a commercial market for corporate (or government) slack-like things self-hosted with data retention, archival, LDAP integration and other enterprisey requirements.
zinidwe can start doing this, but we will eventually end up working on JET and OMEMO :D
ziniddwd: yes, maybe :)
Ge0rGzinid: we don't need to target the nerd audience. They will come to us voluntarily.
zinidGe0rG: but we do this
zinidJET and OMEMO for whom?
zinida girl from instagram never heard about e2e
AlexGe0rG: yes, we had a compliance suite before at the XSF for servers and client, but somehow died
Ge0rGIMO, there are two target groups we should focus on, five years ago:
- Slack-like on-premise / hosted services with a good web and mobile UX
- slightly nerdy normal users (see Conversations' success)
Ge0rGAlex: the current one is defining "mobile" and "IM" use cases. I'm pretty sure that "Slack" would make a good "Multimedia Chat" one or somesuch
Ge0rGAlex: actually, there is not much specification missing to pull off a Slack-like thing, it's only a lack of dev power to make a proper web client and a one-button easy-deployment server.
Ge0rGokay, web client _and_ mobile client
Ge0rGGive me a dozen competent developers, a year and a time machine so I can start in 2012, and I can pull it off.
dwdGe0rG, Actually, Openfire has the latter, for sure. You can install webchat clients just by push-button after Openfire itself is installed.
Ge0rGdwd: will Openfire come with AD/LDAP integration and all the Enterprisey checklist features?
dwdGe0rG, It has had AD integration for a decade or something.
Ge0rGAnd will it scale to (tens) thousands of users?
dwdGe0rG, It doesn't scale as well as some servers. Chokes around 20k users or so, I think, until you do clustering.
edhelaswhat about ejabberd ?
dwdedhelas, I think that has LDAP integration, I don't know about Active Directory SSO (ie, GSSAPI).
Ge0rGdwd: is it possible to buy an Openfire appliance / virtual machine with around-the-clock tech support?
Ge0rGdwd: but we are getting off-topic here.
Ge0rGThere is a German startup, selling enterprise mobile messaging <https://www.teamwire.eu/> - they provide cloud-hosted and on-premise solutions, and ask something like 3€/month/user for a WhatsApp-like experience. And they don't even have federation.
Alexzinid: there is hosted.im if you need that
Ge0rGBut they have a sustainable business for something like five years now.
dwdGe0rG, Sure. We do have a dearth of clients running on Apple-favloured systems.
Ge0rGThe problem is: nobody will buy an XMPP-based IM solution if the UI isn't Slack-like
zinidAlex: me? :) now, thanks, I have my own domain
Ge0rGdwd: business clients running openfire on Apple? Or XMPP clients running on macOS?
edhelasGe0rG what is "slack like" for you ?
ziniddamn, last time I checked slack I didn't understand what its buzz is about
Ge0rGedhelas: easy to use private messages, channels, markup, some integration with websites / web services
dwdzinid, It introduces millenials to IRC.
Ge0rGedhelas: synchronization across all devices. proper push notifications
dwdzinid, That is, pretty much, it. It's IRC with emojis and working file sharing.
Ge0rGI actually like Slack. It's easy to use.
ziniddwd: then what is a problem to build slack-like using existing XEPs? am I missing something?
Ge0rGzinid: lack of developers
dwdzinid, No, we're not missing much to build something Slack-like, but federating and with security.
edhelasI already have a couple of Movim users that deployed it in their business on top of a XMPP server
edhelasyou add some bots to integrate with github/jira and you're good no ?
dwdI do think that our absolute neutrality does handicap us from showcasing good clients, which in turn reduces the appeal of the platform as a whole.
zinidGe0rG: that's true
ziniddwd: so the problem is nobody wants to use XMPP for their apps, ok, we go in circles now :)
dwdzinid, Well, sorta. I think people look at the first client or two they see and assume that the UX they encounter is due to restrictions in XMPP.
AlexGitHubs Gitter is another example, also getting quite popular these days
> I do think that our absolute neutrality does handicap us from showcasing good clients, which in turn reduces the appeal of the platform as a whole.
Very sad, and very true.
edhelasI was thinking of creating a page that showcase those "good clients" actually
edhelasif it's a project that is not officially pushed by the XSF, it could work no ?
zinidwhat good clients? I still use Tkabber...
dwdzinid, Nerd. :-P
edhelaszinid Dino looks very promissing
ziniddwd: but what to use? swift lacks of features, dino is crashing, gajim is buggy (tracebacks are the friends)
dwdzinid, I wonder if - and I know this sounds silly - the XSF should have an awards thing for Best XMPP Client for XYZ Platform, etc? Would that help highlight and motivate?
ziniddwd: no, Durov puts 1M$/month in telegram for instance, I think other major IM players do the same
zinidso if the price will be 1M$ then maybe :)
Ge0rGYou don't need that much for developing a client or two
zinidGe0rG: yes, I understand the most of the money goes into adv, but still there is a lot of guys working on a client
jonaswdwd, awards are not so cool
jonaswthe issue with awards is that only one wins
jonaswor let me put it this way: it would require well thought out terms for people to participate
jonaswalso I’m afraid it might just re-enforce already well-funded (in which way ever) clients
Ge0rGdwd: we really need to revive the Jabber Software Foundation
Alexputting good software projects under an umbrella like Apache does could help
AlexGe0rG: same idea :-)
pep.edhelas, arguably there is no good clients, it's all shapes of bad :)
AlexPandion was this client for windows in the early days of Jabber/XMPP
zinidGe0rG: what will JSF do?
pep.marketing? promotion of clients? and the XSF do standards? (wild guess)
zinidmarketing = $$$
Alex$$$ could be raised easily, when there is good software
ZashAlex: wrong, when there is good marketing
KevOK. I have a strong position on XSF Neutrality, because I don't see a way to do things fairly. So perhaps it'd help to persuade people like me that the XSF can do this stuff if someone came up with a concrete proposal of how the XSF could do recommended clients/servers etc., and do it fairly.
zinidAlex: tons of Conversations user don't even pay for it
Alexzinid: its not easy, but doable
SamWhitedCatching up… fwiw, I don't have a "position" that we should do markdown that likes like plain text in <body>. I have a position that we should deprecate XHTML-IM and then we should take the time top write a good replacement considering the pros and cons of both approaches. Tentatively I suspect that doing it in the body makes sense, but I'm not pushing for that.
SamWhitedSimilarly, this started because I've tried an XHTML-IM impl that was broken … again. So it's hardly a strawman for "other xss'". If I thought other xss' were caused by a poor spec, I would probably try to obsolete that too.
SamWhitedAlex: we have a council vote to advance them next week! https://xmpp.org/extensions/xep-0387.html
SamWhitedOr already did and it's waiting on list votes, I forget.
jonaswSamWhited, good luck trying to obsolete .innerHTML and others :)
KevI think Council just approved an LC on 387, and so vote to advance should be the week after, shouldn't it?
SamWhitedjonasw: exactly, that's why I'm focusing on xhtml-im instead :)
jonaswI think the LC needs to be announced properly on the ML first
jonaswsome editor should do that *cough*
SamWhitedAh yah, that sounds tight
jonaswI can see if I can get around to do that this weekend
KevOh, right. The two weeks don't start until that's announced.
ZashSamWhited: How is "The Web" handling that? Because it's not just XMPP clients having that problem, right?
SamWhitedZash: by not sending raw html around and trusting that developers can implement a white list properly or will do so at all.
jonaswZash, have you looked at the web? it’s injections everywhere.
ZashI looked at the web recently. It said "It is inherently impossible to make a secure web thing."
SamWhitedI assumed we were talking about transmitting formatting over instant messages, but yes, generally it's injections everywhere. I just still fail to see what other problems that aren't xhtml-im have to do with xhtml-im.
ZashI'm thinking, what are other standards orgs or such doing about equivalent injection issues?
ZashI'm assuming here that there's some overlap between xhtml-im and web apps
SamWhitedAh, yah… good idea, someone else must do this.
ZashMore like "has someone done something already that we can steal?"
pep.SamWhited, for me deprecating xhtml-im is just delaying the same kind of issues with another more or less similar XEP. You're complaining about a range of UAs that are either 1. not following the XEP, 2. Not doing their job correctly. I don't think you can't fix 1., and if you see 2. please report it. If you do and and still get no reply, then I don't think we can't do anything either.
pep.Replacing XHTML-IM won't fix this
ZashThe XMPP thing to do is to turn it into XML on the wire. But people will fail at that, whatever we do.
SamWhitedpep.: it won't fix that, you're right. I am not arguing that a new thing will make developers never have any security issues ever again.
SamWhitedI am arguing that I have never seen an xhtmlim impl that worked the first time and that it is especially easy to get wrong. We *can* make something where the default naive implementation does not commonly lead to badness.
SamWhitedThis is the best you can do in any security issue.
mathieuiSamWhited, would bringing up the poezio xhtmlim impl be trollong?
mathieuiSamWhited, would bringing up the poezio xhtmlim impl be trolling?
pep.goffi also has an implementation iirc
jonaswdoesn’t movim have XHTML-IM support?
SamWhitedmathieui: sorry, I didn't include it here because phone, but yes, on the emails I think I specifically said "web based"
jonaswI thought they talked about that (at least partial support, I think they strip @style, but maybe I’m confusing things)
mathieuithat makes sense, np
pep.SamWhited, libervia is a web client.
mathieuiand everything that brings in a webview in a client application too
pep.jonasw, no, edhelas doesn't want xhtml-im for some reason
SamWhitedGetting of the bus… may be slow to respond.
pep.SamWhited, also please don't merge markup in <body> :(
SamWhitedOff… I hate phones.
SamWhitedI give up. Why does everyone think I want to push an alternative format? I will probably volunteer to write one if xhtml-im gets deprecated, but I am not pushing for markdown or whatever in body. I haven't done any research yet.
jonaswI personally do not think that, but we are going to need an alternative, and the alternatives all look bad.
Ge0rGWe won't improve our situation by deprecating XHTML-IM without an alternative, and I still don't believe we will be able to come up with an idiot-proof alternative.
KevAnd if we're not striving for idiot-proof, we should assess whether XHTML-IM can be suitably improved.
ZashIf you do, the universe will spawn a better idiot.
ZashHaving an official safe JS reference implementation does sound somewhat promising tho.
ZashAnd and a bigger better security considerations section
KevAnd blacklisting style.
ZashWhat about the teends that crave colorful messages?
ZashBunch of predefined classes?
KevIf we allow style, I'm not at all sure how even a reasonably diligent implementor avoids issues.
ZashInstead of <span style="color:red">, there would be <span class="fg-red">
jonaswclasses seem like not a good idea either
jonaswit needs sanitization at least
jonaswto avoid that some attacker abusse your clients classes
jonaswbut they’re much easier to sanitise than @style
jonasw(split by " ", make a set, check that only things starting with xhtml-im- are in there or so)
jonaswalternatively (but Sam will shoot me for that), xhtml-im:color="..."
jonasw(hm, still needs sanitisation to prevent injection attacks)
jonaswso using @class and defining a set of classes seems most safe for now
Ge0rGwe should only allow one value for color, which is an integer between `0` and `359` on the XEP-0392 color wheel.
SamWhitedAmusingly, I just sent a short reply to one of Kev's messages that included the example: *<script>alert(123)<script;>* and FastMail rendered that as bold.
ZashWhy not hold a public poll where people can suggest names for them
ZashSamWhited: /nick <script>alert("everyhing is broken");</script>
ZashEspecially my typing
Ge0rGwhat happens if you add unquoted HTML to the <body> text?
ZashWhat happens when you encrypt unquoted HTML in OTR?
jonaswGe0rG, you mean, like, <body><b>foo</b></body>?
ZashInteresting how <b> isn't in xhtml-im
pep.Zash, jonasw, reference implementation, and tests also maybe
pep.Although I'd like to have more than just JS, because it's not just a web issue and it's not just an xhtml-im issue
pep.Basically we need a reference client? :)
mathieui16:20:06 Zash> What happens when you encrypt unquoted HTML in OTR? → let’s not talk about OTR
Zashjonasw: IIRC <b> was added back into html because nobody cares about semantics
ZashAnd nobody is going to fix all the web that uses it
jonaswGe0rG: depends; web clients might joyfully let themselves being XSSed
mathieuibut they do even without xhtml-im
SamWhited> but they do even without xhtml-im
I still can't figure out what the argument is there. Other XSS's aren't related to XHTML-IM… what point are people trying to make when they say that "there are other XSS's too"? If it's obvious to everyone else I apologize, but I'm honestly asking. There is some implied ending to that statement that I don't understand.
pep.That XHTML-IM is not the issue, there's a deeper issue
SamWhitedYes, XSS is a deeper issue, but you can't fix XSS on the web. You can not recommend a spec that actively encourages them though.
SamWhitedIs that what people are suggesting though? XSS is the underlying issue so there's no point in trying to prevent a subset of them?
pep.You can try and improve XEPs all you want, that won't prevent clients from having bugs. I agree XHTML-IM could be improved to try and make people aware of it, but in the end it's mostly reporting bugs to clients
SamWhitedYou won't prevent it, but you can certainly make it less likely that those bugs are catastrophic security issues.
pep.Do you have specific examples of these issues btw? And I suppose you also reported them
SamWhitedHipChat for instance tried to do things right, but had a handful of bugs in their whitelisting (and I'm sure you could easily find more).
SamWhitedWith XHTML-IM almost *any* simple bug leads to a security issue. We can absolutely write a spec where not every single simple logic issue allows a script to be executed.
dwdSamWhited, Quite. The problem with XHTML-IM is that the obvious implementation is the most dangerous one, and insatead you more or less have to add an HTML protocol break in to make it safe.
dwdSamWhited, In fact, edhelas suggested doing so in the server (and I thought of doing so in Metre), but the problem there is that XHTML-IM is often silently extended anyway.
SamWhiteddwd: Indeed. HipChat in theory does things right on the client (whitelist of elements and attributes, don't stick things straight into the DOM), *and* it has server side filtering. Despite that I've found at least two injections in that implementation (both of which are fixed now).
jonaswwhat’s an "HTML protocol break"?
Zashjonasw: <bold> mebbe
Zashdwd: And is that even possible while still being XML?
dwdZash, No, you wouldn't do it with XML, you'd do it with something else. Real protocol breaks would break up all the XMPP traffic into an intermediate form (JSON, maybe) and ship it across a wire to the other side, which then puts it back together. Obviously that's a little overkill for XHTML-IM.
jonaswYou’re saying we can’t use XML for markup, at all.
Zashjonasw: Yes. But we should, but we can't. Nice things and unavailability
pep.Who ever thought web clients would be a good thing :x
SamWhitedWhat is the advantage to doing an XML based markup? I'm not convinced that we can't do it, but I do think a non-XML based thing (which I'm assuming means "in the <body>") has a few nice advantages (single source of truth, better fallback behavior, etc.)
dwdjonasw, No, I'm not. Just that you can't copy it direct to output without very havey mangling.
dwdjonasw, And the level of safety we're talking about in the mangling negates the advantage of it being XML in the first place, IMO.
Zashdwd: people will find a way to lazily search and replace that lets trough evil stuff
dwdjonasw, I mean, two XMPP servers talking across a heavily secured boundary have all their traffic mangled out, and then back into, XML. It's not the XML itself that's the problem, it's that you cannot do a direct copy and guarantee safety.
jonaswSamWhited, whatever we do, not in <body/>.
waqasI agree with everything SamWhited said. Not a single client that I reviewed has gotten xhtml-im secure.
SamWhitedjonasw: what is the advantage of not doing it in the <body/>
jonaswdwd, so, I like your HTML protocol break argument. Indeed, based on that argumentation, I think a something-not-XML-based-but-still-structural representation of the semantics of XHTML-IM would make sense.
SamWhitedthanks waqas; you're running for council next term right?
jonaswSamWhited, doing it in the body ties us to poor plaintext-ish markups like Markdown, Creole or reStructuredText. Those are not extensible, the implementations often vary or are poor in other ways and such.
waqasAs part of my research I'd written a library that implements XHTML-IM protcol break: https://github.com/zeen/xhtml-im.js — it makes an XML DOM to an HTML DOM with a strict whitelist of elements, attributes and possible attribute values.
ZashThe xep talks about rtf. Do that
waqasI need to npm-ify it, to make it easier to use and add docs.
jonaswwaqas, that sounds like the reference implementation I wanted to see in XEP-0071 which everybody can use.
SamWhitedOoh nifty, that's nice to have, thanks
ZashSo, let's pay for an audit?
SamWhitedactually I forgot, I think you sent this to me and challenged me to break it a while back and I never did.
SamWhitednever tried, I mean.
jonaswwaqas, except that I think it also gets the "replace invalid tags with their children thing" wrong
jonasw(i.e. doesn’t implement it)
Zashjonasw: let's kill that plz
waqasIt discards anything invalid first and foremost
waqasi.e., it avoids all positives, but may have false-positives for certain attribute values
waqasAnd the unknown element case
jonaswZash, I think it’s a nice-to-have for extensibility.
jonasw(but I see how it’s hard to implement in anything which isn’t XSLT)
Zashjonasw: you can extend the <message>, it's probably safer
jonaswZash, how’d you extend <message/> with HTML-<video/> elements?
SamWhitedjonasw: why is a plain text protocol not extensible? If I want to add /italics/ later, what is stopping me?
SamWhitedfor that matter, why are "plaintext-ish markups" poor?
dwdSamWhited, It still amuses me that the word italics in your message is rendered in italics for me.
jonaswSamWhited, that clients which don’t know that /italics/ is italics won’t escape it
SamWhitedSee, there we go, it works already!
ZashLet's talk about /etc/foobar/
SamWhitedjonasw: Right, and we have a lovely fallback behavior, they still see the exact same message it just looks like someone added /emphasis/ in a different way
waqasExtensibility is a secondary concern, that we should be thoughtful about. Security is the primary concern. I'm not sure the 'include the children' approach is always correct in HTML.
jonaswso my /path/to/some/file will render italics on your new client supporting italics even though it shouldn’t
dwdZash, Not italics.
SamWhitedZash: Yah, I don't know about the specifics of using /, just an example.
jonaswSamWhited, it’ll work for anything
SamWhited*sigh* so pick a different character, it was an example.
dwdSo /this is italics/ but yet /etc/passwd is not.
jonaswSamWhited, I challenge you, which one which doesn’t look super odd when seen without support?
dwdI mean, right now, in Gajim, this is the actual case.
waqasdwd: Machine learning!
jonaswdwd, what if I want to have only a part of a word in italics?
SamWhitedjonasw: That doesn't look odd to me, _emphasis_, /emphasis/, and *emphasis* all look relatively nice.
SamWhitedwe're diving into specifics that don't matter though. The question is "why isn't it extensible" which you argued was a problem. Implementation details are not relavant at this point.
dwdSamWhited, And they all worked in Gajim. They show the markup, mind, as well as the effect. But they work.
jonaswSamWhited, but that’s exactly why it isn’t extensible.
SamWhiteddwd: Oh that's interesting, I've noticed a few web things (not IM) that keep showing the markup and the effect recently and liked it, I wanted to try it in an IM client. Didn't realize gajim already did it.
jonaswYou are re-defining things which previously were normal characters as meta-characters when extending it.
jonaswThat simply breaks, always.
jonaswanyways, I gotta go for now.
SamWhitedjonasw: ahh, I see. That's fair.
SamWhitedI still think it doesn't matter as far as deprecating XHTML-IM, mind. The security concern comes first like waqas said.
SamWhitedBut it's a fair argument if we're discussing alternatives.
Steve Killehas left
SamWhitedA fair argument that I would love to consider in parallel to deprecating XHTML-IM ;)
pep.So you're going to deprecate XHTML-IM, to then try and find a replacement, that is possibly just XHTML-IM 2.0.
ZashXHTML-IM 2.0, now 200% more security considerations
SamWhitedpep. yes. I certainly hope we don't come up with something that's just as bad, but we *know* XHTML-IM causes problems, it has been for years, so let's stop recommending new implementations of it. Keep in mind that people will still implement it for compatibility, and all existing implementations won't go away. It just means we don't advertise it as the way to do things.
jonaswSamWhited, thanks for seeing my point, that has driven me crazy ;-)
pep.So that doesn't find things at all, deprecating it. Fixing the XEP might be a better option?
pep.So that doesn't fix things at all, deprecating it. Fixing the XEP might be a better option?
SamWhitedpep. we can't fix it without a rewrite, the basic idea is fundamentally broken.
pep.If there is anything to fix
jonaswSamWhited, given the arguments from dwd about protocol breaks and such, I agree that we should do other things. at this point, I’m in favour of something like some simple JSON-based markup or so.
SamWhitedAt least, I don't think we can, if you have a proposal I'd love to be proven wrong.
jonaswbut I’m out of the discussion for now, I’ll write a list to standards@
SamWhitedjonasw: thanks, I look forward to reading it.
pep.SamWhited, no I don't see how what you want to fix can be fixed, and I don't know if we have to worry about this to then come up with something as bad
SamWhitedI don't think we'll come up with something as bad. I'm reasonably sure most of us understand the problem.
dwdZash, Incidentally, `/etc/passwd` works in most IM-based Markdown variants as an escape pattern that also switches to monospaced "code" layout. But not in Gajim.
pep.dwd, that doesn't change jonasw's point
pep.And I agree with him
pep.If you want to do something else please don't use <body>
Zashdwd: Markdown is a html superset. You are doomed.
dwdpep., Oh, I'm somewhat open either way, there.
SamWhitedI wonder if there's a middle ground, body and a hint about the type of markdown being supported. <body>this is *bold*!</body><formatting version="0.2"/>
dwdZash, I'm not proposing using *full* markdown. That would indeed be insane - nobody needs headers and things in IM messages.
pep.SamWhited, so you'd include all different versions?
SamWhitedpep. you'd just give a hint about what the version of the spec was, then the client could implement all or nothing depending on if it understands the version attribute. I'm not sure this is a good idea mind, just spit balling.
Zashdwd: markdown libs are going to pass through html by default
ZashSamWhited: I have this vague feeling that I've seen that proposal before
SamWhitedLike dwd, I'm rather open to either thing though. I do have a vague sense that doing it in body fixes a lot of little problems, but it has the downside that jonasw pointed out. I'm not sure which is the best tradeoff.
dwdZash, Well, I'm not actually sure most of them do, anymore. And in any case, there are loads of cut-down ones. But I'm merely hinting at a direction rather than wanting to compare it with XHTML-IM. It is, as SamWhited says, more a matter of agreeing we need to deprecate XHTML-IM and wondering what the functionality might be replaced with.
pep.SamWhited, right. I don't think this fixes the issue. If the client doesn't handle *foo*, it'll leave the markup here, which is meh, and for each new version you'll just be defining more and more new meta-characters that weren't before. I'm not sure that addresses our issue
pep.(As you just said ^)
SamWhitedI'm pretty okay with leaving the markup there; it may not be ideal, but it's a better fallback behavior than XHTML-IM right now (where the message just doesn't show up if you don't also include a plaintext body, eg. most HipChat extensions)
pep.Well then HipChat is not compliant?
pep.They should follow the XEP
dwdpep., Do you seriously think that anyone using any Internet-based communications medium for the past two decades would be surprised to see *bold* things expressed as such?
pep.dwd, not us no, but I'm pretty sure you don't want to target only us nerds with this
SamWhitedI'm pretty sure that everybody would understand *emphasis*.
dwdpep., No, not just us. Any Twitter user, for example. Probably any Facebook user too.
pep.Also, you will never be able to have any breaking change
ZashShould it be about styling or semantics ? ?
pep.I'd prefer it to be semantics
pep.But I suppose Slack or Facebook users don't care
SamWhitedThat's an interesting distinction. I think a replacement should just deal with text styling. If you want an image you have sims or something, and the client can decide if and how it wants to display that (maybe with an information XEP about displaying media in clients for guidance).
SamWhitedand the XML-based SIMS or OOB or whatever gives the client the semantic meaning of whatever the non-text resource is.
ZashSomeone mentioned push and publish-options being invalid?
Zashhttps://xmpp.org/extensions/xep-0223.html#example-1 has the same issue?
zinidZash: yes, #persist_items is not registered within XEP-0060
Zashdwd: Please don't use JSON as a markup language. Altho maybe that's the reason it's the only choice.
Steve Killehas left
Steve Killehas joined
dwdZash, I wasn't going to suggest JSON at all.
Steve Killehas left
Steve Killehas joined
ZashForget about XHTML-IM and markdown, let's do this https://www.oasis-open.org/committees/download.php/60/HM.Primary-Base-Spec-1.0.html
Steve Killehas left
SamWhitedEmbedded SVG, but the version of the spec that didn't end up making it through where it had functionality to open sockets.
Steve Killehas joined
Steve Killehas left
Steve Killehas joined
Steve Killehas left
SyndaceI really don't get why all examples of XEP-0030: Service Discovery error responses contain the query element mirrored from the request. To me it is as confusing as it looks useless. If a client does not support disco at all it won't know that it has to mirror the query element anyway, so why is it mirrored in all examples and do I really have to mirror it in my implementation? I did not find a MUST or REQUIRED about this in the specification. Do people mirror it? Do people rely on it to exist in error stanzas?
ZashSyndace: I believe that's technically allowed by the specifications, but rarely used.
zinidIt's useful for debugging sometimes
SyndaceZash: So, I can ignore it as being historical? Is there any reason it might be helpfull that I'm missing?
Syndacezinid, okay I can see that, it's easier than matching ids of outgoing and incoming iqs.
zinidI mean when you dump xml on server: you see what exactly your server is replying to with error
zinidSyndace: but there is no requirement in the RFC