XSF Discussion - 2023-01-18

An unavailable presence from the bare jid should mark all resources as unavailable. Is this also true for error presence?

Probably not since I can't find anything explicit in the rfc about that?

good question

It Depends?

from bare JID, probably, from full JID, no idea

Also consider the `<error by=>`

I noticed that the current compliance suites lists some experimental XEPs. I don't think this should be possible with our current official position on what XEP status means

Daniel, I don't know if I'm right, but my instinct says "yes" - I can't think of any cases where "error" isn't practically equivalent to "unavailable"

How do you send from a BareJID? The BareJID corresponds to the account (on the broker), which sends no presence by itself, while the Full JID corresponds to the connection the client has with the broker. The clients sends unavailable as a graceful shutdown.

it certainly doesn't mean "available"

Peter Waher, the client can't send from the bare JID, but their server can

Peter Waher: e.g. bounced probes

aha

a “bounced probe”, what does that mean?

Meaning the server has no information?

yes error == unavailable probably. but I was confused because the rfc has explict wording for unavailable

It could happen to probes if there are s2s failures or policies preventing you sending to that user, etc.

but doesn’t seem to have it for error

instinct wise (or logically) i would agree though

it probably rarely happens

i mean error presence from bare are frequent (failed probes) but you hadn’t receive available presences then before the error one

A client shouldn't receive a presence error from a bare JID most of the time, but sub request/approvals could cause it, and might not mean unavailable, it might just be a subsecond S2S glitch or whatever.

you get them if you enter non existent stuff into your roster, no?

larma, yes, but here we are, I gues.

> you get them if you enter non existent stuff into your roster, no? I'd hope you don't do that most of the time, but yes, probably :)

3. Else, if the contact has no available resources, then the server SHOULD reply to the presence probe by sending to the user a presence stanza of type “unavailable” (although sending Saint-Andre Standards Track Pₐgₑ ₅₃ RFC 6121 XMPP IM March 2011 unavailable presence here is preferable because it results in a deterministic answer to the probe, it is not mandatory because it can greatly increase the number of presence notifications generated by the contact's server). Here the 'from' address is the bare JID because there is no available resource associated with the contact.

Kev, while I agree it might not mean that the JID is actually offline, from the perspective of an observer receiving the error, I think they ought to be treated pretty much the same

I think you can argue either way convincingly.

Some clients (Gajim, for example) have/had a dedicated error status, which was visible in the UI

I found that useful, personally

jonas’, well, I'd also argue those are controversial. Push notifications and Jingle Message Initiation. JMI was already on 2021 and since received major changes and still has major changes pending in PRs.

I haven't closely been following JMI, but drastic breaking changes at this stage concern me a lot

Hearing that Monal can only call Monal is sad

tbf when we added JMI to the compliance suite it seemed more stable than it is now

but that's an argument to remove it this year i guess

Maybe it was necessary, but I'd like to hear that there was absolutely no way to maintain backwards compatibility with everyone who already implements it

For the compliance thing, I think you end up with three choices: 1) Don't reflect reality because XEPs are Experimental that maybe shouldn't be 2) Reflect reality and have Experimental XEPs in the suite 3) Require that the author of the year's compliance XEPs also pushes through everything getting out of Experimental I can see why (3) doesn't garner a lot of interest from the relevant people :)

making a general policy to not have experimental xeps in the compliance suite is sadly not realistic imho

Separate compliance suite into a Vision and a Current State document?

Of course, XEP statuses being appropriate is a good goal, but when it's not the case and you want a compliance suite, that's where you end up with.

but on a case by case basis we might choose not to put them there

or remove them again

Well, it would have prevented this situation if we'd advanced JMI before adding it to the compliance suite

Council approval or a new XEP would have been required instead

or, some other solution to whatever became necessary to update in the new revisions

> Maybe it was necessary, but I'd like to hear that there was absolutely no way to maintain backwards compatibility with everyone who already implements it the current latest NS bump was not necessary at all imho

the upcoming group stuff are probably a different issue tho

Zash, we already have a "keep an eye on …" section

Just practically speaking, we had to add the JMI namespace to Prosody's CSI as a prioritized payload

There is about to be a Debian freeze with this namespace in, and it will be in many Prosody installations for years to come

Pretty sure I added the new JMI namespace ages ago

Precisely to avoid that scenario

Is there only one new one? or is another coming?

Are there more than 2?

there is :0 which is what most clients use

there is :1 which is what monal uses

https://hg.prosody.im/trunk/file/0.12.2/plugins/mod_csi_simple.lua#l72

and then there may or may not be another one that adds group logic to it

Ha, year ago! https://hg.prosody.im/trunk/rev/1c47162dd965

> Council approval or a new XEP would have been required instead For the large changes still pending in PR proposed by me, I originally proposed to create a new XEP for that but Council decided it should go into existing JMI instead ↺

Combining the two policies "new stuff should be merged into old stuff if it fits the topic" and "old stuff should not have large changes" is inherently problematic. We need to decide for one. I personally don't care which one, but I see how this is blocking progress in XMPP.

I don't see right now that it's blocking any progress - but I agree we should have some understanding of what our preferences are

JMI was a widely implemented and deployed XEP that has been updated with breaking changes, and that's not ideal

I don't know what the solution is, since I haven't even studied the changes that were made in this case, but I feel like "re-use existing XEPs that cover a topic" was perhaps too strongly applied here

argubly, one can perfectly run JMI in dual compatibility mode

just like many clients support multiple versions of MAM

I don't know if thilo.molitor hasn't done that in Monal because it's not feasible, or just because he only wants to support the latest version (but I'm curious)

At the end of the day I don't really care about the namespace or XEP number used, but about fragmenting stuff unnecessarily

and a fragmenting change is being introduced here, with Monal incompatible with other calling-capable clients (of which we now have a good number)

If people start updating to the new version only, it's only going to get worse

If everyone agrees to support both in parallel, everything is fine

(for a transition period, at least)

I agree that rejecting call invites was a mistake

I think the XEP number/namespace only comes into the issue because people might look at the latest XEP version and think that's all they need to or should be implementing

> At the end of the day I don't really care about the namespace or XEP number used, but about fragmenting stuff unnecessarily In practice, does it make a diffference if a breaking change is in an existing XEP or a new one? You fragment either way.

If JMI had advanced to Draft, we'd rejected breaking changes and put them in a new XEP, the transition would be easier from that perspective

It's as easy to put a note into an existing XEP saying "Version X was widely deployed as of 2009, you might want to support that too" as it is into a new XEP saying "XEP-X was ..."

The compliance suite would continue to recommend JMI (with a note that it was being replaced) and the new one in "Advanced" or something

Kev, sure, but then the compliance suites should probably clarify the version number(s) people are expected to implement :)

Revert https://xmpp.org/extensions/xep-0353.html#revision-history-v0.4.0 and retroactively accept call invites XEP?

In most cases I don't think that's needed, but in this one I don't think it would be stupid.

The suite saying "While we expect deployment of X.Y to pick up, currently X.Y-1 is more widely deployed, you probably want to support both" seems helpful to me. ✏

regarding transition period: Dino already uses the "next" version of JMI (the one in my PR) in production, but will only use it outgoing for multi-party calls that are unsupported by old/current JMI (but we still accept the next version incoming for 1:1-calls for future compatibility)

I don't much care what solution we go with, as long as we don't end up with a repeat of what happened with file transfers 10-15 years ago

The roll-out of calls to the ecosystem has been pretty successful so far

Which for me comes as a surprise considering how broken everything is 😀

larma, I suspect everything is horribly broken beyond comprehension if you look closely enough

using 'Call invites' for multi party calls and 'jmi:0' for 1:1 would have been a very sane solution imho ✏

larma, so, you've probably looked closely enough 😀

Zash, to be precise, I don't consider 0166, 0167 or 0176 terribly broken (the old things for calls), just about everything else that was added for WebRTC compatibility.

Web = Bad confirmed! 😁️

larma, are those things I voted for while on Council because people well-versed in Jingle said it was good? 😕 ✏

Not sure, sometimes they are also just badly implemented because they XEPs literally come with no business rules

Tho the things that were just "this thing from SIP / SDP" must have had some rules from their RFCs?

Jingle is not SIP/SDP

Jingle has features that do not fully translate to SDP

Or at least there is no XEP on how they translate

The goal of Jingle was never a bug-for-bug translation of SIP/SDP into XMPP.

I feel like I saw something that was basically "here's how to turn this SDP property into XML for Jingle", but can't find now.

Maybe it was some PR to an existing XEP then

or https://xmpp.org/extensions/xep-0293.html ?

Some parts do translate.

XEP-0166 literally asks XEPs for application formats that build on top to specify if and how they translate to SDP ✏

Most of the Jingle stuff was done very early on (before webrtc) and since then a few things were added, mostly by me and Fippo. There are a number of challenges here. This stuff is hard and messy, you need to know a lot about how voice and video works in the real world (Fippo is a world-class mind on this stuff), even the experts disagree and get it wrong sometimes, SDP is incredibly ugly, even the vast majority of webrtc implementations just use what Google hands them in libwebrtc (for interop reasons), etc.

XEP-0338 has the funny line "Note: the identification-tags correspond to the <content/> 'name' attributes. These in turn map to the 'mid' attribute in SDP." - however, nowhere was ever specified that the content name attribute map to SDP mid attribute

stpeter, yes, I totally see that. That's why Jingle has largely degraded to "translate between XML and SDP and do whatever libwebrtc would do with such SDP"

nod

which is increasingly hard if you don't actually use libwebrtc or SDP

(speaking as a Dino dev here who did exactly that)

which is probably also the main reason why I'm the only one complaining 😀

larma: and you are probably the only one in the 'new generation' of jingle a/v devs who understands the issues.

Yes, very much so. As I said, even companies running huge voice and video service like Facebook and Zoom just use the Google library. When I was at Mozilla, I had conversations with a number of these companies and they didn’t like this state of affairs but even they weren’t motivated to put in the enormous amount of energy needed to fix it.

Everyone else just knows how to get libwebrtc working (which is complicated enough)

I guess what I'm saying is that I trust you when you say it's broken. I'm willing to fix it. But I don't have the indeepth knowledge required to fix it

And I guess that's a problem in itself. Because I'm far away from perfectly understanding everything so I'm struggling to even propose changes to what's out there (also many of it is already stable)

But I think we'd add great value to the open protocol ecosystem if we'd specify things beyond "do what libwebrtc does"

We'd probably already do great if we'd just *document* that behavior

Mid of month - Newsletter time! ✉️ Add your news since dec 2022: https://github.com/xsf/xmpp.org/milestone/3 https://yopad.eu/p/xmpp-newsletter-365days

I don’t want to discourage you, but fixing this situation is a big mountain to climb.

And I’m afraid I won’t be of too much help because I’m focused on non-tech pursuits these days.

Speaking of which, I’m going back to work on another project now, bbl.

Also "what WebRTC does" changes by release, and isn't documented

The only constant seems to be "write your software in C++ so you can have a steady stream of vulns'

> I don't know if thilo.molitor hasn't done that in Monal because it's not feasible, or just because he only wants to support the latest version (but I'm curious) Its not feasible, because :0 jmi messages won't get into mam, thus incoming calls could get lost (bad ux)...:1 solves this...and while I'm all in for temporary solutions I won't implement :0 if it becones permanent solution with nobody ever implementing :1...

I don't know exactly why I bumped the namespace back then, but certainly nobody told me "I won't implement :1 because of the ns bump" back then...and given that we did this sort of "support multiple namespaces of a protocol for some time" multiple times in the past (mam for example), I was under the impression the ns bump would not be a problem...

thilo.molitor, :0 messages do get into MAM, right? That's how current clients show things like missed calls

I see two ways forward now: revert the xep to :0 and do the minimal set of changes needed to make it reliable (mam, carbons)...maybe even add the tie breaking stuff and the finish element (better ux), if they don't require a namespace bump...OR implement :1...

Yes they get into MAM. Conversations shows missed calls and even calls that took place on other devices (while Conversations was offline)

The latter w/o duration because the lack of a finish element. But that element could have been added without a bump

MAM is an example of a spec where we intentionally avoided bumping the namespace after it had received wide adoption

It was still experimental, but we added new stuff behind a second disco feature

Yes it's a judgment call every time

we added that new stuff behind a second disco feature because it was possible to do so

Jingle ft didn't handle that great either

that is not always the case

It gets into mam? Jmi messages have no body and there is no exception to put that into mam regardles, no?

Just add a store hint

That's what the hint is for

the store hint is exactly one of the things I added in the :1 version...

yeah, you can add the store hint whenever you like, but if it isn't specified somewhere that becomes some sort of tribal knowledge with some clients adding it and some don't...

thilo.molitor, namespace bumps are always an additional effort and come with certain disadvantages (like fragmentation), they are just from the protocol perspective not an problem

You are allowed to add it everywhere. And adding a recommendation to jmi that a store hint is recommended does not require a bump

flow: yeah...maybe I was a bit naive back then...but then again I wished someone had explained to me that the namespace bump would likely prevent adoption...

yeah I know...this single thing does not require a bump...but I added some more useful things (tie breaking, usage of carbons instead of manually sending out stanzas to all known full jids etc)

well, once stuff is implemented and works, the incentive to add support for a new namespace, let allone support multiple namespaces, goes next to nothing

> well, once stuff is implemented and works, the incentive to add support for a new namespace, let allone support multiple namespaces, goes next to nothing It depends

I have supported sm:2 until a month ago lol

And I support three jingle ft

Ok, well, afaik :1 is not implemented beyond Monal. So maybe we can work out a new XEP version with the :0 namespace that includes additional stuff (that existing clients can incrementally add) and hopefully there isn't too much work for Monal beyond switching to :0

to be honest I don't like the current state of affairs....and it's fine by me if we just remove all my changes (even though I find them really useful) and just add a mam store hint...

But yes once a xep has some adoption regardless of state extra caution should be put into bumping a namespace

> It depends yeah, and nobody told me that jmi :1 is on the wrong side of this "it depends"

I think some changes are good and can easily be applied w/o the bump

> I think some changes are good and can easily be applied w/o the bump then let's sort it out what changes can be applied without bump :)

tmolitor, that's partly my fault, because I remember thinking it at the time, and I guess I didn't speak up

Or in fact I might even think all changes are good but with more care to the Syntax they could have been applied w/o bump ✏

is there a rendered version of v0.4

never mind...

found it

Like the proceed to accept renaming can not be done o/c

But I think we can quietly phase out accept

The old accept

okay...naming does not matter that much imho...

Yes exactly

what about using/relying on carbons instead of manually sending the accept message to all known full jids?

https://xmpp.org/extensions/attic/xep-0353-0.3.1.html#accept

That's what I meant by quietly phasing out the old accept

Even with jmi:0 clients can parse the carbon copied proceed

And Conversations does that

okay good...

same goes for all the other messages (reject etc.)

adding the finish message should not need a namespace bump either...

Like maybe leave a note about accept previously being a thing to get around lack of carbons so new implementors aren't totally confused about a seemingly unnecessary element

> adding the finish message should not need a namespace bump either... Yes

flow: I'd be glad if I would recall that...I guess it was the renaming of some of the elements??

fwiw, which change(s) exactly motiviated the jmi namespace bump? ✏

Motivated dunno. But the renaming of proceed to accept made it syntactically necessary

and that renaming was motivated by?

I whish there where a rendered diff somewhere…

Accept is unnecessary (that's correct). And accept is a cooler name for proceed

That's the entire motivation

Lol

Not on purpose I guess. But in the end that what it boils down to

ok, so now we have a nice example of when to not bump ;)

Yes. Absolutely not blaming anyone but that's probably a bit of lack in experience and nobody else catching it

or, rather, when to not rename an element just for the name

I'm not 100% sure the accept renaming was the only thing that warranted the bump...could be that it was only a side effect of the bump (e.g. allowing the renaming to happen)...

right, I did not want to put blame on anyone, just trying to understand as I did not follow JMI closely in the last months ✏

flow: *last years (the jmi update was done more than a year ago)...

Sure. It might be that I'm missing something

well, the good news is that if the rename was the only reason to bump, then it is easy to "revert" the bump

Last year is days in xsf time

and yes...it was my first xep and thus it may well be I've lacked some sort of experience...

We have reverse dog years here

uh right, nov 2021

I somehow read 2022 in the commit date

Daniel: :D

suggestion: I do a pull request undoing the renaming and namespace bump but keeping everything else and everyone checks if this is okay and nothing more required the namespace bump...

That's probably a good start. Not sure it will work out but that's how I'd start

(instead of starting with the old version)

in any case, we should probably consider bumping to :2 if good reasons to bump come up in the, potential far far, future

yes

okay...I'll do the PR then :)

I think having a html diff between 0.3.1 and 0.4 would be good to double check on the ns bump issue...but I'm not sure how to create that...

hmtldiff

is what I use

works good enough, even for sligthly larger xeps

If you have a quasi 0.4 implementation in Monal and lower the namespace you can do interop testing with Dino and Conversations and that'll be a real world and whether or not this works out

Daniel: good idea :)

Something like Conversations will (right now) obviously ignore finish but the only downside of that is that MAM is lacking the call duration

yes, exactly

but not even that: the xep requires both parties to send finish, but one finish is enough to get the timestamp...(this was to make sure we still have a timestamp in mam even if suddenly one of the devices did a reset/low battery etc.)

okay...I've changed monal to use the :0 namespace now...let's see what happens if I try to call Conversations :D

ah well...I've not yet implemented the jingle stuff (I'm just passing around sdp data using IQs for now)...but well...at least my conversations started ringing :D

A good discussion was had here tonight :)

one that moves xmpp forward, yes :)

0060 question

https://xmpp.org/extensions/xep-0060.html#publisher-publish-success-publisher

Is this implemented ? Is it the only way to get the publisher of an item on a node ?

That would close a big security issue for Movim as someone can easily publish an article as someone else

I'd like to have pubsub service to actually tell me who published the item

Holger do you know how is it handled in ejabberd ?

It's supported in Prosody, behind a config option

Ah !

Privacy implications

expose_publisher

That would be nice to add a per node configuration in 0060 directly

If someone wants to publish to a public node, they don't always want their JID leaked

But yeah, it would make sense as a per-node option, like MUC

Yeah but for social network and commenting actually it makes sense :)

Looks like I'll have to do a PR on 0060 😬

MattJ Zash do you know if the publisher are also returned when doing a disco#items on the node ,

?

There's no any place to put it there, is there?

It's returned in a "get items"

the XEP-0060 one

https://xmpp.org/extensions/xep-0060.html#publisher-publish-success-publisher

Ah yes indeed its there

"If configured to do so the service"

edhelas: Seems ejabberd includes the attribute if the `itemreply` config option is set to `publisher`.

So looks like its a service based configuration, not node based, so Prosody nailed it

edhelas: I *think* that's what "configured to do so" refers to.

Ah, let me check

So tonight I discovered something in 0060, after 10 years of working with it

https://media.tenor.com/j1rz6ft8tTsAAAAC/ratatouille-meme.gif

You'll still be discovering stuff in 2033 :)

Holger indeed with itemreply to publisher I do have the publisher in return \o/

So lets put that as a default and ensure that they are handled and displayed :)

It's not like I was aware of that, I had to check the code of course :-)

I'm sure 0060 is evolving based on ejabberd code

I see no other explanation

They say everytime you turn away from 0060 a new thing appears

Don't blink!

https://upload.nicolosus.chat:5281/file_share/sRSCEoXL804-5CnPkNbyQuDs/queen-doctor-who.gif