XSF Discussion - 2024-06-25

No. I will send him a message.

Hi there. I'm working on a SFU integration in XMPP. I'm doing an implementation with Galène (https://galene.org/), and I'm currently working on a component which will bridge Galène API and Jingle. I'll publish one or two protoXEP for missing parts, notably the way to join an A/V conference group. I plan to make it simple, with a specific category/type, and client joining the conference in a similar way as we do for one2one calls (i.e. by doing a jingle request, without message initiation as it is not relevant here). Does anybody have suggestion for that? One could think about using MUC as a base, but I think that it would complicate for nothing (we can still link a MUC room somewhere).

So basically I thinking about something dead simple: a room would be "room_name@conferences.example.org", and one join by doing a XEP-0167 call there. Ad-hoc commands could be used for configuration, or retrieving list of participants.

Definitely worth speaking with the Tigase and Dino folk about this, as they have done prior work and research in this area

Well that's why I'm asking here :)

Sure :)

I've seen something on OpenFire side too, but it seems specific to their stack.

My component will be server and client independant.

goffi: don't you still need message initiation? Otherwise how does multi device work, how do I know I missed a call, how do I call people I don't have mutual presence with or vice versa?

moparisthebest: in my idea, you join a room (pretty much like with Jitsi or most A/V conferences implementations I've used), you are not called like in one2one calls. Invitations can be sent though.

> Hi there. I'm working on a SFU integration in XMPP. I'm doing an implementation with Galène (https://galene.org/), and I'm currently working on a component which will bridge Galène API and Jingle. I'll publish one or two protoXEP for missing parts, notably the way to join an A/V conference group. I plan to make it simple, with a specific category/type, and client joining the conference in a similar way as we do for one2one calls (i.e. by doing a jingle request, without message initiation as it is not relevant here). Does anybody have suggestion for that? One could think about using MUC as a base, but I think that it would complicate for nothing (we can still link a MUC room somewhere). The ejabberd team is also starting some work on it, and it is exactly why I'm currently getting funding ↺

Looks like it might be wise to have a talk all together, maybe organize a kind of "summit" around this ?

Maybe some sort of sprint? I'm definitely interested in the client side of that. I always said that I wanted someone else to move on the SFU part first (mostly for time reasons) but I'm very interested in having client support

Generally speaking not tying this to MUC and making calling a 'room' on the SFU very similar to how you would call a single contact sounds good to me and is how I would do it

For the record, it's part of my current grant too, but I have little time, and a hard deadline mid-august (with vacations and all, I have only a few weeks to work on it).

edhelas, Daniel: we have a sprint soon in Berlin, as both of you are not attending, maybe we can set a call at this moment?

goffi maybe put somewhere on a pad/wiki/web page what you've worked on, with the list of XEPs and a few schemas and we'll follow on that

edhelas: sure. I'll propose protoXEPs soon on this topic anyway, as I need them and it's one of my remaining grant steps.

I'd prefer to build onto what you did and follow up, then normally at the end of the year we could have a few server and clients SFU implementations :)

2024 is the year of video-conferencing on XMPP

I thought that was 2020? (jitsi)

2020 didn't count 😷

some would say 2020 hasn't even ended yet...

$ sdate --covid 19 Tue 1578 Mar 11:43:02 CEST 2020

This may be relevant: https://www.jsxc.org/blog/2021/08/31/A-group-call-proposal.html

goffi, in Tigase we worked on integration of SFU and we have some documentation of our extensions to support for SFU. I’m not sure if they will be helpful for you, but maybe they are worth to look at https://xeps.tigase.net/docs/meet-group-videocall/

Rooh, I've missed this one. Was there a discussion on standard@? I can't reasonably check every XMPP related project blogs, comments, mucs, and xsf@ muc and standard@.

Andrzej: alright, thank for the links, I'll check it out

(I was answering to previous message)

but in general that's a problem I see more and more often, there are experiment spread over the internets, and it's not centralised somewhere

also after a quick read, it seems that JSXC implemented MESH without using MUJI, I don't get why.

their solution seems really similar.

Is there any way to reliably get a server's entity caps if it doesn't include that in its stream features?

> Rooh, I've missed this one. Was there a discussion on standard@? I can't reasonably check every XMPP related project blogs, comments, mucs, and xsf@ muc and standard@. Are you planning to write some blog-post/documentation explaining all the things you've done and which XEP were used to do it ? ↺

goffi: why do you need something beyond session-initiate for join a call?

edhelas: I'll publish the protoXEP, and probably write a blog post at some point yes.

Perfect 👍

singpolyma: I don't think that we need something beyond session-initial, I'm actually advocating to have the simplest protocol, as close as possible as one2one call.

Do we have some early work on "active-speaker" XEPs ? Basically to tell the audience that someone is taking priority in the video-call, can be useful in a presentation

edhelas: that one of the protoXEP that I'm planning to propose, adding metadata for such information, and to link a stream to a user.

> singpolyma: I don't think that we need something beyond session-initial, I'm actually advocating to have the simplest protocol, as close as possible as one2one call. Good. Yeah. I don't think we should need anything new protocol wise for call to work. Maybe need some metadata stuff to know which streams have what nickname or if you want to put in currently speaking metadata or similar but it could all be optional I think ↺

singpolyma: I think that me need a least a dedicated disco category/type, and probably a way to get current participants (I was thinking about ad-hoc commands).

> edhelas: that one of the protoXEP that I'm planning to propose, adding metadata for such information, and to link a stream to a user. Can we start a small pad where you can list all the XEPs + protoXEP that are and will be used ? ↺

I'll use that as a reference and complete it along my implementation, I'll start in a few weeks to actively work on it

I would use muc for participants, but I can understand why others might not want to do that. But even this could be optional to just have a call and it works

edhelas: I have a rough idea, but it's not fully clear yet, I need to work on implementation to see what is needed. But sure we can write stuff something somewhere to help working on it.

One thing I kind of want is an option to have two jingle sessions for the call livekit-style but if your SFU doesn't need this it's probably overkill to think about it for the prototype

Yes, even if its just ideas :)

singpolyma: What is livekit?

OK this I guess https://meet.livekit.io/

An sfu

and why is it using the equivalent of 2 jingle session?

One for subscribing to incoming streams, one for publishing outgoing streams.

That way either side can add/remove tracks at will and there is no glares problem etc

Can't this be done with a single session? With content-add/content-remove?

Yes but content-add has a race condition if the other side does it simultaneously

Which is why they chose this design

Oh I see.

as I was using Janus SFU, they used similar design, 1 session for incoming streams and 1 for outgoing

Now I think that I've heard about livekit, isn't is the stack used by Matrix?

Yeah. I think it's not crazy to support this as an optional mode. Probably some small jingle extension that that means "this is a one way stream, call me back to publish" or such

Sure. But my goal is currently to have a simple prototype (also due to time constraint), but it could definitely be possible to have this optionally.

Yes for sure, as I said if your SFU doesn't need it do what's best for your prototype. I just want the idea that maybe this will happen "in the air"

> singpolyma: I think that me need a least a dedicated disco category/type, and probably a way to get current participants (I was thinking about ad-hoc commands). Please don't make the disco mistake here again :'( ↺

We're talking about disco on a muc-like jid for the A/V "room"

moparisthebest: ???

You need to know that a jid is an A/V conference service.

My idea for when it's combined with MUC was that when you join the MUC it also calls you. But if you do a MUC-less version I can see the benefit of a new disco item for sure

I was just about to ask for a good disco identity. I was thinking about category="conference" and type="call", but I'm not sure if it's not confusing, as "conference" is currently only used for text conference. Is there any better suggestion?

Text conference is conference/text which definitely implies conference/audio and conference/video as other sensible types to add

A MUC combined room would have all three. Something like a mumble bridge might have /text and /audio but not /video.

The thing is that it can be audio only, video only, and both. That's why I was thinking about the more generic "call".

You can have multiple identities

Though maybe this is thinking too hard and just having var=jingle is enough?

We need an identity, a simple disco feature is not enough to know that it's a conference room.

Yeah. So adding /audio and /video seems like the most obvious way to me

When it is about adding metadata for who is who to the stream, there already is https://xmpp.org/extensions/xep-0298.html, no need for a new protocol

For me having multiple identities, like "conference/audio" + "conference/video" seems unnatural versus a generic term for both (like the "call" I suggested, or something better), but I don't have strong argument against it either.

larma: alright I forgot about this one, I'll check it, thanks.

singpolyma, can you explain the race condition in content-add? Or precisely why content-add has it and session-initiate doesn't?

https://xmpp.org/extensions/xep-0298.html is nothing. it's basically an empty XEP

larma: session-initiate doesn't only because only one side will ever do it for the same one session. If both sides issue a content-add at the same time, then both sides end up needing to renegotiate but have different views of the underlying SDP (since you add to your local SDP before sending the content-add) and it messes up the state machine if not very careful

I don't think content-add is defined in SDP at all, or is it?

No, but my point is you've added to your local view of the session (sorry, saying it as SDP is implementation specific I shouldn't have done that) and sent the offer to change the other side (via content-add)

then you get a content-add before a content-accept has happened

so this confuses the state machine because there are two offers in progress

you need some sort of "tie break"

I don't see why you can't have multiple pending content-add at the same time

The Jingle session state machine only knows three states PENDING, ACTIVE, ENDED

pending content-add is not a state

I'll admit I'm just parroting what I've been told / have read about the WebRTC glares situation generally. I'm not speaking from experience yet

but if two ends at the same time modify content (by content-add) there is a race condition

as to which changes should be applied first and order of contents

there is only a race condition if they try to add the same content name, same as if they try to use the same session id

I don't think the order of contents is part of the session state

when does the order matter?

I suppose when you have to modify SDP ✏

> https://xmpp.org/extensions/xep-0298.html is nothing. it's basically an empty XEP how is it empty? It well defines how to attach RFC 4575 data to a Jingle session. ↺

> I suppose when you have to modify SDP If an implementation for reasons of their use of SDP needs to create an ordering of contents, then this is implementation specific. Which implies it doesn't matter how it does this and thus the ordering can be different from the one the other end chooses. As long as the ordering is not transferred over network, it doesn't matter what your local SDP does. ↺

> You need to know that a jid is an A/V conference service. goffi: that's fine, just don't do disco to user JIDs to determine if the currently online clients support calls ↺

https://slack.com/intl/en-gb/help/articles/29414264463635-Updates-to-message-and-file-history-on-free-workspaces

> starting 26th August 2024, we’ll begin deleting messages and files more than one year old from free workspaces on a rolling basis.

Introducing slack2mam

People let slack retain (more than????) a whole year of their data? 😬

$ update messages set tombstone = true where created_at > '1 year'; ✏

moparisthebest: the whole point of slack is to retain all the messages, yes. That's the main thing slack charges for, it's where they get all their money from

I agree it might make things a bit trickier when mapping Jingle sessions to SDP, but beside that, using a single Jingle session fits much more the Jingle model (a Jingle session being a logical unit of a call). Maybe you can just convert the single Jingle session to two SDP sessions for your underlying implementation?

singpolyma: in my experience companies auto-expire slack messages at like 3 months because... Why would it be sane to expose more than you have to to the inevitable data breaches? https://www.schneier.com/essays/archives/2016/03/data_is_a_toxic_asse.html