XSF Discussion - 2018-11-20

Ge0rG, what is that list of XEPs? anything useful for the compliance suites?

jonas’: you mean the "relevant for mobile" list? I think it's covered by the suite already, plus 0184

good

I wouldn't hold my breath for chatty though

daniel: because of libpurple?

Just by extrapolating the progress of the last four month

daniel: so because of libpurple.

daniel: are you missing XEPs from Compliance Suite 2019?

Ge0rG: I don't recall anything coming up this year

Maybe the bookmark conversion thing?

avatar something-something?

the conversion XEPs would indeed be interesting to have

I was also thinking about bookmarks conversion

Do the xeps that go in there need to be draft or sth?

isn’t even MIX in there?

When nobody implements it and it's not exactly clear what the benefits are over existing protocols?

ah, no, it’s just an informational footnote that MIX might be coming

are you talking about server implementing it or client?

because client its just "dont sync pep and private storage"

A client should Stil know about that

itym: "its just 'dont sync pep and private storage, but only if the server exposes the feature, otherwise you’re as doomed as ever'"

So even it's not a lot of work

yes jonas, i might implement it today

so yeah, clients need to know about it

and even if only to relieve the developers from the nightmares they get when thinking about the mess that synchronization is

I'm toying with an idea, for to XSF to offer up a prize for development of the best/first/somehow-to-be-qualified new (client?) XMPP software, X-Prize style. My primary motivation is to facilitate the development of new, feature complete / representable software for things that are currently missing from the XMPP eco-system. A decent, free IOS client, perhaps? What do you guys think?

I like the idea :)

but maybe put a couple of categories, like the Oscars, best UX, best security, best XMPP support...

I don't have a concrete plan for the shape or form, so yeah, that's all possible.

That said: we have limited resources, so for the incentive to be interesting enough, we might not want to have to split things up to much.

yeah just a few of them, it could be a simple vote and we design a some logos that the winner project can put on their README :D

you have to be prepared that no client reaches the goal, that would be a bit embarasing then

"Best XMPP Client of 2018"

jonas’ [10:22]: > and even if only to relieve the developers from the nightmares they get when thinking about the mess that synchronization is _99 issues with sync in the PEP // 99 issues with sync // One bug gets squashed // 98 issues with sync in the PEP_

I would not split if possible, indeed. If it must be a client, I would say something missing. Either features missing in the modern times or a client for a platform with no clients/up-to-date clients

Guus: that calls for a Jabber Software Foundation

You're miscounting, Ge0rG. Last number should be '100'...

(or more)

lovetox: the challenge need not be time-restricted. If no client reaches the goal, the effort will fizzle out over time, without much harm done.

also, I'm willing to take that chance.

Guus, this will be very frustrating for developers who simply cannot invest the time without reliable source of income from the xmpp development :/

Ge0rG: maybe. I don't yet care at this point. 🙂

jonas' - that's one of my concerns - I'd not like to dissuade others.

I'm pretty sure you won't be allowed to do that within the XSF

but, on the other side: the XMPP community would benefit more from having a limited amount of pretty good clients, than a large amount of very shitty ones...

Guus: that's absolutely true

Ge0rG: at this time, I'm first trying to figure out if the idea itself is viable. We can worry about the appropriate vehicle later.

next worry: client developers will cut corners to achieve the requested milestone

while this might produce a reasonable product *at the time*, it may lead to horrible, unmaintainable code which leads the client into a dead end (at least for a while)

if (xsfCheck) behaveFaster();

hah

#volkswagen

:D

always

jonas' that's valid too. On the one hand, we'd have to come up with criteria that prevent the worst of this. That will be tricky to do, but should be doable

edhelas, are you using https://github.com/hmlb/phpunit-vw in movim yet? ✏

But also, I'm now at a point where I'd love to have a good client on IOS, even if it's unmaintainable and will be shitty in a couple of years time.

Yeah, I'd love to get Swift shipped properly.

Kev: what's keeping you?

Cycles.

Kev, what does blender have to do with that?

Guus, hm, so a prize for an iOS client?

maybe?

that indeed sounds interesting, because that’s the worst platform at the moment

just floating an idea here 🙂

and there’s not too much competition

so less potential for frustration

I’d be totally in favour of that, although that means that my chances from collecting the prize go from 1% to 0% :)

Guus: have you had a look at Monal recently?

No

Guus: you should. MUC and OMEMO aren't there yet, but there is significant progress and Anu is very responsive to feedback

a working XMPP client on iOS probably needs voice chat though

to get privileges for push

to be finally usable

👍

A decent iOS client needs more thinking than it seems.

I'm not sure it's really needed if we properly tackle xmpp push

so… maybe that should be the goal? but then again, the number you’d have to put on the table to have a freelancer implement audio chat from scratch is .. high

We need to get Push properly sorted for it.

Ge0rG, is it? my understanding was that push is unreliable unless you get a voip whitelist mark or whatever ✏

but I’m not familiar with iOS development

The VOIP thing was so that you could leave yourself running in the background to defeat TCP being killed when you're backgrounded.

Maybe Monal has that.

ah, right

that would be better than push, indeed

Not really, I think.

but, so, not a requirement

Kev, why?

Push and quick resync is the better solution. But we don't have a good resync solution at the moment.

except that for push, you need to go through a third-party service

I think you can create a decent iOS client without VoIP

Just by using push properly

daniel, why has nobody managed to do that yet, though?

Except Push was -1ed

jonas’: lack of developer resources?

Ge0rG, hm. "how hard can it be?"

jonas' : wanna win a prize? 🙂

> daniel, why has nobody managed to do that yet, though? Because people don't seem to know how I guess

Guus, no, I’m not touching iOS

jonas’: ChatSecure has a bug where it duplicates incoming messages, and it's unfixed for half a year now already

daniel, do *you* know h ow?

Or don't have enough time to get familiar with it

Ge0rG, yes, but that seems harder than "just" configuring your push server to send the right stuff at the right time

I had a fairly decent picture of what was needed, but it was less trivial than it sounds.

> daniel, do *you* know h ow? I know how I would do it yes

Unless I'm just too stupid to see the simple solution.

No it's not trivial

daniel, maybe get in touch with $iOSClientDeveloper and get it sorted then?

okay

Kev, did you document that?

I’ll shut up then :)

You can get by with just Push, but to get the really good experience you also need much faster resyncs than we can currently do.

Guus: There are many aspects to getting this right, including MAM, MIX, Push and Routing2.

So yes, I've been trying to document the various needed moving parts.

Kev: that's why I hoped for existing documentation 🙂

I think Chris kinda knows it too. But since it wasn't build into chatsecure it requires heavy refactoring

And you probably don't want to use a traditional xmpp library

daniel: what about just writing the magic into the XEP?

The thing that I would get a lot of hate for suggesting is that I think there would be value in ... XMPP over HTTP.

Kev: different from BOSH?

As a quick resync mechanism, so that we could poll in the background.

Well I know how to start and I'm fairly confident I could figure out the missing pieces as I go. But it's not that I could write down a step by step how to

Ge0rG: Yes, just for polling.

Kev: you need cookies / tokens then

Yes.

Kev: also not sure what problem that'll solve, apart from paranoid firewalls

And nobody would then implement it so it would be a wasted effort anyway

Does anyone particularly dislike the idea to have some kind of prize offered up for a good* iOS client?

Ge0rG: It'd let you poll in the background when receiving a bodyless push to just find the content of that message so you could display a notification.

Guus: I do, personally. I think the XSF could better spend its money.

Of which it doesn't have much. ✏

Kev: on what? I'd love to spend money.

> Ge0rG: It'd let you poll in the background when receiving a bodyless push to just find the content of that message so you could display a notification. You have to be fast and recover all state from disk

Kev: how's that different from an XMPP connection?

But that's it basically

daniel: I don't think you need to recover anything at all, actually.

You could do the full resync once the client is foregrounded again.

Of course, this is much easier if you're willing to have your message content go over Push.

But if you're not, you need some mechanism that can wake the client, have it fetch the message that needs displaying, fetch it, display it, and go back to sleep again without the OS getting angry with you.

Right. Yes I see that your approach is valid as well. But I think one has enough time to do a regular sm catchup if we get fast resume

Kev: that sounds like you are trying to get rid of the cruft of XMPP session setup by replacing XMPP with HTTP, instead of replacing the session setup

Aka instant stream resumption and the like

I wouldn't even try, I think. To do it over XMPP I'd just have fast reconnect, not fast resume, and then just poll that one item over MAM.

And leave the fast resume for when the client is woken up by the user.

Kev, that doesn’t sound too bad

Ge0rG: Valid point, see above.

soo… XMPP-over-TLS, SASL2+bind2 to do as much as possible in as few as possible roundtrips, query MAM, close connection?

(or try to keep connection open if possible)

Something like that, yeah.

Yeah I mean in any case it can be done

fg

It's just that the developer has to have in depth knowledge of xmpp and design the client from scratch to do this

jonas’: -EPERM

I'm not sure the client needs designing from scratch (I don't think we'd need to redesign Swift to do this, although it's a chunk of work), but there are certainly designs that would make it harder.

Ge0rG: You say that, but it clearly worked :D

Sure. Depends on the client of course. Maybe swiften was just designed very well

Kev: > The VOIP thing was so that you could leave yourself running in the background to defeat TCP being killed when you're backgrounded. Yes that's how it worked with past iOS versions, AFAIK there's no way to do get that behavior these days. Now you need the VoIP label to get silent notifications, i.e. to have your app woken up rather than notifying the user without waking the app.

I stand corrected.

Kev: > let you poll in the background when receiving a bodyless push You'd still need that VoIP flag to get that 'bodyless push' in the first place. And how does HTTP vs. XMPP help after being woken up?

Keep reading down :)

I did.

It doesn't, you could do the same over XMPP by having a non-session stream making a MAM request.

Ah ok.

As long as you had XMPP-over-TLS, and immediate setup instead of SASL.

Right.

daniel: > I think you can create a decent iOS client without VoIP I dunno how you'd avoid "New message" notifications though.

Presumably by sending the content in the push notif, which some people would hate, and others would be quite happy with.

you can’t do that with OMEMO

With a smarter push mechanism we could avoid sending multiple of those, and once the client *is* woken it can attempt to replace that notification (like ChatSecure already does), but that's not reliable.

Kev: Right.

jonas’: OMEMO is why we can't have nice things :)

no, broken mobile OSes are why we can’t have nice things (in this case)

Yes it would be nice if we could offer a knob to optionally include message contents.

I think the "can't send bodyless push" thing might be broken (didn't know about that), but otherwise the whole "suspect in background" thing seems quite sensible to me (also the way Android seems to be going).

> daniel: > I dunno how you'd avoid "New message" notifications though. You can replace the content of a message

You get the new message push, catch up with the real one and then replace it

But yes you need to make the push a bit smarter

But that's not rocket science

Kev: Yes, the big difference so far is that Android doesn't restrict silent notifications. But who knows where they're going.

s/suspect/suspend/, obviously.

daniel: Right, that's what ChatSecure is doing. But that means in 86,9% of all cases, users see a "new message!" notification.

So what's WhatsApp doing here - registering as a VOIP app so it can receive push notifs?

I think the go the replacing route. They just don't have false positives

daniel: The ChatSecure app server generates both a 'New message' notification *and* a 'silent' notification. The latter one is unreliable, but *if* it's delivered, the "catch up and replace" thing is attempted. If it's not, there's no way to do that.

Kev, daniel: I would've assumed silent VoIP notifications.

But I dunno.

But the new message ones are reliable as well, no?

https://developer.apple.com/documentation/usernotifications/setting_up_a_remote_notification_server/pushing_updates_to_your_app_silently Doesn't say anything about needing to be a VOIP app, AFAICT.

Would be interesting to know what e.g. Signal did before they had VoIP support.

daniel: The 'New message' ones are reliable.

Kev: I'll look it up, it was documented somewhere (and it clearly works that way).

So figure out when a notification should actually trigger a notification, send a reliable one with a dummy body and the modify it to show the actual content

I don't see the problem with that

You get 30s to modify it https://developer.apple.com/documentation/usernotifications/modifying_content_in_newly_delivered_notifications

This only depends on the app server to figure that and then on the client to do the catchup in 30s

That seems achievable

s/catchup/message discovery/

You might do message discovery with catchup, you might employ a different mechanism.

What ever ☺️

What ever meachism you deem fit to fetch a specific message within 30s

Right.

Ahh, right, "Service App Extension". That's a new thing I think, IIRC Chris mentioned that.

That would probably do the trick indeed.

Ok, I've spent a few minutes reading (more than I should, probably), and from what I can tell VOIP pushes will make things better because they're high priority pushes, but shouldn't be necessary for this to work.

Yes yes, I forgot about that thing. Depends on a recent iOS version I think, but apart from that it's probably fine.

I mean 30s is actually *a lot *

I'd be worried if we only had single digit seconds

I mean obviously my users hate metrics and an app 'calling home'. But it'd be fun to get some numbers on the median resume and catch up time in Conversations

Guus, re prices to give to clients, I agree with jonas’ that this will probably discourage others that don't have enough time to allocate. But I also agree that it would be nice to have some kind of incentive

jonas', you here?

kindof

Guus, what’s up?

mind popping over to the iteam muc?

Anyone in here that didn't vote yet? 🙂

if so: you're now looking at your IM-client. You might as well pop off a message to memberbot.

Thanks for the reminder

Thanks, I just did it!

jonas’ kindly commit that missing file 🙂

Guus, did

daniel, Dino merged our pr for the 12 bytes IV already, nice ^^

Yes I saw that

Now we need to do something about chatsecure

Guus: it seems like whatever prize the xsf could offer wouldn't be large enough to pay for development, so you still need a dedicated dev who wants it for themselves

And any Dev like that doesn't want to run a crippled phone OS when they can run Android...

Hey if Android keeps getting worse though...

moparisthebest: prize money is an incentive, but not paying for all of the development.

The real prize is the bragging rights for your marketing department if you win

I think there are basically 2 types of devs that would do it

One is with funding and a marketing plan, and it's not gonna be open, line WhatsApp etc

Other is Dev that wants to scratch their own itch, with capability, and using an iPhone full time

I don't think that second one exists

It shouldn't be overlooked that we have at least two iOS clients already

It's not necessarily a requirement to write a new one from scratch

and therefore it may be feasible to fund shorter specific projects on top of existing iOS clients

Yea that seems more feasible

https://op-co.de/tmp/xep-0184.html#format - the "Note:" below example 4. I think I need to tune the wording some more.

Maybe it's time to do some woodwork instead of writing XEPs.

If you started building a bicycle you might not be able to fix it into a car

Especially not as a third party

In that case, I feel like xmpp is a bicycle car hybrid where one person keeps taking off bike parts and putting car parts on and one person is doing the opposite and others are just randomly bolting airplane parts on wherever they'll fit.

(I missed most context there, but if I interpreted the analogy correctly my point is that I'm starting to think we need a specific mission and set of use cases)

SamWhited: I was talking about clients. I think there is some truth to what you are saying in regards to the protocol but I think it's more severe when it comes to specific clients

A multi use protocol is not an unfixable problem imho

Maybe a hard one

ah yah, that applies even better to most clients, assume I was talking about that too

https://github.com/siacs/Conversations/issues/3125#issuecomment-440398906

there is a need for a simple way to refer to another message

references doesn’t have that use-case

can we get that specified quickly?

or alternatively rename Message Attaching, because it’s not being used for attaching apparently

What uses it and for what then?

Zash, see that link please

What, read what I'm commenting on?! Unpossible!

Mkay

Don't make me submit an <in-reply-to/>

or "How to blockchain using thread/@parent"

im not sure about this for quotation

but what it actually solves nicely is the case where i want to convey a url with a message

but i want to encrypt it

for oob and the sorts, full stanza encryption is necessary

this would solve this nicely for omemo

Why do we have to fix everything else when we can fix omemo :(

daniel said last time, in the last sprint they furhter developed the future omemo version

so i guess it will happen sometime

When it's already too late because the other things around it have been changed?

its never too late for full stanza encryption

also it about a different wireformat to evolve from signal

Yeah there's been discussions around that this weekend as well

"Furthermore, even if 3.4 covered our use case other use cases outside our scope would still have to be implemented in order to be compliant with the whole XEP.", I guess people are also often confused by this.

How many people say they support MUC or Pubsub :) ✏

lovetox [20:48]: > its never too late for full stanza encryption Nobody expects the Full Stanza Encryption!

Zash [20:35]: > Don't make me submit an <in-reply-to/> And then rewrite Receipts and LMC on top of it!

See discussion in jdev@ today

Do you know this https://omemo.im/ ? The website looks like an aggregation of publicly available informations, and it claims to be open source but I don't see any source code. Smells like malicious app

its not malicious

it seems to link for downloads directly to other projects

like gajim, and zom

Except the Android one

yes

thats a bit weird

It links to Conversations in the footer

but i bet its conversations

relabeld

but yeah, i dont know that will help the cause

also here https://www.reddit.com/r/xmpp/comments/9y3ib4/can_you_use_omemo_encryption_across_two_different/ "See omemo.im and test their server and app across yours." by somebody nicknames "omemo82", it doesn't smells good to me

nicknames

nickamed*

arg

nicknamed*

Search for omemo.im on github turns up https://github.com/froghorn82/omemo.im

THe commit history is... interesting 😀

dont understand why people dont try to do something productive

Procrastinating makes the world go round ... later.