XSF Discussion - 2018-11-20


  1. jonas’

    Ge0rG, what is that list of XEPs? anything useful for the compliance suites?

  2. Ge0rG

    jonas’: you mean the "relevant for mobile" list? I think it's covered by the suite already, plus 0184

  3. jonas’

    good

  4. daniel

    I wouldn't hold my breath for chatty though

  5. Ge0rG

    daniel: because of libpurple?

  6. daniel

    Just by extrapolating the progress of the last four month

  7. Ge0rG

    daniel: so because of libpurple.

  8. Ge0rG

    daniel: are you missing XEPs from Compliance Suite 2019?

  9. daniel

    Ge0rG: I don't recall anything coming up this year

  10. daniel

    Maybe the bookmark conversion thing?

  11. Ge0rG

    avatar something-something?

  12. jonas’

    the conversion XEPs would indeed be interesting to have

  13. pep.

    I was also thinking about bookmarks conversion

  14. pep.

    Do the xeps that go in there need to be draft or sth?

  15. jonas’

    isn’t even MIX in there?

  16. pep.

    When nobody implements it and it's not exactly clear what the benefits are over existing protocols?

  17. jonas’

    ah, no, it’s just an informational footnote that MIX might be coming

  18. lovetox

    are you talking about server implementing it or client?

  19. lovetox

    because client its just "dont sync pep and private storage"

  20. daniel

    A client should Stil know about that

  21. jonas’

    itym: "its just 'dont sync pep and private storage, but only if the server exposes the feature, otherwise you’re as doomed as ever'"

  22. daniel

    So even it's not a lot of work

  23. lovetox

    yes jonas, i might implement it today

  24. jonas’

    so yeah, clients need to know about it

  25. jonas’

    and even if only to relieve the developers from the nightmares they get when thinking about the mess that synchronization is

  26. Guus

    I'm toying with an idea, for to XSF to offer up a prize for development of the best/first/somehow-to-be-qualified new (client?) XMPP software, X-Prize style. My primary motivation is to facilitate the development of new, feature complete / representable software for things that are currently missing from the XMPP eco-system. A decent, free IOS client, perhaps? What do you guys think?

  27. edhelas

    I like the idea :)

  28. edhelas

    but maybe put a couple of categories, like the Oscars, best UX, best security, best XMPP support...

  29. Guus

    I don't have a concrete plan for the shape or form, so yeah, that's all possible.

  30. Guus

    That said: we have limited resources, so for the incentive to be interesting enough, we might not want to have to split things up to much.

  31. edhelas

    yeah just a few of them, it could be a simple vote and we design a some logos that the winner project can put on their README :D

  32. lovetox

    you have to be prepared that no client reaches the goal, that would be a bit embarasing then

  33. edhelas

    "Best XMPP Client of 2018"

  34. Ge0rG

    jonas’ [10:22]: > and even if only to relieve the developers from the nightmares they get when thinking about the mess that synchronization is _99 issues with sync in the PEP // 99 issues with sync // One bug gets squashed // 98 issues with sync in the PEP_

  35. Seve

    I would not split if possible, indeed. If it must be a client, I would say something missing. Either features missing in the modern times or a client for a platform with no clients/up-to-date clients

  36. Ge0rG

    Guus: that calls for a Jabber Software Foundation

  37. Guus

    You're miscounting, Ge0rG. Last number should be '100'...

  38. Guus

    (or more)

  39. Guus

    lovetox: the challenge need not be time-restricted. If no client reaches the goal, the effort will fizzle out over time, without much harm done.

  40. Guus

    also, I'm willing to take that chance.

  41. jonas’

    Guus, this will be very frustrating for developers who simply cannot invest the time without reliable source of income from the xmpp development :/

  42. Guus

    Ge0rG: maybe. I don't yet care at this point. 🙂

  43. Guus

    jonas' - that's one of my concerns - I'd not like to dissuade others.

  44. Ge0rG

    I'm pretty sure you won't be allowed to do that within the XSF

  45. Guus

    but, on the other side: the XMPP community would benefit more from having a limited amount of pretty good clients, than a large amount of very shitty ones...

  46. Ge0rG

    Guus: that's absolutely true

  47. Guus

    Ge0rG: at this time, I'm first trying to figure out if the idea itself is viable. We can worry about the appropriate vehicle later.

  48. jonas’

    next worry: client developers will cut corners to achieve the requested milestone

  49. jonas’

    while this might produce a reasonable product *at the time*, it may lead to horrible, unmaintainable code which leads the client into a dead end (at least for a while)

  50. edhelas

    if (xsfCheck) behaveFaster();

  51. jonas’

    hah

  52. jonas’

    #volkswagen

  53. edhelas

    :D

  54. jonas’

    edhelas, are you using https://github.com/auchenberg/volkswagen in movim yet?

  55. edhelas

    always

  56. Guus

    jonas' that's valid too. On the one hand, we'd have to come up with criteria that prevent the worst of this. That will be tricky to do, but should be doable

  57. jonas’

    edhelas, are you using https://github.com/auchenberg/volkswagen in movim yet?

  58. jonas’

    edhelas, are you using https://github.com/hmlb/phpunit-vw in movim yet?

  59. Guus

    But also, I'm now at a point where I'd love to have a good client on IOS, even if it's unmaintainable and will be shitty in a couple of years time.

  60. Kev

    Yeah, I'd love to get Swift shipped properly.

  61. Guus

    Kev: what's keeping you?

  62. Kev

    Cycles.

  63. jonas’

    Kev, what does blender have to do with that?

  64. jonas’

    Guus, hm, so a prize for an iOS client?

  65. Guus

    maybe?

  66. jonas’

    that indeed sounds interesting, because that’s the worst platform at the moment

  67. Guus

    just floating an idea here 🙂

  68. jonas’

    and there’s not too much competition

  69. jonas’

    so less potential for frustration

  70. jonas’

    I’d be totally in favour of that, although that means that my chances from collecting the prize go from 1% to 0% :)

  71. Ge0rG

    Guus: have you had a look at Monal recently?

  72. Guus

    No

  73. Ge0rG

    Guus: you should. MUC and OMEMO aren't there yet, but there is significant progress and Anu is very responsive to feedback

  74. jonas’

    a working XMPP client on iOS probably needs voice chat though

  75. jonas’

    to get privileges for push

  76. jonas’

    to be finally usable

  77. Guus

    👍

  78. Kev

    A decent iOS client needs more thinking than it seems.

  79. Ge0rG

    I'm not sure it's really needed if we properly tackle xmpp push

  80. jonas’

    so… maybe that should be the goal? but then again, the number you’d have to put on the table to have a freelancer implement audio chat from scratch is .. high

  81. Kev

    We need to get Push properly sorted for it.

  82. jonas’

    Ge0rG, is it? my understanding was that push is unreliable unless you get a voip whitelist

  83. jonas’

    Ge0rG, is it? my understanding was that push is unreliable unless you get a voip whitelist mark or whatever

  84. jonas’

    but I’m not familiar with iOS development

  85. Kev

    The VOIP thing was so that you could leave yourself running in the background to defeat TCP being killed when you're backgrounded.

  86. Ge0rG

    Maybe Monal has that.

  87. jonas’

    ah, right

  88. jonas’

    that would be better than push, indeed

  89. Kev

    Not really, I think.

  90. jonas’

    but, so, not a requirement

  91. jonas’

    Kev, why?

  92. Kev

    Push and quick resync is the better solution. But we don't have a good resync solution at the moment.

  93. jonas’

    except that for push, you need to go through a third-party service

  94. daniel

    I think you can create a decent iOS client without VoIP

  95. daniel

    Just by using push properly

  96. jonas’

    daniel, why has nobody managed to do that yet, though?

  97. Ge0rG

    Except Push was -1ed

  98. Ge0rG

    jonas’: lack of developer resources?

  99. jonas’

    Ge0rG, hm. "how hard can it be?"

  100. Guus

    jonas' : wanna win a prize? 🙂

  101. daniel

    > daniel, why has nobody managed to do that yet, though? Because people don't seem to know how I guess

  102. jonas’

    Guus, no, I’m not touching iOS

  103. Ge0rG

    jonas’: ChatSecure has a bug where it duplicates incoming messages, and it's unfixed for half a year now already

  104. jonas’

    daniel, do *you* know h ow?

  105. daniel

    Or don't have enough time to get familiar with it

  106. jonas’

    Ge0rG, yes, but that seems harder than "just" configuring your push server to send the right stuff at the right time

  107. Kev

    I had a fairly decent picture of what was needed, but it was less trivial than it sounds.

  108. daniel

    > daniel, do *you* know h ow? I know how I would do it yes

  109. Kev

    Unless I'm just too stupid to see the simple solution.

  110. daniel

    No it's not trivial

  111. jonas’

    daniel, maybe get in touch with $iOSClientDeveloper and get it sorted then?

  112. jonas’

    okay

  113. Guus

    Kev, did you document that?

  114. jonas’

    I’ll shut up then :)

  115. Kev

    You can get by with just Push, but to get the really good experience you also need much faster resyncs than we can currently do.

  116. Kev

    Guus: There are many aspects to getting this right, including MAM, MIX, Push and Routing2.

  117. Kev

    So yes, I've been trying to document the various needed moving parts.

  118. Guus

    Kev: that's why I hoped for existing documentation 🙂

  119. daniel

    I think Chris kinda knows it too. But since it wasn't build into chatsecure it requires heavy refactoring

  120. daniel

    And you probably don't want to use a traditional xmpp library

  121. Ge0rG

    daniel: what about just writing the magic into the XEP?

  122. Kev

    The thing that I would get a lot of hate for suggesting is that I think there would be value in ... XMPP over HTTP.

  123. Ge0rG

    Kev: different from BOSH?

  124. Kev

    As a quick resync mechanism, so that we could poll in the background.

  125. daniel

    Well I know how to start and I'm fairly confident I could figure out the missing pieces as I go. But it's not that I could write down a step by step how to

  126. Kev

    Ge0rG: Yes, just for polling.

  127. Ge0rG

    Kev: you need cookies / tokens then

  128. Kev

    Yes.

  129. Ge0rG

    Kev: also not sure what problem that'll solve, apart from paranoid firewalls

  130. daniel

    And nobody would then implement it so it would be a wasted effort anyway

  131. Guus

    Does anyone particularly dislike the idea to have some kind of prize offered up for a good* iOS client?

  132. Kev

    Ge0rG: It'd let you poll in the background when receiving a bodyless push to just find the content of that message so you could display a notification.

  133. Kev

    Guus: I do, personally. I think the XSF could better spend its money.

  134. Kev

    Of which it doesn't have muc.

  135. Kev

    Of which it doesn't have much.

  136. Guus

    Kev: on what? I'd love to spend money.

  137. daniel

    > Ge0rG: It'd let you poll in the background when receiving a bodyless push to just find the content of that message so you could display a notification. You have to be fast and recover all state from disk

  138. Ge0rG

    Kev: how's that different from an XMPP connection?

  139. daniel

    But that's it basically

  140. Kev

    daniel: I don't think you need to recover anything at all, actually.

  141. Kev

    You could do the full resync once the client is foregrounded again.

  142. Kev

    Of course, this is much easier if you're willing to have your message content go over Push.

  143. Kev

    But if you're not, you need some mechanism that can wake the client, have it fetch the message that needs displaying, fetch it, display it, and go back to sleep again without the OS getting angry with you.

  144. daniel

    Right. Yes I see that your approach is valid as well. But I think one has enough time to do a regular sm catchup if we get fast resume

  145. Ge0rG

    Kev: that sounds like you are trying to get rid of the cruft of XMPP session setup by replacing XMPP with HTTP, instead of replacing the session setup

  146. daniel

    Aka instant stream resumption and the like

  147. Kev

    I wouldn't even try, I think. To do it over XMPP I'd just have fast reconnect, not fast resume, and then just poll that one item over MAM.

  148. Kev

    And leave the fast resume for when the client is woken up by the user.

  149. jonas’

    Kev, that doesn’t sound too bad

  150. Kev

    Ge0rG: Valid point, see above.

  151. jonas’

    soo… XMPP-over-TLS, SASL2+bind2 to do as much as possible in as few as possible roundtrips, query MAM, close connection?

  152. jonas’

    (or try to keep connection open if possible)

  153. Kev

    Something like that, yeah.

  154. daniel

    Yeah I mean in any case it can be done

  155. Ge0rG &

  156. jonas’

    fg

  157. daniel

    It's just that the developer has to have in depth knowledge of xmpp and design the client from scratch to do this

  158. Ge0rG

    jonas’: -EPERM

  159. Kev

    I'm not sure the client needs designing from scratch (I don't think we'd need to redesign Swift to do this, although it's a chunk of work), but there are certainly designs that would make it harder.

  160. Kev

    Ge0rG: You say that, but it clearly worked :D

  161. daniel

    Sure. Depends on the client of course. Maybe swiften was just designed very well

  162. Holger

    Kev: > The VOIP thing was so that you could leave yourself running in the background to defeat TCP being killed when you're backgrounded. Yes that's how it worked with past iOS versions, AFAIK there's no way to do get that behavior these days. Now you need the VoIP label to get silent notifications, i.e. to have your app woken up rather than notifying the user without waking the app.

  163. Kev

    I stand corrected.

  164. Holger

    Kev: > let you poll in the background when receiving a bodyless push You'd still need that VoIP flag to get that 'bodyless push' in the first place. And how does HTTP vs. XMPP help after being woken up?

  165. Kev

    Keep reading down :)

  166. Holger

    I did.

  167. Kev

    It doesn't, you could do the same over XMPP by having a non-session stream making a MAM request.

  168. Holger

    Ah ok.

  169. Kev

    As long as you had XMPP-over-TLS, and immediate setup instead of SASL.

  170. Holger didn't grasp that you were still tackling the same point here.

  171. Holger

    Right.

  172. Holger

    daniel: > I think you can create a decent iOS client without VoIP I dunno how you'd avoid "New message" notifications though.

  173. Kev

    Presumably by sending the content in the push notif, which some people would hate, and others would be quite happy with.

  174. jonas’

    you can’t do that with OMEMO

  175. Holger

    With a smarter push mechanism we could avoid sending multiple of those, and once the client *is* woken it can attempt to replace that notification (like ChatSecure already does), but that's not reliable.

  176. Holger

    Kev: Right.

  177. Kev

    jonas’: OMEMO is why we can't have nice things :)

  178. jonas’

    no, broken mobile OSes are why we can’t have nice things (in this case)

  179. Holger

    Yes it would be nice if we could offer a knob to optionally include message contents.

  180. Kev

    I think the "can't send bodyless push" thing might be broken (didn't know about that), but otherwise the whole "suspect in background" thing seems quite sensible to me (also the way Android seems to be going).

  181. daniel

    > daniel: > I dunno how you'd avoid "New message" notifications though. You can replace the content of a message

  182. daniel

    You get the new message push, catch up with the real one and then replace it

  183. daniel

    But yes you need to make the push a bit smarter

  184. daniel

    But that's not rocket science

  185. Holger

    Kev: Yes, the big difference so far is that Android doesn't restrict silent notifications. But who knows where they're going.

  186. Kev

    s/suspect/suspend/, obviously.

  187. Holger

    daniel: Right, that's what ChatSecure is doing. But that means in 86,9% of all cases, users see a "new message!" notification.

  188. Kev

    So what's WhatsApp doing here - registering as a VOIP app so it can receive push notifs?

  189. daniel

    I think the go the replacing route. They just don't have false positives

  190. Holger

    daniel: The ChatSecure app server generates both a 'New message' notification *and* a 'silent' notification. The latter one is unreliable, but *if* it's delivered, the "catch up and replace" thing is attempted. If it's not, there's no way to do that.

  191. Holger

    Kev, daniel: I would've assumed silent VoIP notifications.

  192. Holger

    But I dunno.

  193. daniel

    But the new message ones are reliable as well, no?

  194. Kev

    https://developer.apple.com/documentation/usernotifications/setting_up_a_remote_notification_server/pushing_updates_to_your_app_silently Doesn't say anything about needing to be a VOIP app, AFAICT.

  195. Holger

    Would be interesting to know what e.g. Signal did before they had VoIP support.

  196. Holger

    daniel: The 'New message' ones are reliable.

  197. Holger

    Kev: I'll look it up, it was documented somewhere (and it clearly works that way).

  198. daniel

    So figure out when a notification should actually trigger a notification, send a reliable one with a dummy body and the modify it to show the actual content

  199. daniel

    I don't see the problem with that

  200. daniel

    You get 30s to modify it https://developer.apple.com/documentation/usernotifications/modifying_content_in_newly_delivered_notifications

  201. daniel

    This only depends on the app server to figure that and then on the client to do the catchup in 30s

  202. daniel

    That seems achievable

  203. Kev

    s/catchup/message discovery/

  204. Kev

    You might do message discovery with catchup, you might employ a different mechanism.

  205. daniel

    What ever ☺️

  206. daniel

    What ever meachism you deem fit to fetch a specific message within 30s

  207. Kev

    Right.

  208. Holger

    Ahh, right, "Service App Extension". That's a new thing I think, IIRC Chris mentioned that.

  209. Holger

    That would probably do the trick indeed.

  210. Kev

    Ok, I've spent a few minutes reading (more than I should, probably), and from what I can tell VOIP pushes will make things better because they're high priority pushes, but shouldn't be necessary for this to work.

  211. Holger

    Yes yes, I forgot about that thing. Depends on a recent iOS version I think, but apart from that it's probably fine.

  212. daniel

    I mean 30s is actually *a lot *

  213. daniel

    I'd be worried if we only had single digit seconds

  214. daniel

    I mean obviously my users hate metrics and an app 'calling home'. But it'd be fun to get some numbers on the median resume and catch up time in Conversations

  215. pep.

    Guus, re prices to give to clients, I agree with jonas’ that this will probably discourage others that don't have enough time to allocate. But I also agree that it would be nice to have some kind of incentive

  216. Guus

    jonas', you here?

  217. jonas’

    kindof

  218. jonas’

    Guus, what’s up?

  219. Guus

    mind popping over to the iteam muc?

  220. Guus

    Anyone in here that didn't vote yet? 🙂

  221. Guus

    if so: you're now looking at your IM-client. You might as well pop off a message to memberbot.

  222. MattJ scurries

  223. MattJ

    Thanks for the reminder

  224. Link Mauve

    Thanks, I just did it!

  225. Guus

    jonas’ kindly commit that missing file 🙂

  226. jonas’

    Guus, did

  227. Syndace

    daniel, Dino merged our pr for the 12 bytes IV already, nice ^^

  228. daniel

    Yes I saw that

  229. daniel

    Now we need to do something about chatsecure

  230. moparisthebest

    Guus: it seems like whatever prize the xsf could offer wouldn't be large enough to pay for development, so you still need a dedicated dev who wants it for themselves

  231. moparisthebest

    And any Dev like that doesn't want to run a crippled phone OS when they can run Android...

  232. moparisthebest

    Hey if Android keeps getting worse though...

  233. Guus

    moparisthebest: prize money is an incentive, but not paying for all of the development.

  234. Zash

    The real prize is the bragging rights for your marketing department if you win

  235. moparisthebest

    I think there are basically 2 types of devs that would do it

  236. moparisthebest

    One is with funding and a marketing plan, and it's not gonna be open, line WhatsApp etc

  237. moparisthebest

    Other is Dev that wants to scratch their own itch, with capability, and using an iPhone full time

  238. moparisthebest

    I don't think that second one exists

  239. MattJ

    It shouldn't be overlooked that we have at least two iOS clients already

  240. MattJ

    It's not necessarily a requirement to write a new one from scratch

  241. MattJ

    and therefore it may be feasible to fund shorter specific projects on top of existing iOS clients

  242. moparisthebest

    Yea that seems more feasible

  243. Ge0rG

    https://op-co.de/tmp/xep-0184.html#format - the "Note:" below example 4. I think I need to tune the wording some more.

  244. Ge0rG

    Maybe it's time to do some woodwork instead of writing XEPs.

  245. daniel

    If you started building a bicycle you might not be able to fix it into a car

  246. daniel

    Especially not as a third party

  247. SamWhited

    In that case, I feel like xmpp is a bicycle car hybrid where one person keeps taking off bike parts and putting car parts on and one person is doing the opposite and others are just randomly bolting airplane parts on wherever they'll fit.

  248. SamWhited

    (I missed most context there, but if I interpreted the analogy correctly my point is that I'm starting to think we need a specific mission and set of use cases)

  249. daniel

    SamWhited: I was talking about clients. I think there is some truth to what you are saying in regards to the protocol but I think it's more severe when it comes to specific clients

  250. daniel

    A multi use protocol is not an unfixable problem imho

  251. daniel

    Maybe a hard one

  252. SamWhited

    ah yah, that applies even better to most clients, assume I was talking about that too

  253. jonas’

    https://github.com/siacs/Conversations/issues/3125#issuecomment-440398906

  254. jonas’

    there is a need for a simple way to refer to another message

  255. jonas’

    references doesn’t have that use-case

  256. jonas’

    can we get that specified quickly?

  257. jonas’

    or alternatively rename Message Attaching, because it’s not being used for attaching apparently

  258. Zash

    What uses it and for what then?

  259. jonas’

    Zash, see that link please

  260. Zash

    What, read what I'm commenting on?! Unpossible!

  261. Zash

    Mkay

  262. Zash

    Don't make me submit an <in-reply-to/>

  263. Zash

    or "How to blockchain using thread/@parent"

  264. lovetox

    im not sure about this for quotation

  265. lovetox

    but what it actually solves nicely is the case where i want to convey a url with a message

  266. lovetox

    but i want to encrypt it

  267. lovetox

    for oob and the sorts, full stanza encryption is necessary

  268. lovetox

    this would solve this nicely for omemo

  269. pep.

    Why do we have to fix everything else when we can fix omemo :(

  270. lovetox

    daniel said last time, in the last sprint they furhter developed the future omemo version

  271. lovetox

    so i guess it will happen sometime

  272. pep.

    When it's already too late because the other things around it have been changed?

  273. lovetox

    its never too late for full stanza encryption

  274. lovetox

    also it about a different wireformat to evolve from signal

  275. pep.

    Yeah there's been discussions around that this weekend as well

  276. pep.

    "Furthermore, even if 3.4 covered our use case other use cases outside our scope would still have to be implemented in order to be compliant with the whole XEP.", I guess people are also often confused by this.

  277. pep.

    How many people say they support MUC or Pubsub entirely :)

  278. pep.

    How many people say they support MUC or Pubsub :)

  279. Ge0rG

    lovetox [20:48]: > its never too late for full stanza encryption Nobody expects the Full Stanza Encryption!

  280. Ge0rG

    Zash [20:35]: > Don't make me submit an <in-reply-to/> And then rewrite Receipts and LMC on top of it!

  281. Ge0rG

    See discussion in jdev@ today

  282. goffi

    Do you know this https://omemo.im/ ? The website looks like an aggregation of publicly available informations, and it claims to be open source but I don't see any source code. Smells like malicious app

  283. lovetox

    its not malicious

  284. lovetox

    it seems to link for downloads directly to other projects

  285. lovetox

    like gajim, and zom

  286. Zash

    Except the Android one

  287. lovetox

    yes

  288. lovetox

    thats a bit weird

  289. Zash

    It links to Conversations in the footer

  290. lovetox

    but i bet its conversations

  291. lovetox

    relabeld

  292. lovetox

    but yeah, i dont know that will help the cause

  293. goffi

    also here https://www.reddit.com/r/xmpp/comments/9y3ib4/can_you_use_omemo_encryption_across_two_different/ "See omemo.im and test their server and app across yours." by somebody nicknames "omemo82", it doesn't smells good to me

  294. goffi

    nicknames

  295. goffi

    nickamed*

  296. goffi

    arg

  297. goffi

    nicknamed*

  298. Zash

    Search for omemo.im on github turns up https://github.com/froghorn82/omemo.im

  299. vanitasvitae

    THe commit history is... interesting 😀

  300. lovetox

    dont understand why people dont try to do something productive

  301. Zash

    Procrastinating makes the world go round ... later.