jdev - 2024-01-02

  1. ManDay

    Why does conversations.im (monocles) not seem to use my UP distributor (ntfy)?

  2. jonas’

    I don't think that Conversations supports UP in that way.

  3. jonas’

    conversations can *provide* UP, but I don't think it can *use* UP.

  4. ManDay

    Doesn't that defeat the purpose of UP (1 distributor running for multiple apps which need it)?

  5. jonas’

    well, Conversations is supposed to be that app :)

  6. jonas’

    well, Conversations is supposed to be that distributor app :)

  7. Ge0rG

    my experience with ntfy is that it's sometimes extremely laggy, up to multple days

  8. ManDay

    I've only been using ntfy for Matrix (Fluffychat) so far and it works reliably as far as I can see; just got to make sure that battery optimizations are turned off, etc. My only minor issue with it is that the notification renders as "There are messages, open Fluffchat to check", rather than rendering the message itsself. But that may just be a missing feature in fluffychat.

  9. Ge0rG

    ManDay: probably the latter

  10. ManDay

    I don't think Conversations can serve as a generic distributor app? It doesn't offer any configuration of the sort which ntfy does

  11. Ge0rG

    ManDay: now you can get your matrix messages delivered by conversations!

  12. jonas’

    ManDay, it can.

  13. Ge0rG

    ManDay: you just set conversations as the UP service in fluffy. Nothing more to be done

  14. jonas’

    (since November or so?)

  15. jonas’

    maybe monocles doesn't have that tho

  16. ManDay

    I don't think I have to point out how horrible of a design that is? You're probably being sarcastic already

  17. Ge0rG

    jonas’: I thought it was already there half a year ago?

  18. Ge0rG

    I need to read up on UP and maybe one day implement it in yaxim

  19. jonas’

    Ge0rG, I think it was during the great fediverse exodus

  20. jonas’

    Ge0rG, I think it was during the great ~fediverse~ twitter exodus

  21. Ge0rG

    jonas’: that was November a year ago!

  22. jonas’

    Ge0rG, oh lord

  23. jonas’

    is that so

  24. Zash

    I think you have to enable it in Conversations and pick a server endpoint (it needs server support)

  25. jonas’

    shit you are right

  26. Ge0rG

    ManDay: the best design is one with the fewest configurable options

  27. ManDay

    Integrating the UP distributor into a larger chat app? That's frankly silly

  28. jonas’

    ManDay, no

  29. moparisthebest

    > I don't think I have to point out how horrible of a design that is? You're probably being sarcastic already ManDay: it's a great design, you are aware basically all push notifications are XMPP? Google, Apple, and Nintendo push for example?

  30. moparisthebest

    You already have an XMPP connection open, why not use it for everything?

  31. Zash

    XMPP itself is "push" by nature

  32. jonas’


  33. jonas’

    it is rare, but I agree with moparisthebest :D

  34. moparisthebest

    XMPP was push before push was cool in 1999

  35. ManDay

    That's okay, but the *functionality* of a UP distributor should still be contained in a distinct app rather than mangled into an app just because it uses the same protocol.

  36. ManDay

    The fact that monocles apparently isn't to par with conversations and now I can't configure it properly is just one effect this and-the-kitchen-sink-design has

  37. ManDay

    UP is an IPC protocol. Integrating it into a single app is pretty much against its design.

  38. ManDay

    ("and now you use OUR app for everything!"

  39. ManDay

    ("and now you use OUR app for everything!")

  40. moparisthebest

    ManDay: so you think a better design would be for a standalone XMPP client with a different account to send a message to conversations to connect to fetch messages?

  41. moparisthebest

    That seems...wildly inefficient?

  42. ManDay

    If by the "standalone XMPP client" you refer to a (dedicated) UP distributor app (such as ntfy), then yes. Inefficient (not wildly so) at the benefit of interoperability and maintainability.

  43. ManDay

    That's just what UP proposes; it's not my take.

  44. moparisthebest

    This is interoperable, maintainable, and efficient, look you can remove your ntfy thing

  45. ManDay

    What if I don't want to remove my ntfy thing because I think it does a better job at being an UP distributor?

  46. ManDay

    You should see the larger picture here.

  47. moparisthebest

    You'd be wrong, what would make you think so?

  48. ManDay

    "None of your business", and I say that as advocatus diaboli, not because I mean it. You should respect that that's the ideas of separating functionality and leaving the choice to the user.

  49. moparisthebest

    You can keep running ntfy and an XMPP client if you want, that's your choice

  50. ManDay

    Indeed, which again defeats the purpose of UP

  51. moparisthebest

    Right, so you should remove ntfy since it's useless

  52. ManDay

    The design of which assumes one dedicated UP distributor app running

  53. moparisthebest

    No that's incorrect

  54. ManDay

    > Right, so you should remove ntfy since it's useless And who's to say it's ntfy which is useless and not conversations's UP implementation?

  55. ManDay

    Where NTFY implemented UP distribution properly and with conversations it's just the kitchen sink.

  56. moparisthebest

    Again, all real push is implemented with XMPP and having more than one XMPP connection is silly

  57. ManDay

    You twist a good, proficient design principle (UNIX philosophy) around to gain a neglible performance benefit by having it all in the same app.

  58. moparisthebest

    Luckily UPs design accounted for this and so Conversations can do the right thing

  59. moparisthebest

    Ok thanks for your ill informed design, I'm sure you'll take that into consideration when you implement a standalone XMPP UP provider and then also the consumer in Conversations, looking forward to your PRs

  60. ManDay

    And you just assume that Conversations UP functionality (including UI, UX) is as-good as any other UP distributor?

  61. moparisthebest

    Nope, it's better because it's XMPP, that's just a fact

  62. ManDay

    I explicitly mentioned UI and UX.

  63. moparisthebest

    Why would I want my data going over someone else's server etc? That's silly, I already run an XMPP server

  64. ManDay

    What's that got to do with the choice of server? A dedicated UP distributor can be backed by whichevever server you prefer.

  65. moparisthebest

    Like I said, looking forward to you putting your code where your mouth is 😁 I'm convinced this is a far superior design but you do you

  66. pep.

    "Nope, it's better because it's XMPP, that's just a fact" that's a weird shortcut fwiw, without all context above.

  67. pep.

    "Like I said, looking forward to you putting your code where your mouth is 😁 I'm convinced this is a far superior design but you do you" also this is still as ugh as it can be

  68. pep.

    I don't even disagree with the fact that it's ok for Conversations to be an UP provider.

  69. MSavoritias (fae,ve)

    add the "unix philosophy" in that pile also /me hides

  70. ManDay

    It's about (minimal) short-term benefits vs long-term maintainability and other advantages. "Your car's engine gets hot and BBQs are hot" may make you think it would be clever to install a grillage on your engine bay rather than buying a real BBQ, too. That's just too short-sighted thinking, though.

  71. ManDay

    Although, yes, there are short term benefits. You re-use an existing XMPP connection.

  72. ManDay

    I'm facing some of the larger-picture ramifications right now; the UI for configuration and introspectability in monocles, at least, is sub par. I have no idea how this is supposed to be set up (with FluffyChat, for example). Maybe conversations upstream is ahead in that regard, but it doesn't change the fact that they incorporated functionality which would normally be maintained by a dedicated team (like ntfy upstream) into their own sourcebase, leaving the user no choice.

  73. pep.

    I guess that also depend on your usage of XMPP/Conversations. That's not really disturbing to me because that's pretty much the only thing that runs on my phone anyway.

  74. pep.

    (I haven't setup UP yet anyway, but if I did I'd probably try with Conversations)

  75. Zash

    I would note that this was most likely decided and implemented by the Conversations developer, who does not appear to be present. So take the opinions of everyone with an appropriate amount of salt. :)

  76. MSavoritias (fae,ve)

    also for how to set up UP probably the app channel would be better

  77. MSavoritias (fae,ve)

    of the app you are using that is

  78. wgreenhouse

    ManDay: fwiw https://unifiedpush.org/users/distributors/conversations/

  79. wgreenhouse

    would also apply to Monocles, and is the first-party UP documentation

  80. Zash

    wgreenhouse, I got the impression that they want the opposite of that

  81. ManDay

    wgreenhouse Thanks. "You're ready to use UP!" is as far as it goes, though. I think I need to take the rest up with FluffyChat.

  82. ManDay

    Zash "they" want the opposite of "what"? If you're saying that it's not part of their playbook to re-implement UP as XMPP after a "rewrite proxy", then absolutely. Also: "Conversations is NOT an end user application.", ...

  83. ManDay

    Like I said: This whole idea of using an XMPP chat app as a UP distributor twists the idea of UP against its original purpose and longterm goals.

  84. Zash

    As in, I thought you were after using the XMPP app with a different distributor, not as the distributor.

  85. ManDay

    That was my goal initially, yes, because I didn't know that's not how conversations is designed to be used.

  86. MattJ

    I don't think so, but I don't disagree with you in general. The missing piece of logic is simply that XMPP is more well-suited to persistent connectivity than most protocols, and not well-suited to external push notifications (though it can be, and is done on iOS)

  87. MattJ

    For most apps it's push notifications or polling

  88. MattJ

    Or they try and do a persistent connection but do it badly, because it's not trivial

  89. ManDay

    I guess it's more of an Android subject, but why would an XMPP client not be suited to use an external UP notification?

  90. ManDay

    I.e.: Not maintain any background activity (which I suppose is the point of UP, to keep battery and network usage minimal while guaranteing notifications through a centrally configured, priviledged app - the UP distributor)

  91. Zash

    No reason it couldn't work that way. Wouldn't necessarily be a win for battery and radio usage, but you would have to measure to be know for sure.

  92. MattJ

    XMPP is already optimized to keep battery and network usage minimal with persistent connections. I know it's not intuitive, but reconnecting from "cold" frequently will end up using more network than an idle connection that doesn't do anything.

  93. MattJ

    Meanwhile, push notification protocols tend to be more limited than what XMPP can convey. For example, additional context is necessary for end-to-end encrypted messages, etc.

  94. ManDay

    I think there is a fair amount of unintuitive complexities when it comes to Android's "optimizing" an app's resources. That said, while XMPP may be very well suited to maintain a connection at minimal cost, it may solely at the benefit of offloading the complexities of having to deal with Android's optimizations onto a 3rd-party app (the UP distributor) which could warrant using UP.

  95. ManDay

    > Meanwhile, push notification protocols tend to be more limited than what XMPP can convey. For example, additional context is necessary for end-to-end encrypted messages, etc. I think that's what I might be seing on Matrix, too. The fact that the notifications for encrypted rooms don't mention the message but only that there *is* a new message.

  96. MattJ

    What I wrote just now has led to two main approaches to push notifications: 1) try to cram all the necessary context into the notification (a lot of added complexity), or 2) use the notification just to wake up the XMPP app to the fact that there is a message waiting, then the app connects to XMPP and receives it normally

  97. ManDay

    Improving the UP to that end (if it isn't a deliberate design choice - perhaps they only want to notify the app and then expect the app to obtain any further information through it's own connection? I don't know) might be a more promising route, then?

  98. MattJ

    The second approach leads to more battery and network usage

  99. ManDay


  100. MattJ

    For apps that are simpler and just need to know something happened without polling, webpush/UP is a no-brainer

  101. ManDay

    I wager in my personal use case doing it the second way would *still* be an overall improvement. The polling is centralized in the UP distributor and only ever so often an actual event occurs which would cause the app to (re-)fetch the *entire* amount of information.

  102. MattJ

    The point is that there is no polling in XMPP

  103. ManDay

    I actually think that's a sound usecase. UP seems to be about optimizing all those 1000 times of just "checking for new messages", not about the 1 time of where data has to be fetched

  104. ManDay

    MattJ you mean *if* we can maintain an open TCP connection

  105. MattJ


  106. MattJ

    If you can't, UP won't help much

  107. Zash

    If you can't, then how would UP work?

  108. ManDay

    Well I think that's the very problem on Android, so, natively, a lot of apps would have to poll, and UP just centralizes that polling thereby reducing the amoung of connections. If it can work with an open connection without polling, even better, but that's not guaranteed.

  109. Zash

    Didn't UP work by having a persistent connection?

  110. Zash

    or, rather, nftfy

  111. ManDay

    I think it does if it can, but not necessarily.

  112. Zash

    Periodic polling would be kinda meh for Instant Messaging, I would think.

  113. Zash

    Centralized coordination of when to poll does seem like a good thing tho, in theory. But I got the impression that current push systems don't actually provide that.

  114. Zash

    There's some words in a XEP along the lines of "if the radio is already on, do as much work as possible"

  115. ManDay

    No, with UP, no one *but* the UP distributor polls (if necessary).

  116. ManDay

    I mean if we could rely on open connections there really wouldn't be a point of using UP, right?

  117. ManDay

    Wouldn't matter if 99 apps or 1 app kept an open connection. UP only helps with the fact that we can't rely on open connections and might have to poll; in which case 99 polls is worse than 1 poll (per duration). At least that's how I read it.

  118. MattJ

    No, push notifications were introduced because 99% apps were terrible at caring about network/battery utilization, and would just poll

  119. MSavoritias (fae,ve)

    putting that way UP does sound useless unless you can basically make sure that UP app wont be killed

  120. ManDay

    MattJ I think 99% of the apps did care but only 1% went through the effort of going through all the Android shit that was necessary to make it work reliably (and then had to rework the entirety of their network stack every other android version or so).

  121. Zash

    and Google happened to re-use the existing XMPP connection to Google Talk for that purpose :)

  122. MattJ

    ManDay: I'm talking about when they were first introduced by Google, the background restrictions were far less back then

  123. MattJ

    Google had XMPP connections open already, and they implemented push notifications on Android just how Conversations does it with UP

  124. MattJ

    It's far more convenient for most apps to rely on third party APIs and infrastructure for push

  125. Zash

    And then more and more restrictions are added until that's the only way for it to work at all

  126. ManDay

    Right, but nowadays specifically because they won't have to real with optimizations.

  127. Zash

    And then nice things are unavailable!

  128. ManDay

    So these days we isolate the complexities (and the priviledge of being unoptimized) into a single app (the distributor) and make it call out to the apps as needed.

  129. ManDay

    and said distributor would, optimially, work with an open connection but may resort to polling.

  130. MattJ


  131. ManDay

    And I just think the amount of *redundancy* which is introduced in those events where the distributor calls to the XMPP client and the latter has to *re-*fetch stuff, because the UP message wasn't sufficient is easily compensated for by the reduction of polling *if* the XMPP client had to be polling.

  132. ManDay

    If the XMPP client does not care about offloading this stuff into an UP distributor and is fancy enough to deal with all the opmizations such that it can maintain a push-only connection then of course, the UP distributor is useless.

  133. ManDay

    But that would be the same story for any app.

  134. MattJ

    But most apps don't (want to) care about all that

  135. Zash

    UP or whatever push stuff can be used to prod the XMPP app back into life if it gets disconnected or forced into some "optimized" state by the OS, so it does have its place.

  136. MattJ

    In XMPP it's practically our bread and butter for 20 years

  137. ManDay

    Fittingly, there is a German expression "take someone's butter off their bread" :D

  138. MattJ

    And from that perspective, having XMPP (if you use it) be the distributor makes a lot of sense

  139. ManDay

    Well no, that's where I disagree, because UNIX philosophy and all. XMPP clients did care about it and implemented it well, but these days it becomes the domain of a distinct app, the UP distributor!

  140. MattJ

    I love learning things like that 🙂

  141. MattJ

    If an XMPP client dev agrees that that is the way to go, despite the added complexity, then they are free to do so of course

  142. Zash

    More moving parts, more latency...

  143. MattJ

    Being a UP distributor is just a decision of the Conversations developer, not a statement that it's the only way to do things

  144. ManDay

    Why "added complexity"? You can rid your XMPP client of checking for new messages in the background.

  145. MattJ

    XMPP clients don't check for new messages, they receive them "live"

  146. ManDay

    But for that to happen they have to jump through a lot of Android hoops, don't they?

  147. ManDay

    That part could be done away with, if they'd let the UP distributor deal with it.

  148. MattJ

    No, because UP (WebPush) is a subset of what XMPP can convey, as mentioned before

  149. MattJ

    So it's complexity to translate an XMPP message (lossily) to WebPush and then back to XMPP

  150. MattJ

    And it's inefficient to reestablish the XMPP session on every incoming message if you receive more than a few messages per day

  151. Zash

    I'd like to note that last I looked, UP did not have any kind of priority indication, so I'm not sure how it could do any smart push scheduling at all.

  152. Zash

    WebPush has it I think Daniel said, but UP did not.

  153. MattJ

    That sounds familiar, yes

  154. ManDay

    In any case lets assume that WebPush and UP are both open for improvements/augmentations. So lets not use a lack of features in those to argue against more general point.

  155. MattJ


  156. ManDay

    > And it's inefficient to reestablish the XMPP session on every incoming message if you receive more than a few messages per day I get your point.

  157. ManDay

    Are we assuming that *every* message were delivered by UP? I see this coming from the Matrix context where the UP would normally happen on specific events like Mentions.

  158. ManDay

    ...then the client would [connect] and fetch other stuff as needed.

  159. MattJ

    Every message you want to be notified about. So not necessarily messages from public channels that don't mention you, etc.

  160. Zash

    That decision lives in the XMPP server currently.

  161. MattJ

    It always will, ultimately

  162. MattJ

    But I assume you're referring to configuration vs hard-coded decisions

  163. MattJ

    Siskin/Snikket iOS have some level of configuration ability for that stuff already

  164. MattJ

    Because they use push notifications as notifications, so it's more important

  165. ManDay

    So, assuming UP gets the necessary fields/functionality to convey notifications appropriately, would you see how this could help XMPP clients by taking over their background process management - such that the client only gets a foreground process, the connection may or may not remain open, depending on what Android does?

  166. MattJ


  167. Zash

    Foreground and background operation is normally exactly the same (I assume, server developer here) so I don't see what would be simpler.

  168. MattJ

    As I said earlier, it's up to developers to decide that. Personally *I* doubt it would be much help

  169. MattJ

    Because the connection logic has to be implemented anyway, for when the app is in the foreground

  170. ManDay

    Yes, in principle I agree, just from the limited insight I have into Android, that might not be as simple in practice.

  171. ManDay

    I have never done anything with networking on Android myself, just from how difficult it seems to have been to get this right for some clients gives me that impression.

  172. wgreenhouse

    what you describe (a mostly offline client except when the activity is foregrounded) is exactly how iOS forces XMPP clients to be and it's not great

  173. wgreenhouse

    the preferable option is to keep the connection alive if possible

  174. wgreenhouse

    certainly for ui/ux, probably for battery as well

  175. Zash

    If your chat protocol is normally based on polling, like Matrix, then the push notification / UP model works great.

  176. wgreenhouse

    ^ XMPP is inherently connection-based and stateful

  177. wgreenhouse

    deapite stream management and other tricks to elide this away from the underlying network connection

  178. Zash

    I wonder if we could bring back some sort of longer-polling BOSH variant...

  179. singpolyma

    with sufficient SM hibernation polling isn't out of the question, but I'm not sure it's easier/better

  180. Zash

    it'd be terrible, but given the constraints on e.g. iOS, might fit

  181. wgreenhouse

    singpolyma: yeah, that's my impressionistic impression of how Siskin/Snikket iOS does

  182. wgreenhouse

    wakes up and polls just often enough to sate SM before the session expires

  183. wgreenhouse

    or disconnects entirely if the user swipes the activity away

  184. Zash

    maybe SASL2 and FAST will save us :)

  185. singpolyma

    wgreenhouse: tjat

  186. singpolyma

    that's actually an issue with siskin/snikket ios in my view, is that they don't lean on sm much

  187. wgreenhouse

    oh you think they could do more?

  188. wgreenhouse

    (within constraints on iOS packages)

  189. singpolyma

    yes. AFAICT they usually close the connection clean and don't go into sm hibernate for example. and of course sasl2/fast will speed up the "poll" time

  190. wgreenhouse

    ah, thanks

  191. wgreenhouse

    I'm a big weirdo but have been known to use desktop (tui) xmpp clients on android by just putting them on a roaming VPN, which I've experienced staying connected for many days, with negligible battery impact

  192. wgreenhouse

    going very far in the other direction

  193. wgreenhouse

    I bring down emacs/jabber.el about as often as I reboot the phone

  194. Zash

    No IP address changes to break the TCP connections. I had SSH connections survive wifi → wired etc and live for days while disconnected as well, way back when.

  195. wgreenhouse

    exactly like this, yes

  196. moparisthebest

    > Well no, that's where I disagree, because UNIX philosophy and all. XMPP clients did care about it and implemented it well, but these days it becomes the domain of a distinct app, the UP distributor! I'm a big Unix philosophy fan, the one thing an XMPP client does well is receive push notifications, literally every message is one! So distributing them to other apps makes sense.

  197. moparisthebest

    Everyone here is telling you XMPP applications do not poll for messages, ever, so there is zero benefit to them recieving a push notification from another app. You can try to write an XMPP app from the ground up that works like this but you will be in for a lot of pain, ask an iOS developer...