XSF Discussion - 2024-03-13

  1. Guus

    Openfire's MUC service will stamp a Unique&Stable stanza ID on stanzas that are flowing through it. They thus end up in MAM (and in the stanzas delivered to occupants). I'm now having a challenge to integrate with a message store that keeps its own unique stanza IDs (the client populates that store through other means than XMPP). Is there a way to make this work, preferably without mapping Openfire's ID to that of the origin store?

  2. jonas’

    use that store's ID instead?

  3. jonas’

    I think that's basically what prosody does.

  4. Guus

    Does that work? I can disable Openfire's stamping - but that would also mean that clients would only learn about the message-IDs after querying the archive

  5. Guus

    as the 'live' messages won't have them anymore.

  6. jonas’

    oh you should of course stamp the live messages with the ID from the archive

  7. jonas’

    maybe I'm not getting the situation completely

  8. Guus

    Yeah, there's my problem. That ID is now generated outside of my realm of control

  9. MattJ

    Prosody lets the storage backend determine the ID

  10. Guus

    basically, the client, when sending off a message, sends it off through XMPP, but also directly to the message repository (that's not even XMPP).

  11. MattJ

    So if you integrate with an external storage system, that's where the ID still comes from

  12. jonas’

    Guus, the answer to that specific scenario is "don'-"

  13. jonas’

    Guus, the answer to that specific scenario is "don't"

  14. jonas’

    I'd probably not set stanza IDs there *and* not offer MAM. If the client interacts with a third party store, they should do it all the way.

  15. Guus

    MattJ: does that mean that on every live message being routed, there's synchronous interaction with the message store (to obtain the message ID)?

  16. jonas’

    to store the message *and* obtain the ID, yes.

  17. MattJ


  18. MattJ

    It's synchronous from the client's perspective, obviously from the server perspective it's asynchronous (if the store supports that), e.g. we can do other things while waiting for the store to respond, or we can have multiple concurrent requests to the store in flight

  19. Guus

    sure, but the round-trip to the store will add a delay to routing the message

  20. MattJ


  21. Zash

    but you are doing that anyway?

  22. MattJ

    If that store is your source of truth for the user's archive, you can't really avoid that

  23. jonas’

    Guus, ftr, '313 requires you to add the ID to live messages if it's archived: https://xmpp.org/extensions/xep-0313.html#archives_id so either you don't add the ID && don't offer MAM, or you add the ID

  24. Guus

    jonas’ I'm reading that as 'the _archive_ must contain a XEP-359 ID' but not necessarily that this ID must be included in the live message

  25. Guus

    I dont dispute that it'd make sense to do that, I was just hoping for a loophole

  26. MattJ

    Ugh, when you think you worded something clearly

  27. MattJ

    That section is about adding the element to the message that gets delivered to the client

  28. MattJ

    It's a MUST

  29. Kev

    MUST doesn't mean you really have to, if it means changing the code though, right? :)

  30. MattJ

    Not the way I read it ;)

  31. jonas’

    something about XEP-0030 disco#info? :P

  32. Guus

    > MUST doesn't mean you really have to, if it means changing the code though, right? :) ?

  33. Kev

    I was being flippant about you trying to find ways to not implement a MUST :)

  34. Guus

    Ah ok. Well, I don't mind changing the code. In fact, I'm hoping that I get to change code, which is why I'm looking into this in the first place. :) I had only hoped that I could get some work in, without having to require changes to their architecture.

  35. singpolyma

    If you have two IDs for the stanza which someone might want to know, is this not what by= is for? Or its that you have one id but you dnt know what it is?

  36. Guus

    The latter: I don't know what the ID is, when processing the live message.

  37. Guus

    unless I start trusting a client-provided ID, which circumvents the purpose of having the server generate a stable/unique ID.

  38. singpolyma

    so you have to query the archive because you're not doing the insertion, I guess

  39. moparisthebest

    You could just switch the id generation to a uuid or so that is basically guaranteed to always be unique and then you could generate it anywhere?

  40. Guus

    You'd still need to make sure that the id used on the archived message is the same as the one that's added to the live message.

  41. Daniel

    I guess the idea is that you either instruct the backend to use a certain ID or ignore the backend ID entirely and store the generated ID alongside what ever the backend uses internally

  42. Zash

    In Prosody the storage driver is responsible for the archive-id, in order to allow e.g. database-generated primary key to be used, or the time based ID used by xmlarchive.

  43. Zash

    It does support asking to pick a particular ID, which might be used, or not.

  44. Guus

    Yup, I thought of similar constructs. The trouble is that I'd still have to somehow reference the archived message to retrieve its storage-ID from the backend. There's a bit of chicken/egg there.

  45. Guus

    This all is caused by the peculiar separation of storage vs live message paths - not something that I'd consider an omission in the standards

  46. Zash

    SQL libraries usually have some way to retrieve the ID after an INSERT, inspired by that.

  47. Guus

    So, maybe the solution lies there, as Jonas more or less suggested (either provide all functionality, or none).

  48. Guus

    Zash, true, if I'd be the one scheduling a message to be stored, that'd be fine. But in this case, that's not even the case. I'd be basically querying the storage provider again with the full stanza, asking for the ID that it generated for that.

  49. Zash

    Sounds like a weird design.

  50. Guus

    it is. This is XMPP slapped on a larger app.

  51. Guus

    Their clients are proprietary, and it's a closed system. I'm tempted to just require them to generate UUIDs, and use those.

  52. larma

    Anyone plans to attend https://digital-markets-act.ec.europa.eu/events-poolpage/meta-dma-compliance-workshop-2024-03-19_en ?

  53. moparisthebest

    Guus: the live code path always runs though right? So have it stamp a uuid and then it'll always be there if it ends up in the archive path?

  54. Guus

    moparisthebest: the archive path doesn't follow the live path in this backwards setup that I have to deal with.