XMPP Service Operators - 2020-05-01


  1. Martin

    @ping trashserver.net

  2. Martin

    @ping diebesban.de

  3. Ellenor Malik

    A.

  4. Ellenor Malik

    What would cause Prosody to hog CPU?

  5. Martin

    For me it was mam hitting 10000 messages in one archive and using fine storage.

  6. Martin

    For me it was mam hitting 10000 messages in one archive and using internal file storage.

  7. Link Mauve

    For me, smacks going through many queued messages for a given user.

  8. Martin

    > What would cause Prosody to hog CPU? Something enlightening might be found in your logs.

  9. Maranda

    Martin: define "hog"

  10. Maranda has recently upped the archives cap to 23000.. Is still waiting to see the performance hit.

  11. Maranda

    @version myst.blue

  12. Echo1

    Maranda: myst.blue is running Metronome version 3.13.5 on Linux

  13. Martin

    Maranda: > Martin: define "hog" Why are you asking me?

  14. Maranda

    because you mentioned "flat files storage" in the same sentence

  15. Martin

    But the hig came from the citation so the citation is not the same sentence and you should ask Ellenor Malik

  16. Martin

    But the hog came from the citation so the citation is not the same sentence and you should ask Ellenor Malik

  17. Ellenor Malik

    GRR

  18. Maranda

    Martin, should I? Ellenor, didn't mention MAM, didn't mention flat files, didn't mention "after reaching ten thousands of messages in an archive" which is what I just raise the CAP from.... :)

  19. Maranda

    I'm more interested in numbers from that

  20. Martin

    But she brought the term "hog" into the ring, so if you ask definitions for "hog" you should ask her.

  21. Martin

    For me it was: prosody going up to 100% and being unusable (clients disconnected and couldn't log in again) pasting debug log into prosody muc I was asked if I have a huge archive. By counting my users archive it turned out it was exactly 10000, so the limit for archive messages in flat file storage. After stopping prosody, migrating to xmlarchive and restarting it was working/usable again.

  22. Maranda is puzzles by *how actually big* was that _10000_

  23. Maranda is puzzled by *how actually big* was that _10000_

  24. Martin

    Afair it was something between 10 and 20 MB

  25. Martin

    du -sh /var/lib/prosody/mdosch%2ede/archive/ 21M /var/lib/prosody/mdosch%2ede/archive/ du -sh /var/lib/prosody/mdosch%2ede/archive/martin.list 12M /var/lib/prosody/mdosch%2ede/archive/martin.list

  26. Maranda

    -rw-r----- 1 metronome metronome 18M May 1 15:13 maranda.dat ---> I'll have to do some testing on cpu usage when Metronome loads/stores the archive into/from RAM

  27. Martin

    Don't know why but it seems when I hit the 10000 cap prosody freaked out, took 100% CPU and was not usable anymore.

  28. Martin

    Usually it idles around having almost no CPU impact.

  29. jonas’

    Martin, that sounds familiar

  30. jonas’

    verily

  31. jonas’

    cc @ Ge0rG ^

  32. Maranda

    well I think my current storage strategy with atomic writes will be fine until datastores stay within the underlying (os/)hardware IO performance range

  33. Maranda

    well I guess my current storage strategy with atomic writes will be fine until datastores stay within the underlying (os/)hardware IO performance range

  34. Maranda

    well I guess my current storage strategy with atomic read/writes will be fine until datastores stay within the underlying (os/)hardware IO performance range

  35. Ge0rG

    Martin: that issue almost killed yax.im yesterday. Apparently internal storage is not fast at all.

  36. Ge0rG

    Martin: there will be a solution to this soon, I'm sure.

  37. Maranda suspects it's not about internal storage

  38. Ge0rG

    Maranda: it is. deleting one line from internal storage requires fully rewriting the huuuuuge serialized file

  39. Martin

    Ge0rG: I was hoping that xmlstorage might help already. Maybe the structured data is easier to process.

  40. Maranda

    Ge0rG, that's not a problem here tbh, and me thinks it's more about that ".list" file there

  41. Maranda

    and that also you shouldn't keep to rewrite one huge serialized file to disk everytime just one line is popped

  42. MattJ

    Ge0rG [18:49]: > Martin: there will be a solution to this soon, I'm sure. There is no real plan to fix it, it's been known that this does not perform well since before the release, which is why we tell people not to have large archives with internal storage

  43. MattJ

    It's possible that in a future release we will merge xmlarchive or a derivative

  44. MattJ

    And maybe we can tweak e.g. removing more than one item at a time

  45. MattJ

    But rewriting how the archive store works on the stable branch is not on the cards, and would be a large project

  46. Ge0rG

    MattJ: does this affect the stable branch? I've only encountered it after going to trunk

  47. Ge0rG

    and IIRC the 10k limit is the default, and it already makes prosody almost break down ;)

  48. MattJ

    I thought it was in 0.11.x but not certain

  49. Ge0rG

    MattJ: I'm sure I'd have run into it before then.

  50. MattJ

    And the default is not to persist MAM at all actually :)

  51. MattJ

    (I don't believe in shipping software that logs peoples' messages by default)

  52. Ge0rG

    I don't believe in services that don't do everything they can to deliver messages

  53. MattJ

    mod_offline