XSF Discussion - 2019-07-27


  1. Yagiza

    It seems I fixed everything.

  2. Yagiza

    Now OMEMO works all right in my client!

  3. Link Mauve

    “20:49:04 Zash> Actually, you can probably add tags in a custom namespace if you want”, clients will parse the namespaces into their internal structures, then serialise them back without including your extensions, sorry.

  4. Link Mauve

    If you want do extend your bookmarks, you’ll have to use another private PEP node.

  5. jonas’

    unless bookmarks 2 or something specifies that stuff needs to be kept :)

  6. Link Mauve

    jonas’, as far as current implementations are concerned, Bookmarks 2 doesn’t exist.

  7. Ge0rG

    jonas’: keeping unknown elements in a structure that you have to parse is a nasty cross layer challenge.

  8. lovetox

    yeah and i would rather not rely on other clients not overwritting my custom elements

  9. lovetox

    Using a private pep node for that, you dont need to change a xep, you dont need to rely on other clients, you can implement it right now, and it will just work

  10. Zash

    Where do I send patches for the website?

  11. Zash

    editor@xmpp.org or somesuch?

  12. Link Mauve

    Zash, that should be fine, or you could use GitHub and send a pull request.

  13. Link Mauve

    Editor will be the one merging it in the end.

  14. Zash

    I don't feel like interacting with github today, and I strongly believe that it should not be required.

  15. Link Mauve

    Especially now that it’s impossible for people living in countries the USA considers as enemies.

  16. Zash

    Well the XSF is incorporated in the US so that point might be moot :/

  17. Link Mauve

    The XSF isn’t enforcing that ban on any of its infrastructures, AFAIK.

  18. Link Mauve

    GitHub is, since a few days ago.

  19. Zash

    Maybe this merely reminded me to exercise the non-github patch submission path :)

  20. Link Mauve

    :)

  21. Ge0rG

    Zash: how do you obtain a copy of the repository without interacting with github?

  22. Zash

    Ge0rG: Magic already existing local clone.

  23. Zash

    https://xmpp.org/about/technology-overview.html#pubsub has a broken link to what I assume was a local render of https://tools.ietf.org/html/draft-saintandre-atompub-notify-07

  24. Zash

    What's the relation of that and XEP-0277 (or is it 227? I always mix those up)

  25. eevvoor

    hope we can use salut a toi one day to send patches via xmpp :)

  26. Ge0rG

    Store your code in PubSub, not in GitHub!

  27. Link Mauve

    I just got into an argument with someone arguing that XMPP shouldn’t be able to send/store files, even though that was just HTTP. :(

  28. Ge0rG

    XMPP shouldn't be using HTTP for sending files.

  29. Link Mauve

    Agree.

  30. Link Mauve

    Agreed.

  31. lovetox

    hm if set stream lang to german

  32. lovetox

    i should not have to add this to presences if i join a muc or?

  33. Zash

    In theory

  34. Zash

    Which server?

  35. lovetox

    prosody

  36. lovetox

    Zash what does this mean if i set xml:lang=de, and prosody answers me with xml:lang=en

  37. lovetox

    is the lang set on each direction of the stream separate?

  38. Zash

    I don't know. Prosody doesn't do anything with xml:lang

  39. Link Mauve

    lovetox, yes, each direction is separate.

  40. lovetox

    it does, because it sets it to en :D

  41. lovetox

    Zash i hope it adds whatever xml:lang i set to whatever stanza it routes to other servers

  42. Zash

    Nope

  43. Zash

    It does *nothing* with xml:lang

  44. lovetox

    ...

  45. Link Mauve

    But yes, the server SHOULD (IIRC) add the stream’s @xml:lang on each outgoing stanza if you haven’t set one already.

  46. Zash

    Except if it's set on a presence stanza that creates a MUC it uses that as default language in the MUC config

  47. lovetox

    Is there a particular reason prosody ignores stream xml:lang attr?

  48. Link Mauve

    Ah no, it’s not a SHOULD it’s a MAY.

  49. Link Mauve

    “If the initiating entity included the 'xml:lang' attribute in its initial stream header, the receiving entity SHOULD remember that value as the default xml:lang for all stanzas sent by the initiating entity over the current stream. As described under Section 8.1.5, the initiating entity MAY include the 'xml:lang' attribute in any XML stanzas it sends over the stream.”

  50. Link Mauve

    lovetox, it’s allowed in 6120.

  51. Link Mauve

    Section 4.7.4.

  52. lovetox

    no its a SHOULD for the server

  53. Link Mauve

    Ah right, I misread.

  54. Zash

    Link Mauve, Wait inheritance is not mandated by XML itself?

  55. Link Mauve

    Zash, stanzas are sent to other entities than your server, so that wouldn’t be enough.

  56. lovetox

    Zash, im aware that prosody does not provide translations

  57. lovetox

    but other servers do, and it would be nice if prosody adds whatever stream xml:lang i set, to outgoing stanzs

  58. Zash

    Nobody has written code that does that

  59. Ge0rG

    I'm surprised that this is not a MUST in the standard.

  60. Link Mauve

    Same.

  61. Ge0rG

    I've just added xml:lang to yaxim yesterday.

  62. flow

    What would make a MUST better here?

  63. Link Mauve

    flow, clients wouldn’t have to copy it manually on each stanza.

  64. flow

    I assume we are talking about the first SHOULD in the quote link posted

  65. Link Mauve

    I wasn’t aware of this particular issue.

  66. flow

    Link Mauve, well a SHOULD is already pretty strong. I usually read it as "you have to do it, unless all invovled parties aggreed on something else"

  67. flow

    That way, you could write a simpler service implementation, use it in a, more or less closed, envrionment, where that requirement is not needed by every involved party and still claim standard compliance

  68. Ge0rG

    > If an outbound stanza generated by a client does not possess an 'xml:lang' attribute, the client's server SHOULD add an 'xml:lang' attribute whose value is that specified for the client's output stream as defined under Section 4.7.4.

  69. Ge0rG

    This is §8.1.5

  70. Link Mauve

    flow, I actually read MUSTs that way, as long as both entities agree they can do otherwise.

  71. Ge0rG

    Inheritance of xml:lang is a huge can of worms.

  72. flow

    I know that endless discussion have already take place about that topic at various venues :)

  73. flow

    Link Mauve, so what makes a SHOULD different from a MAY and from a MUST in your book?

  74. Ge0rG

    I'm sure there were.

  75. Ge0rG

    flow: prosody developers will implement MUST on request.

  76. Link Mauve

    Ge0rG, SHOULD too.

  77. Ge0rG

    Link Mauve: not in my experience.

  78. Link Mauve

    flow, hmm, SHOULD is more like, it’s possible to not implement it that way if you can’t, for a good enough reason.

  79. Link Mauve

    And MAY, do whatever you want.

  80. flow

    ack on MAY

  81. flow

    but I guess if we ask enough people we sure get two contradictory statements on that too ;)

  82. Link Mauve

    Yeah. ^^'

  83. flow

    It feels like your SHOULD interpretation is not so far away from mine

  84. flow

    so in my book, MUST is something that you can not just not follow, as otherwise some very important aspect will fall apart

  85. flow

    like "MUST be initialized with a CSPRNG"

  86. lovetox

    Ge0rG, can you give an example why this is a can of worms?

  87. flow

    of course you are free to ignore that in your implementation, but then you can't claim standard compliance and you have to live with the consequences

  88. Link Mauve

    Earlier today I was reading the XML specification, and found not respected MUSTs in minidom which are totally fine not to respect in our XMPP environment.

  89. flow

    lovetox, some XML parsers, IIRC the one on Android, will not inhert xml:lang to deeper XML levels

  90. Link Mauve

    Incidentally, I just opened https://gitlab.com/xmpp-rs/minidom-rs/issues/17

  91. Ge0rG

    lovetox: that, and your prosody issue.

  92. flow

    that means you may have to work around that if you want get the effective xml:lang

  93. flow

    note that this is usually also true for namespace prefix bindings

  94. Ge0rG

    Also displaying of the right human readable text from a message with a set of different language texts

  95. lovetox

    i have the feeling you talk about some use cases that never happen

  96. lovetox

    also i dont see why i would need my parser to do some inherit stuff with xml lang

  97. Ge0rG

    They never happen because nobody is making use of xml:lang because they never happen.

  98. lovetox

    this is about telling my server my preferred lang, i dont parse it anywhere, i only send it

  99. flow

    lovetox, well let's say there is a server which optimizes xml:lang by stripped redundant ones

  100. flow

    I think there are some more cases, but yes it obviously hasn't hurt xmpp that much in the past 20 years

  101. Ge0rG

    lovetox: the body inherits the lang from the message, which inherits it from the stream. If your xml library doesn't do that Inheritance, you need to manually traverse the hierarchy

  102. flow

    on the other hand, smack just got support for an xml environment which keeps track of xml:lang

  103. flow

    which is hopefully caused by xmpp becoming more and more used so that those rare cases where xml:lang inheritance becomes important also become important ;)

  104. lovetox

    no you really dont Ge0rG, at least not for the basic use case of getting translated messages from your server

  105. lovetox

    what you are talking about is user to user stuff

  106. lovetox

    yeah i wouldnt even start with xml lang there

  107. Ge0rG

    lovetox: I'm talking about the code needed to display the correct string to the user

  108. Ge0rG

    lovetox: let's say it's not message body but the text field from a message error

  109. Ge0rG

    Same situation

  110. flow

    Ge0rG, aren't you talking about the "here is the same error message in 20 different language strings, please select the best one" case?

  111. lovetox

    whats the problem? you dont work with inheritence there

  112. lovetox

    you just set the xml:lang on the <text>

  113. lovetox

    the server sets it in this case

  114. lovetox

    if its not there, its the lang the server told you in stream init

  115. Ge0rG

    flow: I'm talking about the smack report where no text at all is returned when the languages don't match

  116. flow

    Ge0rG, i guess that is very indirect yes to my question

  117. Ge0rG

    lovetox: you just described Inheritance

  118. Ge0rG

    flow: maybe not 20, but 2, or even just one, but in the wrong language

  119. lovetox

    why would a server return 2?

  120. lovetox

    this is already some theoretical stuff

  121. flow

    lovetox, because the stanza may not originated from the server

  122. flow

    it maybe came from a remote entity which has no knowledge about your stream's xml:lang

  123. Link Mauve

    lovetox, for instance, so you can have a nice localised message for user display, and the English version so they can report to the admin and easily look it up in the code.

  124. lovetox

    yeah still dont see the problem, one has xml:lang=en the other one "de"

  125. Link Mauve

    If I were to implement localised error messages in a server, I’d do it that way.

  126. flow

    Link Mauve, +1

  127. lovetox

    <error> can have "fi"

  128. lovetox

    it does not matter

  129. Link Mauve

    lovetox, the other one may not have it on the <text/> element, but on the stanza.

  130. lovetox

    because the directly set xml:lang on the text will always overrule parent or not?

  131. Link Mauve

    It still MUST be understood as German.

  132. Link Mauve

    lovetox, yes it will.

  133. Link Mauve

    But the @xml:lang might be present on any parent of the <text/> element, and you must inherit it in any child which doesn’t have one defined.

  134. flow

    so basically if you have: stream xml:lang=en, stanza xml:lang=de, error xml:lang not set, your parser should see the error's xml:lang as 'de' and not 'en'

  135. flow

    or, even better, your parser sould provide you mechanism to determine the effective xml:lang of the element

  136. Link Mauve

    And not "" either.

  137. Link Mauve

    And not '' either.

  138. Link Mauve

    And not '' either (which means no language set, as per XML 1.0).

  139. flow

    Link Mauve, yep, today I sadly learned that the empty string is valid for xml:lang

  140. Link Mauve

    TIL too!

  141. flow

    another fun fact: xml 1.0 mentions BCP 47 for xml:lang, xml 1.1 mentions rfc 3066

  142. flow

    now go out and find out the fun :)

  143. lovetox

    does the rfc not define one them self

  144. lovetox

    rfc says BCP 47

  145. flow

    ahh you spoiled the fun for the others, yes rfc 3066 is bcp 47

  146. Link Mauve

    :p

  147. Ge0rG

    it's not as bad as stringprep vs precis, eh?

  148. flow

    I just find it strange that this changed between xml 1.0 and 1.1, and i am curious to get the rationale behind that (if there is any)

  149. Link Mauve

    flow, isn’t BCP47 a live standard, as in it will point to whichever RFC is currently considered the one?

  150. lovetox

    either way, servers in general should get translation going, i see it working in ejabberd, its a nice thing

  151. flow

    Link Mauve, are they? but yes, that could be a reason

  152. Ge0rG

    lovetox: there should also be a translations project for the common error conditions.

  153. flow

    translate your favorite error message!

  154. Ge0rG

    This is especially important for stream errors, where the text element is an extension for the technical details, not the actual explanation

  155. wurstsalat

    > lovetox: there should also be a translations project for the common error conditions. +1

  156. lovetox

    yes also common forms

  157. Ge0rG

    Those too

  158. Link Mauve

    Yes please, Someone™ do that.

  159. Ge0rG

    The JSF should do it.

  160. Ge0rG

    Do we have a complete list of all places where the same translations are needed for all implementations?

  161. Ge0rG

    Forms, error conditions,...?

  162. Link Mauve

    Ge0rG, take a translated server’s .pot file?

  163. lovetox

    forms and error conditions, i didnt come across any other places where a server adds userfacing text

  164. Link Mauve

    lovetox, user configuration maybe?

  165. Ge0rG

    Link Mauve: there is no 1:1 mapping between conditions and server side error messages

  166. lovetox

    whats user configuration?

  167. Link Mauve

    lovetox, some servers may expose e.g. a web interface to their admins.

  168. Link Mauve

    Ge0rG, I know.

  169. Zash

    Create a repo, stick some .po files or whatever in it, pre-seed it with text from the RFC?

  170. Ge0rG

    Zash: on github!

  171. lovetox

    :D

  172. lovetox

    i think one would need some translation website, like pootle or something

  173. Zash

    Not strictly, but such a thing can be used

  174. Ge0rG

    Transiflex?

  175. Ge0rG

    Launchpad?

  176. lovetox

    transifex is pretty expensive

  177. Zash

    Are you supposed to do xml:lang inheritance yourself or can libexpat do it?

  178. Ge0rG

    Zash: yes.

  179. Zash

    .

  180. lovetox

    Why would you care as server?

  181. lovetox

    just set the attr on the top element

  182. lovetox

    and be done

  183. Zash

    lovetox: The question is "Can the XML parser library do the thing you just asked us to do?"

  184. lovetox

    setting an attribute on a stanza?

  185. Zash

    Inheriting xml:lang from the stream to child elements

  186. lovetox

    im pretty sure it can do it

  187. Zash

    I'm not sure you grasp just how deep this can of worms is

  188. Zash

    If something copies one of the child tags of the stanza, they should probably also carry the xml:lang

  189. Zash

    Ie it should work similar to xmlns, which the xml parser supposedly handles for us

  190. lovetox

    I think you overthink that

  191. lovetox

    all you have to do is, add the attr to the message, iq, presence, and leave the rest to client parsers who get the stanza

  192. lovetox

    of course only if its not already there

  193. lovetox

    and this can never be wrong, as we set it on the stream already, so it is inherited in message, iq, presence anyway

  194. lovetox

    you just make it explicit

  195. lovetox

    you dont change anything here

  196. lovetox

    and thats only because these stanzas will reach entitys which dont have the c2s stream context

  197. lovetox

    and nobody asks you should make it explicit all the way down to the last child

  198. lovetox

    only the childs of stream

  199. Link Mauve

    lovetox, think about some user sending the server an xml:lang="fr" message, the server then copies only the body into some other message, maybe to send it to the admin, or to archive, or anything.

  200. Link Mauve

    The original body didn’t contain any @xml:lang, but the copy should contain it as xml:lang="fr".

  201. lovetox

    storing and restoring messages and all of its content correctly is a general issue that does not stop at that attr

  202. Link Mauve

    xmlns, xml:lang, something else?

  203. lovetox

    yes if the server does not preserve attr on the message

  204. lovetox

    there needs to be changes to make this right

  205. Link Mauve

    lovetox, it’s not only on the message, it should be “propagated” to all children of the stanza in that case.

  206. lovetox

    but i would never go the way and add the attr on all childs only because i dont want to store it on the message

  207. lovetox

    No Link Mauve server should replicate the message exactly

  208. lovetox

    if i send a message with lang=de, i expect it to be routed that way

  209. lovetox

    and not with lang=de on all childs set

  210. lovetox

    i mean yes it would be still correct

  211. Zash

    lang=de is implied

  212. lovetox

    but weird to rewrite a stanza that way

  213. lovetox

    and you do it with jabber:client already?

  214. lovetox

    why dont you copy it into every body?

  215. Zash

    magic

  216. Zash

    It's implied

  217. Link Mauve

    lovetox, it’s not about rewriting, it’s about representing in memory or serialised.

  218. Link Mauve

    And having an easy way to access it.

  219. lovetox

    yeah Link Mauve but thats totally different problem, sorry i cant believe that xml:lang attr suddently is the big challenge of serializing

  220. lovetox

    i see that a server dev, maybe has to change or adapt his serialization process

  221. Link Mauve

    lovetox, as for “replicating”, in my example the server would only want to copy the body, not the rest of the message. No matter the way it decides to represent it, it must introduce a xml:lang="fr" on the body once copied alone in another stanza.

  222. lovetox

    and in that way, my pervious statement "just add the attr" is wrong

  223. lovetox

    but i dont think this is a problem hard to solve

  224. Zash

    Actually you have to change jabber:client to jabber:server when sending it over s2s, so that's already messy

  225. lovetox

    also this is all beside the point, users can already add xml:lang to messages, and we expect it to be routed to the destination

  226. lovetox

    are you telling me this does not work currently?

  227. Link Mauve

    lovetox, it does, but I’m talking about internal server processing (or any entity really).

  228. lovetox

    but im not proposing to change anything here

  229. Link Mauve

    If the entity wants to extract one element out of a stanza, the parent’s (parent’s (parent’s …)) @xml:lang should apply.

  230. Zash

    MUC subjects maybe?

  231. lovetox

    what im asking is, that the server sets the attr, not the client, so this change can not lead to problems

  232. Link Mauve

    Zash, yes, that’s a good example.

  233. Zash

    Tho Prosody only saves the text there

  234. Link Mauve

    It should also support setting multiple subjects, each in a different language. :)

  235. Zash

    My head hurts

  236. Link Mauve

    In xmpp-parsers I made each text element a hash map of language to value.

  237. Ge0rG

    Is xml:lang preserved by MAM?

  238. Link Mauve

    Should be.

  239. Ge0rG

    Link Mauve: and then you implemented a getOptimalLanguageValue() based on a doubly weighted list from the Accept-Language header?

  240. Ge0rG walks himself out

  241. Link Mauve

    Ge0rG, only based on a single vector, from preferred to least preferred, defaulting to no lang, or if there is none to the first one.

  242. Link Mauve

    Without any normalisation (so for instance it won’t pick fr-FR if the user prefers fr).