jdev - 2019-09-06


  1. marc0s

    by any chance, do some of you will attend januscon.it later this month? I'm asking because as I read some emails/discussions around jingle/call-related stuff... Wondering if we can meet and have some drinks if that's the case :)

  2. Daniel

    marc0s, no. but let me know in case there is ever a more open source focused conference in italy. i'd love to go, maybe even give a talk

  3. marc0s

    Daniel, no that I'm aware of right now. Will ping you if I know of something. Not living in Italy, though :)

  4. marc0s

    i'm on the process of adding XEP-0402 support to stanzajs and was wondering how a client should actually determine how to store its bookmarks given the server supports both private storage and pep/pubsub. Any ideas/comments/warnings? :)

  5. Daniel

    xep 402 is only specified to work with pep

  6. Daniel

    but pep and publish options as well as private xml have features announcments on the account disco

  7. marc0s

    but, given that disco announcements offer all the options, what should a client use?

  8. pep.

    As not many clients (none?) use 402 yet, you probably also want to support the other pep bookmark thing?

  9. pep.

    (48?)

  10. Daniel

    what i do (but that's only for bookmarks 1); is to check if my server supports the conversion xep and if it does i use pep because it's more efficient. and if not i publish to private xml because that's more widely used

  11. marc0s

    that would be a safe route, yes

  12. Daniel

    i haven’t found a strategy to incorperate bookmarks 2 into the mix

  13. Daniel

    probably have the conversion xep also convert into bookmarks 2

  14. Zash

    Bookmarks Conversion 2: The seriousening

  15. marc0s

    I'm not fully aware of the XEP processes, but it does not sound crazy to me to make 411 take 402 into account

  16. Ge0rG

    I think that 402 should mandate backend-side conversion between all the stores.

  17. Daniel

    402 still lacks a lot of things

  18. Daniel

    it should probably also mention that the node needs to be configured

  19. marc0s

    should we then need Bookmarks 3: The Good One :)

  20. Zash

    Like I mentioned the other day, node item count limits will be fun

  21. Ge0rG

    Bookmarks: Revolution

  22. marc0s

    Zash, yep :)

  23. Zash

    XEP-0927: Bookmarks 2000: This time we finally got it right!

  24. lovetox

    im not convinced on 402, i think it makes the bookmark implementation alot more complex

  25. lovetox

    right now i request on start my bookmarks, i get all, and if another device changes one, i also get all, and thats it there are only these 2 things to think aobut, request, and notification

  26. lovetox

    with a items based approach, you suddenly have to think about stuff like, what if another device deletes on item? do i get a deletion notification once i come online? no .. what if a device adds 2 items while im offline, do i get 2 items when i come online? no .., so im back to requesting all bookmarks items on start anyway, this time with a more overhead as each bookmark is in its own item

  27. jonas’

    lovetox, and with one item, you have to think about: what if another device modifies/adds the same item at the same time, e.g. in response to an invite or something

  28. jonas’

    what if the modifications aren’t identical

  29. lovetox

    so it seems the only benefit is, that if a device adds a bookmark while im online, i get only one item instead of all

  30. jonas’

    or rather, what if two devices concurrently edit two different items

  31. lovetox

    i would consider this if i modify my bookmarks 50 times per hour

  32. lovetox

    but realisticly its probably 3 times a day

  33. lovetox

    jonas’, this is highly unlikely, server processes events in order

  34. lovetox

    invite means both clients modify the same item in the same way

  35. Daniel

    lovetox, i ran into problems with deleting multiple bookmarks in quick sucession

  36. Daniel

    meaning delete the second one while the first one is still in flight

  37. lovetox

    yes Daniel i can see the problem, especially with ejabberd

  38. lovetox

    as it notifys the issuing device with a pep notification

  39. lovetox

    if you take this notification serious, you add back the bookmark that you just deleted

  40. Daniel

    yes

  41. lovetox

    we should fix that in ejabberd though, prosody doesnt do this

  42. Daniel

    i'm not sure that ejabberd is broken in that regard?

  43. lovetox

    yes, if i issue a publish, and i get a result that it was ok

  44. lovetox

    i dont see a reason why i need a pep notification

  45. lovetox

    its not "broken"

  46. lovetox

    its just useless and leads to problems as we can see

  47. jonas’

    lovetox, yes, server processes events in order, but clients have latency to the server

  48. Daniel

    useless maybe. but i'm not willing to by that this is the cause of the problem

  49. Daniel

    this is just what makes you notice the problem

  50. Daniel

    *to buy

  51. lovetox

    i feel we just exchange some kind of problems with other kind of problems with 402

  52. Daniel

    i mean this is just the most obvious race. but as jonas’ pointed out there are other (unlikely?) races in there as well that involve multiple clients

  53. Daniel

    lovetox, what problems do you see with 402 aside from the upgrade path

  54. lovetox

    as i wrote above, it just mentions the benefit, that you can modify one item at a time, but it should have much more on implementation notes, probably because no one implemented it yet

  55. lovetox

    stuff like, if you start, you get the last bookmark item that was published

  56. lovetox

    probably should ignore it, until you requested all bookmarks

  57. lovetox

    stuff like node configuration

  58. Daniel

    lovetox, yes bookmarks 2 is not done yet. and you can just configure the node to not send the last item

  59. Daniel

    which i agree the xep should do

  60. lovetox

    whith what id do we publish, or does the server choose ids

  61. lovetox

    how do we make sure we dont overwrite items

  62. Daniel

    lovetox, the id is the jid. i think the xep says that

  63. lovetox

    ah kk, what i want to say is, i dont see a big problem with the xep, just it obviously was never implemented

  64. lovetox

    and my problems with bookmarks1 are not that big, that i jump into the cold water :)

  65. Daniel

    also the XEP needs to do something about max items

  66. Daniel

    so there are things in the xep that are underspecified a bit

  67. Daniel

    but fixing the race seems to be worth while to me

  68. Daniel

    also; if you ever wanted to do something like shared bookmarks on the server side (which customers ask about all the time) having multiple items that the server can modify or inject without editing xml seems like a big benefit to me as well

  69. guus.der.kinderen

    > also; if you ever wanted to do something like shared bookmarks on the server side (which customers ask about all the time) having multiple items that the server can modify or inject without editing xml seems like a big benefit to me as well Customers ask for this? As in, read-only bookmarks, shared be a group of people?

  70. Daniel

    guus.der.kinderen: well the ask for can we put people into group chats by default

  71. Daniel

    Like when they first open the app

  72. Daniel

    And bookmarks seems like one way of doing that

  73. guus.der.kinderen

    Openfire has a plugin for that. It doesn't do anything fancy, only injects additional bookmarks in a persons bookmarks collection.

  74. guus.der.kinderen

    Daniel: yeah that's what we use it for, by adding autojoin bookmarks

  75. Daniel

    guus.der.kinderen: yes and server side that seems less messy with bookmarks 2

  76. guus.der.kinderen

    It's pretty clean in any form. You simply add a list of shared entries to the personal list, and subtract that list while editing.

  77. Daniel

    You could then properly reject the deletion. Instead of having it just magically Reeapear

  78. guus.der.kinderen

    Right

  79. guus.der.kinderen

    Ok, gotta put the kid to bed. Afk.

  80. lovetox

    Daniel, do you send always set SNI ext, even for starttls?

  81. Daniel

    lovetox: why do you ask? I think I didn't but my last refractor yesterday might have accidentally set it

  82. lovetox

    im asking because i contemplate doing this

  83. lovetox

    gmail xmpp server mandates it

  84. lovetox

    it needs sni even on starttls

  85. lovetox

    and it would make my code less complex

  86. Daniel

    Oh right. Yes now that you mention it I think I did that

  87. lovetox

    i dont really care about gmail

  88. Daniel

    Well since yesterday my setup tls socket code is the same for starttls, direct tls and tor

  89. Daniel

    So it's not more code. Is what I wanted to say

  90. Zash

    I've been trying to make Prosodys certificate and TLS management code treat STARTTLS and TLS+SNI the same.

  91. moparisthebest

    Daniel, how are you doing DNS for tor ?

  92. Daniel

    moparisthebest: not at all. You have to specify the hostname

  93. Daniel

    (if your server doesn't a record to the same machine)

  94. moparisthebest

    hmm, then how do you know direct TLS or STARTTLS

  95. lovetox

    you dont

  96. lovetox

    you expect the server to offer stuff on the standard ports

  97. Daniel

    You can enter port 5223 or 443 and then it will assume that this is direct tls

  98. Daniel

    Which is debatable for 443 but who cares

  99. moparisthebest

    when you run a tor exit node you get to pick what outgoing ports you support, I feel like more might support 443, but I'm not sure

  100. lovetox

    also moparisthebest some servers have .onion adresses

  101. moparisthebest

    which you can put in DNS SRV records

  102. lovetox

    .onion adresses have DNS records?

  103. Daniel

    Yes. I was about to say. If you are serious about tor I'd recommend you put in the onion address in the hostname field

  104. Daniel

    That's what I would recommend to my users

  105. moparisthebest

    lovetox, no, but I can put a .onion in the SRV record for moparisthebest.com for example

  106. lovetox

    how does that help someone that wants connect to a server and only have the .onion adress?

  107. lovetox

    why would a hidden tor service, link itself to a non-hidden srv domain record

  108. moparisthebest

    why not?

  109. lovetox

    because you are not anonym anymore then

  110. Zash

    moparisthebest: someone like that would care about leaking the SRV lookup

  111. moparisthebest

    don't leak it, look it up over TOR

  112. lovetox

    anyway, to support TOR the server admin has to be aware of it

  113. Daniel

    How does that help when you can't do SRV over Tor?

  114. lovetox

    and a onion service even more so

  115. Daniel

    I mean even if you put the onion in dns how are you going to discover it?

  116. Ge0rG

    DoH to the rescue!

  117. Daniel

    No

  118. moparisthebest

    you can, do DoT or DoH to 1.1.1.1 or even regular DNS over TCP to port 53 of dns4torpnlfs2ifuz2s2yf3fc7rdmsbhm6rw75euj35pac6ap25zgqad.onion

  119. moparisthebest

    (which cloudflare runs)

  120. Daniel

    So I should hard code cloud flare ips in my app? Is cloud flare still going to exist tomorrow?

  121. moparisthebest

    maybe, you can hard-code a few, or run your own

  122. lovetox

    moparisthebest, if you are serious about TOR, you will *not* DNS anything

  123. lovetox

    you will just pass the onion adress to the tor proxy

  124. lovetox

    thats it

  125. lovetox

    the moment you involve cloudflare, this degrades TOR seriously

  126. Daniel

    Yeah if you want to use Tor take 3 minutes to figure out the histename / onion address of your server

  127. moparisthebest

    wouldn't it be nice if a user of regularservice.com that happened to have tor could just connect over it automatically without typing .onion addresses though?

  128. Daniel

    And maybe go read the privacy policy of your provider while you are at it

  129. lovetox

    moparisthebest, i dont think you get the idea of TOR

  130. lovetox

    the idea of TOR is that nobody but you yourself and your machine, knows where you gonna connect to

  131. lovetox

    this rules out asking anyone for any information regarding your connection target

  132. lovetox

    because then you leaked your intent

  133. moparisthebest

    I don't think so, after all HTTPS over tor asks for A records right?

  134. moparisthebest

    how is this different?

  135. lovetox

    im not a tor expert but im pretty sure the tor network makes the dns request

  136. lovetox

    not your machine

  137. moparisthebest

    the intent is my ISP doesn't know where I'm connecting to

  138. moparisthebest

    the built in tor DNS supports A and CNAME and nothing else though, asking an .onion address for SRV records is essentially the same

  139. lovetox

    Tor is not only to hide your intent from your ISP

  140. moparisthebest

    it *can* only be for that though?

  141. lovetox

    if that would be the case you would not need tor, just DoH

  142. lovetox

    and a proxy

  143. moparisthebest

    that sounds harder than tor, and also not as secure/the same

  144. lovetox

    if you make a dns request via tor, probably it routes it through the tor network

  145. lovetox

    means nobody in theory can trace it back to you

  146. lovetox

    not even cloudflare

  147. lovetox

    and thats the goal

  148. lovetox

    not exchaning your ISP for cloudflare trustwise

  149. lovetox

    its you trust no one

  150. moparisthebest

    and if you ask cloudflare's .onion for a SRV record they also can't trace it to you, right?

  151. moparisthebest

    in fact it never even crosses the clearnet for anyone

  152. lovetox

    yes, if we could ask a SRV record over the tor proxy this would work

  153. lovetox

    but it doesnt, because TOR just does not support SRV

  154. moparisthebest

    but asking an .onion is asking over the tor proxy

  155. Daniel

    lovetox: it does. If you ask cloud flare over tcp

  156. lovetox

    you propose to do the dns request yourself

  157. Daniel

    Yes

  158. lovetox

    yes that would work, never done anything like that though, so dont know how complex this is

  159. moparisthebest

    yes, just like DNS-over-TLS, DNS-over-HTTPS, and DNS-over-XMPP proposes

  160. lovetox

    but sounds complicated

  161. Daniel

    I'm already dining my normal dns requests myself

  162. Daniel

    I'm already doing my normal dns requests myself

  163. lovetox

    i mean there are libraries and dns lookup tools

  164. moparisthebest

    it's annoying enough that you should probably just use a library

  165. lovetox

    you cant use them, so you have to implement the whole dns request protocl yourself

  166. Daniel

    It's not rocket science. But I won't bother any time soon

  167. Daniel

    Because > Yeah if you want to use Tor take 3 minutes to figure out the histename / onion address of your server

  168. moparisthebest

    asking an .onion should be as quick as possible, it's not going through any exit nodes anyhow

  169. Daniel

    And > So I should hard code cloud flare ips in my app? Is cloud flare still going to exist tomorrow?

  170. lovetox

    moparisthebest, its not about quick, its about implementing another protocol

  171. moparisthebest

    time to expose an unbound port over a tor hidden address ran by conversations.im :D

  172. moparisthebest

    wait why can't you use existing dns lookup libraries/tools lovetox ?

  173. moparisthebest

    conversations uses minidns or something if I recall

  174. lovetox

    i just doubt they let you use them over a tor proxy

  175. lovetox

    but never used one

  176. moparisthebest

    it's just a socks5 proxy, they should...

  177. lovetox

    i know that i cant use python inbuilt one

  178. Daniel

    moparisthebest: fwiw minidns doesn't support dns over TLS or https

  179. moparisthebest

    you can just do regular TCP on an .onion though, or I can put in a PR to swap minidns out for https://github.com/moparisthebest/jDnsProxy lol (not really)

  180. lovetox

    moparisthebest, maybe you missed it, i already agreed with you that it is possible

  181. lovetox

    but as Daniel said, the people who want to use TOR are 1% of the users

  182. lovetox

    and they can take the 2 minutes to get the onion adress

  183. lovetox

    im not going to jump through hoops programming wise to save them those 2 minutes

  184. moparisthebest

    that's fair, I'd kind of like it to be seamless to have regular users connect over tor too, but other people probably disagree

  185. Daniel

    We are also only talking about the subset of tor users on providers that don't listen on the a record

  186. moparisthebest

    FYI this is the cloudflare .onion reference https://developers.cloudflare.com/1.1.1.1/fun-stuff/dns-over-tor/

  187. moparisthebest

    I'll be running a public, anonymous-login-supporting DNS-over-XMPP on clearnet and .onion whenever I get back around to finishing setting it up...

  188. 💋ᵐyᵃᵇᵃᵇᵉᶻ💋

    ?

  189. tom

    whoever 'jdev@muc.xmpp.org/💋ᵐyᵃᵇᵃᵇᵉᶻ💋' is please change your nic

  190. tom

    it's making my software freak out

  191. tom

    how did you even join this muc with that nick? It should be invalid

  192. Zash

    Why?

  193. tom

    because it's using invalid characters or encoding

  194. Zash

    It's UTF-8, but there are barely any other limits

  195. mathieui

    it’s valid

  196. mathieui

    ant it works here

  197. mathieui

    and it works here

  198. Zash

    Valid UTF'8 that passes resourceprep and isn't entirely whitespace, so legal under those rules.

  199. Zash

    Altho, it does not pass Prosodys resourceprep if I recompile it without USPREP_ALLOW_UNASSIGNED, but I think it's using Unicode from 1997 or something then.