jdev - 2022-11-11


  1. emdee

    Ping Syndace

  2. emdee

    Ping Ppjet6

  3. Syndace

    Present!

  4. Syndace

    Ppjet6 is pep. here

  5. emdee

    Tim I switched OMEMO on in a chat with you and immediately hit a Python bug in my client, that I caused. I'll fix it and get back to you (hang head). What's Ppjet6 handle in this group?

  6. Syndace

    I've pinged pep. :) Though he's afk for a while

  7. emdee

    afk?

  8. Syndace

    not available in chat for around an hour

  9. emdee

    No hurry; I think from what you are saying the best way forward is for me to attach to Maxime's slixmpp which I think will just drop into the code because it has more of the XMPP end, and let that codebase follow your rewrites. It also has the SSL bits I want to make sure can be required to use tlsv1.3

  10. Syndace

    Yes, if slixmpp is an option for you, that's the route I would recommend.

  11. emdee

    He has the SSL code cleanly written where I can get into the handshake and enforce the ciphers for tlsv1.3 I hope.

  12. emdee

    What I'm trying to do is dead simple; there's an ancient jabber.py file that is distributed with weechat, a goodish IRC client to dual XMPP it. The code uses the ancient xmppy, but there are only a few xmpp calls in there.

  13. emdee

    So I wanted to just replace the ancient library with a modern one, and them looks at OMEMOing it.

  14. Link Mauve

    Isn’t it qy who’s working on improving XMPP support in Weechat?

  15. emdee

    weechat may be a nice platform as it will speak IRC,XMPP, and Tox all in one client,

  16. Link Mauve

    emdee, see https://github.com/bqv/weechat-xmpp

  17. qy

    emdee: afaict the tox thing may be broken

  18. emdee

    Thanks ‎Link Mauve‎: - I'll try it out, as it's been omemoed. I had been looking at simply updating jabber.py which is is distributed with weechat and hasn't been touched it seems since 2013 https://weechat.org/files/scripts/jabber.py Only a few places to bring up to date with slixmpp and then we could feed it back to weechat so that it's distributed with weechat.

  19. emdee

    qy: I fixed it and added some things to make it work with proxies. Am testing it now

  20. Link Mauve

    emdee, you can’t distribute qy’s thing?

  21. emdee

    I can, but I don't know if weechat will. I completely missed it so there's nothing I could find on a weechat website or issues.

  22. emdee

    I figure if we update the weechat distributed jabber.el to be a modern XMPP they'll just distribute that instead.

  23. emdee

    Then add OMEMO - the webchat IRC people are barely OTR.

  24. emdee

    Given the dependencies I suspect qy’s thing will never make it into weechat's core - they even scare me :-(

  25. qy

    > emdee: > 2022-11-11 01:34 (GMT) > qy: I fixed it and added some things to make it work with proxies. Am testing it now Neat

  26. qy

    Yeah weechat doesnt distribute plugins

  27. qy

    They're bring-your-own

  28. qy

    Only scripts

  29. qy

    But last i had a working version, omemo worked

  30. qy

    Dont ask me what the ref for that was though

  31. qy

    Got bogged down in restructuring

  32. emdee

    And as jabber.el is a standard part then we could update it to a modern XMPP with tlsv1.3 and have it in their core. I'll ask pep to take a look at it with me and see if we can bang something simple out quickly. If they accept it then we can look at OMEMO later.

  33. emdee

    qy: is your "thing" able to be told to enforce tlsv1.3 only if it's talking to a v1.3 server?

  34. qy

    It's jabber.py, and it's very broken, but you do you

  35. qy

    If profanity can do it, i can do it

  36. qy

    Beyond that, im too sleepy to know

  37. moparisthebest

    > to enforce tlsv1.3 only if it's talking to a v1.3 server? Isn't that literally built into TLS for... Near a decade?

  38. moparisthebest

    You don't have to do anything special for it

  39. emdee

    qy: I'm confused I thought your "thing" was https://github.com/bqv/weechat-xmpp - in C++. Do you have a version of jabber.py?

  40. emdee

    moparisthebest‎: NO.

  41. moparisthebest

    emdee: yes, 2009 https://www.exploresecurity.com/poodle-and-the-tls_fallback_scsv-remedy/

  42. qy

    I have nothing to do with the .py

  43. emdee

    I have nothing to do with C++ :-)

  44. emdee

    moparisthebest‎: AFAIK all of the common Python libraries (urlib3 requests) don't enforce v1,3 by default from the client, and it's not that easy to make them do so - I just did one in pyOpenSSL. It's only a few lines of setting the min version and ciphers in openssl, but they are critical, as I'm seeing TLS protocol downgrade attacks on anything less than v1.3.

  45. moparisthebest

    I would complain to your distribution / python maintainers, preventing TLS downgrades has been the default in openssl since 2009

  46. moparisthebest

    Unless you are using redhat 5 what you said should not be the case

  47. emdee

    moparisthebest‎: I think you're assuming that was the only problem. I don't thing v1,3 was finalized in 2009.

  48. emdee

    Link Mauve: I just found your website and see you work on slixmpp too. It's gorgeous code.

  49. emdee

    Link Mauve: could you take a quick look at https://weechat.org/files/scripts/jabber.py and see if you think it will port to slixmpp easily?

  50. moparisthebest

    emdee: it wasn't, but that mechanism prevents TLS version fallbacks, period

  51. moparisthebest

    So you could say all TLS 1.3 libraries have prevented fallbacks from day 1

  52. moparisthebest

    It shouldn't be a thing with 1.3

  53. emdee

    Link Mauve: je pense que ca sera facile a ameliore a slixmpp n'est ce pas?

  54. emdee

    Syndace: when pep comes back could you ask him to take a quick look at https://weechat.org/files/scripts/jabber.py and see if he thinks it will port to slixmpp easily? I'll try but I'm way behind the curve with Python async.

  55. pep.

    !

  56. pep.

    Not entirely sure. I've skimmed through. Basically you'd be porting an entire client to slix. slix is even based also so you may be able to reuse some of your methods, mostly converting them to async and using slix' objects instead.. probably already quite some work

  57. Syndace

    emdee: async is an awesome concept you'll find not only in Python. I'd say it's well worth to learn :)

  58. emdee

    I know, but I've been learning python for 20 years and I'm getting tired of it :-) (It's no only in Python but it came into Python with blistering speed - a hallmark of async code :-)

  59. emdee

    pep.: thanks; I was hoping it was just a case of dropping in your calls for the xmpppy calls in about 5 different places. If it's probably already quite some work I'll back off until I have some more experience with async.

  60. emdee

    pep.: In slixmpp/xmlstream/xmlstream.py L302 you turn off ssl checking of SNI hostnames, or is that done later elsewhere?

  61. pep.

    Hmm these calls can be replaced more or less yes. You'll have to register events though still. And the client needs to be launched in an async runtime

  62. emdee

    Is there an easy way of making async code synchronous? I doubt weechat is async, so there's no advantage to having the XMPP code async (at first glance.)

  63. pep.

    I don't know about SNI, maybe Link Mauve / mathieui would know

  64. pep.

    You can asyncio.ensure_future(async_method) in a sync context and that'll launch the future onto the existing async loop and do whatever it needs to do.

  65. emdee

    So if I did that, whic is simple and straight forward, I could replace xmpppy with slixmpp and look at asyncing the code in the future? (There are lot of advantages to sync slixmpp over xmpppy I'm sure you will agree.)

  66. pep.

    I'm not exactly sure. I guess you'll have to deal with async code anyway in you jabber.py script. Just that you'll wrap it up so that weechat doesn't see it. More and more stuff in slix has async methods/helpers

  67. emdee

    OK - well I'll back off and abandon until I'm up to speed on async and de-asyncing.

  68. emdee

    Thanks for your help anyways.

  69. emdee

    self.ssl_context.check_hostname = False in slixmpp/xmlstream/xmlstream.py L302 says to me you are not checking any SSL certificates.

  70. pep.

    I'm sure we are checking certs, but I'm not well-versed in this part of the code

  71. emdee

    You code tells me you aren't and there's nothing in your tests that tells me you are.

  72. emdee

    There should be a callback to verify the certificates which I'm not seeing.

  73. emdee

    I also don't see any calls to context.set_cipher_list so I don't think you can talk tlsv1.3. Maybe queue up checking for these as todo items in your issues?

  74. pep.

    I'm sure we talk 1.3. I use poezio everyday which uses slix

  75. emdee

    Have you verified the connection after the do_handshake is using a 1.3 cipher, or are you connecting to a 1.3 server and they are argeeing to speak < 1.2?

  76. emdee

    s/< 1.2/< 1.3/

  77. mathieui

    emdee, it is "context.set_ciphers(" and the verification is done automatically by the python "ssl" module after loading cert chains and configuring it

  78. mathieui

    see also xmlstream.py:get_ssl_context

  79. emdee

    mathieui: I'm looking at the line after create_default_context(): self.ssl_context.check_hostname = False

  80. mathieui

    emdee, yes

  81. mathieui

    but the configuring is done before connecting

  82. emdee

    Where?

  83. mathieui

    in get_ssl_context

  84. mathieui

    which is called either in connect_routine or start_tls

  85. emdee

    There is no other call that sets self.ssl_context.check_hostname in the code AFAIK.

  86. emdee

    So please queue up an issue that says "test for v1.3 when connected to a v1,3 server" and find where the connection object is after the do_handshake and put a debug statement to spit out the list of ciphers. Then queue up another one to allow the client end-user to specify the cipher he wants and use the debug statment place to confirm you have it. My read of the code is that you dont even try to do v1,3, or check server certs, but I don;t know the code.

  87. emdee

    (The only real way of telling if you got 1,3 is if the cipher is one of the 3 v1,3 ciphers.)

  88. emdee

    If you are assuming it "is done automatically by the python "ssl" module" you may be decu.

  89. pep.

    connJuaN4ALiuRFP: TLS handshake complete (TLSv1.3 with TLS_AES_256_GCM_SHA384) c2s557ec01bb0b0: Client sent opening <stream:stream> to bouah.net c2s557ec01bb0b0: Sending[c2s_unauthed]: <?xml version='1.0'?> c2s557ec01bb0b0: Sent reply <stream:stream> to client c2s557ec01bb0b0: Channel binding 'tls-unique' undefined in context of TLS 1.3 c2s557ec01bb0b0: SASL mechanisms supported by handler: PLAIN, SCRAM-SHA-1 c2s557ec01bb0b0: Offering usable mechanisms: PLAIN, SCRAM-SHA-1 c2s557ec01bb0b0: Sending[c2s_unauthed]: <stream:features>

  90. pep.

    here

  91. emdee

    Bingo SCRAM-SHA-1 is v1.1 at best - v1 maybe.

  92. pep.

    Can you not just ack first that we're doing tls1.3 instead of jumping right onto another possible issue trying to be right

  93. pep.

    please

  94. emdee

    pep.: Sorry you misunderstood me - I wasn;t jumping. The client sent opening stream... said PLAIN SCRAM-SHA-1

  95. pep.

    Well I'm already on a TLS stream

  96. mathieui

    emdee, I am not sure what that has to do with TLS

  97. emdee

    That means you are speaking tlsv1 maximum, or maybe 1.1 I forget, ie 10 year old crypto.

  98. pep.

    It says right above

  99. mathieui

    those are XMPP processes

  100. mathieui

    inside the TLS stream

  101. pep.

    Granted, I may not have choosen the best part of the logs

  102. emdee

    mathieu: v1.1 = tlsv1.1 - I'm speaking about TLSv1.1 (or maybe even TLS1 which is older weaker).

  103. pep.

    But there's no v1.1 here

  104. pep.

    SCRAM-SHA-1 is a SASL mechanism

  105. pep.

    Not TLS

  106. emdee

    You XMLstream is SSL ssl.create_default_context()

  107. emdee

    The S in SASL is SSL ; SASL is an SSL with regard to ciphers and protocol versions.

  108. pep.

    The S in SASL in Simple :/

  109. pep.

    And Security.

  110. emdee

    OK the other S is the lack of Security in (in)Security Sockets Layer :-)

  111. emdee

    mathieui‎: inside the TLS stream that's not very secure and easy to tamper with. It's the ciphers that make the security and the SCRAM-SHA-1 (I know a hash but still) is considered broken now. The cipher's you want are one of these 3:

  112. emdee

    lOPENSSL_13_CIPHERS = ['TLS_AES_256_GCM_SHA384', 'TLS_CHACHA20_POLY1305_SHA256', 'TLS_AES_128_GCM_SHA256']

  113. emdee

    Notice the difference between SHA-1 and SHA384.

  114. Syndace

    > OK the other S is the lack of Security in (in)Security Sockets Layer :-) LOL

  115. Syndace

    emdee: I'm no expert but I believe you're lacking knowledge about how XMPP streams are secured

  116. Syndace

    There might be two layers of encryption where the inner layer isn't of much relevance

  117. Syndace

    Actually this is the wrong channel to discuss this

  118. Syndace

    Please head over to xsf@muc.xmpp.org where you'll find a lot of experts in TLS, SCRAM, XMPP stream encryption etc.

  119. Syndace

    Maybe we can clear things up there

  120. flow

    I just believe he confused SCRAM with something TLS related

  121. Syndace

    That's what I was thinking - SCRAM is just authentication, right? And it happens in an encrypted TLS channel?

  122. flow

    yes and usually

  123. flow

    SCRAM is on the application layer, whereas TLS is on the transport layer. and you can perform SCRAM over non-TLS connections, but that is usually not the case as the public internet uses TLS (nearly) everywhere these days

  124. emdee

    flow: I'm not talking about TLS - I'm talking about TLSv1.3 - something that's hard to do in Python.

  125. Syndace

    I think this line of pep.'s log > TLS handshake complete (TLSv1.3 with TLS_AES_256_GCM_SHA384) indicates clearly that TLSv1.3 with a secure cipher is used for stream encryption

  126. flow

    emdee, that's like saying "I'm not talking about IP, I'm talking about IPv4", the latter is a subset of the former, so by talking about the latter your are implicitly always talking about the former

  127. Zash

    What's IPv4? Did you mean 'Legacy IP' ?

  128. pep.

    The thing that's used everywhere in the world, Legacy IP

  129. emdee

    Syndace: I have the source code for knowledge. The source code, and mathieu is telling me "and the verification is done automatically by the python "ssl" module" which means the code has never been tested. ssl.create_default_context in Python 3.9 gives you a very wak TLS - TLSv1 I think. I just wrote an IRC Python client to log onto oftc.net with certificate identification where v1.3 is a requirement.

  130. Syndace

    so we have your knowledge of a source code you've looked at for a few minutes vs. pep.'s prosody log

  131. Syndace

    Not trying to be rude but I think the log doesn't lie

  132. mathieui

    emdee, ciphers is up to the client using slixmpp, really

  133. flow

    emdee, I suggest you setup an local experiment where you capture the traffic of poezio connecting to a TLS 1.3 enabled server and check with e.g., wireshark or tcpdump, which TLS version is sued

  134. Syndace

    I appreciate that you look into it though and you're right in that slix should add tests for it

  135. emdee

    By the log, do you mean what you just posted in with SHA-1?

  136. Syndace

    _SCRAM_ SHA-1

  137. Syndace

    SCRAM is the authentication thingy that is spoken inside of the TLS1.3 encrytped stream

  138. flow

    yes, SCRAM is a SASL mechanism, nothing (directly) related to TLS

  139. flow

    the important line is

  140. flow

    connJuaN4ALiuRFP: TLS handshake complete (TLSv1.3 with TLS_AES_256_GCM_SHA384)

  141. flow

    not

  142. flow

    c2s557ec01bb0b0: SASL mechanisms supported by handler: PLAIN, SCRAM-SHA-1

  143. emdee

    OK - reread what I what I just wrote carefully and understand that anything using SHA-1 is rightly considered broken and unsafe for use on the Internet. It's proof you are not even using TLSv1,2 (which was 128 bit), SHA-1 is proof of what I'm been saying.

  144. mathieui

    I give up

  145. flow

    yep, some poeople just want to always be right

  146. Syndace

    dude

  147. flow

    yep, some people just want to always be right

  148. emdee

    OK - I give up on slixmpp - reread what I wrote and get some tests together, You don't need wireshark -just learn the python ssl or pyOpenSSL code and how to get the ciphers,

  149. flow

    emdee, multiple people are telling you that the SHA-1 that you see in the logs is happening on a different layer from TLS

  150. flow

    yes SHA-1 is insecure, for certain definitions of "insecure". but just becasue SHA-1 does appear in the log is not a proof that the connection is *not* using TLS 1.3

  151. Syndace

    emdee: what about the PLAIN next to the SHA-1 thing? Doesn't that bother you as well?

  152. pep.

    Syndace, :D

  153. Syndace

    If I send you a SHA-1 hash in an OMEMO-encrypted chat, does it matter?

  154. Syndace

    It does not, because OMEMO is not broken - SHA-1 is

  155. emdee

    Sure but I'm asking about the SSL layer. The OMEMO over is I haven't ever referred to.

  156. Syndace

    On the SSL layer, SHA-1 is not used

  157. Syndace

    SCRAM is not part of the SSL layer

  158. emdee

    Where did pep.'s log > TLS handshake complete (TLSv1.3 with TLS_AES_256_GCM_SHA384)

  159. emdee

    come from?

  160. pep.

    My server

  161. Syndace

    That log line is about the SSL layer

  162. Syndace

    The following log lines are not about the SSL layer

  163. emdee

    And the client was a slixmpp client.

  164. flow

    I am pretty sure tcpdump would also indicate that TLS 1.3 is used

  165. emdee

    pep.: And the client was a slixmpp client?

  166. pep.

    emdee, it was poezio, yes it's using slix

  167. emdee

    Great - end of story - sorry I missed the log paste earlier. It;s a v1,3 cipher and I was mistaken.

  168. Zash

    flow, doesn't TLS 1.3 look just like TLS 1.2 in order to not trip up middleboxes?

  169. pep.

    emdee, fwiw, you may have pointed out an issue anyway, not this one though. We're checking it :)

  170. flow

    Zash, yes, but I somehow hoped that tcpdump would still be able to detect 1.3 being used, but that is probably not the case :(

  171. flow

    or wasn't it just "we signal 1.2 as version, but here is the 1.3 extension?" or something?

  172. Zash

    maybe? I remember it as TLS 1.3 looking like a TLS 1.2 session resumption, but maybe that's the 0RTT thing?

  173. emdee

    It's 1.2 as a protocol and specific cipher choice I think - tcpdump won;t see the difference.

  174. emdee

    pep.: the other thing to queue up, and easyish to do is the client users of the library will want to specify the ciphers that they will accept for whatever reason.

  175. pep.

    emdee, looks like it's already handled: https://lab.louiz.org/poezio/slixmpp/-/blob/master/slixmpp/xmlstream/xmlstream.py#L164

  176. emdee

    I saw that and L775 - but I don't see where a use can do anything with them.

  177. pep.

    Not sure I understand

  178. pep.

    What more do you need

  179. emdee

    A getter and setter method that advertises they're settable was what I was thinking which usually brings them up to the surface,

  180. pep.

    This property is editable, that's as good as a get/set

  181. pep.

    It's even documented!

  182. emdee

    documented where? I grepped the docs/* and didn;t find it.

  183. emdee

    I guess I'm wondering if it should come up to the level of api.py

  184. emdee

    (I'm having a hard time getting at the ciphers in urllib3 and requests so I guess it's a peeve of mine.)

  185. emdee

    (I didn't read that code as saying it's a property. Am I out-of-date = does listing them in the class preample turn them into properties?)

  186. moparisthebest

    that's good, you'd probably mess them up, as I said TLS 1.3 has *always* prevented downgrades out of the box

  187. pep.

    not sorry I didn't mean it as @property, it's just a class attribute. And you can edit it

  188. pep.

    no sorry I didn't mean it as @property, it's just a class attribute. And you can edit it

  189. emdee

    Of course - if you can reach that far down, just like anything else. But I was suggesting users will want to specify this at the top-level using the library, maybe at the api level. I tried to follow down how I could reach that instance from the top and go lost.

  190. emdee

    (I'd still queue up a todo of testing that you are actually checking the SNI name. I dug into the gajim code and find their test accepts all certificates regardless of the name on the certificate :-)

  191. emdee

    (Their test code had one return: True !)

  192. pep.

    It's accessible from ClientXMPP, it's a grandparent class

  193. emdee

    Which is accessible how from api.py?

  194. pep.

    api.py?

  195. pep.

    `from Slixmpp import ClientXMPP` and that's what you use as your entrypoint for the lib

  196. pep.

    https://slixmpp.readthedocs.io/en/latest/

  197. emdee

    OK - so you don't use slixmpp/api.py ?

  198. pep.

    No

  199. emdee

    https://slixmpp.readthedocs.io/en/latest/api/clientxmpp.html has no mention of it which is where I looked. And I'm still scratching my head over: use_ssl (bool) – Indicates if the older SSL connection method should be used. Defaults to False!

  200. pep.

    I guess that's not exactly the first thing we want to advertize. I'm happy it's not so obvious to change

  201. emdee

    Your call but you should expect people to ask for it. It's supported in the code but not easy to get at. The other part I feel I'm missing in in the openssl s_client which has the notion of --connect-to separate from target host.

  202. emdee

    This comes up with XMPP sites that support onion addresses - you have to give it a SNO name to validate against the certificate that may not be what you are connecting to ( the onion),

  203. emdee

    s/SNO/SNI/

  204. pep.

    Yeah it's there as well, I know poezio supports it. Provide `address` on connect()

  205. pep.

    It can be different from the jid you pass when instanciating ClientXMPP

  206. emdee

    It may be there but it's not at the level I as a user would find it: from the doc

  207. emdee

    connect(address=None, use_ssl=False, force_starttls=True, disable_starttls=False)

  208. pep.

    "address (Optional[Tuple[str, int]]) – A tuple containing the server’s host and port."

  209. pep.

    Right below

  210. pep.

    "When no address is given, a SRV lookup for the server will be attempted. If that fails, the server user in the JID will be used." and this as a description.

  211. pep.

    Maybe you have better wording to provide

  212. emdee

    (Pleas change the use_ssl=False !)

  213. pep.

    The name is confusing, this parameter allows using direct tls

  214. pep.

    starttls will be attempted by default, see `force_starttls=True` in the same method signature

  215. pep.

    "Indicates if the older SSL connection method should be used." I guess this could be updated

  216. pep.

    What is old is new again

  217. emdee

    No I think that misses what's needed I want to provide the address tuple but not connect to the address - I want to verify the certificate against the address, but connect to something else.

  218. pep.

    Yes

  219. emdee

    E,g in openssl it would be s_client --connect-to longonionname.onion -tls1_3 address:port

  220. moparisthebest

    why are you supplying an address and connection type manually instead of looking it up? seems very wrong

  221. pep.

    emdee, there's also `-starttls xmpp-server` if you use starttls and `-xmpphost thehost` for what you're trying to do

  222. emdee

    What's -xmpphost?

  223. pep.

    it's the domain part in the jid

  224. emdee

    I don't see it or its equivalent in https://slixmpp.readthedocs.io/en/latest/api/clientxmpp.html

  225. pep.

    That you provide to openssl s_client

  226. pep.

    foo = ClientXMPP('user@thehost', ...); foo.connect(address=('longonionname.onion', 5222), ...)

  227. emdee

    You think the connect will use the address from the ClientXMPP for the SNI, and not the address from the connect?

  228. pep.

    Try it out?

  229. emdee

    Sure - in the future - pun intended :-) I have to figure out your future siggestion to see if I can deasync the code. My complements it's gorgeous code and I really want to try using it for *something*. I looked at the xmpppy code on sourceforge after a full meal and almost lost it.

  230. emdee

    ‎Zash‎: flow: come to think of it there maybe a difference between v1.2 and v1.3 at the tcpdump/wireshark level. I seem to remember tha there is an information leak with v1.2 so that the certificates are sent in the clear, so the information of who you are connecting to is visible with 1.2 but not 1.3.