XSF Discussion - 2018-10-11

  1. jonas’


  2. Maranda


  3. Ge0rG


  4. jonas’


  5. Link Mauve

    The 35c3 voucher system has started, did you get in touch with them to get some?

  6. Ge0rG

    Link Mauve: this should be for SCAM team, right?

  7. Link Mauve

    Ah, maybe.

  8. pep.

    Scam or not maybe somebody can send something, so we're not too late

  9. jonas’

    @SCAM: 35c3-tickets@cccv.de

  10. jonas’

    @SCAM: mailto:35c3-tickets@cccv.de

  11. jonas’

    Ge0rG, Link Mauve, ^ I don’t konw if SCAM has a MUC or something. forward it to them.

  12. MattJ


  13. jonas’

    (something something too many tabs something)

  14. MattJ


  15. jonas’

    hah, relevant backlog in that muc

  16. MattJ


  17. nyco

    yep ;-)

  18. nyco

    who chairs? who minutes?

  19. guusdk

    Go for it

  20. MattJ

    I'll chair if someone else can minute

  21. guusdk

    I'll do afterwards

  22. nyco


  23. MattJ


  24. MattJ

    0) Role call: nyco, guusdk and myself

  25. MattJ

    1) Topics for agenda

  26. MattJ

    Anyone have anything not on Trello?

  27. guusdk


  28. nyco


  29. MattJ

    2) Decisions: board priorities

  30. nyco

    last time we talked prios, didn't have time to talk executive

  31. nyco

    oh ok

  32. MattJ

    Didn't we decide on this already?

  33. nyco

    according to the minutes, we did not decide (we were not meant to)

  34. MattJ

    Gah, browser with Trello and minutes just crashed, one moment

  35. nyco

    we agreed we would not take decisions for the next board

  36. MattJ


  37. MattJ

    So are we done with this for now?

  38. nyco

    that's my feeling

  39. guusdk

    Nyco, didn't you offer to draft a nice recommendation for next board?

  40. nyco


  41. MattJ

    Moving on...

  42. guusdk

    I'm not trying to make nyco donor, btw. Though he volunteered

  43. guusdk

    Donor/do it

  44. nyco

    that's fine, I'm happy to contribute this

  45. MattJ

    3) Commitment for the week (ha!) ahead: search for new ED

  46. nyco

    (which week? ;-) )

  47. MattJ

    I don't think we should be expecting Martin back before the end of the term

  48. MattJ

    and this was assigned to him

  49. MattJ

    Nope, we already have one

  50. nyco

    there is a subject with Peter on sponsoring as well, right?

  51. nyco

    modifying the bylaws need some work I guess, like validate the changes by a lawyer?

  52. MattJ

    guusdk, added to Trello

  53. guusdk


  54. MattJ

    5) Discussion: Change board member tenure

  55. MattJ

    Did we come to some decision on this?

  56. guusdk

    I don't believe that there's consensus to change anything here.

  57. guusdk

    Leave as-is?

  58. MattJ

    I'll archive the card then

  59. nyco

    agree, I feel we don't want to change

  60. MattJ

    Ok, that's cleared up a few things in Trello

  61. MattJ

    We're left with our two long-running themes of the year (fundraising/financing, and ED replacement), and one new action item (call with Peter)

  62. MattJ

    6) AOB?

  63. MattJ

    I have none

  64. guusdk

    Board/council elections process was started by the Secretary

  65. MattJ

    Oh, wasn't aware of that

  66. guusdk

    Please find candidates and/or apply on wiki

  67. MattJ

    Shall do

  68. nyco


  69. guusdk

    See mail sent on October 1

  70. nyco

    @SCAM can/should we tweet that?

  71. MattJ

    Doesn't really fall under SCAM, does it? There's nothing meetup-related

  72. guusdk

    nyco: good idea

  73. guusdk

    But it should be tweeted nonetheless

  74. MattJ


  75. nyco

    oh commteam

  76. MattJ

    Yeah, sorry, I realised it was probably just a typo after I wrote my message :)

  77. nyco

    we also should tweet every call for membership

  78. MattJ

    Ok, I think we're done then

  79. MattJ

    7) Next meeting

  80. MattJ

    +1 week

  81. guusdk


  82. nyco


  83. guusdk


  84. MattJ

    8) The End

  85. MattJ

    Thanks all

  86. nyco

    thx all!

  87. guusdk


  88. MattJ


  89. MattJ

    I think I found what's up with xmpp.org

  90. MattJ

    Well, the XMPP service anyway

  91. MattJ

    Currently holds 1020 open fds, limit is 1024

  92. Zash

    It uses select???

  93. Ge0rG

    Le Sigh.

  94. MattJ


  95. Zash


  96. Ge0rG


  97. MattJ

    Ah, fun

  98. MattJ

    The majority of open fds are UDP sockets "connected" to the DNS server

  99. MattJ

    which was down earlier in the week, so I guess that constitutes a bug

  100. Zash

    https://issues.prosody.im/1170 ?

  101. MattJ

    Except this server is many versions out of date, so probably just needs an upgrade

  102. Ge0rG

    I remember having a server silently running out of FDs. It sucks.

  103. MattJ

    Oh no, it's not out of date, apparently I upgraded it at some point

  104. MattJ

    Zash, so I guess you can take NeedInfo off that issue :)

  105. Zash

    Is this even 0.10.x?

  106. MattJ


  107. Zash

    MattJ: That change isn't in 0.10.2

  108. MattJ


  109. MattJ

    Ok, service should be restored for now

  110. Ge0rG

    did you throw in some more sockets?

  111. jonas’

    prosody should just re-exec itself when it runs out of sockets /s

  112. Ge0rG

    jonas’: message to all admins and reexec

  113. Zash


  114. Zash

    And how

  115. Ge0rG

    Zash: it will close all sockets.

  116. flow

    Ge0rG why not simply monitor the open fd externally and send an (XMPP) message?

  117. flow

    *open fd count

  118. jonas’

    Ge0rG, let me tell you about my botnet opening 65535 TCP connections to your prosody instance :>

  119. jonas’

    flow, "simply" and "monitor" and "prosody" in a single statement is daring

  120. Zash

    "fd count" is painful as well

  121. flow

    jonas’, I think it could be done prosody agnostic, i.e. not specific to a particular process binary, you just monitor if your process is running against the open fd limit

  122. jonas’

    flow, picking the right process to monitor isn’t easy either

  123. Ge0rG

    flow: feel free to PR against https://issues.prosody.im/536

  124. MattJ

    Ge0rG, PR for an external script?

  125. MattJ

    I'll repeat what's already said in the issue: there is no API for asking the OS how many fds you have open

  126. MattJ

    So file an issue for Linux

  127. Zash

    I haven't even seen anyhing in the epoll API that tells you how many FDs you are watching

  128. Zash

    Easy enough with select() since you have to manually add them for every call

  129. jonas’

    open /proc/self/fd, count number of entries?

  130. Zash

    But then you're limited to sizeof(fd_set) * (bits per byte)

  131. Zash

    jonas’: That's mentioned in a comment in the issue already

  132. jonas’

    sorry :)

  133. Ge0rG

    MattJ: the external script could be part of the OS-level packaging and tooling

  134. Zash

    File an issue for systemd

  135. MattJ

    Talking about epoll/etc. is besides the point... fds != connections

  136. Ge0rG

    isn't this all about fds only?

  137. MattJ

    You suggested select/epoll in comment 6

  138. MattJ

    and I pointed out what I just pointed out (fds != connections) in comment 7

  139. Zash

    All this has happened before, and all of it will happen again, and again, and again.

  140. Zash

    What jonas’ said is comment 8

  141. MattJ

    The only way to correctly implement this is an O(N) loop, run $frequently

  142. Zash

    Can you get the size of a directory in O(less than n) in Linux?

  143. Zash

    Or does that just move the O(n) elsewhere?

  144. Ge0rG

    MattJ: you are the one bringing up connections all the time.

  145. Zash

    Or are /proc directory entries too magical for that

  146. Ge0rG

    Zash: opening /proc/self/fd requires... a free fd

  147. Zash

    Dun dun DUN

  148. Ge0rG

    I wouldn't be surpsised if sending a message to admins would also require a free fd.

  149. Ge0rG

    or multiple ones.

  150. Zash

    Might, yes

  151. Ge0rG

    Can't you just detect EMFILE from any fd you open and trigger an error condition on that?

  152. Ge0rG

    BTW, this MUC is exceptionally laggy today.

  153. Zash

    Storage might need a FD to write to eg offline storage or archive

  154. Zash

    Don't we already?

  155. Zash

    Wow, that is laggy

  156. Ge0rG

    MattJ: did prosody run out of descriptors again?

  157. Zash

    But not now?

  158. Zash


  159. Zash

    Apparently it is

  160. Zash

    17:00 now

  161. Zash

    25s lag

  162. Zash

    Catching up with something?

  163. Ge0rG

    xsf@muc.xmpp.org did not respond to ping after 10s: timeout

  164. Ge0rG shakes fist at poezio

  165. jonas’


  166. jonas’


  167. jonas’

    very fast from here

  168. Zash


  169. Ge0rG

    sometimes it's fast, and sometimes it lags. swap disk thrashing?

  170. waqas


  171. jonas’

    fun :)

  172. Zash


  173. Ge0rG

    . . , -

  174. jonas’

    fertig ist das mondgesich,.

  175. jonas’

    fertig ist das mondgesicht

  176. Zash

    u wut m8?

  177. MattJ

    Everything seems fine to me

  178. jonas’

    maybe bad internet weather?

  179. Ge0rG

    do I need to add xmpp.org to my smokeping?

  180. jonas’

    do it

  181. Ge0rG

    all is looking well on the icmp front

  182. Ge0rG

    it must be a host latency issue.

  183. MattJ

    Ah, postgres 100% CPU, user: xmppoke

  184. jonas’


  185. MattJ


  186. MattJ


  187. waqas


  188. Zash

    Is xmpp.net still broken?

  189. Zash

    Looks liket here are recent results that worked

  190. Zash

    Looks like there are recent results that worked

  191. MattJ

    `docker nice`...

  192. jonas’


  193. jonas’

    (pun intended)

  194. MattJ

    It doesn't exist

  195. MattJ

    I wish it did

  196. SamWhited

    jonas’: re my question earlier, can you expand on your decision to move 0392 to HSLuv? You claim it has widespread library support, but the ones I've found are *really* terrible, and I think there was some value in using a standard color space. HSLuv may look nice, but it has no real applications in insdustry and isn't likely to be widely supported, so I'm skeptical of that claim

  197. SamWhited

    I'm not specifically for YCbCr to be clear, I don't care what colorspace is used it just seems like there's value in using a standardized one

  198. SamWhited

    not just some random persons personal project that's not used anywhere

  199. jonas’


  200. jonas’

    so, what we did on top of YCbCr is essentially what HSLuv does on top of CIE XYZ, but worse

  201. jonas’

    so it’s a home-brew solution either way

  202. jonas’

    except that HSLuv gives better results

  203. SamWhited

    I didn't remember you making changes to YCbCr, what did you do on top of it?

  204. jonas’

    the algorithm to determine CbCr based on the angle

  205. jonas’

    or rather, based on the 16 bit input extracted from the hash

  206. SamWhited

    That's just picking a color out of the space, that seems fine; you'll have to do that no matter what space you use

  207. jonas’

    that’s what HSLuv does on top of XYZ anyways

  208. jonas’

    but the results are much more uniform than what we had with YCbCr

  209. SamWhited

    Hmm, I didn't understand that; it's an entirely new colorspace as far as I could tell, maybe I need to go reread their definition of it

  210. jonas’


  211. jonas’

    oh, it works on top of CIE LUV LCh, not XYZ

  212. jonas’

    not sure where I got the XYZ from

  213. jonas’

    it is really pretty much exactly what we did with YCbCr: use Hue as angle, cast a ray to the border, take value

  214. SamWhited

    Yah, that's not the same thing at all; it's a brand new color space, they've just defined it as a transformation of CIE LUV

  215. SamWhited

    You couldn't just pick a color from an eisting CIE LUV implementation, as far as I can tell from this

  216. jonas’

    so, old XEP-0392 was just an entirely new color space which can be defined as a transformation of YCbCr; that’s wordplay

  217. SamWhited

    I don't think that's true, these are two different things

  218. SamWhited

    If I had the set of all possible colors in YCbCr I could do you algorithm and get colors in that set (I think?)

  219. jonas’


  220. SamWhited

    If I have the set of all colors in CIE LUV I can't do 0392 with that, I have to implement HSLuv first

  221. jonas’

    no wait, I don’t understand what you’re saying

  222. SamWhited

    I'm saying that if I have a CIELuv implementation I can't implement 0392 with that; it will be different colors than if I start from an HSLuv implementation.

  223. SamWhited

    And HSLuv isn't actually a standard that anyone recognizes, and contrary to your claim doesn't have very good library support as far as I can tell

  224. jonas’

    SamWhited, I didn’t review all the implementations in http://www.hsluv.org/implementations/ but at least the Python and Java ones seem to be fine

  225. jonas’

    the C one, too

  226. SamWhited

    I reviewed a few; they were mostly terrible, unidiomatic, and unmaintained.

  227. SamWhited

    They're examples, but not actually something I'd use in prod

  228. SamWhited

    And either way it seems like there's valule in using a widely recognized standard and not something where the only support will come from a single party

  229. jonas’


  230. jonas’

    however, the results with YCbCr are not as nice as HSLuv

  231. SamWhited

    Maybe if you want those specific colors it would make sense to define 0392 in terms of CIE Luv which I think is widely standardsized and just include the math for going to HSLuv (and you could mention that if you have an HSLuv implementation you can just start from their)?

  232. jonas’

    that would make 0392 unnecessarily long

  233. SamWhited

    They look about he same to me, if anything I've gotten more gross mustard yellow since Conversations updated, but that's probably just bad luck

  234. jonas’

    the difference is in the uniformity of brightness

  235. jonas’

    lemme grab you a screenshot

  236. SamWhited

    Yah, the brightness uniformity is nice, but it doesn't outweigh the downside in my opinion; there has to be a standardized colorspace or subset of one that has uniform brightness?

  237. jonas’

    not really, unless you do exactly what HSLuv does

  238. jonas’

    maybe on top of something else

  239. jonas’

    but you end up using one of the photometric ones (XYZ, LUV, you name it), and do a solve() to determine the right color for a given hue and saturation

  240. SamWhited

    I dunno, personally if the only benefit is more uniform brightness that's nice, but doesn't make it worth using a colorspace that has a total of one implementation in a few languages and isn't actually used anywhere and that no one has any experience with

  241. SamWhited

    It doesn't seem like it would be that much more to describe how to construct this colorspace though, but maybe I'm being over optimistic, I need to look over the math and actually try to understand it

  242. SamWhited

    Since I'd have to do that to implement it anyways

  243. jonas’

    the golang implementation doesn’t look too bad to me, but I don’t know lots of go either way

  244. SamWhited

    It's bad

  245. SamWhited

    That one in particular is more or less unusably bad; there are standard interfaces for doing colors in go which it doesn't implement.

  246. SamWhited

    Also the code just isn't idiomatic

  247. SamWhited

    And is impossible to follow or reason about because of the way they've defined it as transformations through multiple other colorspaces

  248. jonas’

    color spaces are all a mess, unless you go to the photometric ones

  249. jonas’

    and converting the good photometric ones (CIE LUV in this case) to one you can use to display something on a computer is where you get the mess you save by doing stuff in photometrics

  250. SamWhited

    I get that that's a pain, but it still seems worth using an actual standard

  251. SamWhited

    Even if it doesn't look as great; YCbCr seemed fine to me personally

  252. SamWhited

    Actually, specifically looking at Go right now (I'd like to implement this in Go and Rust) I can't find a CIELUV implementation either, which suprises me

  253. jonas’

    I’m not surprised.

  254. SamWhited

    I guess that one's not all that widely used, even if it has been standardized for a while

  255. SamWhited

    So maybe starting from that wouldn't help, I haven't looked at other languages yet though