XSF Discussion - 2017-03-07

This is kind of nifty if true: https://twitter.com/Midar3/status/839059229289943041

(TL;DR — libstrophe is listed in the Nintendo Switch's open source license credits)

libstrophe. nice :)

and somebody didn't bother enough to rotate that picture.

heh

Ge0rG: you can rotate your screen, can’t you :P

jonasw: I tried, but it turned out to be attached to the laptop body.

xrandr --output $OUTPUT --rotate left

alternative version: I did rotate it, but then the desktop manager autorotated it back.

Tangentially related: I didn't realize Jack Moffitt worked for Mozilla or was in charge of Servo these days; that's fantastic. I wish he'd revamp libstrophe in Rust.

or maybe I did realize that, since I apparently follow him on a bunch of Rust stuff, but didn't realize he was the same person who wrote libstrophe.

SamWhited, similar thing with some of Cisco's original jabber devs :)

I wouldn't really expect Mozilla to do anything with XMPP.

they probably went there for non-XMPP related things

yah, I once got asked at a conference by someone on the Hello team (or whatever that short lived firefox messenger was) what the point of XMPP or using standards was; I dunno about the rest of Mozilla, but I more or less gave up on the Firefox team then and there.

Zash, although they should do more with it

SamWhited: wtf

I think his exact words were "why would anyone bother using standards?"

wtf

have they even SEEN internet explorer?

Granted, I doubt he's representative of the rest of the Firefox people given their involvement with all things web-standards related; maybe that was just the Hello team.

jonasw, come on...with web assembler you can finnally render your IE6 pages they way they are supposed to look on every platform :)

Tobias: you make me sad.

I wonder if they still remember that "The Internet" does not equal "The Web"

SamWhited: https://www.mozilla.org/en-US/about/manifesto/#principle-06

Zash: Hah, I should have just pointed him at that; thanks.

gahaha

slap ’em in the face with that manifesto

Zash, nah...just use Hello

those are the same people who didn’t fight (enough) against WebDRM

They fought a lot, they just didn't win.

Zash, or allo https://twitter.com/burnflare/status/838966485011685376 :)

I don't think it's fair to say they didn't fight enough; they were against it all the way through.

SamWhited: okay

I admit I haven’t followed it in detail, but what I saw from news coverage it didn’t seem too great.

wow, i am just now realizing how much work there is to be done

anyone here touched xml regex?

XML regex?

yes

arc: Wait for it

what the heck is XML regex

I'm about to have to flip my table, aren't I?

http://www.xmlschemareference.com/regularExpression.html

https://stackoverflow.com/a/1732454/1248008

SamWhited, oh you have some of those new flipping desks

SamWhited: relevant for tableflip: https://www.youtube.com/watch?v=eob7V_WtAVg

its a method to constrain acceptable string values within xml

Tobias: Nah, they went for the sit-stand ones, but wouldn't spring for the flipping ones

arc: seems like a regex variant used by XML?

im still reading into it, but yes. thankfully its a simplified variant

EXI uses it for strings.

Why and where?

all strings.

CH notably

without it, an unconstrained string in the schema is transmitted as an unsigned int per char representing the unicode codepoint. no UTF-8

Shouldn't most strings fit in either enums or user-provided strings with no restrictions?

oh yes. but you'd be insane to do so

Whta

No UTF-8?

no UTF-8

KILL IT WITH FIRE

not as far as ive found. admittedly ive just started

that was my gut reaction too. however the more i read into this, the more i understand why.

why would one want to do that?

> representing the unicode codepoint

Yup, now my desk and things are all over the floor. Saw it coming.

So, UTF-32

SamWhited: did you do it with the excellent stare of Alan Rickman?

... yes. but as i said, you'd be insane to not use the regex to restrict the character map

jonasw: No, I am nowhere near that fantastic; that was amazing.

arc: so assuming this is used for standard desktop clients, I either have to restrict what codepoints users can use or the text is blown up to factor two to four of the bytes needed with UTF-8?

unfortunetly the EXI spec doesn't go into deep detail on this, it refers to other documentation on xml regex, but it appears with bitpacked encoding you can compress it down a lot better than UTF-8

jonasw: nope. you can craft a method to support the entire breadth of unicode in a much tighter format than UTF-8 because you're no longer constrained to byte boundaries.

mhm

Like, huffman code

i wouldn't go that far with it.

im trying to track down whether a codepoint can represent multi-character sequences now.

i would not be suprised.

unlike using DEFLATE tho, this would not be dynamic, but encoded as part of the schema.

Define "multi-character sequences?"

i mean, you could allocate the values 128-255 to represent the 127 most common words in the english language

i do not know if this is true yet or not.

Well, Hangul?

You could probably do that, you won't be able to do that for all languages though

ive only been reading into this for the past 2 hours.

Not without canonicalizing inputs first

SamWhited: but as far as I understand it, your client could choose a schema specialised for the locale you’re using

SamWhited: since the client dictates the schema, the client could adjust this per selected language for the user

wee, I understood EXI \o/

also, there's nothing stopping you from using 9 bits

im just commenting that we have a shortcoming in the XEP schemas. strings can and should be validated

also this could be extremely useful for client-side data forms validation

But let's say one of those words is "café"; is that caf + Unicode character LATIN SMALL LETTER E WITH ACUTE (U+00E9) or cafe + Unicode character COMBINING ACUTE ACCENT (U+0301)?

Are there really strings that are not logically enums, while being user controlled?

before 8:30am this morning i wasnt aware that XML regex even existed, so my understanding is still very crude, and further what subset of this applies to EXI

Unicode provides ways to do canonicalization of things like that, you'd just have to make sure you were doing it before building the string table and to any words you compare against the string table

SamWhited: *IF* this supports multi-character sequences, and not simply constraining which unicode codepoints are acceptable in regex format, then its whatever is defined in the schema. but this is entirely separate from the string table provided by EXI.

That way lies madness

Oh, I thought this had something to do with the string table. Either way, if you're searching for things, you'll need to do canonicalization too if you want to actually find things (since the same thing may be encoded differently on different machines, but be the same as far as the user is concerned)

SamWhited: yes, on the machine side this is evaluated to a unicode codepoint per character in any case.

goodbye char*

goodbye stdlib

goodnight #import "string.h"

hey, were will I get my memset from, arc? ;)

jonasw: heh

jonasw: dd in=/dev/mem of=/dev/mem start=x count=y

or was it seek=

*blink*

I’m done for today.

LOL

Zash: also, that’s memcpy, not memset.

Ok I'm now officially over G+

having the alert bubble show a new message while on a google search, and see people arguing implied consent for sexual penetration by the TSA by their choice to fly... break time.

wat

https://plus.google.com/+JohnWarthog9Hawley/posts/LEvErfQnajc

off topic much?

that's the problem with G+

it shows up in google searches for unrelated things.

i'm sure that's not standard behavior

anyway. yes im starting to suspect that the way this works, multi-character sequences can be implied, but it might be even more devious. more like a smartphone dictionary predictor

if you have both the letter "c" and a number of whole words that "c" could be grouped properly, you could resolve whole words in a minimum number of bits. and that can be optimized by the client in the chosen schema

arc: so basically the string is encoded by the states of a regex automaton which gets the string fed as input?

I think so.

actually i should go back to what i did in the early days with this work, grab the reference implementation and try some things on it, then read the bits

clever and devious at the same time

you might even be able to, if you are very clever, recreate UTF-8 using an XML regex.

that's not even work, that'd be pure joy for some weekend.

Wat

well remember that the top bit of UTF-8 determines whether its a 1-byte or multi-byte sequence. and if the first byte has bit 128 set, then the next byte will have the top two bits set appropriately to show a continuation, etc

if you are very very clever, and if this works the way im starting to understand, then you could build a regex that recreates UTF-8 precisely such that the string value encoded by EXI would be precisely UTF-8

such that if you encoded EXI byte-aligned, and you read the raw stream, you would find the UTF-8 encoded strings within

it might not be possible but im fairly certain it is, because the bits in UTF-8 are always meaningful, you would just have to nest your atoms appropriately.

but UTF-8 encoding is a hack for ascii backwards compatibility, i believe in almost every case you could craft a better one. which is kind of cool if you think about it, even with a limited dictionary, Zipf's Law will ensure extremely tight compression, and without the encryption concerns

https://www.youtube.com/watch?v=fCn8zs912OE

Out of all Unicode related things, UTF-8 is the last thing I'd complain about

oh im not complaining about UTF-8. i love UTF-8. but I can see now why UTF-32 was acceptable.

Why not UTF-64? Surely it'll be more efficient on modern machines ;)

heh

i think actually most uses of this would be about as fast as UTF-8 decoding

a very simple regex could be something like """[\p{BasicLatin}|(he)|(se)|(re)|(hat)|.]*"""

EXI usually follows schema semantics literally, so i would assume 3 bits would be used to determine whether its a chr(0:127), one of the four provided common word segments, or a full unicode character

"The" would be then be encoded as "000 1010100 001"