XSF Discussion - 2018-03-04

wait icq is still useable?

Wait Telegram does everything right?

Should I even bother reading the rest?

jonasw, after re-reading xep390 I wonder if the ASCII separators are valid in the separated strings itself. I know it likely qualifies as nit, but if they are valid then couldn't one could generate indistinguishable feature sets?

flow: they are forbidden in xml if that answers the question

daniel, probably does, thanks

although https://www.w3.org/TR/xml11/#NT-Char only discourages those

OK let me rephrase. It was the intention of the xep to use something that is forbidden in xml or xmpp (can remember which). I didn't actually check that myself. Maybe jonasw messed up 😀

Na, it's probably a XML 1.0 vs 1.1 thingy

in 1.1 the separators appear to be valid but discouraged, and in 1.0 they are invalid

fun

why do we have that in 0313

These IDs are strings that servers may construct in any manner, and clients must treat as opaque strings (e.g. there is no requirement for them to be numeric, sequenced or GUIDs).

is it not the most natural thing for a database to have a numeric index

lovetox, might be since you have to query for timestamp and interface with different DBs it might look different

SaltyBones, 1. you dont have to query with a timestamp

2. this has nothing to do with why it would be some bad constraint for the server to have a numeric index as ID

i bet every server has

lovetox, http://logs.xmpp.org/xsf/2018-03-01/ 16:50

lovetox, according to MattJ Prosody alone uses multiple formats.

i know that servers use different formats, its just not clear why

there is no faster search in an archive as with a numeric index

also i see no need to encode other informations into the index if i have a row with columns that can hold these information

as i see it using a numeric sequential index as id, would open up more possibilites with MAM, and i dont see the downside for the server

Which possibilities

it currently makes no sense to request a certain id from an archive

because you dont know where to place it

because there is no order information in the ID

lovetox: it makes sense for ranged requests. Give me the first 50 messages after X. Then you'll have a new last ID of Y, and ask for the next 50 after Y

thats how we use MAM now yes

this would not change

it doesnt depend on the ID beeing numeric or not, this works with any kind of ID

and with numeric i mean sequenced

lovetox: do you see any benefit in enforcing a certain implementation optimization into the XEP?

flow, all XML parsers I tried rejected them even in entity form

so it would at least be massively unsafe to use them in XMPP and get you disconnected very likely

yep, because they are disallowed in XML 1.0 (but not 1.1)

jonasw ^

flow, yeah...

RFC 6120 depends on XML 1.0 though

I wonder if xep30 should specify string preperation for the various values

jonasw, I'd love to add some rationale to xep390 explaining the situation. What do you think?

flow, like this? https://xmpp.org/extensions/xep-0390.html#security-separators

one could add a mention that RFC 6120 bases on XML 1.0 and not 1.1

jonasw, what about merging my PRs for 401?

can do that later today I guess

remind me tonight maybe

okay

At least Ge0rG and Kev have no objections anymore :)

Haha looking for backup solution and see jonasw commenting, do you still use Borg? :)

moparisthebest, yes

Btrfs send/receive solves all my problems except on this one machine that is ext4 :( Borg looks like a good candidate though

moparisthebest: btrfs will solve all your problems by corrupting your data and laughing in your face.

hah

Ge0rG: my years of using it everywhere say otherwise, so far I've had corrupt (restore from backup) ext4 twice and only partially broken (still readable) btrfs once

All those failures happened on a machine with a bad PSU that caused them btw

moparisthebest: btrfs has some nice checksumming features, but they tend to fail on EMEM and have a huge scary warning instead of just fixing their RAID: https://btrfs.wiki.kernel.org/index.php/RAID56

jonasw: thanks for #585

jonasw, thanks for merging

Oh yea 5/6 has never been useable, 0/1/10 has been solid for years though

moparisthebest: not if you want to have 16TB of data on a 2GB RAM machine.

I have a 32tb array on a 16gb machine and it uses hardly any ram

I can't imagine running a server with 2gb ram

All my servers have 2GB RAM

Most of my servers have 1 GB of ram…

Giant storage servers?

No, I don't remember how much is in my big storage server. It's probably 2 or 4.

I mean if you are saying btrfs uses more memory then ext4 then sure, Linux uses more memory than an Arduino too, it's got a few more features though

I also have never had a problem with memory due to ZFS (which is broadly similar and people use the same complaint about it), so I dispute both of these arguments (that 16 gigs of ram can be expected, but also that the filesystem uses up too much ram)

Yea I agree I don't think btrfs uses much ram

SamWhited: the usual ZFS tutorials all begin with a huge red warning that your data is already as good as gone if you don't have 16GB or more RAM

Depends how big your storage pool is; 1GB of RAM per TB of storage is a good general guess for how much space you'll want for a cache.

SamWhited: and regarding bitter btrfs experiences, I happen to have a documented case: https://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg48375.html

...with too little RAM

SamWhited: the question is actually how much *kernel memory* is needed to operate the fs, because you can't swap the kernel

I don't run swap either meh

oh yah, that's poor

Does Cisco Jabber comply with the XMPP standard?

huhu

marc: at least on cisco.com they still dont have TLS

Looks like they do on c2s?

Ge0rG, is Cisco Jabber SaaS or do they provide a server software?

Security and Cisco - what do you dream at night?

marc, [19:58:22] ‎Echo1‎: Maranda: cisco.com is running jabberd version 7.26.0.40895 on an unknown platform

haha

https://check.messaging.one/result.php?domain=cisco.com&type=client

marc, don't ask me what kind of a hack of jabberd14 that is because I don't know and I don't want to know.

jabberd14 or jabberd that is.

Maranda, according to Holger it is not related to the jabberd project all atl

Jabber XCP or something

Zash, who knows. https://www.cisco.com/c/dam/en/us/products/collateral/unified-communications/jabber-windows/jabber-xcp-retirement-message.pdf

Yeah it's no longer called XCP.

https://www.cisco.com/c/en/us/products/unified-communications/jabber/

marc: They sell the server software.

(Usually integrated with their VoIP stuff these days.)

Holger, ah okay. yes, the VoIP stuff is the reason why some people want to move from ejabberd(!) to cisco jabber ôÔ

Yup.

Because it works well with the Cisco Phones :D

:(

Yes I get how this is nice for the enterprise use case.

Unfortunately their XMPP support degraded over the past few years.

They never implemented the modern XEPs and now slowly come up with proprietary extensions to support the multi-device/mobile stuff.

Holger, does it mean that the phones will use VoIP via the jabber server or what's the use case?

marc: No it's mostly just an integrated address book for both (which can be Microsoft AD) and that you see the phone presence (or initiate calls) in your chat app. Stuff like that.

Their VoIP is just SIP.

Holger, Sounds like very boring stuff to me. What does phone presence mean?

You're currently in a call or not.

My co-workers love it.

Well the administration people.

Staff council not so much.

Hmm, something XMPP-optimized similar to https://tools.ietf.org/html/draft-ietf-acme-email-tls-02 ?

I feel I'm missing something with this MAM thread, as he seems to be asking for a whole bunch of things that MAM can already do.

Has anyone else got a better grasp on what he's asking? He asked me in private too, and I suggested he take it to the list, but it's not explained it to me any better.

If you have some random MAM id, how do you retrieve that specific message?

If you have some random MAM id, why do you need that particular message? The holes he talks about are bounded by ids, and you can request the messages between two ids.

he just finds it inefficient

Yes, that's the confusing thing. If you want the messages, then first asking for the ids and then asking for the messages is obviously inefficient.

Assuming you're getting the same ids and messages in both cases - which you seem to be able to do for all his use cases.

i guess he doenst want to load all these messages

He says he does.

instead he wants to load them later when looked at or something like that

also i think he has plotted out his client architecture without looking how the xep works first

I skimmed through the thread. His mental model is a bit different than our normal one.

His model makes sense if this was REST/HATEOS

Or in cases where IDs have some significance or order is missing

But MAM isn't syncing of a key-value store, its syncing of a log, where content matters and IDs only play a supporting role.

MAM is like git, and unlike e.g., file system sync.

How is MAM like git?

It's sequenced.

You use the content of one item to find the previous item, linked-list stylee.

Yes, and you'd notice similarities if you look at the git/hg/etc sync protocols

Where commit hashes (ids) are used more like database cursors, even if you'd otherwise think of them as database primary keys.

This is very common in log sync protocols

I'd like to think that MAM is much simpler than dvcs sync protocols

I'm not sure why. Optimisations aside, git sync is very straightforward.

(And then the optimisations and details make it all much less pleasant, naturally)

Yes, git sync is very simple at its core

I don't know how git sync works. I only read the mailing list posts where it was said to be just scp/rsync/whatever

rsync is different, and more like what that person wants

By "rsync" I mean "just transfer the files"

If it were truly a content-addressable thing, that would work

The protocol requires both sides to scan the entire dataset in rsync :)

Because there is no order

But that'd be rsyncs problem, not git

Yep, because git can have a much simpler protocol due to commit ordering, "give me everything since this hash"

a la MAM

I'd imagine it'd be "between this hash and this branch/label/named commit"

Other examples of log sync protocols include kafka's sync, db replication protocols used by mysql/postgres/mongodb/etc all of which implement a write ahead log, etc

At their core if you ignore implementation specific bells and whistles, it's the same logic

Fetching stuff from one DAG to another ought to be more complicated

What I've seen of the hg protocol seems to agree

The main compleixity beyond MAM is multiple local and remote heads (or hg branches). It's largely the client communicating what it has and what it wants up to, give_me(from_hash_list, to_hash_list), plus some protocol elements and local git config to figure out what the server has.