XSF Discussion - 2018-04-03

hello everyone

o/

https://blog.torproject.org/sunsetting-tor-messenger

^ *yawn* tbh

METADATA!!!1

Ge0rG stop leaking all my precious metas ™️😪

got a question for you

Marandata!

what do you think about using MAM to "discover" Pubsub nodes

\o/

edhelas, > https://blog.torproject.org/sunsetting-tor-messenger Good riddance

<iq to='pubsub.shakespeare.lit' type='set' id='juliet1'> <query xmlns='urn:xmpp:mam:2' queryid='f28' /> </iq>

without the "node" attribute, then I can paginate on those nodes

That's more about adding some result filtering option to disco#items queries (and something simpler than having to implement connection nodes) imho

s/connection/collection/

edhelas, indeed, aren't you just looking for RSM on disco#items? Which is one of the things RSM was created for

But no-one* implements, unfortunately.

I'd like to also order

[* I've not seen it in the wild, in any case]

Aww

RSM would be nice for pagination indeed

I was hoping for *except we almost have an implementation in Swift

We've got RSM in Swift, but not plumbed into disco.

*soon to be released

also filtering

but it would be the same kind of things for conferences

"give me the top conferences of this service", "conferences with the most users"

and I want only the top 50 ones

And while RSM does it for paging I'm not sure it does it to filter results by <insert something, e.g. name> as long as I didn't miss any bit in {xep 30} which does

Maranda: Service Discovery (Standards Track, Final, 2017-10-03) See: https://xmpp.org/extensions/xep-0030.html

I suppose "hierarchies"?

You could use disco nodes for the filtering, and RSM on that :)

Kev what do you mean ?

If you wanted to build filtering into disco results, you could define a way of encoding the filters in the disco node, and then use RSM in the 'usual' way for the disco results.

I don't see how you can do that

Kev, I think the problem here was having the server filter result items by say a portion of the name e.g. "urn:xmpp:microblog" and return only the matching items

Not the client retrieving everything and then filter

Indeed.

I repeat, you could just define a syntax for using nodes as a filter.

)

:)

I know that there's the pubsub#type attribute in Pubsub

I guess you could arbitrarily use type or something else, but whatever needs implementation both sides

Aka hax

Something more definite in the spec itself would be better imho (but then that would fall into the same issues)

yeah

I'm kinda lost

And we're talking about filtering the containers aka the nodes 'emselves so can't use nodes, pubsub doesn't allow for hierarchy/leaf types afair

s/hierarchy/branch/

As long as not using collections which is overly convoluted

do we have one Pubsub Collection implementation in the wild that is actuavelly used ?

ejabberd has code for it.

I'm still finishing the 0060 pubsub implementation in ejabberd

No idea whether it's actually used by anyone :-)

Holger as far as I saw, it's far from be done

Ah. No idea.

Problem here we have 3 xeps to deal with disco, pubsub and microblogging

I know it is used, but personally I think the current spec is not great. Except, maybe, the root node.

😎

Maranda it's more than Pubsub

again, how can I discover conferences on my own server without listing 10K conferences from the service ?

I think that in most cases where you might think that collection nodes are a good idea, you're probably better of with node-as-code.

I said "disco, pubsub and microblogging"...

ah yeah sorry

ralphm maybe we can deprecate Collection ?

I guess we could, but who would benefit from this exactly?

who is benefiting from it actually ?

The specification is already deferred

Collections is a bit of a mess when you end up trying to use it, so I'm not sure it's an altogether stupid suggestion.

I had a look at collection many times, also for social network usage, I never ended up to find a nice way to use AND implement i

*it

I.e. you can't go to deferred without going to Draft, first.

I think that we better design Pubsub as a flat tree behind the services, but with a good filtering/ordering/pagination system to explore the nodes

then a client that want to explore the network will be able to grab lots of interesting data quickly

and not doing recursive exploration, loading entire dump of nodes and exploring each of thems

edhelas: without XEP-0248, which is Deferred, there is only a bag of nodes, without any organization. If that aligns with your idea of a flat tree, yay.

yes

I like doing nothing and achieve results :-D

MAM got a lot of nice features, including filtering on results, pagination

too bad it's called "message archive"

because it could be used for many other use cases

like discovery

Well the pagination is 0059, it's just the filtering syntax you're missing, no?

as well

Or sorting, which MAM also doesn't have.

yup :D

I.e. "sort by number of participants/subscribers" or so.

both 3

So I don't quite buy "MAM is perfect just has the wrong name" :-P

I totally see your use case and would think this just[tm] needs a bit new syntax.

or we can revive SQL over XMPP

edhelas: I propose we use an HTTP-REST abstraction in between. OData is my favorite for that.

“11:57:15 ralphm> I.e. you can't go to deferred without going to Draft, first.”, IIRC we did that at least once, and I’d like to propose a change to the process to allow it more.

more seriously if we can achieve, in a XEP, a way to deal with those 3 things, pagination (RSM), filtering (MAM) and sorting, we can be good

We’ve been telling people to implement deferred XEPs for a while, because a lot of them are useful, yet there are a ton of useless deferred ones.

also MAM can then reuse that, I mean, in ejabberd MAM is basically querying the archives table :p

edhelas: I see the idea, I'm just not sure a generic filtering/sorting syntax really simplifies things compared to adding whatever you need to 0045/0060.

this XEP can be reused in lots of places "give me all the Holger messages, sorted by date in this chatroom, by pages of 20"

Holger yeah maybe

I'm not coming wih a proper solution here, just that I can't scale Pubsub for now

especially because of this disco issue

Maybe. I'm just saying that it's not obvious to me that this works as nicely for filtering/sorting as it does for pagination. You want different filtering/sorting fields for the different use cases.

but that's the same thing with disco overall :D XMPP is just not that good with discovery, too "raw"

indeed

Holger what about filtering/ordering by https://xmpp.org/extensions/xep-0060.html#entity-metadata

I have to do a PR in that thing as well, to expose more metadata

including pubsub#access_model and pubsub#last_update

Link Mauve: I strongly believe that if you need to encourage people to implement Deferred specifications, you should make an effort to bring it to Draft.

Going from Deferred to Deprecated doesn't make sense to me.

I agree, but that rarely happens.

Or at least not in a timely manner.

Deferred simply means: nobody's touched this in 12 months (used to be 6 I think) and nobody cares to move it forward in the process.

also you cannot ask developpers to implement a new XEP, they are implementing it if they find it valuable

XEP-0248 hasn't been touched since 2010, so I'd say it is rather dead. There doesn't seem any value in making it Deprecated.

edhelas: a new XEP is not Deferred

sorry, defered

Sure, people that have a particular use case might not care about the actual status.

We currently only have 11 Deprecated XEPs and most of them were superseded by either RFCs or new protocol. The only exception, arguably, is XHTML-IM.

edhelas, we make sure these developers will continue to implement deprecated things, by making experimental ones scary and never advancing them, and by deferring them.

To be honest, the XHTML-IM spec should have a description of why it was deprecated.

what is the state of bookmark2 by the way ?

(and probably also mark XEP-0393 as a successor if that was the intent of the Council)

I see that XEP-0393 does mark itself as a successor to XEP-0071, so maybe it is just an editorial issue. Dave Cridland, SamWhited what is the idea here?

Funny thing, what I find more difficult is keeping track of when the spec changes

Short of being in these groups I don’t know how I would know that something I implemented years ago has been updated or changed

Well that and someone telling me something is broken

Are you on the standards mailing list?

No

That's the central place for notifications about updates

I have been in the past but since I do stuff as a hobby more than professionally

I’m often out of the loop or behind

Any other protocol has an overall version of the api

What protocols?

Anu, there was a proposal for that recently on the standards list ;)

There are many closed protocols that use xmpp as a backend but expose it as a rest Api

They make xmpp easier to use and develop for. We debate the restful part but the fact that there is an api version means there less of a moving target

The largest closed garden messaging services are or started as xmpp servers with push support and rest interfaces

Api versioning could be as simple as having a release schedule. Xep are always in development but X times a year we publish the new spec .

There are a lot of xeps and they all look equally important

But in reality some are heavily supported and others aren’t

Anu: XEP-0387 is supposed to provide an up-to-date overview of the most important ones for IM

Anu: as somebody who's not following closely, it would be nice if you could point out where you struggle. I'd love to provide resources for newcomers in the wiki or maybe even on the main page in the future

Regarding disco#items filtering, sorry but had no cell where I was, problem is we're dealing with 3 different xeps each one wanting a different thing.. As usual.

Maranda I'm currently talking with order

we maybe have an idea :) define two new XEPs Result Order Management and Result Filtering Management

the service expose trough the disco#info the field that you can order and filter on

and boom, you apply just RSM like things

*Holger

Order?

Yes, with me.

Oh.

Seems my client failed to change my nick to 'Order' before making that statement.

Result order management is needed for what beside the obvious?

Received error packet [conflict] from <xsf@muc.xmpp.org/Order>

Changing order based on x?

Multiple devices under the same nick

Maranda that's it

MattJ: Ah, right.

I have a Jappix dejavu for some reason.

MSN nicknames are a problem still.

TBH I'm still not convinced we need a new XEP. As opposed to just adding a data form to the disco request like MAM does today.

Yes, agreed.

Ge0rG: We have no problems, we have opportunities.

I came out with something even simpler than using DFs, but that's the idea.

(Sorry already had a bullshit bingo meeting today.)

Holger: let's make an MVP!

Prosody trunk == your MVP

From MVP to ICO in only 0.5 Bitcoins!

A coworker of mine is quitting to do something with the blockchain. Let's see if I can talk him out of it again.

Ge0rG I suggest just blocking him with a chain.

is that like clubbing him with books?

On the nearest lamp post

Irregardless of the nature of the article http://www.dailymail.co.uk/news/article-1281541/Chinese-boy-chained-lamp-post-father-tried-sell-street.html this gives an idea Ge0rG

🤣

Hm. https://wiki.xmpp.org/web/Special:Random is rather worthless because you always end up on random membership application pages.

Maranda: was that boy to be sold or to be rented out?

"Irregardless of the nature of the article" you know what to do with the "chain" and your colleague

Or google failed bringing up a good piccie

Hm. https://wiki.xmpp.org/web/Facebook links to a non-existent page.

Ge0rG, more adapt http://www.mannequin-man.com/machinemart6.jpg

For some reason the first results of "chained to a lamp post" all brought either bikes or stuff that is all but humouristical

So... I still wonder if we can solve multi-client/mobile MUC by implementing: - account joins all bookmarks on behalf of user - account determines notification-worthiness - account sends to-be-defined (push) notification to client - client transparently joins and gets backlog from MAM

"Account joins all bookmarks on behalf of client"?

Is that still the bnc idea?

Maranda: yes

It's a good, strong idea.

It's an opportunity.

Questions... when "is it supposed to join" and "how long is it supposed to stay joined"

how is the POC prosody module to do that working out?

Maranda: when bookmarks

"When bookmarked" and "for eternity from that" has its issues ™️

Maranda: I know. Tell the MIX.

Maranda: I would probably go with something like "at most N (=14) days after the last client disconnected"

But obviously, having a zombie in a MUC is not desirable

that'd just be an account/server setting

Maybe also "when the last client session is killed"

14 days on very busy room is a lot but still better than forever

Anyone tested my minimix?

moparisthebest: users won't be able to figure it out. Probably not even server admins

Zash: tell us how

Ge0rG: I'm sure I put it in prosody-modules

Zash: I'm not sure you told us that before

so https://modules.prosody.im/mod_minimix.html

https://blog.torproject.org/sunsetting-tor-messenger what do they mean with centralized metadata problem? isn't it decentralized?

oh that's different than the one I (and I think you) were thinking of where each contact has their own .onion address, that looks like 'pidgin over tor'

I think they just brought a random word to add between the last one and "problem"

But that's me

The aim was to provide a chat client that supported a wide variety of transport networks like Jabber (XMPP), IRC, Google Talk, Facebook, Twitter; had an easy-to-use graphical interface; and configured most of the security and privacy settings automatically with minimal user intervention.

so they are just saying all of those are centralized (xmpp least of which, but still has a good amount of metadata)

> https://blog.torproject.org/sunsetting-tor-messenger what do they mean with centralized metadata problem? isn't it decentralized? Iwas thinking the same.

vanitasvitae: it's more centralized than in p2p messengers

in other words it is not https://en.wikipedia.org/wiki/TorChat

moparisthebest, do you realize the intrinsical dumbness of that article though, "you don't want to scatter anything around" but you use an intermediary entity, oh rly?

Aka If you're so concerned, just go serverless and GTFO with it.

I still think much can be done with XMPP towards the 'no metadata' goal, or at least 'minimal metadata'

Some things can, but it's hard to get away from an envelope, I think

If you want E2EE and minimal meta-data, XMPP is just not the right protocol.

the initial plan I have in my mind, in a 2 server scenario, prevents any 1 server from knowing the endpoints

moparisthebest: you want to reinvent tor at the message routing layer.

it's not actually that complicated, but perhaps that's a good summary

basically bob@bob.com communicates with tom@tom.com, but server bob.com only knows bob@bob.com is communicating with tom.com, and tom.com only knows tom@tom.com is communicating with bob.com

moparisthebest: and then you also need to tie the identity of users into cryptographic identifiers, and at that moment you could just use Tor for the plumbing

yep, but we already have that with omemo and pgp so

going with tor is already done with torchat, it doesn't give me all the nice things xmpp does though

moparisthebest: both OMEMO and PGP suck regarding identity management. There is no strong binding between your JID and your key.

moparisthebest: you could do serverless XMPP between onion hosts.

sounds like it'd eat battery though

moparisthebest: I'm not so sure. You need to get some tor routing DHT data, that's some hundred megs a week or so

one of the downsides of my plan is it makes any type of content or metadata based spam blocking worthless, but that sounds like someone else's problem ¯\_(ツ)_/¯ :)

Ge0rG, having no strong binding between your JID and your key is a bonus in the context of the metadata thing

I could message you from a new random JID every day, but you would still know it was me

MattJ: except I would need to analyze your key to decrypt, and attackers could do so as well to identify you

only the thing meant to decrypt it should be able to do that

Ge0rG: unless it's sign then encrypt.

And how am I supposed to respond in this scheme?

You don't bother, it was probably spam anyway :D

Lol

Ge0rG, who is the attacker in that case?

in my mind, the server keys are in DNS, so you look up the key for otherserver.net, encrypt the fact that you are responding to bob@otherserver.net, and send it to otherserver.net

The premise is that I can use a different server for each message if I wanted

Can you?

DXMPP!

I guess you could, the premise is a 'person' would be the owner of a certain key, and you'd just respond to whatever JID they told you to at the time

moparisthebest, So the problem with hiding metadata to the server level is that once you add in things like blocklists, it starts to behave pretty badly.

moparisthebest, On the other hand, you can use anonymous MUCs to get what you want quite usefully, if you choose. Still needs sign->encrypt, but MLS actively hides the participants, for example.

not from the server though Dave Cridland ?

but yes, you'd have to be careful not to use bookmarks/blocklists/etc when hiding metadata from your server

moparisthebest, MLS hides the metadata entirely, but XMPP needs it visible for routing. So if you use an anonymous MUC, your server knows only that you're talking to a person in the MUC, as does theirs. The MUC server knows both ends, though.

That's not going to scale tho

Every DXMPP server allows anonymous login and anonymous MUCs. Users each choose a login server at random, and then agree on a MUC JID on a random MUC server...

MattJ: and then they talk about random stuff in that MUC.

What else do you talk about after going to all that effort?

that reminds me of amateur radio. You build complicated devices to talk to other nerds about the complicated devices you built.

Tacos.

devices, tacos and knee implants.

I'd need a shoulder implant more atm probably :P

Sounds like how GNU Social was a social network for talking about GNU Social

I need a gatling gun implant.

Or how 3D printers are devices for producing parts for 3D printers

Dave Cridland, xmpp needs only certain parts for routing, not all the parts it has now

it just needs the last and next hop, not both endpoints

moparisthebest, Sure. That's basically Tor, right?

it's xmpp just with an extra bit

but, kinda

I'm dead curious to see the practical result.

is there a xep or is anyone working on one for emoji reactions?

I think there's more than one

-xep attaching

Zash: Message Attaching (Standards Track, Deferred, 2017-09-11) See: https://xmpp.org/extensions/xep-0367.html

-xep references

Zash: References (Standards Track, Deferred, 2017-09-11) See: https://xmpp.org/extensions/xep-0372.html

Those, I think?

Those may give you the building blocks to build such a system, but I don't think there's anything standardizing that feature in particular.

(and I'm not aware of anyone working on it)

buddy sent me https://api.slack.com/docs/message-buttons

which is a interesting feature

SamWhited: We're not quite there yet, but we're starting to build up References stuff.

The issue with emoji reactions is that while sending them is straightforward, dealing with them very much isn't.

We need a situation where you request a message from MAM, and you get all the references to it as well, correlated into a useful format so you don't get 30 'likes' in your archive a day later that you somehow have to correlate client-side.

So References is going to do everything?

Depends what 'everything' is.

I don't see a reason not to use References as the framework for everything that's a reference.

But I'm not intending squeezing URL references, emoji reactions, quotes, etc. all into the one XEP.

I mean, it looks like References and Attaching have some overlap

They do, yeah.

I have heard from multiple people that emoji reactions are the killer feature of slack

I still don't get what they give you over a well timed

people use them for stuff like "do we have quorum for the XSF boardmeeting?" and you do +1 or whatever

>:)

or whatever

I've learned to take what people say the killer feature is with some salt.

but I guess I'm oldschool or something

well it's more "is emoji reactions the killer feature" and instead of agreeing or disagreeing you do thumbs up or thumbs down. and in theory we can go back in time and go "yeah everyone thought it was a shit idea" without reading the following messages

I mean, emoji reactions don't seem like much, but people like them and get used to using them. They're great for quick agreements on things. Sending smilies and frownies creates a lot more noise.

Zash: On the one hand, yes.

seems like it'd be useful on something like twitter, not so much instant messaging

On the other, people really do get these little features into their workflow and they add value.

not saying it's THE killer feature - but it's a feature I hear - to my surprise - a lot of people use.

I don't use them as much as some folks, but it's useful.

Also, people tend to differ on which feature is the killer feature. Everyone has their own favorite.

And one message with "30👍" beats a message with 30 replies each just saying 'great' or something by a long margin.

moparisthebest, a lot of people use that feature with instant messaging. It's shockingly useful. I wouldn't say it's the killer feature, but it's definitely a good feature that's well liked and well used.

Last week it was stickers. XMPP can't succeed without stickers! It's the killer feature!

Before that it was something else.

what are stickers?

I'd certainly implement it if someone writes a xep

Exactly.

daniel: Or you could implement it and XEP-ify what you did.

daniel: It's one of the oh-so-many things on my list.

The age old question, which comes first, implementation or specification :)

MAM is the real killer for it, otherwise it's trivial.

Kev: The problem you talk of, it's basically the same as with corrections, right?

And receipts.

Yes.

Both of which I'm considering reframing in terms of References in the hope that MAM can just be references-aware, and not million-different-things-without-future-proofing aware.

Aka "graph-like queries on log-style data is messy"

Because message logs are actually message forests where each IM message is the root of a tree with leaves of emoji, corrections, receipts, URLs, ...

Thankfully not at all messy.

Encode messages into a DAG structure, switch to JSON and transport them over HTTP

https://github.com/jabbercat/jabbercat/issues/80 proto XEP for reactions be here

jjrh: ^

> Fitzpatrick modifiers; Aggregate or show separately? If aggregate, show unmodified version or most-voted-for version? (I tend to unmodified since the "unrealistic skin tone" is probably least controversial) On Mattermost (and probably slack?) you can click on the reactions that people already sent, to send one of the same type, aggregation here would prevent this

pep.: what if you make a happy white and a sad black Emoji (slack supports that)? Is that Racial Segregation?

We need blue skin tone modifiers

Ge0rG, do I care?

:P

Scale the font size? ✏

Ge0rG, that was sarcastic right?

pep.: you will care, as soon as the Twitter SJW mob finds out

pep.: I wish it was

Well it's already possible on any of the other platforms I'm sure

hmmm how many spim hits consist enough of an offense to justify a s2s ban? 🤔

Ge0rG, hmm I take that back, I'm not able to find fitzpatrick modified emojis in Mattermost's selection.

... and I know whatever I pick is probably too low :P

You can input one yourself of course, but not as a reaction I think. Or you'd have to add it manually as a custom emoji maybe

math.random(100)?

85!

Maranda: you don't need to ban domains if you can ban users fast enough

Maranda, I would say increase the number of parameters you take into account, not just the number of hits. On what period of time, how often etc.

I'm not sure I really need to ban anything, but I'm implementing a logic for modules to temporarily ban remote servers based on hits.

(default is 2 days)

Any local users having any users on the offending server in their roster?

If not, ban instantly and forever!!!11!!eleven

🤣

85 sounds like a nice number anyways

(my year of birth also)

and random()'ed too!

Ge0rG, in any case if we aggregate we'll get reports like: "My reaction doesn't show up"

also that's a client impl. detail, right?

Not a protocol issue

pep.: how do you want to achieve consistency without writing such details into the XEP?

You can suggest that sure

On the other hand, not being precise gives us the opportunity to differentiate on quality of implementation.