jdev - 2022-05-17

Is there an XEP for file sharing that isn't http-based, that is actually well supported among clients, yet?

The answer is something like "of these three properties you want, pick any two"

😔

There's Jingle FT, which is supported, not sure how widely exactly. It was plagued by interop problems before, tho might have settled by now. Doesn't work with multiple devices or MUC tho. Transfer methods have reliability issues.

:/

Why would you not want it to be HTTP-based?

There are others like https://xmpp.org/extensions/xep-0385.html and https://xmpp.org/extensions/xep-0447.html but I don't think support is there yet.

Obligatory confusion between negotiation and transfer!

I'm mostly just curious. I know theres a tonne of XEPs around, at least two of which i really want to go somewhere fast

Goffi has a jingle component fwiw

Most of the benefits of http upload isn't that it's http anyway, it's that it's a component

And it can do offline / multiparty stuff

Store and forward not dependent on origin availbility.

Component?

Also

I am disappointed to not see http://samy.pl/pwnat/ used more often for p2p things (and in this case it could be a simplified version of it if the xmpp connection can be used to agree on tcp or udp port numbers)

The basic idea is "send packets pretending that a connection is already open, and since the NAT/firewall on either side usually assumes all outgoing connections are allowed, it will actually work"

Write a XEP? But then, p2p and multi-party distribution is still an issue

Bittorrent is another thing. If you rope in the server as a peer you might get similar benefits as http upload

hot take: http upload is only good as a negociated transport behind jingle anyway :-° ✏

Im slightly keen to have ipfs as the backing proto for file transfer, but that'd need servers as ipfs nodes

Or maybe not, clients could be

Actually realistically it'd need neither, but that'd help for better resolution

p2p and mobile is also problematic wrt battery usage

So yeah, serverside

Possible to use go-ipfs externally as the server, and hook in by a mod_ipfs, and expose that to clients as some ipfs-native representation, and also http upload

I know people like bittorrent, but i really want ipfs to work, and to do that it just needs more projects with it baked in

But in this case that'd need an xep probably and i don't wanna get bogged down in beaurocracy if nobody else likes the idea

But whats the benefit since the server is gonna be involved either way 🤔

msavoritias, what do you mean? this is XMPP, the server is always involved

it's often *not* people not liking the idea, but people not having the resources to explore and implement it ✏

moparisthebest: i was talking about what qy said with ipfs

right but how does it change anything I mean

msavoritias: the server hosts the original and seeds to ipfs, but clients dont need to fetch it from the server, they fetch it from any ipfs gateway or ipfs native, and if the server goes down, it's all still there because ipfs caching

well, it *may* still be there

qy, but I like the concept, sounds like you can base much of the XEP off http upload, give it a shot ?

it's not so much bureaucracy

Not to be a downer but this seems incompatible with gdpr. Unless specific case

not sure why it would be, you upload your file knowing all this in advance

if it is, then so is XMPP, because you sent that message not knowing how long I'll keep it (might be forever)

The difference is that in XMPP its not known how long true. With ipfs though its by design to be kept forever.

Or at least they try to

not *really*, it's just as long as each node caches it, which in practice isn't so long

at least last time I played with it a couple years ago...

That is what they advertise thougb

try it, run an ipfs node with your website on it, when you turn your node off your website will be gone

> moparisthebest wrote: > not sure why it would be, you upload your file knowing all this in advance You expect it to be on the server and the recipient/s With ipfs its in the server, the recipients and an unknown number of computers which have nothing to do with your server or the people you talk to

moparisthebest: as stated. It could be. But they advertise it for resiliency.

Just like most xmpp operators are not honey pots

as far as I know ipfs doesn't really have a pre-fetching thing by default, you can only fetch things you know the unique hash of

exactly

nothing is fetched until advertised by a seeking node

so hence, nothing is cached unless wanted

What would that do for latency?

and caching is stronger with things more wanted

it would be shit latency for anything not heavily cached, hence shit latency for anything not heavily wanted

but, the latency is almost always just a one-time thing, that's why for my ipfscat script, i prefetch from public gateways, so once it's on one it's pretty much on them all, and once it's there it's not going away for at least a day, usually longer

with a bunch of xmpp nodes on it, i'd wager it'd get a lot better anyway

the ipfs network provenly gets better with more nodes

what would that be used for in xmpp world? sharing a picture with a MUC? that's maybe ~1000 users max? or do you have some more interesting applications in mind? or is this just offtopic discussion now? (not that I mind offtopic things, just trying to follow the ideas)

i'm mainly thinking just any random file sharing. i don't like HTTP and i'd rather support IPFS over that

the access pattern with http upload from what I can tell is that most hits is just after the referencing message was sent, then maybe the odd hit a bit later. something that improves with time doesn't sound great for that

i already kinda use ipfs for file sharing anyway

Zash, that's the thing, if servers are nodes, it wouldn't matter

if servers are nodes, they'll end up well connected to cdn gateways, because so much file sharing is occuring, so latency would be next to none in real terms

it WOULD "improve with time", but in realistic terms, "time" would be pretty short because the links between servers and e.g. cloudflare-ipfs.com, would pretty much never be dropped due to constant access

I don't really understand, for file sharing over XMPP usually I just want to send one file from one sender to one receiver, and I think I wouldn't want the files to be stored on random ipfs nodes along the way?

it would be terrible if servers weren't nodes, yes

pulkomandy, imagine bittorrent, but ephemeral. that's all this is

yes but that's what ipfs is already, so I don't see the link with xmpp

the link is just that more nodes makes ipfs better, and more ipfs makes xmpp more file-resilient

e.g. if a server goes down, currently, that makes all files "Gone"

ipfs: that's not the case

I consider that being able to delete a file easily and being sure it is not still there on some server is a feature. I guess we have very different uses for file sharing

deleting anything on the internet is always a misfeature

so we should preserve everything and drown in a sea of outdated and irrelevant information?

Servers probably don't go down in the few second window when the wast majority of downloads occur.

nothing is preserved indefinitely, but ipfs allows elastic caching that means one server going down is next-to irrelevant

you can't delete anything you share on the internet, that's a rule of the internet

despite GDPR whining

^

https://en.wikipedia.org/wiki/Streisand_effect

as long as someone is interested in it

my quest to recover and archive lost BeOS software says as much. Most of it was not on the internet anymore. People trying to track down software for PalmOS would say as much too

i'm keen to get ipfs more public and embedded in more software. as long as servers are interested in adding it, and the xep process wouldn't be too beaurocratic and people are happy for it to be a thing, i don't mind writing one, but i won't bother if it won't be used

also, my private conversations in XMPP are not exactly "the internet"

i'm happy to write an xep and a mod_ipfs for prosody in lua, at best

maybe an erlang module at a stretch

I wonder if this couldn't be done with HTTP Upload and OOB?

i mean, it could, with custom p2p, but then you'd be forsaking the existing ipfs p2p network

it would be a no brainer to use the existing p2p network for extra dataresilience

As in, server provides a HTTP upload interface that gets stored into ipfs

oh

Return ipfs:// URL

probably. i've been eyeing that for some time, the http external upload interface is woefully incompatible, and http upload trivially so without modification

Or return some https://well-known-ipfs-gateway.example/ URL

but i'd also like to be able to modify one of those to suit, if happy

up to you, as the server dev

if prosody supports it, i'm sure more could happen

Prosody supports it if you write a XEP-0114 external component 😉

or a module 🙂

wait, wouldn't require an xep?

i'm in

would you be happy to advertise it as much as generic http upload?

(i'm thinking just some comment in the default config, at best)

If it's done as a thing that exposes a http upload service but stores the file in ipfs, I'm not sure if any protocol changes are needed

a module that accepts https uploads and returns https URLs to ipfs gateways would work without any protocol work easily

but is that what you want ?

fab, i'll try it

at a start, it's better than nothing

PoC or more thinking over a whiteboard might help, but now I'm going to be busy with pancake coma

i can then rework that to include something more embedded with ipfs later

If the client has ipfs, it could just put stuff there directly and send, no server needed. Servers could provide (via XEP-0215 maybe or some other discovery method) ipfs gateways locally.

I'll see what i can draft

yea that seems more like what you want, and 0 server work required

send a regular "I uploaded this https file for you" @ an ipfs gateway for backward compat, with an added "or fetch with ipfs here" element, done

Here's where SIMS or similar would be nice, as it allows alternative transfer methods to be listed

would it be more forward-thinking to implement it over SIMS?

SIMS is what the sending client would say, so if you're implementing both client and server bits then it might be sensible to look into

That is, https://xmpp.org/extensions/xep-0385.html or maybe https://xmpp.org/extensions/xep-0447.html