XSF Discussion - 2018-10-03

:D

I would like to know if there are boundaries on how often, how in generel it is allowed to crawl the extensions website? I build a bot which is able to grab the info header from the each xep extension. But there are no threshholds on how often one could issue a request. Or maybe if theres a neat api I could use :)

robots.txt ?

Why don't you just use a local git clone for that?

or https://xmpp.org/extensions/xeplist.xml

mightyBroccoli, if there’s anything more you need which isn’t in xeplist.xml, let me know

I wasnt aware of the xml :) thats everything I need :) caching will do the reset :) thats even better then my bs4 garbage :D jonas’ it seems that the xml lists even the deferred xeps, are there some not listed in there?

mightyBroccoli, the XML lists all the things

even inbox

ahh ok, thats nice. I guess the accepted tag could be used to redact the inbox xeps from the results

exactly

mightyBroccoli, use If-Modified-Since

the webserver supports that

Why not ETag/If-None-Match ?

or that

firefox does both and gets 304

Both work exactly the same anyways

probably

depends on how the etag is generated

Which means you can use them as supercookies

I made a thing once that used the timestamp of the last request in If-Modified-Since. It doesn't work then.

It has to be exactly what the server sent in whatever header it was.

I will try both ideas and use whatever is easiest and working :)

Hi! Who was running planet.jabber.org again?

ah found it 😀

Won't need to point you to ralphm then ;)

😀

Zash, I am just wondering the xml you posted is actually invalid. XEP 225 talks about <domain> and <hostname> tags which are unescaped.

Who what when?

-ENOCTX

It's like 5 messages up 😂

Then it's scrolled out of view and out of mind, flushed away by quitjoins

Also what

I don't see how what you just said has to do with xeplist.xml, if that's what you were talking about

“It does not enable a component to bind multiple hostnames to one stream (as, for example, a client can bind multiple resource identifiers).”, a client can do that?!

Is this the reason why @from exists on sent stanzas?

> <remark>Modified namespace to incorporate namespace versioning; clarified that the value of the &lt;hostname/&gt; element Are you just looking at the Firefox rendering of the XML?

ok so the xeplist.xml lists all currently known xeps inbox and accepted. Inside in line 5958 this line is invalid. > Modified namespace to incorporate namespace versioning; clarified that the value of the <hostname/> element can be either <domain> or <domain/resource>. I did requests.get the xml and read it line directly from that no browser involved

$ curl https://xmpp.org/extensions/xeplist.xml | grep -o '.......hostname........' he &lt;hostname/&gt; el

I think you're holding it wrong

Or you're printing the text content, after unescaping.

damn i found the error. I need to reply.content.decode to get the correct format :) thank you though :)

etag header is way easier then Last-Modified header. I choose etag and it works like a charm :) thanks :)