XSF Editor Team - 2024-03-08

That would be ideal, yes.

- So my Editor flow is now, approximately: Open 3 terminals in xeps.git T1: `git pull --rebase && ./tools/merge-helper.sh` Continue until it says to do merges, then T2: `./tools/local-triage.py` each time I merge one of the PRs T3: `git pull --rebase` then when I'm done with triage/merge close T2 Back in T1 continue until it tells me to generate docker images, at which point T3: `../upload-xeps.sh` once that completes, follow T1 to completion. - And upload-xeps.sh is 1 │ #!/bin/bash 2 │ set -e 3 │ XEPIMAGE=registry.gitlab.com/xsf/xeps/packed:main-latest 4 │ ATTICIMAGE=registry.gitlab.com/xsf/xep-attic/packed:main-latest 5 │ pushd path/to/xeps 6 │ docker build --build-arg NCORES=64 -t $XEPIMAGE . 7 │ docker push $XEPIMAGE 8 │ popd 9 │ pushd path/to/xep-attic 10 │ docker build --build-arg NCORES=64 -t $ATTICIMAGE . 11 │ docker push $ATTICIMAGE 12 │ popd 13 │ ssh -t xmpp.org "sudo ./xeps-up.sh"

Kev: and these steps also send out the email notifications?

https://github.com/xsf/xeps/blob/master/tools/merge-helper.sh

They do.

And instead of 'do your merge now' I guess I can also just do modifications? For example if I wanted to start a last call that's not based on an actual PR?

LC process isn't defined.

(In terms of Editor actions)

moparisthebest, do I recall rightly that you were working on a Github action for the triage stuff?

Kev, can you make sure if have access to git and the necessary servers? https://gultsch.de/sshkey.txt I don’t have today but I'll probably try my luck at my first editor task tomorrow

I think you'll need to sign up for gitlab or something, I don't remember details.

And thank you, BTW, very much for sharing the Editor pain.

Me being the only one, and not having cycles to do anything with it, really isn't tenable.

`iNPUTmice` on GH, right?

> moparisthebest, do I recall rightly that you were working on a Github action for the triage stuff? MattJ, I had planned to but then I think singpolyma did it? oh he's not in here :D ↺

yes I think as Council chair it makes a lot of sense to also be editor. that actually removes some of the work because i don’t have to tell myself to do something. plus my hope is that actually doing the editor tasks would help me understand which parts need automating

Kev, yes that's the gh username

I am on gitlab with the same username

Ok, you're in the editors group on GH now, at least.

the only thing automation-wise I'm not sure should be done is sending out emails otherwise I think there should be a command to generate all html files from the git repo, and then it can just be set to run nightly (or whatever) on the server and run on new commits to master

also though I think that makes the attic both hard and useless, so maybe some blessing from the board to get rid of that? in exchange for a link to git history for the file or something

moparisthebest, right. i agree that this should be the goal. but (due to not having done them yet) i don’t understand the steps necessary that would take us there

I believe we've talked about this in various issues on github but I can't go looking at the moment :D maybe MattJ remembers

Kev, did you also add my ssh key? or is this not necessary?

yep makes sense Daniel , but to be clear I'll write the required code if I can get some clear requirements and decisions (from board?) about attic etc

You control your own ssh keys for GH.

I'm trying to work out what I need to do on GL at the moment.

Then maybe see if I can persuade Matthew to work out what needs to happen on XSF servers.

I meant for the ` ssh -t xmpp.org "sudo ./xeps-up.sh"` step

Ah, I don't have access on GL anyway.

and some decisions re: email, I think trying to detect changes once and send emails from a cronjob is asking for trouble at least, so I'd suggest no emails (subscribe to changes on github if you want them) or editors do that one part manually, unless others have great ideas :D

MattJ You seem to be owner of the GL group, can you do the needful for Daniel, please?

*or* now that we have fancy mailing list UI, maybe we could set the editor email to moderated or whatever, have mails go out automatically, and a person "approve" them before going out to protect against bad cronjob

moparisthebest, i *think* part of the 'email problem' is that you need to record your changes when you make them and that is then used to trigger the email. you can’t generate diff emails just from looking at a static status quo

right, and in theory you could do that by storing the last commit you saw, checking differences, sending appropriate emails etc etc, but it's all very error prone and I'd rather not do it

Can you manipulate tags?

Or git annotations?

if the "look at git HEAD and generate html" script gets messed up, no big deal, fix it and run it again if the "auto send emails" script gets messed up and you send 10,000 emails to every list member and get blacklisted from gmail.com that's a bigger deal :D

Or maybe I don't mean annotations, maybe it's some other word.

Kev, yes I believe singpolyma already did that work and maybe made a PR, will ping

So it wouldn't be too bad to record a label each time an email is sent, maybe?

I don't know, I don't feel like me trying to (micro)manage a solution to the automation is the right thing to do :)

still the failure modes scares me, I'd rather do everything else than send emails, and then maybe work on that if it's valuable

(re: failure modes: what does the script do if github is down etc etc)

Well, if the mail send script can do the same thing with labels that the automation would do, that already means that it can be run manually after merges are done, instead of needing to be part of the manual merge process, right?

I think these are seperate automations: 1. triage script to analyze PRs and apply tags to github, the script has been done but doesn't auto apply tags, I think singpolyma did this, otherwise I plan to 2. script to run in a cronjob on xmpp.org that grabs git master and generates all the XEP html 3. script to send emails when changes are detected (this is the only dangerous one I'd really prefer not to do)

Agree on all three.

We already have the mail autosending script, if it could work off git commits instead of working trees, it wouldn't be a hardship for Editor to just run the script locally, I think.

But the current setup of making sure you don't merge unless you've got your local setup ready so the mails will get sent when you're done is a bit inconvenient.

And means it has to be an Editor with the ability to run the script that does the merges, and etc. etc.

3 wouldn't worry me at all if there was only a human approval somewhere/somehow in the flow

So if e.g. the script would fetch the repo, including tags, work out the diff, tag the current commit, push the tag, and if all that succeeds, then send the emails, that would make it 'atomic' rather than ingrained in how 1/2 are done.

so that if it said "ok I'm gonna send 500 emails" someone could say "nooooo"

Yes, the script already shows a preview and says 'is this right?'.

actually I think github actions probably has these kind of pipelines, I know gitlab does

but I really hate to build github specific things too :'(

I can understand that. Although in this instance, I would take holding my noise for such things :)

not even from some moral standpoint mind you, it's just a pain to develop for :D

change, push, wait, click 80 things, repeat

Ok :)

> MattJ You seem to be owner of the GL group, can you do the needful for Daniel, please? GL == GitLab?

What do we use GitLab for?

It’s where the docker images live, no?

I didn't think so, but as I also can't say exactly where they do live, I could very well be wrong :)

I log in and see an xsf org, with the description: "INOFFICIAL experiment for replacing some of the XSF’s GitHub usage with GitLab"

I'll figure it out

Yeah, xmpp.org has a container running from an image updated 6 weeks ago with the tag 'registry.gitlab.com/xsf/xeps/packed'

But I didn't set this up and don't really use Gitlab for anything, so I'll have to poke around

I just see a lot of repos with last commit >years ago

There is a 'xeps' repo with the last commit and pipeline run 3 years ago

Also 'xep-buildspace' and 'xep-buildspace-slim', also last built 3 years ago

Looks like we have a build runner registered with "last contact 25 minutes ago", but it has not run any jobs

So if Gitlab is part of our XEP publication process, I really don't know how

MattJ: kev said his upload script is - And upload-xeps.sh is 1 │ #!/bin/bash 2 │ set -e 3 │ XEPIMAGE=registry.gitlab.com/xsf/xeps/packed:main-latest 4 │ ATTICIMAGE=registry.gitlab.com/xsf/xep-attic/packed:main-latest 5 │ pushd path/to/xeps 6 │ docker build --build-arg NCORES=64 -t $XEPIMAGE . 7 │ docker push $XEPIMAGE 8 │ popd 9 │ pushd path/to/xep-attic 10 │ docker build --build-arg NCORES=64 -t $ATTICIMAGE . 11 │ docker push $ATTICIMAGE 12 │ popd 13 │ ssh -t xmpp.org "sudo ./xeps-up.sh"

But maybe he copy pasted the wrong thing?

Ah okay, so it's built locally and then pushed to Gitlab

I didn't know that was a service they offered

Ah yep, under Container Registry I see the latest image is from a month ago

You should have access now

Yes I got an email. If I can actually push to the registry is something I will have to check tomorrow

Can you also add my ssh key to xmpp.org?

I don't know if Editor has a separate account on that machine?

I only know the last line of that script

I created daniel@xmpp.org and dropped the same script in your home folder

Yes login works. Thanks.

I‘m not going to be watching stuff over the weekend, I’ll be back Monday if there’s questions for me.

And thanks both.