*** dviroel is now known as dviroel|out | 00:28 | |
ysandeep | good morning folks o/ | 04:40 |
---|---|---|
*** yadnesh|away is now known as yadnesh | 04:45 | |
*** ysandeep is now known as ysandeep|afk | 05:47 | |
*** marios is now known as marios|ruck | 06:08 | |
marios|ruck | o/ | 06:13 |
marios|ruck | frenzy_friday: o/ for container push/promoter creds issue http://promoter.rdoproject.org/promoter_logs/centos8_wallaby.log can we have a quick call when you have a minute | 06:41 |
*** ysandeep|afk is now known as ysandeep | 07:00 | |
*** yadnesh is now known as yadnesh|afk | 08:13 | |
marios|ruck | needs reviews please https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/861692 | 08:17 |
reviewbot | Do you want me to add your patch to the Review list? Please type something like add to review list <your_patch> so that I can understand. Thanks. | 08:17 |
ysandeep | marios|ruck, looking | 08:18 |
marios|ruck | ysandeep: thanks its from takashi needs some eyes (not easy/big review but needs a few mins staring thanks) | 08:18 |
marios|ruck | ysandeep: looks like chandankumar voted just now | 08:19 |
jm1 | good morning #oooq :) | 08:19 |
jm1 | what a nice quasi-monday :D | 08:19 |
marios|ruck | o/ | 08:19 |
ysandeep | marios|ruck, yeah chandankumar already +wed, chandankumar++ | 08:19 |
frenzy_friday | marios|ruck, good morning. Lemme check the logs | 08:26 |
frenzy_friday | Is it only for wallaby c8 or all the releases? | 08:26 |
marios|ruck | frenzy_friday: o/ i see the creds were updated on promoter and it works for me locally for login so not sure why it fails to push | 08:26 |
marios|ruck | frenzy_friday: well wallaby8 is the one trying to promote now so that is why we see it there | 08:27 |
frenzy_friday | hm... wallabyc8 is trying to login with tripleozedcentos9+cirobot creds | 08:27 |
frenzy_friday | Lemme check what recent patches merged | 08:27 |
dpawlik | hey folks. Yesterday I was generating secrets via zuul encrypt secret script: python3 ./encrypt_secret.py --tenant rdoproject.org --infile secret.data --outfile test.yml https://review.rdoproject.org/zuul config | 08:28 |
dpawlik | usually all was working fine | 08:28 |
dpawlik | but I remember that last time frenzy_friday was regenerating token after me :> | 08:29 |
marios|ruck | dpawlik: frenzy_friday: invite sent, lets talk on phone in half hour will be easier than discussing here? | 08:29 |
frenzy_friday | dpawlik, yep, it is not an issue with the zuul secrets patch. | 08:29 |
marios|ruck | dpawlik: i can login with the new secret (docker login) so seems OK | 08:30 |
dpawlik | so you can login to the Quay but push does not work? | 08:31 |
marios|ruck | dpawlik: yeah but looks to be an issue with our code, probably not creds related? not sure yet | 08:31 |
marios|ruck | frenzy_friday: maybe one of the config patches for the container jobs (rdo registry decommission) | 08:32 |
frenzy_friday | yeah, that is my guess. I thought it was https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/45898/9/ci-scripts/container-push/roles/containers-promote/defaults/main.yml that was causing trouble but nope. Looking for the other merged patches | 08:32 |
dpawlik | I made a test: login + pull + push new image -> all was working for quay.rdoproject.org/tripleozedcentos9/nop:dpawlik . Checking if it got permissions to create new repository | 08:35 |
dpawlik | also got permissions to create new repo: https://quay.rdoproject.org/repository/tripleozedcentos9/dpawlik | 08:36 |
dpawlik | could you send me some log url? | 08:36 |
marios|ruck | dpawlik: wallaby/8 creds are OK (issue currently is pusing to w8) ... logs there http://promoter.rdoproject.org/promoter_logs/centos8_wallaby.log looks like it tries to push with wrong user (using zed pass) | 08:37 |
dpawlik | Added PS for adding new volume: https://softwarefactory-project.io/r/c/software-factory/sf-infra/+/26933 | 08:37 |
dpawlik | oh | 08:38 |
frenzy_friday | hey akahat , which role/playbook does the promoter use to promote containers? Is it only ci-scripts/container-push ? | 08:38 |
dpawlik | did you check in the project logs history? | 08:38 |
marios|ruck | frenzy_friday: yes that play invoked containers-promote role | 08:39 |
dpawlik | https://quay.rdoproject.org/organization/tripleowallabycentos8?tab=logs -> it seems that it works normally | 08:40 |
frenzy_friday | marios|ruck, https://github.com/rdo-infra/ci-config/blob/master/ci-scripts/dlrnapi_promoter/config_environments/rdo/CentOS-8/wallaby.yaml#L8 this is wrong as well. I'll put up a patch to update. | 09:08 |
frenzy_friday | But it is not the reason for our current failure | 09:08 |
marios|ruck | frenzy_friday: ack | 09:08 |
frenzy_friday | actually it may not be wrong. Sorry, a stupid question: where do we promote wallaby to? Like I know the source registry is quay.rdo, what is the target registry? | 09:10 |
*** yadnesh|afk is now known as yadnesh | 09:20 | |
marios|ruck | frenzy_friday: we 'promote' by tagging the hash as current-tripleo and pushing back to quay.rdoproject.org ... we used to also push to other registries in that code https://github.com/rdo-infra/ci-config/blob/00e78ee915ce8f19ef0661f8c78358f67e3ded8f/ci-scripts/container-push/container-push.yml#L50 (quay.io nowadays only) but i know we have made that even more complicated with an additional copy quay tool | 09:20 |
marios|ruck | so... | 09:20 |
marios|ruck | :/ | 09:20 |
*** ysandeep is now known as ysandeep|lunch | 09:21 | |
dpawlik | folks, we are adding new volume to the Quay | 09:31 |
dpawlik | service should work normally, but just FYI | 09:31 |
akahat | frenzy_friday, we use ci-scripts/container-push role. | 09:31 |
frenzy_friday | akahat, hey, need one help - In wallaby c8, promoter is using the wron username | 09:32 |
frenzy_friday | If you check http://promoter.rdoproject.org/promoter_logs/centos8_wallaby.log - unauthorized: Could not find robot with username: tripleozedcentos9+cirobot | 09:33 |
akahat | frenzy_friday, yes. i've checked it. looking for the issue.. what exactly caused it. | 09:33 |
frenzy_friday | It is wallaby c8 , it should be using unauthorized: Could not find robot with username: tripleowallabycentos8+cirobot | 09:33 |
frenzy_friday | cool, thanks | 09:33 |
frenzy_friday | dpawlik, ack, thanks | 09:33 |
frenzy_friday | marios|ruck, akahat this is what I found: try running docker manifest inspect --insecure quay.rdoproject.org/v2/tripleowallabycentos8/openstack-aodh-api:47c581dacb7a7b35ad3aa337c08e1703 from your local (or anywhere) | 09:46 |
frenzy_friday | Then try running it from thr promoter (as promoter user) | 09:46 |
frenzy_friday | This is a get call, and should not need authorization. But on the promoter somehow the default docker login is zed | 09:47 |
* frenzy_friday trying to find more | 09:47 | |
akahat | frenzy_friday, this is strange. | 09:47 |
marios|ruck | frenzy_friday: looking | 09:47 |
marios|ruck | frenzy_friday: yes i see so something in the promoter env is overriding | 09:50 |
frenzy_friday | marios|ruck, ~/.docker/config.json | 09:50 |
frenzy_friday | I renamed this , and now the auth is gone | 09:51 |
marios|ruck | frenzy_friday: good, that might be it. but why/how for this config.json | 09:51 |
frenzy_friday | marios|ruck, did we try docker login manually from the promoter vm? | 09:52 |
frenzy_friday | * docker login to zed registry | 09:52 |
frenzy_friday | that would have saved the creds in the config | 09:52 |
marios|ruck | frenzy_friday: could be | 09:54 |
marios|ruck | frenzy_friday: few mins promoter should try again lets see | 09:54 |
marios|ruck | but :/ | 09:54 |
frenzy_friday | can we run the promoter now? | 09:55 |
frenzy_friday | Or will it runin some time? | 09:55 |
frenzy_friday | oh, you answered already :P | 09:55 |
marios|ruck | frenzy_friday: it will rerun in a few | 09:55 |
marios|ruck | frenzy_friday: looks good running | 10:01 |
marios|ruck | damn exception :/ | 10:01 |
marios|ruck | ffs | 10:02 |
* frenzy_friday looks | 10:02 | |
frenzy_friday | Generate list of containers to push passed this time | 10:05 |
marios|ruck | yeah | 10:05 |
marios|ruck | frenzy_friday: also Login to source registry | 10:05 |
marios|ruck | attempt a push fails | 10:05 |
frenzy_friday | ah, bad gateway | 10:06 |
marios|ruck | hmm | 10:06 |
frenzy_friday | saw a ping from daniel, they are adding a new volume | 10:07 |
marios|ruck | right maybe related | 10:07 |
marios|ruck | k thanks frenzy_friday going to get some food and lets see in next run then | 10:07 |
* marios|ruck food biab | 10:07 | |
frenzy_friday | cool | 10:07 |
jpodivin_ | marios|ruck: Hi. I'm having some issues with the override-checkout parameter. It seems that jobs don't really care for it? Even when it's defined the normal branch matching rules take over. | 10:13 |
*** jpodivin_ is now known as jpodivin | 10:14 | |
jpodivin | It makes work with non-traditional branch names rather painful. | 10:15 |
jpodivin | that being said, I can work around it, by either making my own jobs with right nodeset or by changing the matching rule in openstack-zuul-jobs | 10:16 |
jpodivin | That second option isn't a what I would go for however. | 10:17 |
*** ysandeep|lunch is now known as ysandeep | 10:18 | |
marios|ruck | jpodivin: ok if you give me more information (where are you seeing this, logs, patch etc) and i can try check later | 10:27 |
jpodivin | marios|ruck: thanks, see this buildset https://zuul.opendev.org/t/openstack/buildset/5d653e1a3e3149c7a7e5a62ad2bd7110 And the associated patch https://review.opendev.org/c/openstack/validations-common/+/866343/ | 10:28 |
jpodivin | search for 'openstack-tox-linters' that's the one I'm trying to bring back on line. | 10:29 |
marios|ruck | jpodivin: k | 10:30 |
marios|ruck | dpawlik: still adding voluem or done? | 10:31 |
marios|ruck | dpawlik: we hit 502 like yesterday (assuming this is why you are adding new vol, out of space) | 10:31 |
marios|ruck | damn again frenzy_friday running now... so lets see what dpawlik says | 10:35 |
marios|ruck | FAILED! => {"changed": true, "cmd": "docker build -t quay.rdoproject.org/tripleowallabycentos8/nop - && docker push quay.rdoproject.org/tripleowallabycentos8/nop\n", "delta": "0:00:00.670944", "end": "2022-12-06 10:34:36.145891", "msg": "non-zero return code", "rc": 1, "start": "2022-12-06 10:34:35.474947", "stderr": "received unexpected HTTP status: 502 Bad Gateway", | 10:35 |
dpawlik | not yet | 10:35 |
marios|ruck | dpawlik: ok thanks, we are hitting 502 now for container push ^^^ | 10:35 |
dpawlik | there was small issue | 10:35 |
dpawlik | it can be: /dev/mapper/vgimage-image 1.0T 1.0T 128K 100% /var/data | 10:36 |
marios|ruck | dpawlik: ack | 10:36 |
dpawlik | we are checking one thing due we don't want to loose data | 10:45 |
*** dviroel|out is now known as dviroel | 11:12 | |
dpawlik | marios|ruck: partition resized | 11:16 |
dpawlik | marios|ruck: now should be OK | 11:16 |
marios|ruck | dpawlik: yes, looks like it is pushing now lets see http://promoter.rdoproject.org/promoter_logs/centos8_wallaby.log | 11:17 |
dpawlik | sorry for delay, one feature were added and took some time to get information if it can be done with my fix | 11:17 |
marios|ruck | thanks for helping me frenzy_friday ^^^^ | 11:17 |
marios|ruck | dpawlik: thx, seems it was multiple issues same time (creds leak, disk space, local config on promoter... last one is probably related to debug from creds change) | 11:18 |
*** rlandy|out is now known as rlandy|rover | 11:18 | |
frenzy_friday | \0/ Hope this time it promotes | 11:18 |
marios|ruck | frenzy_friday: well its been pushing for ~ half hour should be ok | 11:19 |
frenzy_friday | awesome! I'll delete the backup of docker creds json | 11:19 |
marios|ruck | frenzy_friday: leave it for now since you renamed should be OK, maybe add 'delme' to the name or something | 11:20 |
marios|ruck | frenzy_friday: just incase we want/need debug there | 11:20 |
dpawlik | I thought I would come back from PTO and it would be so quiet in infra.... | 11:20 |
marios|ruck | frenzy_friday: hm actually not sure it would be useful we already updated the timestamp on it... k whatever delete is fine | 11:20 |
frenzy_friday | cool | 11:21 |
rlandy|rover | marios|ruck: frenzy_friday: hey - reading back on promoter/container push issue | 11:21 |
rlandy|rover | marios|ruck: pls ping when you have time to sync | 11:21 |
marios|ruck | rlandy|rover: sure lets sync in a few | 11:21 |
frenzy_friday | rlandy|rover, tldr: it was an issue on the promoter server vm. Solved now | 11:21 |
rlandy|rover | marios|ruck: ok - chceking what happened with 17.1 | 11:21 |
marios|ruck | rlandy|rover: good there we got green run | 11:22 |
marios|ruck | rlandy|rover: was updating that right now thats why need few mins | 11:22 |
rlandy|rover | failed periodic-tripleo-ci-rhel-9-ovb-3ctlr_1comp-featureset001-internal-rhos-17.1 - ah O see your rerun | 11:22 |
rlandy|rover | cool - last night's run passed | 11:23 |
rlandy|rover | yay | 11:23 |
rlandy|rover | ysandeep: chandankumar: hi - if you have any outstanding travel expenses, pls submit today | 11:26 |
chandankumar | rlandy|rover: all submitted already. | 11:26 |
rlandy|rover | great | 11:26 |
ysandeep | rlandy|rover, same for me, all submitted already | 11:29 |
rlandy|rover | great | 11:30 |
*** yadnesh is now known as yadnesh|afk | 12:00 | |
rlandy|rover | dpawlik++ frenzy_friday++ marios|ruck++ for working through promoter issue | 12:00 |
rlandy|rover | ysandeep: hey - do we need to chat re: nova operator? | 12:07 |
ysandeep | rlandy|rover, ack we can and we need a bigger discussion about inrepoconfig, would be good if dviroel can also join. | 12:08 |
dviroel | i can join now | 12:09 |
ysandeep | dviroel, rlandy|rover meet.google.com/xta-zxuf-cvn | 12:10 |
chandankumar | dviroel: ysandeep rlandy|rover on a side note, crc takes approx 5 mins to start up in zuul job https://review.rdoproject.org/r/c/testproject/+/46362/1#message-66d25756952f8e35a1fbfb24b4d63b69b2da5656 | 12:12 |
marios|ruck | rlandy|rover: especially frenzy_friday helped me a lot today | 12:12 |
*** frenzy_friday is now known as frenzy_friday|food | 12:21 | |
rlandy|rover | marios|ruck: hmm ... did you recheck the fs001 patch for 17.1? | 12:49 |
rlandy|rover | looks like it reset | 12:49 |
marios|ruck | rlandy|rover: yeah rechecked cos the fail was inconsistent - looking | 12:49 |
rlandy|rover | ah | 12:50 |
marios|ruck | rlandy|rover: yeah see links https://code.engineering.redhat.com/gerrit/c/testproject/+/437138/1#message-d157ff5f9bb9b31fc59a8e0f1d0513d48536a875 | 12:50 |
rlandy|rover | marios|ruck: also - line kicked again | 12:50 |
marios|ruck | rlandy|rover: may still be a thing but not sure yet | 12:50 |
marios|ruck | rlandy|rover: both in provisioning, but different issue in those two runs so far ^^ | 12:50 |
rlandy|rover | ok - there will be your run plus the line in general | 12:51 |
*** yadnesh|afk is now known as yadnesh | 13:10 | |
marios|ruck | jpodivin: had a look - commented there fyi https://review.opendev.org/c/openstack/validations-common/+/866343/12#message-831919147cd28d75c2d6dbf7c98d8eaf04ae4e46 | 13:18 |
marios|ruck | jpodivin: mainly that branches: != override-checkout (looking at https://zuul.opendev.org/t/openstack/buildset/5d653e1a3e3149c7a7e5a62ad2bd7110 and 'openstack-tox-linters' .. the variants there are determined by branches: | 13:18 |
marios|ruck | jpodivin: so it does seem to honour the override-checkout - see https://c978daf28b4ba7352ce1-f4b3e6a1d73b0d4e13079b70a1af7060.ssl.cf2.rackcdn.com/866343/12/check/openstack-tox-linters/f915a35/zuul-info/inventory.yaml projects: e.g. opendev.org/openstack/requirements checkout: stable/train | 13:20 |
*** frenzy_friday|food is now known as frenzy_friday | 13:30 | |
*** ysandeep is now known as ysandeep|dinner | 13:34 | |
marios|ruck | rlandy|rover: master promoting | 13:34 |
*** dasm|off is now known as dasm | 14:04 | |
dasm | o/ | 14:04 |
rlandy|rover | marios|ruck: zed is promoting | 14:06 |
marios|ruck | nice | 14:08 |
marios|ruck | rlandy|rover: fs1 is now running tempest (no consistent provision issue then .. we can see if we get it again anyway) | 14:16 |
marios|ruck | rlandy|rover: (17.1/9) | 14:16 |
rlandy|rover | marios|ruck: also jenkins didn't kick again | 14:16 |
rlandy|rover | so I manually trggered that | 14:16 |
marios|ruck | rlandy|rover: cool | 14:16 |
jpodivin_ | marios|ruck: right, but the documentation implies that it also works for job selection | 14:17 |
jpodivin_ | https://zuul-ci.org/docs/zuul/latest/config/job.html#attr-job.override-checkout | 14:17 |
jpodivin_ | the second paragraph | 14:17 |
*** jpodivin_ is now known as jpodivin | 14:17 | |
marios|ruck | jpodivin: reading (zuul docs are always hard to read :/ i mean i usually have to re-read things a few times ) | 14:19 |
marios|ruck | jpodivin: didn't realise this overrides branch for job selection | 14:21 |
marios|ruck | jpodivin: in your logs though, the override-checkout does seem to work ok did you see my comment? | 14:21 |
marios|ruck | jpodivin: it checks out train for required-projects | 14:21 |
jpodivin | marios|ruck: yeah, that part is working, but in this context what I really want it to get the "stable/train" version of the job | 14:23 |
marios|ruck | jpodivin: i see i thought the problem was with required-projects checkout. ok i'll have another pass and see if can spot something else | 14:23 |
jpodivin | marios|ruck: thanks | 14:28 |
jpodivin | Worst case scenario I'll have to define our own jobs. | 14:28 |
rlandy|rover | chandankumar++ | 14:43 |
rlandy|rover | nicely done | 14:43 |
chandankumar | rlandy|rover: thank you :-) | 14:44 |
chandankumar | ysandeep|dinner++ thank you for figuring out how to make job required and testing together | 14:45 |
*** ysandeep|dinner is now known as ysandeep__ | 14:46 | |
ysandeep__ | chandankumar++ great work on Zuul POC and today' demo | 14:47 |
ysandeep__ | rlandy|rover, chandankumar dviroel we have gha/prow/zuul already, we should throw some jenkins into the picture too :D | 14:49 |
rlandy|rover | ysandeep__: don't even :) ... it's coming | 14:50 |
rlandy|rover | we don;t know what QE wants yet | 14:50 |
ysandeep__ | hahaha.. true | 14:50 |
dviroel | ~o~ | 14:52 |
rlandy|rover | pojadhav: you around? | 14:56 |
rlandy|rover | community call in a few | 14:57 |
rlandy|rover | chandankumar: hey - we have no agenda items for community call - will you be there? maybe you can show your demo again | 14:57 |
chandankumar | rlandy|rover: yes sure, I will be there | 14:57 |
rlandy|rover | chandankumar: let's see if we get a guest | 14:58 |
pojadhav | rlandy|rover, yep | 14:59 |
rlandy|rover | akahat: frenzy_friday: pojadhav: bhagyashris: dviroel: ysandeep__: chandankumar: marios|ruck: rcastillo: jm1: dasm: community call | 15:00 |
dasm | thx | 15:00 |
dasm | joining | 15:00 |
pojadhav | folks, https://hackmd.io/MMg4WDbYSqOQUhU2Kj8zNg#2022-12-06-Community-Call | 15:00 |
ysandeep__ | dasm: o/ your light setup looks great | 15:01 |
dasm | ysandeep__: thanks! | 15:02 |
dasm | ysandeep__: it's ligth strip at the wall in front of my eyes | 15:02 |
dpawlik | frenzy_friday, marios|ruck: on the end, what was the issue with authentication to the quay? | 15:03 |
pojadhav | rlandy|rover, sorry was out for dr appt | 15:03 |
frenzy_friday | dpawlik, we did a docker login -u ... -p ... from the promoter vm (which retags and pushes the containers) | 15:03 |
dpawlik | aha | 15:04 |
frenzy_friday | this created a file ~/.docker/config.json - which overrides every other auth | 15:05 |
rlandy|rover | marios|ruck: going to retrigger wallaby c9 - since that is the only one not promoted | 15:14 |
marios|ruck | rlandy|rover: k thx | 15:16 |
dasm | pojadhav: o/ did you get my pm? | 15:19 |
pojadhav | dasm, yes.. will check after community call | 15:20 |
dasm | pojadhav: ack, thx | 15:20 |
Tengu | rlandy|rover: heya! quick headsup: https://review.opendev.org/c/opendev/system-config/+/866175 will be merged shortly, therefore I should be able to finalize the tripleo-ansible change to use the galaxy proxy. | 15:23 |
Tengu | rlandy|rover: also: https://bugzilla.redhat.com/show_bug.cgi?id=2150283 container-selinux issue should hopefully be sorted out shortly. | 15:24 |
Tengu | (meaning openstack-selinux should properly install, so the vTPM boolean should be good as well) | 15:24 |
marios|ruck | rlandy|rover: ysandeep__: merge this please https://review.rdoproject.org/r/c/rdo-jobs/+/46334 | 15:29 |
marios|ruck | rlandy|rover: i'll abandon that one https://code.engineering.redhat.com/gerrit/c/openstack/tripleo-ci-internal-config/+/436919 | 15:29 |
rlandy|rover | marios|ruck: done | 15:30 |
*** dviroel is now known as dviroel|lunch | 15:31 | |
ysandeep__ | marios|ruck, done double +wed :D | 15:31 |
marios|ruck | totally merged thanks ysandeep__ | 15:31 |
chandankumar | see ya! | 15:31 |
*** pojadhav is now known as pojadhav|afk | 15:32 | |
* ysandeep__ out, see you all tomorrow, have a great rest of your day/evening o/ | 15:37 | |
*** ysandeep__ is now known as ysandeep|out | 15:50 | |
rlandy|rover | dasm: happy RH anniversary!!!!!!!!!!! | 15:51 |
*** yadnesh is now known as yadnesh|away | 15:51 | |
dasm | rlandy|rover: thank you :) | 16:05 |
dasm | 4 more years and I'm gonna get a puck! :) | 16:06 |
*** dviroel|lunch is now known as dviroel | 16:34 | |
marios|ruck | rlandy|rover: off in a minute need sthing? | 16:43 |
marios|ruck | rlandy|rover: i think we got all the things, maybe just watch these for me please? https://review.opendev.org/c/openstack/tripleo-ansible/+/866566 for train/cix and https://review.rdoproject.org/r/c/rdo-jobs/+/46334 still waiting in the queue for some reason | 16:44 |
rlandy|rover | marios|ruck: should be fine | 16:44 |
rlandy|rover | sure - thanks | 16:44 |
rlandy|rover | lunch - brb | 16:49 |
*** marios|ruck is now known as marios|out | 16:55 | |
*** pojadhav|afk is now known as pojadhav | 18:37 | |
rlandy|rover | akahat: we tried to review your registry patches ... | 19:07 |
rlandy|rover | there are two sets - one today and one from yesterday | 19:08 |
rlandy|rover | I added both sets to tomorrow | 19:08 |
rlandy|rover | can you present the order and logic at tomorrow's review meeting | 19:08 |
rlandy|rover | so can go ahead? | 19:08 |
dasm | oh, review meeting. | 19:28 |
dasm | i forgot about that. | 19:28 |
dasm | sorry | 19:28 |
*** dviroel is now known as dviroel|biab | 19:48 | |
* dasm => offline | 22:51 | |
dasm | o/ | 22:51 |
*** dasm is now known as dasm|off | 22:51 | |
*** rlandy|rover is now known as rlandy|out | 23:01 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!