Monday, 2020-07-13

*** jmasud has quit IRC00:48
*** jmasud has joined #oooq00:50
*** EmilienM|PTO is now known as EmilienM01:27
*** pojadhav|pto is now known as pojadhav02:56
*** weshay_ruck is now known as weshay_pto03:04
*** Tengu has quit IRC03:21
*** Tengu has joined #oooq03:22
*** skramaja has joined #oooq03:54
*** raukadah is now known as chandankumar04:20
*** Tengu has quit IRC04:21
*** Tengu has joined #oooq04:22
*** soniya29 has joined #oooq04:38
*** Tengu has quit IRC04:41
*** bhagyashris|away is now known as bhagyashris04:41
*** Tengu has joined #oooq04:42
*** ykarel has joined #oooq04:43
*** ykarel has quit IRC05:11
*** ykarel has joined #oooq05:12
*** marios has joined #oooq05:35
*** udesale has joined #oooq05:38
*** ysandeep|away is now known as ysandeep05:50
*** ysandeep is now known as ysandeep|afk05:51
*** ykarel has quit IRC06:15
*** Tengu has quit IRC06:21
*** Tengu has joined #oooq06:23
*** tosky has joined #oooq06:39
bhagyashrisysandeep|afk, hey jfyi akahat and me picked some jobs to work on it from this task https://tree.taiga.io/project/tripleo-ci-board/task/1813?kanban-status=144727406:46
*** ykarel has joined #oooq06:50
marioslooks like github.com is down (at least is here)06:55
bhagyashrismarios, yes07:00
mariosbhagyashris: thanks07:01
*** amoralej|off is now known as amoralej07:04
*** yolanda has joined #oooq07:07
mariosysandeep|afk: o/ hey can you check https://review.opendev.org/#/c/740440/3 don't we need it on ussuri too?07:11
marios(first and then train)07:11
*** ccamacho has joined #oooq07:13
mariosysandeep|afk: found it it didn't have the 'cherrypicked from' so it doesn't show up in the 'Cherry-Picks' on the right hand side of the Gerrit UI07:13
*** pojadhav is now known as pojadhav|afk07:17
chandankumarysandeep|afk: Hello07:31
chandankumarysandeep|afk: https://e0852df78584e7be60ec-9b6cbe136e0e918c88bba727d81cded7.ssl.cf2.rackcdn.com/739710/6/check/tripleo-build-containers-centos-7-rocky/db8a29b/logs/containers-build-errors.log is it a known issue07:31
*** ysandeep|afk is now known as ysandeep|rover07:53
ysandeep|roverchandankumar, will check07:54
ysandeep|rovermarios, thanks!07:54
ysandeep|roverbhagyashris, ack07:54
*** ysandeep|rover is now known as ysandeep|lunch08:16
*** dtantsur|afk is now known as dtantsur08:34
*** jbadiapa has joined #oooq08:40
*** ysandeep|lunch is now known as ysandeep|rover08:55
*** pojadhav|afk is now known as pojadhav08:58
*** derekh has joined #oooq09:00
*** sshnaidm|afk is now known as sshnaidm09:09
ysandeep|roverchandankumar, that error is not consistent.. I am running a test job to reproduce the failure - https://review.rdoproject.org/r/#/c/28446/09:20
chandankumarysandeep|rover: yes, it was random, after rebasing it passed, thanks for looking into it09:24
ysandeep|roverack :)09:29
*** ccamacho has quit IRC09:58
ykarelysandeep|rover, that issue is likely related to github outage09:58
ysandeep|roverykarel, yes its working fine now09:59
ykarelwhich is back to normal now https://www.githubstatus.com/09:59
*** zbr has joined #oooq10:34
chandankumarsshnaidm: marios https://review.opendev.org/#/c/739710/ when free, thanks!10:45
marioschandankumar: adding to queue10:45
chandankumarmarios: thanks!10:46
chandankumararxcruz: Hello11:11
arxcruzchandankumar: namaste11:11
chandankumararxcruz: regarding tempest skipped standalone scenario jobs, do we want to run upstream in periodic queue, as they will be constantly failing and they will not like it, do we want to move it to rdo side?11:11
arxcruzchandankumar: I though it would be on rdo side11:12
chandankumararxcruz: ok then let me put it on rdo side11:12
arxcruzchandankumar: there's already a tempest-skipped job just need to create the branch11:12
chandankumararxcruz: it is just plain standalone na?11:12
chandankumararxcruz: https://hackmd.io/fIOKlEBHQfeTZjZmrUaEYQ?view#2020-07-10 -> different scenarios11:13
arxcruzchandankumar: we can have a base, and create branch and override the scenario11:14
chandankumararxcruz: yes11:14
arxcruzi was about t o work on that but if you already started, i'm okay11:14
chandankumararxcruz: I am on it, you can guide me via review :-)11:14
arxcruzsure11:14
bhagyashrisysandeep|rover, hey fyi i just take up this task https://tree.taiga.io/project/tripleo-ci-board/task/1844?kanban-status=144727511:17
ysandeep|roverbhagyashris, i haven't taken a deeper look at logs, but you might need backport of this - https://review.opendev.org/#/c/724103/ for centos811:19
bhagyashrisysandeep|rover, ok for sc012. I just updated i mean to say rebase the patch i will take look11:25
bhagyashrisysandeep|rover, thank you :)11:25
ysandeep|roverbhagyashris, sure..11:25
ysandeep|roverplease let me know if you need any help11:25
bhagyashrisysandeep|rover, sure11:26
*** pojadhav is now known as pojadhav|afk11:31
*** bhagyashris is now known as bhagyashris|afk11:33
*** ccamacho has joined #oooq11:34
chandankumarmarios: pleae enqueue this also https://review.rdoproject.org/r/#/c/28451/11:35
marioschandankumar: ack11:39
*** amoralej is now known as amoralej|lunch12:03
*** rfolco has joined #oooq12:04
*** rlandy has joined #oooq12:05
*** rlandy is now known as rlandy|ruck12:06
rlandy|ruckzbr|ruck, ysandeep|rover: hey ... how are things today?12:08
*** udesale_ has joined #oooq12:09
rlandy|ruckzbr|ruck, ysandeep|rover: wrt master promotions, we may need to turn the wait time back12:11
ysandeep|roverrlandy|ruck, gate is good12:11
rlandy|ruckysandeep|rover: good :)12:11
rlandy|ruckysandeep|rover: master promotions are out for 5 days12:11
rlandy|ruckweshay_pto changes the wait time to 12 hours to give the mirrors a rest12:11
rlandy|ruckI think we are missing the hashes to promote now12:12
*** udesale has quit IRC12:12
rlandy|ruckwe have push fails12:12
rlandy|ruckysandeep|rover: zbr|ruck: ^^ which means I think that we need to bring the wait time down or keep more hashes12:13
rlandy|ruckwill ask panda about that at the promoter meeting12:13
rlandy|ruckthe rest is ok12:13
ysandeep|roverrlandy|ruck, okay , we are already missing panda :)12:14
rlandy|ruckysandeep|rover: I know :(12:14
rlandy|ruckysandeep|rover: we all need to learn the panda-ways now12:14
rlandy|ruckysandeep|rover: you haven't seen this anywhere on OVB have you? https://sf.hosted.upshift.rdu2.redhat.com/logs/openstack-component-tripleo/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-rhel-8-bm_envE-3ctlr_1comp-featureset001-tripleo-rhos-17/0a5d41b/logs/undercloud/home/zuul/overcloud_prep_flavors.log12:15
rlandy|ruckhitting both baremetal jobs12:15
rlandy|rucknot ovb downstream stream12:15
ysandeep|roverrlandy|ruck, i haven't noticed this12:16
zbrrlandy|ruck: wasn't a storage issue that forced us to limit the number of hashes?12:19
rlandy|ruckzbr: iiuc we don;t have the hash anymore to promote12:19
rlandy|ruckbecause of the 12 hours wait12:19
rlandy|ruckzbr: if we keep the promotions running so fast, the mirrors can't keep up and we get gate errors12:20
rlandy|ruckso we need to do something here to get master to promote12:20
rlandy|ruckzbr: are you coming to the promoter meeting?12:21
rlandy|ruckin about two hours?12:21
zbryes12:21
chandankumarrlandy|ruck:  https://review.opendev.org/#/c/736999/ and https://review.opendev.org/#/c/737015/ this12:21
chandankumarrlandy|ruck: wanted to talk about cr repo component pipeline12:21
chandankumarlet me when free12:22
chandankumarmarios: merging this https://review.opendev.org/#/c/732969/ if no objection12:24
*** bogdando has joined #oooq12:24
rlandy|ruckchandankumar: ok - let's talk about these two reviews and cr repo component pipeline after promoter meeting (in 2.5 hrs) - is that too late for you?12:24
rlandy|ruckchandankumar: if CIX call ends earlier - will ping you12:25
marioschandankumar: ack if you're sure per my comment otherwise could bring to scrum too12:25
chandankumarrlandy|ruck: sounds good to me12:25
chandankumarmarios: let's bring it to scrum12:25
*** derekh has quit IRC12:26
rlandy|ruckchandankumar: we should talk about the  cr repo in general12:26
rlandy|ruckI'll be at scrum12:26
chandankumarrlandy|ruck: sure12:26
ysandeep|roverrlandy|ruck, fyi.. pooja was trying to create periodic mutinode-ipa job for train branch https://review.rdoproject.org/r/#/c/28284/ , she hit some issue puppet duplicate decalaration issue, i wrote bz https://bugs.launchpad.net/tripleo/+bug/1887376 and cherry picked https://review.opendev.org/#/c/740679/ which i think should solve it. (testing currently)12:27
openstackLaunchpad bug 1887376 in tripleo "periodic-tripleo-ci-centos-8-standalone-on-multinode-ipa-train is failing with Error while evaluating a Resource Statement, Duplicate declaration: Exec[/etc/pki/CA/certs/vnc.crt]" [Medium,Triaged]12:27
rlandy|ruckzbr: CIX call12:32
rlandy|ruckI'm on it12:32
zbrrlandy|ruck: super, tx. ping me after, maybe we can have a meeting so i get a better view.12:33
*** bhagyashris|afk is now known as bhagyashris12:35
*** pojadhav|afk is now known as pojadhav12:35
rlandy|ruckzbr: will ping you after scrum - need to attend that meeting12:57
rfolcoping scrum13:00
rfolcoarxcruz, rfolco, zbr, panda, sshnaidm, rlandy, marios, ysandeep, bhagyashris, soniya29, pojadhav, akahat, weshay, chandankumar13:00
*** rlandy|ruck is now known as rlandy|ruck|mtg13:01
*** amoralej|lunch is now known as amoralej13:04
*** bogdando_ has joined #oooq13:05
rlandy|ruck|mtgbhagyashris: https://trello.com/c/SH18qURV/1479-cixlp1876893tripleociproa-error-error-removing-container-device-or-resource-busy13:06
*** bogdando has quit IRC13:07
*** bogdando_ is now known as bogdando13:07
*** derekh has joined #oooq13:09
mariostrying to merge https://review.opendev.org/#/c/739939/ topmost13:10
*** Tengu has quit IRC13:13
*** Tengu has joined #oooq13:15
*** soniya29 has quit IRC13:22
*** Goneri has joined #oooq13:26
*** udesale_ has quit IRC13:27
*** rlandy|ruck|mtg has quit IRC13:27
*** rfolco has quit IRC13:27
*** jbadiapa has quit IRC13:27
*** irclogbot_0 has quit IRC13:27
*** zbr|ruck has quit IRC13:27
*** TrevorV has joined #oooq13:27
*** udesale_ has joined #oooq13:28
*** rlandy|ruck|mtg has joined #oooq13:28
*** rfolco has joined #oooq13:28
*** jbadiapa has joined #oooq13:28
*** irclogbot_0 has joined #oooq13:28
*** zbr|ruck has joined #oooq13:28
chandankumarhttps://review.opendev.org/#/c/732969/13:38
chandankumar@all ^^ please review13:39
rlandy|ruck|mtgchandankumar: want to meet now?13:41
chandankumarrlandy|ruck|mtg: yes sure13:41
chandankumargetting gmeet link13:42
rlandy|ruck|mtgchandankumar: https://meet.google.com/hyi-ckhy-egg13:42
*** soniya29 has joined #oooq13:44
chandankumarrlandy|ruck|mtg: https://hackmd.io/iA6AyTn8Rn2ximODx6FmKg?view I will be add the thoughts here13:49
bhagyashrisrlandy|ruck|mtg,13:54
bhagyashrisHi13:54
bhagyashrisi will be away13:54
bhagyashrisonce will be back i will take a look and will discuss on https://trello.com/c/SH18qURV/1479-cixlp1876893tripleociproa-error-error-removing-container-device-or-resource-busy13:55
*** bhagyashris is now known as bhagyashris|afk13:56
rlandy|ruck|mtgbhagyashris: sure - no rush - before wednesday would be great13:56
bhagyashris|afkrlandy|ruck|mtg, sure .. ack13:57
bhagyashris|afk:)13:57
*** ykarel has quit IRC14:11
weshay_ptorlandy|ruck|mtg, master tried to promote over the weekend... some failure there14:19
weshay_ptoperfect thing to raise here14:19
rlandy|ruck|mtgweshay_pto: yep - already discussed with zbr to bring up jere14:19
rlandy|ruck|mtghere14:19
rlandy|ruck|mtgwe are missing hashes14:19
rlandy|ruck|mtgwe have seen this before14:20
weshay_ptomissing hashes or containers?14:20
rlandy|ruck|mtgpanda: ad7d4b111c58647200378977824cba7a14:20
weshay_ptothis one as well14:21
weshay_pto2020-07-13 06:37:21,886 2620 ERROR    promoter Containers promote 'aggregate: 5bc6464e2646a823e67beaf351fda5ff, commit: e93f9ff2b9693a390bd9ecc474a30d14e3152da0, distro: 5c42f9ed7923a234b746a6aa3b09cdeb3355c237, component: tempest, timestamp: 1594599168' to current-tripleo: Failed promotion end logs -----------------------------14:21
rlandy|ruck|mtgcommit_hash: f8c9720499600baad488b956210e4bbe3b5da5bc14:21
rlandy|ruck|mtg distro_hash: 8c0e50eaa860d4e4b316ba0f7cba8ef6142b5e0914:21
rlandy|ruck|mtghttp://osp-trunk.hosted.upshift.rdu2.redhat.com/rhel8-osp17/component/ui/f8/c9/f8c9720499600baad488b956210e4bbe3b5da5bc_8c0e50ea_5ef92eb1/commit.yaml14:22
*** ykarel has joined #oooq14:22
rlandy|ruck|mtg2020-07-13 00:09:32,352 8518 INFO     promoter Candidate hashes younger than target label current: [<DlrnAggregateHash object aggregate: ad7d4b111c58647200378977824cba7a, commit: f8c9720499600baad488b956210e4bbe3b5da5bc, distro: 8c0e50eaa860d4e4b316ba0f7cba8ef6142b5e09,14:24
rlandy|ruck|mtghttp://10.0.110.45/rhel8_rhos-17.log-2020071214:24
*** ysandeep|rover is now known as ysandeep|away14:32
pandahttps://hackmd.io/KyIJPdghQEqX1MFV974fnA14:35
chandankumarrlandy|ruck|mtg: still around14:42
rlandy|ruck|mtgchandankumar, ack on promoter meeting14:42
chandankumarrlandy|ruck|mtg: https://hackmd.io/iA6AyTn8Rn2ximODx6FmKg?view added the stuffs here, feel free to improve it14:43
chandankumarand add more questions14:43
*** ykarel is now known as ykarel|away14:45
*** skramaja has quit IRC14:45
*** Goneri has quit IRC14:50
rlandy|ruck|mtgok15:01
*** Goneri has joined #oooq15:03
rlandy|ruck|mtgzbr: ysandeep|away: note state of vexxhost15:05
rlandy|ruck|mtgdown atm15:05
*** udesale_ has quit IRC15:07
*** soniya29 has quit IRC15:17
rlandy|ruck|mtgweshay_pto: zbr: going onto promoter to restart service15:28
rlandy|ruck|mtgsending tmate15:28
zbrokey15:28
rlandy|ruck|mtgzbr: let's see if we get a successful master promotion now15:31
*** rlandy|ruck|mtg is now known as rlandy|ruck15:31
rlandy|ruck  promoter Running: env ANSIBLE_LOG_PATH=/home/centos/promoter_logs/container-push/20200713-153053.log ANSIBLE_DEBUG=False ansible-playbook -v -e @/tmp/tmpytoaqprm.yaml /home/centos/ci-config-refactored/ci-scripts/container-push/container-push.yml15:32
rlandy|ruckchandankumar: thanks - will comment on hackmd15:32
rlandy|ruck    output=stdout, stderr=stderr)15:37
rlandy|rucksubprocess.CalledProcessError: Command '['env', 'ANSIBLE_LOG_PATH=/home/centos/promoter_logs/container-push/20200713-153053.log', 'ANSIBLE_DEBUG=False', 'ansible-playbook', '-v', '-e', '@/tmp/tmpytoaqprm.yaml', '/home/centos/ci-config-refactored/ci-scripts/container-push/container-push.yml']' returned non-zero exit status 2.15:37
rlandy|ruck2020-07-13 15:33:13,860 24996 ERROR    promoter Containers promote 'aggregate: d5bc59bc3b7156b960d1ee790dc6afa1, commit: f8c9720499600baad488b956210e4bbe3b5da5bc, distro: 8c0e50eaa860d4e4b316ba0f7cba8ef6142b5e09, component: ui, timestamp: 1594627964' to current-tripleo: Failed promotion end logs -----------------------------15:37
rlandy|ruck2020-07-13 15:33:13,860 24996 ERROR    promoter Candidate hash 'aggregate: d5bc59bc3b7156b960d1ee790dc6afa1, commit: f8c9720499600baad488b956210e4bbe3b5da5bc, distro: 8c0e50eaa860d4e4b316ba0f7cba8ef6142b5e09, component: ui, timestamp: 1594627964': client registries_client FAILED promotion attempt to current-tripleo15:37
rlandy|ruck2020-07-13 15:33:13,860 24996 ERROR    promoter Failed to promote overcloud images15:37
rlandy|ruckzbr: ^^ no dice on container push again15:37
rlandy|ruckzbr: going to testproject the failure from last hash15:39
rlandy|ruckmaybe we can get that one to run15:39
rlandy|ruckzbr: actually the main line should kick again in about 20 mins15:43
rlandy|rucklets see if we get those containers15:44
rlandy|ruck2020-07-13 15:33:13,860 24996 ERROR    promoter Candidate hash 'aggregate: d5bc59bc3b7156b960d1ee790dc6afa1, commit: f8c9720499600baad488b956210e4bbe3b5da5bc, distro: 8c0e50eaa860d4e4b316ba0f7cba8ef6142b5e09, component: ui, timestamp: 1594627964': client registries_client FAILED promotion attempt to current-tripleo15:45
rlandy|ruck^^ was the last hash run15:45
* rlandy|ruck wondering if we still build rsyslog15:46
rlandy|ruckwe do https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-build-containers-ubi-8-push/25c5bbf/logs/buildah-builds/container-builds/d377f389-d2c1-41e8-9e40-b0dbf740fdb5/build-tree.yaml15:47
*** dsneddon has joined #oooq15:48
zbrrlandy|ruck: need one more vote on https://review.opendev.org/#/c/740299/815:48
zbror rfolco marios ^15:48
zbrfixing the last week logstash.txt issue15:49
zbrfixing 3 lines of bash, required adding 50 more of tests.15:50
zbras the original one had 0 testing15:50
rlandy|ruckdone15:50
rlandy|ruckmarios: was I on Wed's meeting invite, I don't see it - no prod chain meeting btw15:50
mariosrlandy|ruck: checking15:52
sshnaidmzbr, don't you want to see ci results before? https://review.opendev.org/#/c/74029915:52
mariosrlandy|ruck: my apologies you were not added now15:52
mariosrlandy|ruck: thanks for checking15:52
* marios EOD15:52
rlandy|rucknp15:53
*** marios is now known as marios|out15:54
*** marios|out has quit IRC16:13
*** bogdando has quit IRC16:20
*** amoralej is now known as amoralej|off16:22
rlandy|ruckzbr: pls ping me when you are EOD16:31
*** ykarel|away has quit IRC16:39
*** holser has quit IRC16:45
*** holser has joined #oooq16:45
*** derekh has quit IRC17:00
rfolcochandankumar, pls let me know if questions on jira - the workflow should the same and pretty simple. Speak up otherwise.17:12
*** dtantsur is now known as dtantsur|afk17:15
zbrrlandy|ruck: i am leaving17:19
rlandy|ruckzbr: k - have a good night17:19
rlandy|ruckwill try to promote master17:19
zbrthanks17:20
*** ccamacho has quit IRC20:52
rlandy|ruckrfolco: you around?21:09
rfolcorlandy|ruck, it depends :)21:10
rfolcorlandy|ruck, whazzup21:10
rlandy|ruckrfolco:  what interface should we use access rdocloud registry21:10
rlandy|ruckI just want to check that containers are there21:10
rlandy|ruckthe console is not responding21:10
rfolcothere is no console anymore21:10
rfolcohttps://trunk.registry.rdoproject.org:8443/oapi/v1/namespaces/tripleotraincentos8/imagestreamtags/21:10
rfolcocheck the hashes there ^21:11
rfolcoits the only way21:11
rlandy|ruckok so _i'm looking for master21:12
rlandy|ruckhttps://trunk.registry.rdoproject.org:8443/oapi/v1/namespaces/tripleomaster/imagestreamtags/21:12
rlandy|ruckrfolco: is there a way to get to the container itself21:12
rlandy|ruckor I just need to open every link21:13
rlandy|ruckah - raw data21:13
rfolcorlandy|ruck, replace the namespace and all the hashes will be there21:13
rfolcoyou have to do in rough mode21:13
rfolco:)21:13
rlandy|ruck"name": "centos-binary-aodh-api:a1045cf70c73c04420cf03821bb829d0",21:14
rlandy|ruckrfolco: thanks21:14
rfolcoyw21:15
rlandy|ruckrfolco: the promoter says the hash does not exist ... but it does21:15
rlandy|ruck2020-07-13 20:39:49,870 8668 ERROR    promoter changed: [localhost] => (item=aodh-api) => {"ansible_loop_var": "item", "changed": true, "cmd": "echo -n \"Checking trunk.registry.rdoproject.org/tripleomaster/centos-binary-aodh-api:a1045cf70c73c04420cf03821bb829d0 ... \";\ndocker manifest inspect    --insecure \"trunk.registry.rdoproject.org/tripleomaster/centos-binary-aodh-api:a1045cf70c73c04420cf03821bb829d0\"\nif [[ \"$?\"21:16
rlandy|ruck== \"0\" ]]; then\n    echo aodh-api >> /tmp/parsed_containers-centos-8-master.txt;\n    echo \"OK\";\nelse\n    echo \"FAIL\";\n    echo \"ERROR ========== centos-binary-aodh-api IS NOT BUILT! FIX THIS ASAP! ==========\";\n    exit 1\nfi\n", "delta": "0:00:01.484770", "end":21:16
rlandy|ruck^^ above21:16
rlandy|ruckrfolco: ^^ seen that?21:17
rfolcorlandy|ruck, hmmm... let me check21:17
rlandy|ruckrfolco: it's got the new token21:17
rlandy|ruckfor vexxhost21:17
rfolcoyeah docker manifest works from my staging env too21:19
rfolcorlandy|ruck, and that command also works from vexxhost-promoter... docker manifest inspect    --insecure trunk.registry.rdoproject.org/tripleomaster/centos-binary-aodh-api:a1045cf70c73c04420cf03821bb829d021:22
rlandy|ruckthis promoter will do me in21:22
rlandy|ruckrfolco: that worked from rdocloud promoter21:22
rlandy|ruckas well21:23
rfolcorlandy|ruck, maybe transient issue? I don't know :(21:23
rlandy|ruckno - I mentioned it this morning21:24
rlandy|ruckthought we were missing the hashes21:24
rlandy|ruckbut now I promoted right away21:24
rlandy|rucksame issue21:24
rlandy|ruckrfolco: ah ...  trunk.registry.rdoproject.org/tripleomaster/centos-binary-rsyslog:a1045cf70c73c04420cf03821bb829d021:26
rlandy|ruck "name": "centos-binary-rsyslog:a1045cf70c73c04420cf03821bb829d0",21:27
rlandy|ruckit's there21:27
*** TrevorV has quit IRC21:28
rlandy|ruckthe containers are all there21:30
*** TrevorV has joined #oooq21:31
*** TrevorV has quit IRC21:32
*** rlandy|ruck is now known as rlandy|bbl22:34
*** tosky has quit IRC22:42
*** holser has quit IRC23:33
*** holser has joined #oooq23:33
*** holser has quit IRC23:34
*** holser has joined #oooq23:34
*** holser has quit IRC23:36
*** holser has joined #oooq23:38

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!