Monday, 2022-04-25

*** rlandy is now known as rlandy|PTO00:12
*** jparker is now known as jparker|afk01:48
*** ysandeep|out is now known as ysandeep03:51
*** bhagyashris is now known as bhagyashris|ruck05:46
*** ysandeep is now known as ysandeep|afk06:01
*** amoralej|off is now known as amoralej06:05
*** ysandeep|afk is now known as ysandeep06:48
*** pojadhav is now known as pojadhav|afk07:31
*** jpena|off is now known as jpena07:35
* bhagyashris|ruck lunch brb07:41
*** pojadhav|afk is now known as pojadhav\08:26
*** pojadhav\ is now known as pojadhav08:26
akahatbhagyashris|ruck, o/ is it known issue: https://logserver.rdoproject.org/27/41827/3/check/tripleo-ci-centos-8-ovb-1ctlr_1comp-featureset002-upload/521ad6b/logs/undercloud/home/zuul/undercloud_install.log.txt.gz09:05
bhagyashris|ruckakahat, let me check09:46
bhagyashris|ruckakahat, yes09:46
*** rlandy|PTO is now known as rlandy10:33
rlandyhello all10:33
rlandybhagyashris|ruck: hello - how are things?10:34
rlandyyou're on with dviroel_?10:34
bhagyashris|ruckrlandy, hi yes with dviroel_ 10:34
bhagyashris|ruckrlandy, master cs9 is not in the good shape10:34
rlandybhagyashris|ruck: ok - just catching up on stuff10:35
bhagyashris|rucklooking into that 10:35
bhagyashris|ruckrlandy, ok10:35
rlandybhagyashris|ruck: let's sync when dviroel_ is in10:36
bhagyashris|rucksure10:37
chandankumarrlandy: welcome back, hope you had a great vacation :-)10:38
rlandychandankumar: I did - thank you10:39
rlandychandankumar: anything you need from my side?10:39
chandankumarbhagyashris|ruck: https://review.rdoproject.org/zuul/buildset/69072cfc9f2f4d93b42e14c55daada1710:44
chandankumarrlandy: nope, will let you know10:47
*** dviroel_ is now known as dviroel11:07
*** dviroel is now known as dviroel|rover11:07
dviroel|roverrlandy: bhagyashris|ruck i am ready 11:07
bhagyashris|ruckdviroel|rover, hey we have review time after 7 mins11:08
bhagyashris|ruckmay be after that ?11:08
dviroel|roverworks for me too11:10
rlandydviroel|rover: hello11:12
rlandychandankumar: ysandeep: added pSI resources to prod chain discussion11:12
rlandyanything else to raise today?11:13
dviroel|roverrlandy: welcome back 11:13
rlandychandankumar: ysandeep: any vexxhost issues?11:13
rlandydviroel|rover: thanks - will sync with you and bhagyashris|ruck after review time11:13
dviroel|roverok11:15
ysandeeprlandy, chandankumar review time11:16
bhagyashris|ruckrlandy, dviroel|rover chandankumar review time?11:16
bhagyashris|ruckakahat, ^11:16
bhagyashris|ruckpojadhav, ^11:16
dviroel|roverjoinning11:17
pojadhavbhagyashris|ruck, sorry cant join, need to go clinic having dentist appointment.11:38
* pojadhav brb for hr11:38
bhagyashris|ruckpojadhav, np11:38
ysandeepchandankumar, left a comment for you: https://review.opendev.org/c/openstack/tripleo-quickstart/+/838990/1#message-d26b955d4b71d4d027913177338e71db41c8264f12:14
ysandeepI can change as per your comment, but image is old: http://images.rdoproject.org/CentOS-9-Stream-x86_64-GenericCloud.qcow2.md5 12:18
* bhagyashris|ruck tea brb12:19
*** ysandeep is now known as ysandeep|afk12:25
*** amoralej is now known as amoralej|lunch12:27
rlandydviroel|rover: bhagyashris|ruck: looking at CIX board12:34
rlandythree cards in triage column12:34
rlandycan we review quickly12:35
rlandyhttps://bugzilla.redhat.com/show_bug.cgi?id=2077514 closed?12:36
dviroel|roverhum, last node_failure was on 22th12:37
dviroel|roverhttps://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/builds?result=NODE_FAILURE12:37
rlandydviroel|rover: bhagyashris|ruck: let's run though these quickly12:39
rlandyhttps://meet.google.com/pds-itzv-jov12:40
chandankumardpawlik: hello12:48
chandankumardpawlik: cs9 images on image server are not updated periodically https://review.opendev.org/c/openstack/tripleo-quickstart/+/838990/1#message-d26b955d4b71d4d027913177338e71db41c8264f12:49
chandankumardpawlik: can we add a job to do that?12:49
rlandychandankumar: are https://trello.com/c/PdYa8Bta/2464-cixbz2076450osp170rhel9rhos17all-osp17-rhel9-jobs-are-failed-with-retrylimit-having-error-nothing-provides-network-scripts-openv and https://trello.com/c/EZmictvR/2456-cixlp1968732tripleociproa-testupdaterouteradminstate-test-failed-with-unable-to-connect-to-port-22 related?12:51
Tenguhello there! can anyone point me where openstack-INPUT chain is defined in CI? I checked in oooq, oooq-extras and tripleo-ci, but apparently it's not in any of them12:52
Tenguissue is: that chain is wrongly defined imho, and clashes with https://review.opendev.org/c/openstack/tripleo-ansible/+/839163 due to the way it behaves.12:52
Tenguissue can be seen in the iptables-save here: https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_410/839163/1/check/tripleo-ci-centos-9-containers-multinode/410ba8b/logs/undercloud/var/log/extra/network.txt12:53
Tenguthe -A INPUT -j openstack-INPUT comes before the A INPUT -m comment --comment "Redirect to TRIPLEO_INPUT" -g TRIPLEO_INPUT (and some more rules actually)12:53
chandankumarrlandy: both are different issues12:53
Tenguand that openstack-INPUT has a   -A openstack-INPUT -j REJECT --reject-with icmp-host-prohibited  at the end of it. leading to network issues.12:54
chandankumarrlandy:  one is about missing packages related to new ovn version12:54
rlandychandankumar: anything needed from our side?12:54
chandankumarand another is regression12:54
chandankumarrlandy: nothing12:54
chandankumarone will be fixed by release delivery and another by ovn team12:54
chandankumarTengu: not sure you are looking for this https://opendev.org/openstack/tripleo-ci/src/branch/master/roles/build-containers/tasks/pre.yaml#L9412:57
* chandankumar has no idea how and why i added that12:58
chandankumarhttps://codesearch.opendev.org/?q=openstack-INPUT&i=nope&literal=nope&files=&excludeFiles=&repos=12:59
rlandy when: ansible_distribution_major_version is version(9, '<')12:59
rlandychandankumar: ^^12:59
chandankumarit is not linked with above job12:59
bhagyashris|ruckdviroel|rover, hey re - running rhos16.2 fs035  https://code.engineering.redhat.com/gerrit/c/testproject/+/31528512:59
dpawlikchandankumar: hey, I was downloading that image manually13:01
dpawlikshould I update?13:01
chandankumardpawlik: yes please13:02
dpawlikack, will do in few min13:02
Tenguchandankumar: that's a rule applied to the chain, not the one creating it13:03
rlandyTengu: as chandankumar mentions, there are a bunch of rules added13:03
Tenguah......13:03
Tenguare we able to change some of those rules?13:03
Tengujust to make it more compatible with a cleaner way to handle the rules13:03
dviroel|roverbhagyashris|ruck: ack, we can wait the results, and skip depending on the failure13:04
rlandyTengu: we can always change stuff - but there may be fallout13:04
rlandyI would guess the order was not well thought of13:04
Tengurlandy: ok. lemme submit a proposal.13:04
rlandywe may need a task to reorder or check the order13:04
Tengurlandy: I won't change the order actually. just remove the REJECT and use the INPUT chain policy.13:04
Tenguthat would have the same effect.13:05
Tengubut cleaner13:05
rlandyTengu: only an issue in multinode jobs?13:05
Tenguand more future-proof13:05
Tenguyeah13:05
rlandyk - pls submit the patch and we will review13:05
Tengu++13:05
rlandymay have something to do with the bridge setup13:05
rlandyif it's multnode only13:05
Tengunot really either :)13:05
Tengulemme create an LP for a proper tracking13:05
Tenguthen submit a patch13:05
chandankumarTengu: https://paste.centos.org/view/raw/7e0497b113:06
Tenguchandankumar: yeah - I see what I have to do :)13:07
Tenguthanks for the pointer - Iforgot about the codesearch.13:07
chandankumarTengu: codesearch was not helpful13:07
Tenguactually yes - pointed to https://opendev.org/opendev/system-config/src/branch/master/playbooks/roles/iptables/templates/rules.v4.j213:08
chandankumartricked via bridge comment from rlandy :-)13:08
chandankumarI was looking into that file13:08
dpawlikchandankumar: should I update c8 stream and c7?13:10
chandankumardpawlik: c8, we shoudl test first then we can update13:10
chandankumarI will let you know13:11
dpawlikchandankumar: k. Let me know later also which images I can remove: https://paste.openstack.org/show/bSJFGLqqAvQSDy6k0aSx/13:11
dpawlikchandankumar: and I guess centos 8 non stream can be remove, right?13:11
Tengurlandy, chandankumar https://bugs.launchpad.net/tripleo/+bug/197021113:13
Tengufyi13:13
TenguI'll submit a patch shortly.13:13
dpawlikchandankumar: c9 stream updated13:13
chandankumardpawlik:  yes centos8 is now centos stream13:13
rlandyTengu: thanks13:14
chandankumardpawlik: thanks!13:14
chandankumardpawlik: please remove cs8 images13:14
chandankumar*c813:14
dpawlikdone13:15
chandankumardpawlik: thanks!13:17
chandankumarTengu: thanks!13:17
*** amoralej|lunch is now known as amoralej13:18
Tenguchandankumar, rlandy https://review.opendev.org/c/opendev/system-config/+/83921013:22
Tengulemme know your thoughts.13:22
Tenguhan. there's a test.13:24
Tenguupdated.13:25
rlandyTengu: thanks  - adding to our review list13:26
Tenguoh. wait.13:26
Tenguhttps://opendev.org/openstack/project-config/src/branch/master/nodepool/elements/nodepool-base/install.d/20-iptables13:26
Tenguthere's... what ?!13:26
Tenguguess I'll need to update that on as well.13:26
Tengurlandy: I think that one has more chance to get the wanted effect: https://review.opendev.org/c/openstack/project-config/+/839212 (cc chandankumar )13:29
* chandankumar not sure how to test it in our CI, it is a trusted repo13:31
Tenguso fungi validated I had to hit the openstack/project-config one13:32
rlandyno way to do that13:32
rlandyunless we faked out that change elsewhere13:32
Tenguso we can't depends-on openstack/project-config changes?13:33
chandankumarTengu: yes correct, we cannot it use a depends on13:33
Tengudang13:33
Tengupretty sure I can't go with a "well, just trust me" :)13:33
rlandyno speculative testing on config repos13:34
rlandyyou can but you may break the world :(13:34
Tenguso, in this case, there is ONE difference: packets will be dropped instead of rejected [with a nice "message"]13:35
Tenguso if anything is expecting a REJECT, it will probably create some issue, such as time-out.13:35
Tenguthat's the only thing I can think about with that change.13:35
Tengulet's get more ppl on it so that we can get a clearer view, and some other ideas.13:36
*** ysandeep|afk is now known as ysandeep13:41
dasmo/13:41
rlandyTengu: thanks13:48
rlandychandankumar: dviroel|rover: did you send your slide to kieran?13:50
chandankumarrlandy: not yet, there are few suggestions on slides we are waiting for marios13:51
chandankumarreply, He will be back tomorrow.13:51
chandankumarLet's talk during scrum on when to send13:51
dviroel|roverchandankumar: rlandy lets sync before sending13:51
rlandyk - chatting with keiran now13:51
rlandylet's confirm at sprint meeting13:52
bhagyashris|ruckakahat, chandankumar rlandy pojadhav- 14:01
bhagyashris|ruckscrum time14:01
pojadhav-bhagyashris|ruck, joining14:01
* pojadhav- facing power failure issue from morning.. connected through mobile data (limited speed internet) :(14:02
dasmpojadhav-: if you can call in, that should spare you some data. you could just listen to the meeting14:03
Tengurlandy, chandankumar I think I have a better solution that won't involve any modification of the base firewall.14:04
dasmpojadhav-: gmeet can call you back on provided phone number14:04
pojadhav-dasm :) thank you I can hear properly, no need right now. if needed surely will try 14:06
*** pojadhav- is now known as pojadhav14:06
dasmpojadhav: ack. i had to use it several times in the past. it's subpar, but at least gives some alternative :)14:08
pojadhavdasm, yep :)14:08
ysandeeppojadhav-, https://github.com/rdo-infra/ci-config/blob/master/ci-scripts/infra-setup/roles/rrcockpit/files/telegraf_py3/telegraf.d/jobs_blocking_promotion.conf#L75-L90 please remove these in followup14:21
*** pojadhav- is now known as pojadhav14:22
pojadhavysandeep, yes14:22
ysandeeppojadhav, thanks i have +wed the patch14:23
pojadhavysandeep, thank you for +W:)14:23
ysandeepchandankumar, dviroel|rover at what time you want to meet tomorrow.14:51
chandankumarysandeep: when dviroel|rover arrives14:52
dviroel|roverysandeep: chandankumar 12pm utc?14:54
frenzy_fridayhey arxcruz, rcastillo, Jakob and I were discussing if it is possible to use ansible-test to test the openstack collections and check the code coverage. I have not used ansible-test before but the doc says it has different level of testing (sanity, integration etc..) Do you guys have some time tomorrow to discuss? Maybe after community call?14:54
chandankumarworks for me 14:54
arxcruzfrenzy_friday yes 14:55
ysandeepchandankumar, dviroel|rover works for me14:55
rcastillofrenzy_friday: after community call is fine for me14:56
frenzy_fridayack, thanks, sending invite14:56
ysandeepchandankumar, you sending invite?14:58
chandankumarysandeep: sending the invite14:59
ysandeepchandankumar, thanks.. we can also meet marios in our morning14:59
chandankumarysandeep: yes15:00
chandankumarysandeep: let's check with marios in the morning15:00
rlandyysandeep: chandankumar: dviroel|rover: marios: sending you the link to the main deck15:00
chandankumarysandeep: dviroel|rover rlandy invite sent15:00
rlandykieran asked that you add the slides to that deck15:00
ysandeeprlandy: thanks!15:01
rlandyysandeep: chandankumar: dviroel|rover: marios: see slide 2 for the order15:02
rlandyyou're after LT update15:02
rlandyand DFG info sharing15:02
dviroel|roverack15:03
chandankumarsee ya!15:21
rlandydviroel|rover: bhagyashris|ruck: rerunning periodic-tripleo-ci-centos-8-standalone-wallaby15:24
rlandyfailing test in current line15:24
rlandydviroel|rover: bhagyashris|ruck: need help with master?15:24
rlandybhagyashris|ruck: I see you are rerunning 16.2 fs03515:25
rlandydid we decide that is safe to skip and promote?15:25
rlandyif this fails?15:25
dviroel|roverrlandy: we can check this latest run on 16.2 and decide to skip or not, wdyt?15:26
rlandydviroel|rover: ack15:26
dviroel|roverrlandy: i am rerunning latest tested hash on master, will do a deep investigation after lunch15:27
dviroel|roverrlandy: the hash that i mentioned earlier, for master, is this one: https://trunk.rdoproject.org/api-centos9-master-uc/api/civotes_agg_detail.html?ref_hash=84357a6fbb3ee36ac0539f7c37bde60d15:27
dviroel|rovermissing fs039 only15:30
* dviroel|rover lunch, biab15:30
*** dviroel|rover is now known as dviroel|rover|lunch15:30
rlandyrcastillo: alan said vexx has enabled nested virt15:37
*** ysandeep is now known as ysandeep|out15:38
rcastillorlandy: I can try a manual run there if that's better than PSI15:38
rlandyrcastillo: pls hang on that15:39
rlandyI want to test if kvm works first15:39
rcastillorlandy: k15:39
rlandyif that works, we know we have nested virt15:39
rcastillothat'd be sweet15:40
*** amoralej is now known as amoralej|off15:49
dpawlikfrenzy_friday: Quay rdo has been updated to new release. Let me know if you have some troubles 15:57
frenzy_fridaydpawlik, thanks for the info. rerunning our testjob patch for train and master15:58
rlandydviroel|rover|lunch: cool - ping if there is something I need to pick up16:10
*** dviroel|rover|lunch is now known as dviroel|rover16:17
rlandyarxcruz: 1-1?16:30
rlandyarxcruz: dropping - ping if you need something16:34
* rlandy going to lunch16:35
*** jpena is now known as jpena|off16:46
rlandyrcastillo: hello17:19
rlandyTengu said LADA is not accessible17:20
rlandycan you look into that?17:20
rlandydviroel|rover: can you nick in internal channel17:39
rlandysays you are out17:40
rlandyjm1: you still around?17:40
dviroel|roverrlandy: done17:42
jm1rlandy: yes17:43
rlandyjm1: Tengu says LADA is down17:43
rlandyI can't access https://wimp.usersys.redhat.com/17:43
jm1rlandy: lemme check17:44
rlandycan you check that vm?17:44
rlandythanks17:44
rcastillorlandy: sorry for missed ping, went to make coffee17:45
rlandyrcastillo: np - asked jm1 to look at it17:45
rlandyrcastillo: jm1: if it's still hosted on the same box as the hardawe sheet says, I can't ping it17:46
jm1rlandy: host is down, sshnaidm is looking17:46
jm1rcastillo: ^17:46
rlandyjm1: we can't keep pinging sshnaidm 17:48
rlandyjm1: rcastillo: how is the internal cockpit up then>17:49
rlandysame box17:49
rlandyor did that move?17:49
jm1rlandy: its his box. so i wont change things without asking him17:51
rcastillodns issue?17:52
rlandyjm1: rcastillo: we need to get this on a box that sshnaidm does not own or get ownership of that box17:56
rlandyit was on my box17:57
jm1rlandy: my last update from Tengu was that wimp.usersys.redhat.com is now at lada.lab4.eng.bos.redhat.com17:57
Tengui may consider hosting lada back on my own infra. might be easier for everyone.17:57
Tengujm1: both should resolve to the same ip. seems the usersys wasnt updated for some time17:58
rlandyTengu: sshnaidm is no longer with us - so anywhere as long as it's not a box that belongs to someone outside17:58
Tenguleading to its (temporary) removal17:58
Tengurlandy: yeah. ill check that tomorrow.17:59
jm1Tengu: does dns resolution still work for you for wimp.usersys.redhat.com?17:59
rlandyTengu: do you have appropriate hardware to host it?17:59
*** rlandy is now known as rlandy|mtg18:00
Tengujm1: not on the right network atm, wasnt working earlier. but was ok last week18:00
Tengurlandy|mtg: yeah. my local gw. should be enough18:00
Tenguor any nuck behind it18:00
rlandy|mtgTengu: ok - so best to get that off sshnaidm's box18:01
Tenguyup. will work on that this week.18:01
rcastillodig @$INTERNAL_DNS_SERVER $INTERNAL_COCKPIT_URL isn't giving me any A records either18:01
jm1rcastillo: internal cockpit is working with static dns entry18:03
jm1Tengu: the other vm on that host is working although we cannot access the host and your vm atm. maybe the host is dying. its idrac does not look healthly: Correctable memory error rate exceeded for DIMM_B6.18:04
Tenguhmmm. if i move lada to my local 3nv, ill stick to the usersys18:04
jm1Tengu: @usersys good luck 😉18:04
Tengujm1: ah. yeah. great..... a reboot might kill it for good18:04
Tenguwell, usersys isnt dead yet. but its future is unsure18:05
Tenguwould be great to get the official app maintained by the right team....18:05
jm1Tengu: not dead yet? looks pretty dead atm at least ..18:06
* Tengu ducks and hides18:06
Tengunah18:06
jm1Tengu: just temporary hickups? ^^18:06
Tenguare you able to dig cjeanner.usersys.redhat.com ?18:06
jm1Tengu: yep. do they drop dns entries from usersys.redhat.com if the dont get updated regularely?18:07
Tenguexactly18:08
jm1Tengu: pff...18:09
Tenguyou have to update each 5 minutes, else it's dropped until it's refreshed18:09
Tenguunreliable18:09
Tengubut that's how it currently works18:09
jm1Tengu: probably makes sense somehow18:10
Tenguanyway. will move the app on my env. Will probably dedicate a NUC18:10
jm1Tengu: ack. do you need anything from the old vm?18:11
Tengunope18:11
jm1Tengu: ack. one probably cannot trust it anyway... with ram errors18:11
Tenguyep18:11
Tengufun time ahead18:13
Tenguanyway. I'll qoek on that tomorow.18:14
Tenguthanks for checking, jm1, rlandy|mtg 18:14
jm1Tengu, rlandy|mtg: we are in now, will have a look18:17
rlandy|mtgjm1: thanks - let me know if that stuff needs to move18:18
Tengujm1: ah. try rebooting the vm.18:26
Tenguthat will give me some delay to move the app around18:26
jm1Tengu, rlandy|mtg: found and fixed the issue, also for the future.18:51
rlandy|mtgjm1++ - what was it?18:51
rlandy|mtgjm1: also - can we move that to a box we own?18:52
rlandy|mtgdviroel|rover: how are you with master? need any help there?18:52
dviroel|roverrlandy|mtg: last testproject finished with 3 ovbs missing - https://review.rdoproject.org/r/c/testproject/+/36976/31#message-80582d280a4b7a15915b751d5a33edbfacb36d2718:53
jm1rlandy|mtg: there was a outage or something in the datacenter. when the server was rebooted, it did not have network, so dhclient failed. followed this link to let dhclient try to get an ip for ever https://access.redhat.com/solutions/244518:53
jm1rlandy|mtg: sshnaidm gave me his login credentials, so we will be able to solve issues like that in future immediately ;)18:54
jm1rlandy|mtg: lets talk about moving the vm tomorrow. not sure it pays off18:56
rlandy|mtgjm1: ack - thanks - let's talk then18:57
rlandy|mtgjm1: also - maybe sshnaidm can give you ownership of the box18:57
jm1rlandy|mtg: you are owner iirc18:57
rlandy|mtgthen I can transfer to you18:57
jm1rlandy|mtg: or something18:58
rlandy|mtglet's confirm tomorrow18:58
sshnaidmrlandy|mtg, currently it's loaned by you18:58
sshnaidmbut it doesn't matter18:58
sshnaidmwho is the owner18:58
rlandy|mtgsshnaidm: hey - ok - just need to get this off your plate :)19:01
rlandy|mtgyou have other duties now19:01
*** rcastillo_ is now known as rcastillo19:02
rlandy|mtgdviroel|rover: nice 16.2 promoted19:02
rlandy|mtgand rhel 9 1719:03
sshnaidmrlandy|mtg, yeah, I gave my creds to jm1, so he can do all I can do19:03
rlandy|mtgsshnaidm++ nice thank you19:03
rlandy|mtgsshnaidm: hope all is going well on the new team19:03
*** rlandy|mtg is now known as rlandy19:03
sshnaidmrlandy, pretty much, thanks :)19:06
Tengujm1: ack - let's sync tomorrow19:09
dviroel|roverrlandy: yeah, without skipping19:10
rlandynice19:10
* jm1 out for today. have a nice evening :)19:17
rlandydviroel|rover: need help with master ovb?19:24
rlandyor wallaby c8?19:24
dviroel|roverrlandy: yeah, starting with master ovb19:28
dviroel|roverrlandy: fs020 seams to be a known issue - https://logserver.rdoproject.org/76/36976/31/check/periodic-tripleo-ci-centos-9-ovb-1ctlr_2comp-featureset020-master/03e01f5/logs/undercloud/var/log/tempest/stestr_results.html.gz19:28
* dviroel|rover looking cirros bug19:28
rlandyk - ping of you need help - otherwise working on some planning docs19:28
dviroel|roverrlandy: so, on fs020 we have different tempest tests failing, but the errors looks like the same (https://bugs.launchpad.net/tripleo/+bug/1968732)19:42
dviroel|roverrlandy: fs039 and fs064 - failing on overcloud deploy, same point of failure - this is new for me, but also happenned on previous run today19:43
dviroel|roverwe are missing those 3 for promotion19:44
dviroel|roverrlandy: there is a new hash running now, deploying overcloud on some ovb jobs - that is my hope for now :(19:46
dviroel|roverrlandy: https://bugs.launchpad.net/tripleo/+bug/197026220:04
rlandydviroel|rover: thanks20:17
dviroel|roveroh, featureset064-master just passed in the newest hash 20:30
dviroel|roverwill rerun 001, and 039 on this new hash20:30
rcastillolunch, brb20:47
*** dasm is now known as dasm|off22:06
dasm|offhave a good afternoon team!22:06
dasm|offo/22:06
*** rlandy is now known as rlandy|bbl22:16
*** dviroel|rover is now known as dviroel|rover|afk22:36

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!