Monday, 2020-03-23

« Sunday, 2020-03-22 Index Tuesday, 2020-03-24 »
*** jmasud has quit IRC00:54
*** jmasud has joined #oooq00:55
*** udesale has joined #oooq04:13
*** ykarel|away is now known as ykarel04:32
*** jmasud has quit IRC04:53
*** jmasud has joined #oooq04:56
*** ratailor has joined #oooq05:46
*** jfrancoa has joined #oooq05:57
*** saneax has joined #oooq06:39
*** matbu has joined #oooq07:10
*** dtantsur|afk is now known as dtantsur07:18
*** dpawlik has joined #oooq07:44
*** jbadiapa has joined #oooq08:08
*** ykarel is now known as ykarel|lunch08:31
*** tosky has joined #oooq08:32
*** bogdando has joined #oooq08:33
*** arxcruz|off is now known as arxcruz|rover08:37
*** matbu has quit IRC08:39
*** matbu has joined #oooq08:40
*** amoralej|off is now known as amoralej08:41
*** jpena|off is now known as jpena08:52
*** jmasud has quit IRC08:53
*** jmasud has joined #oooq08:56
*** ykarel|lunch is now known as ykarel09:14
*** ccamacho has joined #oooq09:14
*** saneax has quit IRC09:15
*** apetrich has joined #oooq09:16
*** chem has joined #oooq09:19
*** saneax has joined #oooq09:32
*** holser has joined #oooq09:35
*** derekh has joined #oooq09:43
*** dpawlik has quit IRC10:11
*** dpawlik has joined #oooq10:13
*** sshnaidm is now known as sshnaidm|pto10:53
arxcruz|roverchandankumar: around ?10:56
chandankumararxcruz|rover, yes10:57
arxcruz|roverchandankumar: i'm getting an error on rhel8 build image10:57
arxcruz|roverhttps://logserver.rdoproject.org/97/714297/1/openstack-check/tripleo-rhel-8-buildimage-ironic-python-agent/e8282db/ironic-python-agent.log10:57
arxcruz|roverchandankumar: I really don't know why the permission denied here, it's copying the hooks properly, but dib-run-parts fail with permission denied, and the log doesn't show exactly where10:57
arxcruz|rovercan you help me out ?10:58
chandankumararxcruz|rover, checking10:58
chandankumar2020-03-22 13:30:18.004 | /home/zuul/workspace/venv/lib/python3.6/site-packages/diskimage_builder/lib/common-functions: line 264: /home/zuul/workspace/venv/lib/python3.6/site-packages/diskimage_builder/lib/dib-run-parts: Permission denied10:59
chandankumarwill be looking in 10 mins10:59
arxcruz|roverchandankumar: yeah, so, i check the dib-run-parts, there a bunch of hooks installed there10:59
*** matbu has quit IRC11:01
*** matbu has joined #oooq11:07
arxcruz|roverykarel: around?11:21
arxcruz|roverykarel: https://bugs.launchpad.net/tripleo/+bug/186733311:21
openstackLaunchpad bug 1867333 in tripleo "[tripleo-build-containers-rhel-8-master]No match for argument: collectd-pcie-errors" [Critical,Triaged] - Assigned to Arx Cruz (arxcruz)11:21
ykarelarxcruz|rover, yes11:22
ykarelarxcruz|rover, rhel8 master still needed?11:22
arxcruz|roverykarel: i guess so, weshay|ruck asked me to take a look on this11:23
ykarelarxcruz|rover, https://review.opendev.org/#/c/710029/11:24
ykarelrecently added on 16th march11:24
ykarelso please check if rhel8 is still relevant, as it seems that package is missing from rhel811:25
chandankumararxcruz|rover, we have removed rhel-8 master jobs in favor of c8, if still present remove it from pipeline11:28
arxcruz|roverweshay|ruck: ^11:34
ykarelarxcruz|rover, i checked collectd-5.8 from rhui custom deps don't have pcie-error package11:39
chandankumararxcruz|rover, I am on pto today, please take over this patch https://review.rdoproject.org/r/#/c/26035/11:40
*** skramaja has joined #oooq11:40
*** rlandy has joined #oooq11:44
*** udesale_ has joined #oooq11:44
rlandychandankumar: hi ...11:46
rlandyhow's it looking today?11:46
*** udesale has quit IRC11:47
*** skramaja has quit IRC11:54
*** skramaja has joined #oooq11:54
*** rfolco has joined #oooq11:58
rlandyarxcruz|rover: hi .. re escalation meeting and cards12:02
rlandyhttps://bugs.launchpad.net/tripleo/+bug/1868079 - see we still have that failure in the integration line12:03
openstackLaunchpad bug 1868079 in tripleo "puppet-neutron container failed" [Critical,Triaged]12:03
rlandyzbr: hi - can we close https://trello.com/c/6QFim7tI/1399-cixlp1868120tripleociproa-centos-7-ovb-jobs-failing-in-build-test-packages-due-to-pip-install-pip-failure out?12:07
arxcruz|roverykarel: can we add it ?12:11
ykarelarxcruz|rover, yes we can think of it if that job is needed still, but last i heard they are gone, so good to confirm that first12:11
arxcruz|roverrlandy: do you know if rhel-8 jobs are still relevant?12:12
arxcruz|roverrlandy: wasn't supposed to happen the puppet-neutron issues?12:12
rlandyarxcruz|rover: rhel jobs are not relevant in master12:13
rlandywhat failures are you looking at?12:13
rlandyarxcruz|rover: the neutron issues were supposed to be cleared - let me check the tht patch12:13
zbrrlandy: yeah12:14
weshay|ruckarxcruz|rover, 0/12:15
rlandyarxcruz|rover: the last comment says to see what happens in the next build - which was friday - so we still have that in the integration line12:17
rlandyweshay|ruck: hey12:17
weshay|ruckhey12:17
rlandyweshay|ruck: you attending escalation meeting? if not, I will12:17
weshay|ruckrlandy, arxcruz|rover and I can cover12:18
weshay|ruckbut thanks12:18
rlandyk12:18
arxcruz|roverweshay|ruck: hey12:21
arxcruz|roverweshay|ruck: so, you asked about the di-builder failure on rhel-8, is it still relevant ?12:21
weshay|ruckarxcruz|rover, what do you mean relevant?12:24
weshay|ruckarxcruz|rover, re: the last one worked?12:25
weshay|ruckif it's working.. we're good12:25
arxcruz|roverweshay|ruck: i mean, it's rhel-8 job12:25
weshay|ruckarxcruz|rover, train is relevant12:26
arxcruz|roverweshay|ruck: ack12:26
arxcruz|roverweshay|ruck: so https://bugs.launchpad.net/tripleo/+bug/1867333 isn't relevant anymore right ?12:27
openstackLaunchpad bug 1867333 in tripleo "[tripleo-build-containers-rhel-8-master]No match for argument: collectd-pcie-errors" [Critical,Triaged] - Assigned to Arx Cruz (arxcruz)12:27
rlandy2020-03-18T17:33:27.340748053+00:00 stderr F <13>Mar 18 17:33:27 puppet-user: No such file or directory @ rb_sysopen - /etc/neutron/plugins/networking-ovn/networking-ovn-metadata-agent.ini .. hmmm12:28
weshay|ruckarxcruz|rover, correct.. remove that job https://review.rdoproject.org/zuul/job/tripleo-build-containers-rhel-8-master12:29
weshay|ruckarxcruz|rover, put up changes to remove the job https://bugs.launchpad.net/tripleo/+bug/1867333 added comment12:29
openstackLaunchpad bug 1867333 in tripleo "[tripleo-build-containers-rhel-8-master]No match for argument: collectd-pcie-errors" [Critical,Won't fix] - Assigned to Arx Cruz (arxcruz)12:29
rlandyrfolco: hey - welcome back12:31
rfolcorlandy, thanks :)12:31
arxcruz|roverrfolco: welcome my friend, welcome to the machine...12:31
rfolcoarxcruz|rover, how crazy rover week is ?12:32
arxcruz|roverrfolco: counting the days to end :)12:33
rfolcoarxcruz|rover, I'm probably the next, so need a proper hand off12:34
arxcruz|roverrfolco: it's not so bad tbh...12:34
arxcruz|roverykarel: so, pcie-errors isn't required :)12:35
ykarelarxcruz|rover, yes if no rhel8 in master, no pcie-errors needed to be updated12:38
EmilienMrlandy: hey12:41
EmilienMrlandy: I was curious, how useful (or not?) did you find the logging output on https://bugs.launchpad.net/tripleo/+bug/186766412:41
openstackLaunchpad bug 1867664 in tripleo "Master periodic jobs are failing overcloud deploy with ''Container(s) with bad ExitCode: [''container-puppet-neutron''], check logs in /var/log/containers/stdouts/'" [Critical,Triaged]12:41
EmilienMrlandy: it's something new with paunch's replacement, I'm seeking for feedback12:41
rlandylooking12:42
rlandyEmilienM: good - the error was clear12:43
EmilienMack12:43
rlandyEmilienM: in this case , the log wasn't long - I think the issue is in some cases, you have to scroll through a ton of lines to find the failure12:44
EmilienMyeah :/ that's what I'm trying to improve12:44
rlandywhen there is no clear 'Error', that's the hard part12:44
arxcruz|roverweshay|ruck: https://review.rdoproject.org/r/#/c/26050/12:45
*** ratailor has quit IRC12:50
weshay|ruckarxcruz|rover, ya.. so don't confuse the master rhel8 container build issues w/ train.. only train is required12:53
weshay|ruckstill failing12:53
*** jmasud has quit IRC12:53
arxcruz|roverweshay|ruck: ack12:54
weshay|ruckarxcruz|rover, still failing in your patch too12:54
weshay|ruckarxcruz|rover, grep: /tmp/kolla-*/docker: No such file or directory12:54
arxcruz|roverweshay|ruck: not only that, there's a --work-dir error as well, i'm checking12:55
weshay|ruckarxcruz|rover, ah.. think we need to cherry pick a couple patches from EmilienM12:55
EmilienMwhich ones?12:55
*** jpena is now known as jpena|lunch12:56
weshay|ruck--work-dir /me finds12:56
*** jmasud has joined #oooq12:56
EmilienMah that12:56
EmilienMtripleoclient/tripleo-common12:56
ykarelmmm but why only rhel8 fails?12:56
ykarelfor work-dir12:56
ykarelthere may be some other issue if centos7 train is passing12:57
ykareland that shouldn't be failing else patches would have blocked since then12:57
weshay|ruckhttps://review.opendev.org/#/c/709039/12:57
weshay|ruckykarel, I'm looking at results from arx's patch.. not periodic atm12:58
weshay|ruckhttps://review.opendev.org/#/c/713676/12:58
weshay|ruckarxcruz|rover, fix conflicts and cherry pick to train please https://review.opendev.org/#/c/709039/12:59
ykarelweshay|ruck, ack i think it fails without that patch too, but good to check12:59
weshay|ruckarxcruz|rover, and rerun your patch w/ a depends-on to the cherrypick12:59
weshay|ruckscrum time.. will be short13:00
ykarelso likely ci_branch is not set correctly13:00
rlandyrfolco: panda: chandankumar: scrum13:00
rlandyarxcruz|rover: ^^13:01
weshay|ruckchandankumar, get off irc13:02
ykarelweshay|ruck, https://github.com/rdo-infra/rdo-jobs/blob/master/zuul.d/build-containers.yaml#L4313:05
ykarelrelease missing13:05
ykarelso ci_branch is wrongly set to master, and thus --work-dir wrongly used13:05
rlandyweshay|ruck: https://bugs.launchpad.net/tripleo/+bug/186793113:09
openstackLaunchpad bug 1867931 in tripleo "periodic-tripleo-ci-centos-8-scenario010-standalone-network-master deploy fails with package conflict - "openstack-neutron-common-1:16.0.0-0.20200318094753.8c7ca77.el8.noarch conflicts"" [Critical,Triaged]13:09
*** amoralej is now known as amoralej|lunch13:09
*** ykarel is now known as ykarel|away13:10
* ykarel|away out, need to leave13:11
weshay|ruckarxcruz|rover, https://review.opendev.org/#/c/709117/5/roles/build-containers/templates/build.sh.j213:19
*** ysandeep is now known as ysandeep|away13:21
*** TrevorV has joined #oooq13:38
weshay|ruckrlandy, so.. the network component did not promote properly13:39
rlandyweshay|ruck: it promoted last week13:42
weshay|ruckrlandy, well.. it didn't13:42
weshay|ruckrlandy, I see the tick in the cockpit for the promotion.. but the promoted_components rpms did not update13:43
weshay|ruckrlandy, arxcruz|rover and I will manually promote13:43
weshay|ruckif that's ok13:43
rlandyweshay|ruck: yeah13:43
weshay|ruckrlandy, see https://trunk.rdoproject.org/centos8-master/component/network/component-ci-testing/openstack-neutron-16.0.0-0.20200321090239.b319d64.el8.src.rpm13:43
rlandybut we need to check why it didn't promote13:43
weshay|ruckhttps://trunk.rdoproject.org/centos8-master/component/network/tripleo-ci-testing/openstack-neutron-15.1.0-0.20191218200030.8375f4c.el8.noarch.rpm13:44
weshay|ruckrlandy, aye13:44
rlandylooking into it13:44
rlandyweird - says it promoted on the 20 and 21st13:45
rlandyweshay|ruck: have you seen that with any other components?13:46
*** pojadhav is now known as pojadhav|away13:48
rlandyweshay|ruck: did you guys promote yet? https://trunk.rdoproject.org/centos8-master/component/network/promoted-components/ still looks old13:55
weshay|ruckrlandy, no..  rlandy dlrnapi thinks it promoted13:56
weshay|ruckwe're on the promotion server tmux now13:56
weshay|ruckrlandy, if you want insight.. https://meet.google.com/oiv-geho-mai?authuser=113:57
*** jpena|lunch is now known as jpena13:59
*** amoralej|lunch is now known as amoralej14:06
arxcruz|roverweshay|ruck: did you die?14:06
weshay|ruckrlandy, arxcruz|rover sorry network14:09
weshay|ruckrlandy, arxcruz|rover http://dashboard-ci.tripleo.org/d/UDA4H3aZk/component-pipeline?orgId=114:15
*** Trevor_V has joined #oooq14:23
*** TrevorV has quit IRC14:27
weshay|ruckarxcruz|rover, hrm..14:31
weshay|ruckarxcruz|rover, we may already have this14:31
arxcruz|roverweshay|ruck: ?14:31
weshay|ruckarxcruz|rover, https://meet.google.com/kzx-hnid-xyv?authuser=114:31
* weshay|ruck shows u14:31
weshay|ruckarxcruz|rover, JfV9AdPZP3kKwXCSxtqPRsJHs@sfo2.tmate.io14:32
*** TrevorV has joined #oooq14:34
*** Trevor_V has quit IRC14:37
amoralejis fs033 used anywhere? https://opendev.org/openstack/tripleo-quickstart/src/branch/master/config/general_config/featureset033.yml14:46
amoralejit uses some repos in https://cbs.centos.org/repos which will be removed on wednesday14:47
rlandyfs033 - scenario00914:50
rlandyI don't think we run that anymore - checking14:50
rlandyamoralej: it does run ... http://zuul.openstack.org/builds?job_name=tripleo-ci-centos-7-scenario009-multinode-oooq-container14:55
rlandyalthough always failing14:55
amoraleji'll see if i can replace those repos by the ones in mirrors14:55
amoralejit's using origin311 from cbs which is not very updated i'd say14:56
*** bogdando has quit IRC14:57
weshay|ruckrlandy, thinking about switching to this view for promotions https://snapshot.raintank.io/dashboard/snapshot/qVNWJSslKvDPb7Rw1YeiGvHhtju1Bj5q?orgId=215:00
weshay|ruckrlandy, see baremetal.. wdyt?15:00
rlandyweshay|ruck: so I see the baremetal component ... no promotions diagram15:02
rlandyso there is no history on promotions15:02
weshay|ruckrlandy, switching from a graph to a table in baremetal components promotions15:02
weshay|ruckrlandy, there will be... we only have on in my dev env15:03
rlandyoh I see15:03
weshay|ruckrlandy, so we'd have a list w/ date and hash promoted15:03
rlandyweshay|ruck: that's file15:03
rlandyfine15:03
rlandyweshay|ruck: so the hash shown is the hash that should match promoted_components15:04
rlandynot component-ci-testing15:04
rlandyI think with clients, it rev'ed after client tests ran15:04
rlandystill checking tempest15:04
rlandyI think we should have promoted there15:04
EmilienMrlandy: hey15:31
EmilienMhow often do we have master promotions nowadays?15:31
rlandyEmilienM: hi15:31
EmilienMsorry you're my go-to person today, weshay|ruck is ignoring me as usual15:31
rlandyEmilienM: it's not automatic atm - we turned off the promoter15:32
EmilienMah ok15:32
rlandyso we promoted once last week15:32
EmilienMwhat should I do if I need a new ansible-pacemaker in master?15:32
rlandyand we will turn it on again15:32
rlandyin current-tripleo?15:32
EmilienMyeah15:32
rlandyEmilienM: we can manually promote15:32
EmilienMWOW15:32
EmilienMthat would be awesome15:33
EmilienMi realized my patch isn't working though15:33
EmilienMmatbu: https://trunk.rdoproject.org/centos8-master/report.html15:33
EmilienMseems like ansible-pacemaker fails to build.15:33
rlandylet me check the next integration run15:33
EmilienMoh I know15:34
EmilienMmatbu: https://review.rdoproject.org/r/2605515:35
rlandyEmilienM: master pipeline is due to kick again at 16:00 utc15:35
rlandyabout half an hour15:35
EmilienMyea15:35
EmilienMI'll get this thing merged before (tm)15:35
rlandyif that run shows neutron to be passing now, we can try promote15:35
weshay|ruckEmilienM, ya.. master is blocked on a neutron issue15:36
rlandyso that change will have to hit component first15:36
weshay|ruckso give us a day.. and I need panda back to check our tooling15:36
EmilienMweshay|ruck: sorry I only talk to rlandy from now15:36
weshay|ruckEmilienM, :)15:36
weshay|ruckEmilienM, good15:37
rlandyif neutron runs clean, we have a good shot15:37
EmilienMcool15:39
pandaweshay|ruck: rlandy EmilienM to promote manually this needs to merge: https://review.rdoproject.org/r/2599215:46
*** panda is now known as panda|off15:46
rlandylooking15:46
weshay|ruckrlandy, EmilienM master is not going to promote today15:47
EmilienM:( ok15:47
*** skramaja has quit IRC15:52
rlandyweshay|ruck: any issue with merging https://review.rdoproject.org/r/25992?15:53
*** dsneddon has joined #oooq15:57
weshay|ruckrlandy, /me looking but I don't care much about the cli atm16:00
weshay|ruckrlandy, /me looking for panda's latest bug fix16:00
weshay|ruckso we promote the right tripleo-ci-testing16:00
rlandyweshay|ruck: meaning?16:00
weshay|ruckmaybe that merged16:01
weshay|ruckrlandy, remember the last current-tripleo promotion?16:01
panda|offweshay|ruck: yes it merged.16:01
weshay|ruckhttps://review.rdoproject.org/r/#/c/25938/16:01
weshay|ruckya.. k16:01
rlandyyeah16:01
weshay|ruckguess I don't yet understand why this is needed for a promotion16:01
weshay|ruckhttps://review.rdoproject.org/r/#/c/25992/16:01
* weshay|ruck reads16:01
rlandyopenstack-periodic-master should kick16:02
rlandyor have started 2 minutes ago16:02
weshay|ruckrlandy opion on https://snapshot.raintank.io/dashboard/snapshot/Cxtk9CjssLrwiLcmZcuxAUCNn3wqWkqc?orgId=216:04
panda|offweshay|ruck: not for a promotion, for a manual promotion specifiying the hashes. For the promotion cycle we should be already good16:04
weshay|ruckrlandy, look at baremetal and ocatavia please16:04
weshay|ruckpanda|off, aye.. makes sense16:04
weshay|ruckpanda|off, is the promoter turned back on?16:04
* weshay|ruck looks16:04
weshay|ruckno16:04
rlandyweshay|ruck: on cockpit? on dlrn?16:04
weshay|ruckrlandy opion on https://snapshot.raintank.io/dashboard/snapshot/Cxtk9CjssLrwiLcmZcuxAUCNn3wqWkqc?orgId=216:05
weshay|ruckcomponent pipeline16:05
weshay|ruckpanda|off, is the promoter up to date w/ what's merged?16:05
weshay|ruckand can we turn it on?16:05
rlandyweshay|ruck: it's good - scrolling though to see the hash16:06
weshay|ruckrlandy, k.. and the sorting?16:06
weshay|ruckre: promotion vs. by date16:06
weshay|ruckrlandy, sort so the latest promotion is on top?16:07
rlandyweshay|ruck: I prefer the last promotion on top16:07
weshay|ruckk.. will update all of them.. thanks16:07
rlandynot sure of we need all the dates we didn't promote16:07
weshay|ruckrlandy, we're not manually promoting to current-tripleo today I hope16:07
rlandyweshay|ruck: ^^ it could be a lot16:08
weshay|ruckaye.. not sure if we can hide that tbh16:08
weshay|ruckwill look16:08
rlandyweshay|ruck: we're almost a week out on promotion though - tomorrow?16:08
weshay|ruckrlandy, do we have a clean set of jobs?16:09
*** panda|off has quit IRC16:09
rlandyweshay|ruck: agreed we can't promote without a clean run16:09
weshay|ruckfrom the weekend?16:09
weshay|ruckaye16:09
weshay|ruckso /me looks16:10
* rlandy is hopeful network component will do better now16:10
rlandyweshay|ruck: there is no good run from the weekend16:10
rlandywe need next run16:10
rlandyjust kicked16:10
weshay|ruckright..16:10
weshay|ruckk16:10
rlandyno worries - I have NO intention of promoting with a bunch of failed jobs16:11
weshay|ruck:)16:11
weshay|ruckrfolco, ok.. let's chat16:11
weshay|ruckrfolco, https://meet.google.com/kzx-hnid-xyv?authuser=116:12
*** sshnaidm|pto has quit IRC16:15
*** sshnaidm|pto has joined #oooq16:17
*** panda has joined #oooq16:22
weshay|ruckarxcruz|rover, got it :)16:22
weshay|ruckrfolco, you there?16:23
pandaweshay|ruck: just verify the criteria, it's ready to be started16:24
weshay|ruckcool.. thank you16:24
arxcruz|roverrfolco: got what?16:27
*** udesale_ has quit IRC16:27
*** panda is now known as gabriele16:32
*** gabriele has quit IRC16:36
weshay|ruckrlandy, arxcruz|rover ok.. https://review.rdoproject.org/r/2606316:45
rlandyweshay|ruck: hard to tell from the review but the links before looked good16:47
weshay|ruckrlandy, added another snapshot in the commit msg16:47
rlandy+216:47
rlandyI see16:47
*** panda has joined #oooq16:47
arxcruz|roverweshay|ruck: https://review.opendev.org/#/c/713676/ train passes :)16:47
pandachoo-choo16:48
weshay|ruckarxcruz|rover, NICE16:48
*** panda is now known as panda|off16:48
weshay|ruckpanda|off, thanks dude.. :)16:48
weshay|ruckrlandy, can chat bm whene ver16:49
weshay|ruckrfolco, ?16:49
rlandyweshay|ruck: may have solved the BM issue16:49
rlandychecking with the current run16:50
weshay|ruckgreat16:50
*** rfolco has quit IRC16:51
*** rfolco has joined #oooq16:51
rfolcosorry weshay|ruck, back now from lunch16:52
weshay|ruckrfolco, cool.. ready to chat?16:52
rfolcoyep16:53
weshay|ruckhttps://meet.google.com/swr-bpbs-adz?authuser=116:53
*** jmasud has quit IRC16:55
*** jmasud has joined #oooq16:56
weshay|ruckrfolco, https://hackmd.io/HrQd03c9SxOMtFPFrq50tg16:58
weshay|ruckrfolco, https://docs.google.com/document/d/1XxTxgXGdWntjURJBVP2n9ZZ_By8NqRmkbtY3VWcUUcQ/edit#heading=h.d21vw474q5aw16:59
weshay|ruckrfolco, https://hackmd.io/7MBqFHurTA2e5H8kYRwgag#UNPINNING-NOTES--pipeline17:11
*** dtantsur is now known as dtantsur|afk17:36
*** dpawlik has quit IRC17:37
*** TrevorV has quit IRC17:45
*** pojadhav|away is now known as pojadhav17:48
weshay|ruckrlandy, looks like I should sort on date http://tripleo-cockpit.usersys.redhat.com/d/2tivP9BWz/component-pipeline?orgId=117:55
rlandyweshay|ruck: yeah - it's a bit populated17:58
weshay|ruckfixing17:59
weshay|ruckrlandy, fyi.. promoter is turned on17:59
weshay|ruckfor centos-8 master17:59
rlandyweshay|ruck: we need to review the criteria then18:00
rlandyin case jobs pass18:00
rlandygetting18:00
rlandyweshay|ruck: https://github.com/rdo-infra/ci-config/blob/master/ci-scripts/dlrnapi_promoter/config/CentOS-8/master.ini18:01
*** dpawlik has joined #oooq18:02
rlandyfs001 should be in, right?18:02
*** derekh has quit IRC18:02
weshay|ruckrlandy, in criteria?18:02
rlandyyes18:02
weshay|ruckrlandy, well.. I'd be willing to wave the ovb stuff one more time18:03
weshay|ruckso we can get a promotion.. then I'll add it back18:03
weshay|ruckhttps://review.rdoproject.org/zuul/builds?job_name=tripleo-ci-centos-8-ovb-1ctlr_1comp-featureset001&job_name=tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset00118:03
weshay|ruckwe're doing ok there18:03
rlandyweshay|ruck: and the other commented jobs?18:04
weshay|ruckperiodic-tripleo-centos-8-master-containers-build-push18:04
weshay|ruckperiodic-tripleo-centos-8-buildimage-overcloud-full-master18:04
weshay|ruckperiodic-tripleo-centos-8-buildimage-ironic-python-agent-master18:04
weshay|ruckperiodic-tripleo-ci-centos-8-standalone-master18:04
weshay|ruckperiodic-tripleo-ci-centos-8-scenario001-standalone-master18:04
weshay|ruckperiodic-tripleo-ci-centos-8-scenario002-standalone-master18:04
weshay|ruckperiodic-tripleo-ci-centos-8-scenario003-standalone-master18:04
weshay|ruckperiodic-tripleo-ci-centos-8-scenario004-standalone-master18:04
weshay|ruckperiodic-tripleo-ci-centos-8-scenario007-standalone-master18:04
weshay|ruckperiodic-tripleo-ci-centos-8-singlenode-featureset027-master18:04
weshay|ruckperiodic-tripleo-ci-centos-8-multinode-1ctlr-featureset010-master18:04
weshay|ruckperiodic-tripleo-ci-centos-8-containers-undercloud-minion-master18:04
weshay|ruckperiodic-tripleo-ci-centos-8-scenario000-multinode-oooq-container-updates-master18:04
weshay|ruckis what I see atm18:04
weshay|rucklet's see what happens w/ this build18:05
rlandycorrect18:05
rlandywishful you mean, that this will pass??18:05
weshay|ruckI guess so18:08
weshay|ruckrlandy, they are getting through deploy :)18:14
rlandyweshay|ruck: ack  - I see tempest18:14
rlandyweshay|ruck: so we're going to do a regular promotion now on master?18:16
rlandyif so, I would put fs001 back after this18:16
weshay|ruckrlandy, if these pass18:16
weshay|ruckyes18:16
rlandymakes me nervous18:16
weshay|ruckyou have enough to worry about18:17
*** jpena is now known as jpena|off18:27
rlandyweshay|ruck: baremetal is on undercloud install now ... https://review.opendev.org/#/c/714208/2/roles/baremetal-prep-virthost/templates/add-provisioning-interface.sh.j2 were made to the virthost setup18:29
rlandyI had to bring eth3 down to bring the brovc up18:29
rlandyidk how that will play out when we come to provisioning18:30
rlandyI dont see how we add a bridge interface and keep the original interface up18:30
weshay|ruckrlandy, eth3 down on the virthost?18:33
weshay|ruckthat should be fine18:33
weshay|ruckvirthost I think should only be using two nics18:33
rlandyweshay|ruck: ack18:35
rlandyto add that interface to brovc18:35
rlandy2020-03-23 18:26:04.690233 | primary | Unable to change directory before execution: [Errno 2] No such file or directory: b'/home/zuul/tempest'18:36
rlandyscenario01218:36
weshay|ruckrlandy, ya.. scenario012 failure is expected18:37
rlandy  msg: |-18:38
rlandy    Unable to start service network: Job for network.service failed because the control process exited with error code.18:38
rlandy    See "systemctl status network.service" and "journalctl -xe" for details.18:38
rlandy^^ failure on bm undercloud18:38
rlandyCalled as ovs-vsctl -t 10 -- --if-exists del-port br-ctlplane eth1 -- add-port br-ctlplane eth118:40
rlandyPMD: net_mlx4: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx4)18:40
rlandynet_mlx5: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory18:40
rlandy[zuul@undercloud ~]$ cat /etc/redhat-release18:40
rlandyCentOS Linux release 8.1.1911 (Core)18:40
*** amoralej is now known as amoralej|off18:44
*** panda|off has quit IRC18:50
*** panda has joined #oooq18:51
rlandyweshay|ruck: have you seen any issues restart service on centos 8?18:54
rlandymay be virt only18:54
weshay|ruckrlandy, which service?18:57
rlandynetwork18:57
rlandyit might just be that networkManager is also running18:58
rlandyand I had to install libibverbs18:58
weshay|ruckrlandy, we may need to convert and network service commands to NetworkManager at this point19:00
rlandyweshay|ruck: also I had to install libibverbs19:01
rlandythis is the undercloud install19:01
rlandyso the virt must be different from ovb19:01
weshay|rucklet me know if you need help.. don't have enough info to know yet19:02
rlandyweshay|ruck: in summary ... we should not need to change the undercloud install19:03
rlandyit should work on a centos 8 vm19:03
weshay|ruckcool19:03
rlandyas it does on a centos 8 node19:03
rlandyBUT we do as there are failures19:03
* rlandy will try patch for the moment and see how far we get19:03
*** saneax has quit IRC19:07
rlandyweshay|ruck: looking like we will promote today19:15
weshay|ruckaye19:15
rlandywe passed criteria19:15
weshay|ruckrlandy, it's running now19:16
rlandyshould be interesting19:16
weshay|ruck8362bba39607f045bff5df31657cbaf319:17
rlandywoohoo fs020 got to tempest19:52
rlandyso feploy much have worked19:52
rlandydeploy19:52
*** ccamacho has quit IRC20:05
weshay|ruckrlandy,  https://review.opendev.org/#/c/714437/ https://review.opendev.org/#/c/713676/5 when you have a sec20:06
rlandylooking20:06
weshay|ruckrfolco, can you move the planning mtg 1/2 hour earlier please20:10
rfolcoweshay|ruck, sure20:10
weshay|ruckthanks20:10
rfolcodone20:11
rfolcoweshay|ruck, looking at get-hash thing, the hole is deeper than I thought20:11
rfolcodocumenting in a hackmd and adding to the task in taiga20:12
*** dpawlik has quit IRC20:43
rlandyweshay|ruck: so ... https://review.opendev.org/#/c/713979/1/config/release/tripleo-ci/CentOS-8/master.yml20:53
rlandyyou +2'ed it20:53
*** dpawlik has joined #oooq20:53
rlandydlrn_hahs_path should always be set, right?20:53
*** jmasud has quit IRC20:53
rlandydlrn_hash_path ... if we ever evaluated to dlrn_hash that would not work20:54
*** jmasud has joined #oooq20:55
weshay|ruckrlandy, this is about https://review.opendev.org/#/c/713753/3/roles/repo-setup/tasks/get-dlrn-hash-newest.yml ?20:59
*** jfrancoa has quit IRC20:59
rlandyweshay|ruck: no ... https://review.opendev.org/#/c/713979/1/config/release/tripleo-ci/CentOS-8/master.yml21:00
panda8362bba39607f045bff5df31657cbaf3 promoted.21:05
*** panda is now known as panda|off21:05
weshay|ruckit sure did21:11
weshay|ruckrlandy, so two hats..21:11
weshay|ruckrlandy, one for ci one w/o21:11
weshay|ruckre: drln_hash_path I think21:11
weshay|ruckso.. I guess I'm saying that may need to work w/ either current-tripleo as the label or the exact hash21:13
weshay|rucknot sure if I'm helping or not.. probably not21:13
rlandyhttps://logserver.rdoproject.org/79/713979/1/openstack-check/tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001/2298a62/logs/undercloud/home/zuul/repo_setup.sh.txt.gz21:13
*** rfolco has quit IRC21:17
rlandyweshay|ruck: commented on the review21:18
weshay|ruckrlandy, dlrn_hash_tag_newest is the aggregated hash?21:18
rlandyweshay|ruck: it's current21:19
rlandyie: the current hash21:19
weshay|rucka little code comments go a long way21:19
*** dpawlik has quit IRC21:20
rlandyweshay|ruck: think we're talking past the point - chat?21:23
weshay|ruckhttps://meet.google.com/ovy-whjr-rhm?authuser=121:24
*** holser has quit IRC21:42
weshay|rucktripleo-rhel-8-buildimage-overcloud-fullFAILURE in 29m 01s22:29
weshay|rucktripleo-rhel-8-buildimage-ironic-python-agent22:29
rlandyweshay|ruck: https://review.rdoproject.org/zuul/stream/525a796a21cc4682910be17802ef6db2?logfile=console.log at tempest - may pass23:34
rlandyperiodic-tripleo-ci-centos-8-multinode-1ctlr-featureset010-tripleo-master23:34
« Sunday, 2020-03-22 Index Tuesday, 2020-03-24 »

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!