Tuesday, 2019-04-09

hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-fedora-28-standalone, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039 @ https://review.openstack.org/602248,  (2 more messages)00:34
weshayoooh.. nice00:43
*** aakarsh has joined #oooq01:43
*** apetrich has quit IRC01:58
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario010-multinode-oooq-container, tripleo-ci-fedora-28-standalone @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039 @ https://review.openstack.org/602248,  (2 more messages)02:34
*** sanjayu_ has quit IRC02:47
*** ykarel|away has joined #oooq02:51
*** dsneddon has quit IRC03:28
*** dsneddon has joined #oooq03:53
*** dsneddon has quit IRC03:58
*** udesale has joined #oooq04:04
*** ykarel|away is now known as ykarel04:07
*** Vorrtex has joined #oooq04:13
*** Vorrtex has quit IRC04:22
*** dsneddon has joined #oooq04:30
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-fedora-28-standalone, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp- (3 more messages)04:34
*** ykarel has quit IRC04:42
*** ykarel has joined #oooq04:56
*** ratailor has joined #oooq05:11
*** skramaja has joined #oooq05:19
*** marios has joined #oooq05:30
*** dsneddon has quit IRC05:34
*** quiquell|off is now known as quiquell|rover05:43
*** udesale has quit IRC05:52
*** ykarel is now known as ykarel|afk05:58
*** jbadiapa has joined #oooq06:01
*** ratailor has quit IRC06:01
*** ykarel|afk has quit IRC06:02
*** jfrancoa has joined #oooq06:03
*** kopecmartin|off is now known as kopecmartin06:05
*** dsneddon has joined #oooq06:06
chandankumarquiquell|rover: hello06:11
chandankumarquiquell|rover: please have a look at this review https://review.rdoproject.org/r/#/c/19916/06:11
*** dsneddon has quit IRC06:11
quiquell|roverchandankumar: o/ will do06:11
chandankumarquiquell|rover: thanks!06:12
*** ccamacho has joined #oooq06:12
*** ratailor has joined #oooq06:13
*** ykarel|afk has joined #oooq06:13
*** dsneddon has joined #oooq06:15
*** dsneddon has quit IRC06:19
*** udesale has joined #oooq06:33
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-fedora-28-standalone, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp- (3 more messages)06:34
*** ykarel|afk is now known as ykarel06:35
*** quiquell|rover is now known as quique|rover|brb06:41
*** dsneddon has joined #oooq06:46
*** dsneddon has quit IRC06:51
*** bogdando has joined #oooq07:07
*** apetrich has joined #oooq07:10
*** quique|rover|brb is now known as quiquell|rover07:13
*** chem` has joined #oooq07:13
*** chem has quit IRC07:14
*** dsneddon has joined #oooq07:17
*** tosky has joined #oooq07:20
*** dsneddon has quit IRC07:21
*** sshnaidm|off is now known as sshnaidm|pto07:23
*** dsneddon has joined #oooq07:24
chandankumarzbr: please have a look at this patch https://review.openstack.org/#/c/650613/ I am not sure where it is failing07:26
*** dsneddon has quit IRC07:28
*** panda has joined #oooq07:29
*** ykarel is now known as ykarel|lunch07:31
*** tosky has quit IRC07:33
*** tosky has joined #oooq07:33
quiquell|roverpanda: good morning07:33
quiquell|roverpanda: question can we ignore newton ocata at upstream CI ?07:34
pandaquiquell|rover: yes, newton certainly, ocata had still some relevance fro fast forward upgrades up to a point, now we can ignore it.07:36
mariospanda: newton is the ffu one07:37
mariospanda: N..Q ffu07:37
pandamarios: fuuuuuuuuuuuuuu07:37
mariospanda: well exactly07:37
quiquell|roverok so newton is important07:39
quiquell|roverand ocata too ?07:39
quiquell|roverTengu: ^07:39
quiquell|roverlooks like backport for newton and ocata still needed07:39
*** holser_ has joined #oooq07:40
pandaquiquell|rover: newton is EOL, not even accepting changes.07:41
quiquell|roverok07:42
pandamarios: n -> o is a single release, ffu was n -> n+ 307:42
quiquell|roverocata ?07:42
*** amoralej|off is now known as amoralej07:43
quiquell|roverpanda: another question, why don't we use fedora28 standalone as criteria to promotion ?07:43
mariospanda: yeah 10:37 < marios> panda: N..Q ffu07:43
pandaquiquell|rover: ocata is in extended maintenance, soon to be unmaintained, as far as I know, ther may be still some community patches, but priority is under the shoes.07:44
Tenguquiquell|rover: ok.07:44
Tenguso pike and newton that is.07:44
pandamarios: ah, my font doesn't show the small sign under the Q very well, seemed a O to me07:44
Tengun o p ?07:45
pandaTengu: r s t ?07:45
pandaTengu:  m d ?07:45
mariosu wot m807:45
tosky(and pike is almost under Extended Maintenance too)07:46
pandaan intruder!07:46
Tenguso ocata has its review as well: https://review.openstack.org/65112607:46
Tengunewton on the way.07:46
pandaTengu: I don't think newton will accept any change07:47
*** jpena|off is now known as jpena07:47
Tenguhumpf.... so we let that one fail the units?07:48
pandaTengu: who's still watching the units in newton ?07:48
Tengudunno. some weirdo that stick to that old release? :)07:48
Tengubut if we don't need to patch newton, fine for me :).07:49
toskyTengu: there is are stable/newton branch anymore around, so that settles the question07:49
toskynewton is *gone*07:49
toskyargh07:49
marioslong live newton!07:49
toskythere are *no* stable/newton branches...07:49
mariosTengu: would have to be d/stream only tosky is right there is nothing to post it to upstream07:49
pandaWe'll branch it again.07:50
marioslol07:50
mariosnewton207:50
pandanewton will rise from its ashes07:50
marios2.007:50
panda*hashes07:50
Tenguhmmm. I do see remotes/origin/stable/newton07:50
Tenguanyway.07:51
Tenguno newton then :).07:51
pandaTengu: if you see it locally it's because you didn't prune07:51
pandaTengu: you unpruner you!07:51
mariosTengu: eg see branches at https://github.com/openstack/tripleo-heat-templates07:51
mariosTengu: you have to go to tags to find newton things07:51
toskyand http://git.openstack.org/cgit/openstack/puppet-tripleo07:51
tosky(or better, https://opendev.org/openstack/puppet-tripleo/branches/ )07:52
Tenguoh well.07:53
quiquell|roverpanda: about fedora28 promotions, can we add the fedora28 standalone to the promoter criteria ? or since it's non voting it does not make sense ?07:57
pandaquiquell|rover: what's currently in the promoter criteria for fedora ? is standalone stable enough ?07:58
quiquell|roverpanda: no criteria07:58
pandafedora28 will probably never be voting at this point07:59
quiquell|roverIn case fedora28 is voting could we add it as criteria for master and stein ?07:59
quiquell|roverI see07:59
quiquell|roverbut does not make sense to promote if the only job we have is not working though07:59
pandaunless it's so stable that it doesn't bother anyone07:59
pandaquiquell|rover: no it's still not stable08:00
*** dtantsur|afk is now known as dtantsur08:00
*** dsneddon has joined #oooq08:00
*** ccamacho has quit IRC08:01
*** sanjayu_ has joined #oooq08:06
*** dsneddon has quit IRC08:06
*** marios has quit IRC08:11
*** zbr has quit IRC08:28
quiquell|roverpanda: this will fix current fedora28 standalone issue https://review.rdoproject.org/r/#/c/20039/08:29
quiquell|roverpanda: we were using wrong parsed_containers file08:29
quiquell|roverpanda: so it can be partial08:29
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-fedora-28-standalone, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp- (3 more messages)08:34
*** derekh has joined #oooq08:40
*** dsneddon has joined #oooq08:42
*** dsneddon has quit IRC08:47
*** ykarel|lunch is now known as ykarel08:48
chandankumararxcruz: Hello08:49
chandankumararxcruz: for porting periodic jobs, we need to use this userstory na https://tree.taiga.io/project/tripleo-ci-board/us/972 ?08:49
arxcruzchandankumar: i think yes08:57
*** marios_ has joined #oooq09:08
*** dsneddon has joined #oooq09:13
*** dsneddon has quit IRC09:19
quiquell|rovermarios_: fix for promotion-blocker https://review.rdoproject.org/r/#/c/20039/09:21
*** jaosorior has quit IRC09:24
marios_quiquell|rover: ack09:25
*** marios_ is now known as marios09:28
*** jaosorior has joined #oooq09:31
*** matbu has quit IRC09:36
*** yolanda_ has quit IRC09:47
*** jfrancoa has quit IRC09:53
*** jfrancoa has joined #oooq09:55
*** holser_ is now known as holser|lunch09:55
*** udesale has quit IRC09:57
*** udesale has joined #oooq09:58
*** skramaja has quit IRC10:03
quiquell|roverweshay: do you happend to know why we don't fail on partial container push at promotions ?10:28
quiquell|roverweshay: https://github.com/rdo-infra/ci-config/commit/f81c91283560db5bf8f1c4125e5a1f1e6742f12a10:28
weshayquiquell|rover can you clarify that.. are you asking why does a promotion still happen if the containers fail to push accurately?10:30
weshaythat would be a bug not a feature, the tooling used to NOT promote if a failure was detected in the container push10:30
quiquell|roverweshay: nope if we have a promotion push runing in the middle of container build10:30
quiquell|roverweshay: not all the containers are going to be at RDO registry10:31
quiquell|roverweshay: and we still make a promotion with a partial number of containers10:31
quiquell|roverweshay: https://review.rdoproject.org/r/2003910:31
quiquell|roverweshay: partial fedora28 promotion here https://launchpadlibrarian.net/418481718/fedora28_master.log-2019040810:31
quiquell|rover2019-04-08 23:03:34,579 22611 INFO10:32
quiquell|roverpanda: ^10:32
weshayquiquell|rover hrm.. can I get on blue and get a few more details?10:32
quiquell|roversure i can be wrong there though10:32
quiquell|roverpanda: can you join ?10:32
weshayhttps://bluejeans.com/u/whayutin/10:32
weshayI'll join but let's wait for panda10:33
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-fedora-28-standalone, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp- (3 more messages)10:34
quiquell|roverhttps://launchpadlibrarian.net/418481718/fedora28_master.log-2019040810:36
chandankumararxcruz: please add the scenario 002 card here https://tree.taiga.io/project/tripleo-ci-board/us/97210:36
chandankumarkopecmartin: can I add you as a qe on this user story https://tree.taiga.io/project/tripleo-ci-board/us/972 ?10:38
arxcruzchandankumar: ack10:38
chandankumarmarios: Can I get +w on https://review.rdoproject.org/r/#/c/19916/ on scenario 001 port?10:40
*** ykarel is now known as ykarel|afk10:41
marioschandankumar: not from you10:41
marioschandankumar: you cannot10:41
marioschandankumar: i have it not10:41
marioschandankumar: to give to you10:41
marioschandankumar: yououou10:41
mariosou10:41
chandankumarok ok :-)10:41
kopecmartinchandankumar, sure10:50
*** udesale has quit IRC10:56
pandaweshay: quiquell|rover https://review.rdoproject.org/r/2004211:04
quiquell|roverpanda: can we unit test it ?11:05
pandaquiquell|rover: that's a different story ...11:05
*** holser|lunch is now known as holser_11:05
quiquell|roverpanda: ack +211:06
quiquell|roverwill workflow when ci passes11:06
pandaquiquell|rover: we are not testing anything currently for the promoter.11:06
quiquell|roverpanda: we have some unit tests already11:06
pandaquiquell|rover: and that's bad.11:06
pandaquiquell|rover: we do ?11:06
quiquell|roverpanda: yep11:06
quiquell|roverI added some11:06
pandaquiquell|rover: where ?11:07
quiquell|roverpanda: https://github.com/rdo-infra/ci-config/blob/master/ci-scripts/dlrnapi_promoter/test_get_latest_hashes.py11:07
pandaquiquell|rover: I'm blind11:07
*** zbr has joined #oooq11:08
pandaquiquell|rover: ok, I'll add it then11:08
quiquell|roverpanda: ack, let me know if you are missing something11:09
quiquell|roverpanda: at one moment we should have enough unit test to refactor the script11:09
zbrquiquell|rover: can you help me wf https://review.openstack.org/#/c/650913/ and https://review.openstack.org/#/c/650921/ ?11:10
pandaquiquell|rover: it's not the lack of unit tests that is preventing us from refactoring ... but they would probably make it easier.11:11
quiquell|roverzbr: to test that we need more test reviews from different branches11:12
*** ykarel|afk is now known as ykarel11:12
quiquell|roverzbr: at least for uc  upgrade job11:12
quiquell|roverjfrancoa: ^ is that rigth ?11:12
quiquell|roverpanda: they make it easier11:12
quiquell|roverpanda: and time of course11:13
zbrquiquell|rover: the mixed config files were made obsolete by the emit-release changes which takes care of that.11:13
quiquell|roverzbr: well not really emit releases still references them11:14
quiquell|roveror they have change it ?11:14
quiquell|roverlet me check11:14
quiquell|roverthey were fixing it11:14
zbrplease do, as you know better tham me where to look on that.11:14
jfrancoaquiquell|rover zbr: well, I lost the current track on the upgrades jobs being covered, but it's true it would be nice to test the patch in queens and rocky branches11:15
quiquell|roverzbr: ^ just do a pair of test reviews to see if it works11:15
jfrancoazbr: and it's something easy to do...take a noop patch in tripleo-quickstart and include a depends-on this patch, if you want I can do it11:15
*** amoralej is now known as amoralej|lunch11:16
quiquell|roverjfrancoa: uc upgrade is master->master ? or N-1->N ?11:16
quiquell|roverI see master master11:16
quiquell|roverhttp://logs.openstack.org/13/650913/2/check/tripleo-ci-centos-7-containerized-undercloud-upgrades/3eb82e1/logs/quickstart_files/releases.sh11:16
quiquell|roverA no11:17
quiquell|roverhumm I see it no longer uses those file11:17
zbrquiquell|rover: that is what i was told by two people last week.11:18
quiquell|roverzbr: let's do the test review to be sure11:18
quiquell|roverzbr: And we cleanup11:19
quiquell|roverzbr: we can even cleanup emit releases script11:19
jfrancoaquiquell|rover: uc upgrade is doing N-1 to N if I remember correctly11:19
quiquell|roverjfrancoa: emit releases no longer needs the code that concatenant relases to compose release file name ?11:19
*** panda is now known as panda|lunch11:20
jfrancoaquiquell|rover: I'm not sure about it...that's why I'd like to test the patch in former releases11:20
quiquell|rover ack11:21
chandankumarhttps://review.openstack.org/#/c/635478/ Can we get +w on this?11:22
zbrjfrancoa: on which project should I open the branch change? tq does not have branches.11:23
jfrancoazbr: I would use tripleo-heat-templates, I think it does the best test coverage... quiquell|rover wdyt?11:23
quiquell|roverzbr, jfrancoa: tht it's we can even change zuul layout to explictily run the jobs we want (meaning upgrades)11:24
*** bogdando has quit IRC11:25
zbrquiquell|rover: we do not want to run them all the time, but triggered by magic comment, clearly yes. we had this for upgrades downstream.11:25
quiquell|roverzbr: I mean a DNM THT review just riggering them11:26
zbri am adding one on ocata now11:26
weshayquiquell|rover panda|lunch ok.. promotion criteria across releases and distro's is now clean11:27
quiquell|roverweshay: I think it's missing f28 master https://review.rdoproject.org/r/#/c/20043/11:28
weshayquiquell|rover https://review.rdoproject.org/r/#/c/20041/1/ci-scripts/dlrnapi_promoter/config/Fedora-28/master.ini11:29
zbrquiquell|rover: weshay : please help me workflow the promote images multi distro support at https://review.rdoproject.org/r/#/c/19966/911:30
quiquell|roverweshay: ack11:30
zbrquiquell|rover: do you know which magic files I need to touch to make tht run all jobs or at least the upgrade ones, adding a placeholder file did not trigger  much... in fact very few.11:35
zbrquiquell|rover: see https://review.openstack.org/#/c/651198/ --11:35
quiquell|roverzbr: let me add a ps there11:37
quiquell|roverzbr: so you see11:37
*** jpena is now known as jpena|lunch11:37
zbrsure, always welcomed.11:37
*** dsneddon has joined #oooq11:56
*** aakarsh has quit IRC12:01
*** aakarsh has joined #oooq12:01
*** dsneddon has quit IRC12:01
*** dsneddon has joined #oooq12:02
*** dsneddon has quit IRC12:07
*** quiquell|rover is now known as quique|rover|eat12:07
*** jfrancoa has quit IRC12:09
*** dsneddon has joined #oooq12:12
*** aakarsh has quit IRC12:13
chandankumarmarios: scenario 003 https://logs.rdoproject.org/38/20038/2/check/periodic-tripleo-ci-centos-7-scenario003-standalone-master/473e7e9/logs/stestr_results.html with os_tempest12:15
quique|rover|eatpanda|lunch, weshay: f28 standalone is working fine https://softwarefactory-project.io/zuul/t/rdoproject.org/build/361e6d8b0caa483fbb4ba4fdd87598a612:15
quique|rover|eatmaybe we can set it non-voting12:15
weshaygood :)12:15
weshayquique|rover|eat you mean to voting?12:15
quique|rover|eatI am pusing the containers with the newest hash though12:16
quique|rover|eatvoting12:16
quique|rover|eatyep12:16
weshayfaker12:16
*** dsneddon has quit IRC12:16
*** jfrancoa has joined #oooq12:23
*** ratailor has quit IRC12:24
weshaymarios fyi.. rlandy got osp-14 working yesterday12:27
weshaymarios internal rh cert :)12:27
mariosweshay: i know i saw :D12:27
mariosrlandy++12:27
*** matbu has joined #oooq12:27
weshayit's worth now trying w/ 1512:28
mariosweshay: yup12:28
mariosweshay: its #3 in my priority list12:28
weshaybusy guy12:28
mariosweshay: 1 is 1* https://tree.taiga.io/project/tripleo-ci-board/us/846 split upgrades jobs r->s s->master 2* https://tree.taiga.io/project/tripleo-ci-board/us/959 standalone upgrade job green voting and gating 3* https://tree.taiga.io/project/tripleo-ci-board/us/952 OSP 15 job... post some shit the job definition etc...  4* https://tree.taiga.io/project/tripleo-ci-board/us/958 document emit-releases...12:28
mariosexisting patch catch up with quique about that and take over or assign to him...12:29
mariosweshay: not really they're lots of minor things so far12:29
mariosweshay: and moving to the 15 stuff just now coincidentally as you ping12:29
*** jpena|lunch is now known as jpena12:29
weshaymarios /me adds https://bugs.launchpad.net/tripleo/+bug/1823882 to the taiga us12:29
openstackLaunchpad bug 1823882 in tripleo "tripleo_ovn_northd_healthcheck.service loaded failed failed ovn_northd healthcheck" [Critical,Triaged]12:29
mariosweshay: for that, i was thinking we don't need tasks.. like the story is a task. is that acceptable sire12:30
mariosweshay: so i'll add https://review.openstack.org/651213 there too12:30
weshaymarios however you want to work it12:30
mariosweshay: ack12:30
* weshay looking .. I see enough info12:30
* weshay watches us, as I was working this a bit too12:30
mariosweshay: added bug and review there for now https://tree.taiga.io/project/tripleo-ci-board/us/95912:32
mariosunder dependencies12:32
zbrjfrancoa: any hints on how to trigger the magic upgrade jobs? quique|rover|eat tried to update my testing THT patch but I doubt it would work that way https://review.openstack.org/#/c/651198/12:32
mariosk?12:32
*** rlandy has joined #oooq12:32
weshaymarios would we really want to remove validations?12:32
mariosweshay: no12:32
weshaymaybe you spoke w/ Tengu?12:32
quique|rover|eatzbr: no luck12:32
mariosweshay: i hve12:32
mariosweshay: in tripleo12:32
*** apetrich has quit IRC12:32
weshayk12:32
*** rlandy is now known as rlandy|ruck12:32
mariosweshay: its just that regardless if that is a valid fail or not, tengu only activated that on a non voting job and we are moving that to voting. see his comment in https://review.openstack.org/#/c/651213/1/config/general_config/featureset056.yml that i'm removing12:33
quique|rover|eatzbr: has try to override files section but is not working12:33
quique|rover|eatzbr: https://review.openstack.org/#/c/651198/12:33
quique|rover|eatmarios: ^ this should work ?12:33
zbrquique|rover|eat: look at https://review.openstack.org/#/c/644400/ -- i think that forking the gate check could work.12:33
mariosweshay: iadded him to review and -1 it for now. i want to see if the job gets much further/green without that validation12:33
mariosquique|rover|eat: looking12:33
weshaymarios aye.. makes sense12:33
quique|rover|eatzbr: that should work too12:33
Tengumarios, weshay: well, we can enable them maybe? Any stats about false-positives with that new validation?12:34
mariosquique|rover|eat:  that needs mre time than i can give it right now i'll add for next reviews12:34
zbrquique|rover|eat: what i would like to know is the name of the job(s) that are supposed to test  my code. if i know their name I can dig and find a way to trigger them.12:34
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario010-multinode-oooq-container, tripleo-ci-fedora-28-standalone @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp- (3 more messages)12:34
quique|rover|eatzbr: the ones in the review12:35
rlandy|ruckquique|rover|eat: https://review.rdoproject.org/r/#/c/20039/ ok to merge?12:35
weshayTengu we have no such data atm, https://bugs.launchpad.net/tripleo/+bug/1823882 looks like a real issue12:35
openstackLaunchpad bug 1823882 in tripleo "tripleo_ovn_northd_healthcheck.service loaded failed failed ovn_northd healthcheck" [Critical,Triaged]12:35
quique|rover|eatzbr: https://review.openstack.org/#/c/651198/12:35
quique|rover|eatrlandy|ruck: let's wait for panda|lunch and weshay vote12:35
quique|rover|eatrlandy|ruck: issue was different I totally misslead it12:35
weshayrfolco can you please send an email to the tripleo list re: the community call mtg time12:35
weshaymove12:35
weshayrfolco and also get on #tripleo and let folks know there12:36
rlandy|ruckquique|rover|eat: weshay already +2'ed but you are right - let's let panda|lunch have a say12:36
rfolcoweshay, ok will do12:36
*** aakarsh has joined #oooq12:38
*** apetrich has joined #oooq12:39
mariosTengu: not yt lets see i'll revisit that task waitint to see what happens on the ci run there too12:39
mariosTengu: will discuss with you before doing anything thanks12:39
mariosTengu: added you on review too12:39
weshayarxcruz what do you want to do w/ https://trello.com/c/im5WRPw4/926-cixlp1821690tripleo-ci-tempest-testautoallocatenetwork-tempestapicomputeimagestestimages12:42
weshayshould we chat about that for a sec?12:42
rfolcoweshay, how about updating ci community mtg time in https://wiki.openstack.org/wiki/Meetings/TripleO ?12:43
Tengumarios: fine for me :)12:43
arxcruzweshay: sure12:44
arxcruzweshay: i don't know, as i point in the card, we need someone who understand network to take a look12:44
weshayrfolco would be cool if we could get in #topic in #tripleo12:44
weshayI lost my ops12:44
*** quique|rover|eat is now known as quiquell|rover12:44
*** apetrich has quit IRC12:45
jfrancoazbr: I guess we don't have too many jobs running in stable/ocata any more...that's why we only see the undercloud-oooq job, but still that's enough. at least to see we don't break other branch jobs12:45
rfolcoweshay, k12:45
*** dtantsur is now known as dtantsur|brb12:46
weshayarxcruz k.. I updated the card12:46
zbrjfrancoa: ok, i cherrypicked the testing change to rocky and queens, hopefully we will see something.12:46
rlandy|ruckarxcruz: fs021 is also failing everywhere -https://logs.rdoproject.org/openstack-periodic-24hr/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset021-rocky/86ef5a0/logs/undercloud/home/zuul/tempest.log.txt.gz  related to https://bugs.launchpad.net/tripleo/+bug/1821690?12:46
openstackLaunchpad bug 1821690 in tripleo "periodic-ovb-1ctlr_1comp-featureset020-master fails tempest test_auto_allocate_network tempest.api.compute.images.test_images" [High,In progress] - Assigned to Arx Cruz (arxcruz)12:46
weshayrlandy|ruck do you need anything regarding hardware?12:46
rlandy|ruckweshay: ack - the hardware12:46
weshayneed a hand w/ updating ips12:46
quiquell|roverzbr: issue with my review was ocata I thin those jobs don't run oin acata12:46
quiquell|roverzbr: same thing at rock/queens should work12:46
rlandy|ruckweshay: no - should be fine - just busy work - planned to do wit yesterday when worked on standlaone - will do that today12:47
jfrancoazbr: great, thanks12:47
arxcruzweshay: you want to chat about it ?12:47
weshayarxcruz so.. I think the sum is.. we are ready, just need someone from networking?12:47
arxcruzweshay: yes12:48
arxcruzIF12:48
arxcruzwe want to execute this test12:48
arxcruzotherwise, we keep it forever in the skip list12:48
weshaywhat's the calculus on the IF12:48
weshayis it a good valid test?12:48
weshayI see the resource issue12:48
arxcruzfor us, not really imho12:49
weshayperhaps.. we need to increase the compute node specs?12:49
arxcruzyeah, this might be problem on my environment12:49
arxcruzmaybe12:49
weshayarxcruz is it failing in fs021?12:49
arxcruzweshay: probably yes, because as I said in the comments, it requires to create some networks, pool and subnets that we are not creating12:50
*** Goneri has joined #oooq12:50
quiquell|roverweshay: I will bring https://bugs.launchpad.net/tripleo/+bug/1815744 to the community meeting12:50
openstackLaunchpad bug 1815744 in tripleo "build-test-package does not fail on partial builds" [Critical,In progress] - Assigned to Quique Llorente (quiquell)12:50
quiquell|roverweshay: Don't really know what to do with it12:50
weshayDetails: {u'message': u'Unable to automatically allocate a network for project b378b5a042ee429fb81e5cc69bcbe499', u'code': 400}12:52
weshayarxcruz aye k12:52
arxcruzweshay: yes, this error is because in order to run this particular test, it requires to create some subnets and pools with automatic allocation enabled12:52
weshayhttp://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset021-master/6f17825/logs/tempest.html.gz12:53
weshayarxcruz seems like we could remove quite a bit from master/stein skip list12:54
weshayonly 9 failures12:54
rlandy|ruckquiquell|rover, can you catch me up - what's still needed for stein phase 1?12:54
arxcruzwow yeah cool12:54
*** apetrich has joined #oooq12:54
*** sanjayu_ has quit IRC12:55
arxcruzapetrich: hey, volkspark this weekend? :D12:55
apetricharxcruz, rain and cold?12:55
apetricharxcruz, at least it was the forecast this morning for the weekend :)12:55
quiquell|roverrlandy|ruck: no clear, there were error few days ago, but I see that then it passes https://ci.centos.org/job/tripleo-quickstart-promote-stein-current-tripleo-delorean-minimal/12:55
arxcruzapetrich: it's not rain and cold, perfect weather for a beer12:56
quiquell|roverrlandy|ruck: but latest is failing again12:56
rlandy|ruckquiquell|rover: will take a look after meeting - sorry been so absent12:56
quiquell|roverrlandy|ruck: no problem, is not that bad for now the rr'ing12:56
rlandy|ruckmarios: you ok with the standalone changes?12:56
mariosrlandy|ruck: o/12:57
mariosrlandy|ruck: which one? i saw you merged 14 job12:57
rlandy|ruckmerged so that other working on internal could get that work12:57
mariosrlandy|ruck: ceph things?12:57
mariosrlandy++12:57
rlandy|ruckmarios: the ceph things were not needed to pass the job12:57
rlandy|ruckbut I think they are still good to have12:57
mariosrlandy|ruck: ack i left relevant comment on the review do we need ceph team to bless it or they already did12:57
rlandy|ruckmarios: it is w/o blessing atm12:58
mariosweshay: hmm.. we can't get away with 'no task' cos that way we cant move it across status parked->in progress etc12:58
mariosweshay: will add it but not right now anyway fyi12:58
mariosweshay: in time for the call tomorrow anyway12:58
zbrmarios: weshay: reminder about image promoter: https://review.rdoproject.org/r/#/c/19966/ :p13:00
marioszbr: ack noted for next reviews13:00
rfolcobe aware!! The CI Community Meeting (office open hours) has been moved to 13:30 UTC (30 minutes before the TripleO meeting).13:01
*** ykarel is now known as ykarel|afk13:02
panda|lunchoh, yeah we are in the limbo of the daylight savings.13:03
panda|lunchnot for long13:03
*** panda|lunch is now known as panda13:03
rlandy|ruckquiquell|rover: k, panda blessed your review  -https://review.rdoproject.org/r/#/c/20039/ merging13:04
*** mjturek has joined #oooq13:07
*** ykarel|afk has quit IRC13:09
rfolcobe aware!! The CI Community Meeting (office open hours) has been moved to 13:30 UTC (30 minutes before the TripleO meeting).13:11
pandaquiquell|rover: I have no idea what assert to use to understand that the function skipped promotion. The best way I see is mock logger.info and check assert_calls with the log message that we skipped promotion13:15
rlandy|ruckweshay: adding missing rocky standalone - check and/or periodic?13:24
*** amoralej|lunch is now known as amoralej13:27
rlandy|ruckquiquell|rover: weshay: anything to bring up at tripleo meeting?13:30
quiquell|roverI want to talk about https://bugs.launchpad.net/tripleo/+bug/1815744 b13:31
openstackLaunchpad bug 1815744 in tripleo "build-test-package does not fail on partial builds" [Critical,In progress] - Assigned to Quique Llorente (quiquell)13:31
quiquell|roverDon't know what to do with it13:31
quiquell|roverI have a partial solution13:31
rfolcoThe CI Community Meeting (office open hours) start now at https://bluejeans.com/4113567798 - agenda: https://etherpad.openstack.org/p/tripleo-ci-squad-meeting13:31
*** ykarel has joined #oooq13:32
*** mjturek has quit IRC13:35
*** mjturek has joined #oooq13:35
*** udesale has joined #oooq13:36
*** vinaykns has joined #oooq13:38
*** Vorrtex has joined #oooq13:58
zbrrlandy|ruck: can you help with https://review.openstack.org/#/c/645626/20 ?14:22
rlandy|ruckzbr: on three meetings atm - sec14:23
mariosrfolco: wil you send invite please?14:26
mariosrfolco: for the call14:26
mariostomorrow14:26
mariosquiquell|rover: ^14:26
rfolcoyes will do marios quiquell|rover14:26
mariosrfolco: thanks14:26
rfolcomarios, there is no scrum tomorrow, what time works best for you ?14:27
rfolco2pm utc ?14:27
mariosrfolco: fine for me14:27
mariosrfolco: sec14:27
mariosmaybe there is ruck call then14:27
rfolcoprog call is 1pm utc14:28
mariosrfolco: ah no should be ok14:28
mariosrfolco: ack14:28
quiquell|roverrfolco: you can takeover https://review.openstack.org/#/c/64895014:28
quiquell|roverrfolco: my idea was to first refactor unit tests14:28
quiquell|roverrfolco: to reduce redundancy14:28
rfolcoquiquell|rover, ok but still need you on the 15 min call to explain me the rationale behind that refactoring14:28
quiquell|roverrfolco: like the stuff that is there14:28
quiquell|roverrfolco: ack will do14:29
rfolcoquiquell|rover, marios thx for your collaboration14:29
*** dtantsur|brb is now known as dtantsur14:30
mariosrfolco: https://www.youtube.com/watch?v=nx2G3OCVRNk14:31
rfolcomarios, :)14:31
quiquell|rovernice14:31
quiquell|roverrobofolcop14:32
rfolcohahaha14:33
rfolcocome on, my speech is always sincere and natural14:33
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario010-multinode-oooq-container, tripleo-ci-fedora-28-standalone @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp- (3 more messages)14:34
rfolcomarios, I'm "SORRYIN'", I'd never "KICK A" guy like you.14:38
rfolcook I stop14:38
mariosrfolco: WE14:38
quiquell|roveranother leason to leave from here14:38
quiquell|rover:-D14:38
mariosr  sorren14:38
*** quiquell|rover is now known as quiquell|off14:38
quiquell|offreason14:39
rfolcohave a good one quiquell|off14:39
rfolco;)14:39
*** panda has quit IRC15:22
*** bogdando has joined #oooq15:25
*** panda has joined #oooq15:34
*** udesale has quit IRC15:37
*** apetrich has quit IRC15:50
*** jfrancoa has quit IRC16:01
*** mjturek has quit IRC16:05
*** marios has quit IRC16:14
zbrquiquell|off: while you are off, maybe you can help me remove few files https://review.openstack.org/#/c/650913/ and its dep sister. -- finally got confirmation that it does not affect rocky/queens.16:20
*** dsneddon has joined #oooq16:23
*** dsneddon has quit IRC16:28
*** dsneddon has joined #oooq16:29
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario010-multinode-oooq-container, tripleo-ci-fedora-28-standalone @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp- (3 more messages)16:34
*** jpena is now known as jpena|away16:50
*** mjturek has joined #oooq16:50
*** derekh has quit IRC17:00
*** dtantsur is now known as dtantsur|afk17:03
rlandy|ruckweshay: hi - can we catch up on the ruck/rover stuff?17:07
*** ykarel is now known as ykarel|away17:12
*** dsneddon has quit IRC17:17
*** ykarel|away has quit IRC17:17
weshayrlandy|ruck ya, I'm avail17:23
rlandy|ruckweshay: sec - working with sorin to get his reviews in atm17:23
*** holser_ has quit IRC17:30
rlandy|ruckweshay: can we delete newton release files now?17:31
rlandy|ruckstill any support for this release?17:32
weshaydepends on what we do w/ upgrades17:32
rlandy|ruckweshay: looking at https://review.openstack.org/#/c/65092117:32
weshayrlandy|ruck you talking to sorin17:32
weshay?17:32
rlandy|ruckI am17:32
weshayI'll jump in17:32
rlandy|ruckrelease support is a PM question17:33
rlandy|ruckjoin bj17:33
weshaywhich?17:33
rlandy|ruck<zbr> https://bluejeans.com/265541792817:33
*** bogdando has quit IRC17:34
*** kopecmartin is now known as kopecmartin|off17:36
*** jpena|away is now known as jpena17:43
*** panda has quit IRC17:43
*** holser_ has joined #oooq17:50
*** dsneddon has joined #oooq17:51
weshayzbr thanks17:52
zbrrlandy|ruck: weshay : thanks! really cool to ho hands on reviewing with you.17:52
*** jpena is now known as jpena|off17:53
*** aakarsh has quit IRC18:08
*** amoralej is now known as amoralej|off18:15
*** rlandy|ruck is now known as rlandy|ruck|hw18:19
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario010-multinode-oooq-container, tripleo-ci-fedora-28-standalone @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp- (2 more messages)18:34
weshayrlandy|ruck|hw stein rdo phase 1 should promote, deploy worked, tempest running18:35
rlandy|ruck|hwweshay: \o/18:36
*** dsneddon has quit IRC18:40
*** dsneddon has joined #oooq18:41
*** aakarsh has joined #oooq18:41
rfolcoweshay, in the scope of buildah for master/stein upstream, we want 1) f28 buildah jobs (emilien confirmed) 2) make sure buildah is voting 3) branchful stein buildah job just liked we did for rocky: https://github.com/openstack-infra/tripleo-ci/commit/e7317c7a58d14f37345e504a9462658b27cbfed9 ... so my question is: should we remove/replace old docker container build job and keep them ? If not replacing, master and stein will run both buildah one and docker18:47
rfolcojobs.18:47
rfolcoreplace OR*** keep them :)18:48
* weshay reads18:55
weshayrfolco once we have the buildah jobs voting and gating, bring the question to the #tripleo meeting and ask... I think they can be removed at that point18:57
rfolcoweshay, ok, sounds like a plan18:57
weshayrlandy|ruck|hw fyi.. I figured out stein rdo-1 is not promoting20:06
* weshay sends review20:06
rlandy|ruck|hwok - looking ofr review20:06
weshayrlandy|ruck|hw https://review.rdoproject.org/r/2005120:08
weshayhttps://ci.centos.org/view/rdo/view/promotion-pipeline/job/rdo_trunk-promote-stein-current-tripleo/20:08
weshaypackstack002 is gone20:08
rlandy|ruck|hw+2'ed20:09
vinayknsweshay: workaround for this one http://pastebin.test.redhat.com/75140420:13
*** aakarsh has quit IRC20:15
rlandy|ruck|hwweshay: ha - half our hw is missing :(20:15
weshayvinaykns that's odd..  /me wonders if the regular expression changed20:16
weshayvinaykns does curl -s https://trunk.rdoproject.org/centos7-master/current-tripleo/delorean.repo | grep baseurl20:16
weshaywork for oyu?20:16
weshayrlandy|ruck|hw say what?20:16
weshay2019-04-09 20:16:48,687 7584 INFO     promoter Promoting the container images for dlrn hash 1b93d589fed8dbe50017af88d987b32c44e20b21 on stein to current-tripleo-rdo20:16
rlandy|ruck|hwracks 05 and 05 are not listed20:17
* rlandy|ruck|hw asks phil20:17
weshaythanks20:17
*** mjturek has quit IRC20:18
weshayvinaykns what release is that?20:18
weshayugh "ERROR ========== centos-binary-neutron-server-opendaylight IS NOT BUILT! FIX THIS ASAP! =========="]}20:19
vinayknsits the latest master branch20:19
weshayvinaykns so.. -R master ?  or -R tripleo-ci/CentOS-7/master20:20
vinayknsmaster-tripleo-ci20:21
*** Goneri has quit IRC20:24
weshayvinaykns shit.. that string has 41 characters20:26
weshayrlandy|ruck|hw we have a problem houston20:26
*** rlandy|ruck|hw is now known as rlandy|ruck20:27
rlandy|ruckweshay: ack - how can I help?20:27
weshayrlandy|ruck it seems that the dlrn_hash just changed from 40 chars long to 4120:27
rlandy|ruckoh shoot20:27
weshaythus blowing up https://github.com/openstack/tripleo-quickstart/blob/master/roles/repo-setup/tasks/get-dlrn-hash.yml20:27
rlandy|ruckthat assumption is everywhere20:27
* weshay seeing how real this is20:27
weshayI know20:27
*** holser_ has quit IRC20:28
* rlandy|ruck stands by20:28
weshayrlandy|ruck but that promoted 3 days ago20:28
weshayI don't see jobs failing20:28
vinayknsthx..weshay for now I'll update manually20:28
weshayhrm20:29
weshayecho "c16eb31ee0e6e2f01d50794c90a19c85793515a2" | wc -c                                                                master✔ thinkdoe ⌚ 14:30:0020:30
weshay4120:30
weshayvinaykns I may be wrong20:30
weshaybecause20:30
weshaycurl -s https://trunk.rdoproject.org/centos7-master/current-tripleo/delorean.repo | grep baseurl | grep -Eo \"[a-zA-Z0-9]{41}_[a-zA-Z0-9]{8}\"20:30
weshayis also not working20:30
rlandy|ruckweshay: should we bj and figure this out?20:30
weshayif you have a sec20:31
rlandy|ruckyeah20:31
rlandy|ruckjoining20:31
*** aakarsh has joined #oooq20:32
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci- (2 more messages)20:34
*** aakarsh has quit IRC20:37
*** aakarsh has joined #oooq20:40
weshayvinaykns hey.. does curl -s https://trunk.rdoproject.org/centos7-master/current-tripleo/delorean.repo | grep baseurl | grep -Eo '[a-zA-Z0-9]{40}_[a-zA-Z0-9]{8}' work for you20:43
*** holser_ has joined #oooq20:44
*** aakarsh has quit IRC20:45
vinayknsweshay nope20:45
weshaycan you try from a diff network or something20:46
vinayknsyeah in the internal network it doens't work for me20:47
vinayknsbut it works outside20:47
*** Vorrtex has quit IRC20:47
*** holser_ has quit IRC20:59
vinayknsweshay: am I getting blocked or something on that url..?21:00
vinayknsweshay: http://pastebin.test.redhat.com/75142221:11
*** Goneri has joined #oooq21:27
*** chem` has quit IRC21:45
weshayvinaykns yup21:57
*** vinaykns has quit IRC22:00
rlandy|ruckweshay:  https://review.rdoproject.org/r/2005222:00
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci- (2 more messages)22:34
*** rlandy|ruck is now known as rlandy23:06
*** rlandy has quit IRC23:09
*** tosky has quit IRC23:36

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!