Wednesday, 2018-05-16

*** saneax is now known as saneax-_-|AFK00:00
*** agopi has joined #oooq00:03
rlandystable_release is not defined00:09
hubbotFAILING CHECK JOBS on stable/ocata: gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-ocata, tripleo-ci-centos-7-undercloud-upgrades @ https://review.openstack.org/564291, master: tripleo-ci-centos-7-scenario002-multinode-oooq-container, tripleo-ci-centos-7-3nodes-multinode, gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master, tripleo-ci-centos-7-scenario001-multinode-oooq-container, gate-tripleo- (2 more messages)00:13
*** myoung|ruck|afk is now known as myoung|ruck00:14
myoung|rucksshnaidm|rover: when do you want to swap ruck/rover?00:14
rlandybbl00:16
*** rlandy is now known as rlandy|bbl00:16
*** agopi has quit IRC00:19
*** dtrainor has quit IRC00:52
*** atoth has quit IRC01:59
rlandy|bblscript fails02:01
hubbotFAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-scenario002-multinode-oooq-container, tripleo-ci-centos-7-scenario003-multinode-oooq-container, tripleo-ci-centos-7-containers-multinode, tripleo-ci-centos-7-3nodes-multinode, tripleo-ci-centos-7-scenario001-multinode-oooq-container, tripleo-ci-centos-7-scenario004-multinode-oooq-container, gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-queens,  (2 more messages)02:13
*** rlandy|bbl has quit IRC02:16
*** gkadam has joined #oooq02:36
*** agopi has joined #oooq02:50
*** agopi has quit IRC02:54
*** agopi has joined #oooq02:58
*** gkadam has quit IRC03:07
*** agopi has quit IRC03:10
*** agopi has joined #oooq03:10
*** links has joined #oooq03:41
*** sanjay__u has joined #oooq04:03
hubbotFAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-containers-multinode, tripleo-ci-centos-7-3nodes-multinode, gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-queens, gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-queens @ https://review.openstack.org/567224, stable/ocata: gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-ocata, tripleo-ci-centos-7-undercloud-upgrades @  (2 more messages)04:13
*** moguimar has quit IRC04:38
*** dtantsur|afk has quit IRC04:38
*** moguimar has joined #oooq04:39
*** jaosorior has quit IRC04:40
*** jaosorior has joined #oooq04:47
*** ykarel|away has joined #oooq04:50
*** saneax-_-|AFK is now known as saneax04:51
*** jaosorior has quit IRC04:54
*** jaosorior has joined #oooq05:02
*** udesale has joined #oooq05:11
*** ratailor has joined #oooq05:23
*** saneax is now known as saneax-_-|AFK05:23
*** jtomasek has joined #oooq05:36
*** marios has joined #oooq05:43
*** quiquell|off is now known as quiquell05:43
*** jfrancoa has joined #oooq06:03
*** pliu has quit IRC06:04
*** lucas-afk has quit IRC06:05
*** pliu has joined #oooq06:06
*** lucasagomes has joined #oooq06:07
*** moguimar has quit IRC06:11
hubbotFAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-containers-multinode, tripleo-ci-centos-7-3nodes-multinode, gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-queens, gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-queens @ https://review.openstack.org/567224, stable/ocata: gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-ocata, tripleo-ci-centos-7-undercloud-upgrades @  (2 more messages)06:13
*** moguimar has joined #oooq06:13
quiquellsshnaidm|rover: I see grafana kind of limiting to express some stuff06:15
quiquellhttps://github.com/grafana/grafana/issues/1141806:15
quiquellMaybe we can try ELK (elasticsearch, logstash, kibana)06:15
*** skramaja has joined #oooq06:26
*** udesale has quit IRC06:27
*** udesale has joined #oooq06:29
*** holser__ has joined #oooq06:33
*** chandankumar has quit IRC06:38
*** apetrich has quit IRC06:38
*** chandankumar has joined #oooq06:39
sshnaidm|roverquiquell, ELK? for which purpose?06:39
quiquellFor the ruck rover monitoring tool06:44
*** saneax-_-|AFK is now known as saneax06:47
*** quiquell is now known as quiquell|afk06:49
*** apetrich has joined #oooq06:50
sshnaidm|roverquiquell|afk, but for which task? what doesn't grafana support that elk does support?06:57
*** udesale has quit IRC06:58
*** udesale has joined #oooq07:07
*** ccamacho has joined #oooq07:08
*** agopi has quit IRC07:17
*** tesseract has joined #oooq07:17
*** udesale has quit IRC07:19
*** ykarel|away is now known as ykarel07:19
*** udesale has joined #oooq07:24
*** zoli is now known as zoli|sickday07:26
*** zoli|sickday is now known as zoli07:27
*** quiquell|afk is now known as quiquell07:33
quiquellsshnaidm|rover: coloring table rows only possible with int values07:33
quiquellWe can bypass that without much effort07:34
quiquellNot big deal07:34
*** florianf has joined #oooq07:34
sshnaidm|roverquiquell, what do you want to use tables for?07:37
quiquellDLRN promoter history07:37
quiquellsshnaidm|rover: nevermind, found a solution07:37
sshnaidm|roverok, great07:37
*** tosky has joined #oooq07:38
sshnaidm|roverquiquell, btw, we have elk in openstack: http://logstash.openstack.org/#/dashboard/file/logstash.json07:41
sshnaidm|roverquiquell, and it even works sometimes07:41
sshnaidm|roverquiquell, so generally speaking we can use it07:42
*** agopi has joined #oooq07:43
quiquellsshnaidm|rover: Cool, will take a look07:44
*** kopecmartin has joined #oooq07:53
*** amoralej|off is now known as amoralej07:53
*** dalvarez has joined #oooq07:55
*** ykarel is now known as ykarel|lunch07:56
quiquellsshnaidm|rover: https://review.openstack.org/#/c/567521/  ready for +2 ?08:07
sshnaidm|roverquiquell, will look08:08
hubbotFAILING CHECK JOBS on stable/ocata: gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-ocata, tripleo-ci-centos-7-undercloud-upgrades @ https://review.openstack.org/564291, master: tripleo-ci-centos-7-scenario002-multinode-oooq-container, tripleo-ci-centos-7-3nodes-multinode, gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master, tripleo-ci-centos-7-scenario001-multinode-oooq-container, gate-tripleo- (2 more messages)08:13
*** ykarel|lunch is now known as ykarel08:37
*** agopi has quit IRC08:44
*** gkadam has joined #oooq08:53
*** links has quit IRC09:07
*** panda|bbl is now known as panda09:18
*** bogdando has joined #oooq09:23
*** links has joined #oooq09:24
quiquellpanda: Is this https://review.openstack.org/#/c/567521 missing something to have a +2 ?09:30
quiquell"09:30
quiquellAdd python script to dynamically compose releases09:30
quiquellpanda, sshnaidm|rover: It's possible to rebase a review that depends on other reviews not already merged ?09:33
sshnaidm|roverquiquell, yeah, and it will rebase all dependencies too09:34
quiquellgit pull --rebase will do ? or do I have to fetch the latest refspecs ?09:34
sshnaidm|roverquiquell, you can rebase by clicking rebase button in gerrit09:38
sshnaidm|roverquiquell, if it has merge conflicts, then you need to fetch latest master, and rebase on it in the patch09:38
sshnaidm|roverquiquell, commented on review09:38
quiquellsshnaidm|rover: I mean doing for not merged dependencies09:39
sshnaidm|roverquiquell, when you rebase top patch, it will rebase also all dependencies09:39
sshnaidm|roverquiquell, what do you want to get?09:39
quiquellsshnaidm|rover: Want to rebase https://review.openstack.org/#/c/568285 to the lastest PS from its dependencies09:40
quiquellrebase is not ding that09:40
sshnaidm|roverquiquell, which dependencies?09:42
quiquellThis review is done in top of other reviews09:42
quiquellThose reviews have newer PSs09:42
quiquellI want to do that09:42
sshnaidm|roverquiquell, oh, I see09:43
sshnaidm|roverquiquell, well, manual rebase09:43
sshnaidm|rovergit rebase, git rebase --skip, git rebase , etc..09:43
*** dtantsur has joined #oooq09:43
quiquellOk...09:43
quiquellThanks sshnaidm|rover09:43
sshnaidm|roverquiquell, yeah, it's not easy, especially if it has merge conflicts..09:44
pandaquiquell: you have to address sshnaidm|rover comments on that change09:47
quiquellpanda: Yep, just found it after send it to you09:47
pandaquiquell: and you can't +1 your own review :) It's like liking your own post on facebook09:48
quiquellpanda: There is trown work there I was +1 his work09:48
pandaquiquell: trown's ps was 12, you +1 PS 13. Anyway adding a comment helps understanding what you're +1 and why.09:50
quiquellpanda: Ohh ok, will fix my +109:51
quiquellThanks09:51
quiquellpanda: Sagi's comment fixed, let's try to merge this thing https://review.openstack.org/#/c/567521/10:11
quiquellEverything is in place now10:11
hubbotFAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-3nodes-multinode, gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-queens, gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-queens, tripleo-ci-centos-7-containers-multinode @ https://review.openstack.org/567224, stable/ocata: gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-ocata, tripleo-ci-centos-7-undercloud-upgrades @  (2 more messages)10:13
panda%mor10:15
hubbotpanda: Error: "mor" is not a valid command.10:15
panda%more10:15
hubbotpanda: Error: You haven't asked me a command; perhaps you want to see someone else's more.  To do so, call this command with that person's nick.10:15
panda%more | adarazs10:15
hubbotpanda: Error: I haven't seen |.10:15
panda%more adarazs10:15
hubbotpanda: Error: I haven't seen adarazs.10:15
quiquellpanda: Leave the pet alone10:16
pandaquiquell: I'm giving the patch the big review.10:21
quiquellpanda: \o/ !!!!10:21
pandaquiquell: stay with me for a while10:21
quiquellbj you mean ?10:21
pandaquiquell: not needed. IRC is fine10:22
pandaquiquell: LONG_TERM_SUPPORT_RELEASE contains only queens10:23
pandaquiquell: newton is also LTS, but I understand that it will not receive changes in the foreseeable future10:23
pandaquiquell: still seems weeird we're not mentioning it10:24
quiquellpanda: We are going to have more than one LTS relase ?10:25
quiquells/relase/release/10:25
quiquellRight now we are doing ffu from newton10:25
pandaquiquell: that's the idea with LTS usually10:26
*** jaganathan_ has quit IRC10:26
quiquellpanda: Wait... it says queens, but really is refering to newton10:27
pandaquiquell: ...10:28
quiquellqueens is the target,10:28
pandaquiquell: ok, now bj is needed :)10:28
quiquellSure10:28
quiquellhttps://bluejeans.com/789106523210:28
*** links has quit IRC10:44
sshnaidm|rovermyoung|ruck, Containers multinode jobs fails on stable pike because of pacemaker  https://bugs.launchpad.net/tripleo/+bug/177155110:52
openstackLaunchpad bug 1771551 in tripleo "Containers multinode jobs fails on stable pike because of pacemaker" [Critical,Triaged] - Assigned to Sagi (Sergey) Shnaidman (sshnaidm)10:52
sshnaidm|rovermyoung|ruck, https://bugs.launchpad.net/tripleo/+bug/1771549  https://bugs.launchpad.net/tripleo/+bug/177154910:52
openstackLaunchpad bug 1771549 in tripleo "Containers multinode jobs fails on stable queens" [Critical,Triaged] - Assigned to Sagi (Sergey) Shnaidman (sshnaidm)10:52
pandaquiquell: how do I test this ? just launch pytest on the current dir ?10:54
quiquellyep10:54
quiquellinstall the stuff at test-requirements.txt10:54
quiquellyou can even run pytest directly under tripleo-ci/10:54
quiquellIt will discover all the tests10:55
panda24 passed in 0.03 seconds10:55
quiquellWe are good then10:57
quiquellCheck the test suite10:57
quiquellTo see if we cover all the cases10:57
pandaI'll propose a patch to the jobs10:57
quiquellBut we can check this out after merging it10:57
quiquellok10:58
*** links has joined #oooq10:58
pandaquiquell: why do you mean with "encapsulate" job  in 567649 ?11:01
quiquellpanda: I think zuul have the concept of base jobs, that you can inheritance from11:02
quiquellmaybe we can put the filter expression there an inherit from the job11:02
pandaquiquell: so you mean specifying the files paramter only once ?11:02
quiquellpanda: Yep11:03
*** lucasagomes is now known as lucas-hungry11:04
pandaquiquell: +211:08
quiquellYipi !!!11:08
quiquellsshnaidm|rover: Give it another +2 :) https://review.openstack.org/#/c/56752111:09
pandaquiquell: I have a note for you11:13
quiquellpanda, sshnaidm|rover: for the rebase of in-review changes 'git review -i' is your friend11:15
quiquellreplacing manually the commit-id is working11:15
sshnaidm|roverquiquell, nice11:15
sshnaidm|roverquiquell, I can review only from python view, leaving the logic for sprint folks11:16
quiquellpanda: Agree with the note, let's do a more declarative approach in a later refactoring11:17
quiquellsshnaidm|rover: So you cannot +211:17
quiquellOk11:17
sshnaidm|roverI don't fully understand what happens there11:17
quiquellLet's wait for the other folks11:17
pandaquiquell: regarding 567320, trown stated that using other method for the retries was breaking his persistence approach11:18
pandaquiquell: and it may be not worth for such a small download11:18
quiquellI have rebase all the maze to the latest maze parts (I was not expecting that)11:19
quiquellpanda: What persistence ?11:19
pandaquiquell: was what not expecting ?11:19
quiquellpanda: the git rebase -i command updating all the reviews11:20
pandaquiquell: I'm not sure, you'll have to ask him. For me the patch is good enough. I really want to start testing as soon as possible11:20
pandatomorrow the latest11:20
quiquellpanda: Agree with that, just feels weird to do a manual retry11:20
quiquellLet's get this merge I am going to add a +1 to state that11:21
quiquellpanda: We can have this merge today, we are not going to break anything11:23
*** jaosorior has quit IRC11:23
pandaquiquell: w/r/t 56793611:24
pandaquiquell: what's the goal of your comments ?11:25
quiquellpanda: the +1 comment or the code comment ?11:26
pandaquiquell: I see no +111:26
quiquellhttps://review.openstack.org/#/c/56793611:26
quiquellIs there11:26
pandaquiquell: ok, gertty synced now11:27
*** quiquell is now known as quique|lunch11:32
weshaysshnaidm|rover, morning https://review.openstack.org/#/c/568701/11:33
weshaydid it update the contentdir file correctly on both subnode0 and 1 for you?11:34
sshnaidm|roverweshay, didn't test it, but should11:34
sshnaidm|roverthis role runs on all subnode11:35
*** panda is now known as panda|lunch11:36
weshaypanda|lunch, for next sprint.. one of the goals should be to get the libvirt repro into ci.centos https://review.openstack.org/#/c/568701/11:40
weshayhttps://bugs.launchpad.net/tripleo/+bug/177149811:40
openstackLaunchpad bug 1771498 in tripleo "libvirt reproducer script fails on yum var $contentdir" [High,Triaged]11:40
*** jaosorior has joined #oooq11:42
*** gkadam has quit IRC11:43
*** gkadam has joined #oooq11:43
weshaysshnaidm|rover, any further thoughts on the image corruption for the bm images?11:48
sshnaidm|roverweshay, running now  https://review.openstack.org/#/c/568805/11:49
weshayinteresting.. on a new slave each run you think that would be an issue?11:50
*** atoth has joined #oooq11:51
weshaypanda|lunch, https://bugs.centos.org/view.php?id=1478611:58
*** trown|outtypewww is now known as trown11:59
panda|lunchweshay: you mean test reproducer itself12:06
weshaypanda|lunch, yes sir12:06
panda|lunchhttps://trello.com/c/eRx0S8ip/763-add-job-to-test-libvirt-reproducer-in-cicentos12:07
*** amoralej is now known as amoralej|lunch12:07
*** lucas-hungry is now known as lucasagomes12:09
hubbotFAILING CHECK JOBS on stable/ocata: gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-ocata, tripleo-ci-centos-7-undercloud-upgrades @ https://review.openstack.org/564291, master: tripleo-ci-centos-7-3nodes-multinode, gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-queens-branch, gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master, gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001 (1 more message)12:13
*** tcw has quit IRC12:21
*** rlandy has joined #oooq12:29
rlandypanda: hello12:31
rlandypanda: I tried to do some integration testing last night12:31
rlandyweshay wanted us to investigate a bug with releases being passed12:31
rlandyhttps://review.openstack.org/#/c/568717/2/toci_gate_test-oooq.sh12:32
rlandyI added this ...12:32
rlandyto test12:32
rlandywe need to define the script location and output location12:33
weshaythanks rlandy12:34
rlandyalso stable release12:34
weshayrlandy, I'm still a little bit on fire atm12:34
rlandyweshay: I understand12:34
weshaybut if you need to chat, now is a decent time12:34
*** panda|lunch is now known as panda12:34
weshayor anyone12:34
rlandyweshay: I need to chat with panda now12:35
rlandywe will get back to you12:35
rlandywith output results12:35
weshayk.. thanks12:35
pandarlandy: hhmm12:41
arxcruzweshay: 1-1 moved to tomorrow ?12:41
rlandypanda: http://logs.openstack.org/17/568717/2/check/tripleo-ci-centos-7-containerized-undercloud-upgrades/069525f/job-output.txt.gz#_2018-05-16_00_25_02_85297112:41
rlandywe actually do know where the script will land12:42
pandarlandy: yes, I was trying to look at the script log12:42
rlandyand we can decide where the output will be12:42
rlandyalso the definition of stable release is an issue12:42
rlandyI made some assumptions in the script12:42
rlandyanother issue is that when the script fails, there is no error output12:42
rlandyas failing the script does not get log collected12:43
rlandythe rest works :)12:43
pandarlandy: what rest ? :)12:43
rlandypanda: the recognition of what fs we are using12:44
rlandyprinting out the args12:44
pandarlandy: one problem at a time. What are your assumptions ?12:44
rlandysingle sourcing them12:44
rlandypanda: ok - let's start at the top12:44
rlandyhttp://logs.openstack.org/17/568717/2/check/tripleo-ci-centos-7-containerized-undercloud-upgrades/069525f/job-output.txt.gz#_2018-05-16_00_25_02_85297112:45
pandarlandy: avil to chat if it's easier for you12:45
rlandypanda: ok - might be easier12:45
*** tcw has joined #oooq12:45
pandarlandy: I'm already in my channel and in your channel too12:46
rlandyjoining your channel12:46
*** Goneri has joined #oooq12:51
*** moguimar has quit IRC12:57
*** ratailor has quit IRC13:01
myoung|rucksshnaidm|rover: swap ruck/rover?13:02
myoung|ruck(caveat - we're all hands on deck anyway and this sprint the lines have been blurred anyway)13:03
sshnaidm|rovermyoung|ruck, I don't know, I have a lot of things in progress.. maybe next week?13:03
myoung|rucksshnaidm|rover: ok...as sprint ends in 6/7 days, I'm going to be working on some rfe/bugs as time allows in parallel then.  Also need to make progress on rhos-12/rhos-13 TQ/TQE gates downstream.  You do have a lot of critical things in progress.13:08
myoung|ruckit's been an exciting sprint in ruck/rover land :)13:08
sshnaidm|rovermyoung|ruck, oh yeah))13:08
sshnaidm|rovermyoung|ruck, seem fine to me, I don't have time for downstream at all13:09
myoung|ruckweshay: do you want to sync with sshnaidm|rover and myself before the day get's long in the tooth?  Feel like we're juggling quite a bit, and time/joules/calories/synapse_sparks are finite.13:10
myoung|ruck^^ (briefly)13:10
weshaymyoung|ruck, sshnaidm|rover I can chat in irc but on a 4hr manager call in blue atm13:11
myoung|ruckweshay: 4 hour manager meeting!  Please know that you are in our thoughts and prayers.  Stay the course.  Be strong.13:12
pandarlandy: https://review.openstack.org/566565 upodated13:14
rlandypanda: cool - looking13:15
sshnaidm|rovermyoung|ruck, weshay so latest possible solution for introspection is:  https://review.openstack.org/56883813:18
ssbarneahi! i got a "brctl: command not found" on http://logs.openstack.org/89/568789/1/check/tripleo-ci-centos-7-undercloud-containers/9c8dce1/ara-report/ which seems totally unrelated to my first quickstart patch. is this caused by a known issue?13:19
*** quique|lunch is now known as quiquell13:23
*** amoralej|lunch is now known as amoralej13:26
rlandypanda: looks good - rebasing my patch on that13:27
quiquellrlandy, trown: By mistake (but it's a good thing) I have rebase all the maze commits13:28
quiquellYou will see they are in top of the latest PS13:28
rlandyquiquell: works for me :)13:28
quiquellcool13:28
trownquiquell: ya I pushed the bottom one, just working on lots of unittests for later patches today13:29
trownmock mock mockety mock mock13:29
myoung|rucksshnaidm|rover: looking13:29
quiquelltrown: Looks like having fun :-)13:29
quiquellbtw git rebase -i allow you to manually put the commit id13:30
quiquellThis way you can rebase in-review dependencies13:30
quiquelltrown: what do you think about runnign some integration testing with real featuresets and real output ?13:33
quiquellto test main13:33
trownquiquell: ya can always do `git commit -a --amend` before git review to manually put change id in13:33
trownquiquell: ya makes sense really, just need to put some "fixture" featuresets there to test with13:34
quiquelltrown: this work when dependencencies are not merged ?13:34
trownquiquell: not exactly sure what you mean... but Change-Id is 1to1 mapping to gerrit review13:35
trownquiquell: so if I put up a totally unrelated change with the same change-id, it will put it up as a new patchset on that gerrit review13:35
quiquelltrown: It doesn't fail ?13:36
myoung|rucksshnaidm|rover: nice debug / proposed solution, I've updated the bug.  Regarding the 'ts' issue, do we think that is a additional (discreet) issue (different LP needed?) https://bugs.launchpad.net/tripleo/+bug/1770972/comments/1413:36
openstackLaunchpad bug 1770972 in tripleo "CI: Images introspection fails in OVB jobs" [Critical,Triaged] - Assigned to Derek Higgins (derekh)13:36
quiquelltrown: Damn so it was just easier argg13:36
myoung|rucksshnaidm|rover: mentioned this bug at program call, and will talk in 1 hr at CIX call about it13:36
quiquell:-)13:36
trownquiquell: maybe if you have some bad dependency graph it will fail... like a loop13:36
trownquiquell: but as long as the change could merge, I think that always works13:37
quiquelltrown: Cool, will try that next time instead of rebase -i and manual commit id13:37
sshnaidm|rovermyoung|ruck, I suspect it's the reason for introspection failures13:37
trownrebase -i can still be useful with these big patch chains though13:37
sshnaidm|rovermyoung|ruck, let's leave it in the same bug atm..13:37
myoung|rucksshnaidm|rover: ack, agreed13:37
trownquiquell: but I tend to just checkout the bottom patch, then cherry-pick one at a time to get to where I want to work13:38
quiquelltrown: And also rebased the dependencies for free13:38
quiquelltrown: noted13:38
quiquell Let's get the big patch merged13:38
quiquellVerified state already13:38
quiquellmyoung|ruck, sshnaidm|rover: upstream check jobs are ok ?13:41
myoung|ruckquiquell: umm...define "ok" :)13:41
quiquellmyoung|ruck: The commit to add the new script is failing at check13:42
quiquellThat's kinda of a noop for them13:42
myoung|ruckquiquell: https://trello.com/c/c3U7NbsN/587-cixlp1770972tripleociproa-ci-images-introspection-fails-in-ovb-jobs is blocking OVB introspection across board... sshnaidm|rover has a patch that might resolve that13:42
quiquell%gatestatus13:42
hubbotFAILING CHECK JOBS on stable/ocata: gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-ocata, tripleo-ci-centos-7-undercloud-upgrades @ https://review.openstack.org/564291, master: tripleo-ci-centos-7-3nodes-multinode, gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-queens-branch, gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master, gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001 (1 more message)13:42
quiquellmyoung|ruck: Ok, thanks13:42
myoung|ruckquiquell: (juggling) - link to patch you're talking about?13:43
quiquellmyoung|ruck: https://review.openstack.org/#/c/56752113:43
myoung|ruckthx...looking13:43
sshnaidm|roverquiquell, this may affect too: https://bugs.launchpad.net/tripleo/+bug/177150813:45
openstackLaunchpad bug 1771508 in tripleo "Telemetry tests fail in scenario-001 and 002 jobs" [Critical,Triaged] - Assigned to Pradeep Kilambi (pkilambi)13:45
sshnaidm|rovermyoung|ruck, ^^13:45
sshnaidm|roverbut the fix was merged13:45
sshnaidm|rovers/fix/workaround/13:46
quiquellsshnaidm|rover: Can we bypass the check ?13:46
myoung|ruckquiquell: hrm http://logs.openstack.org/21/567521/14/check/tripleo-ci-centos-7-undercloud-containers/a4df2f9/logs/undercloud/home/zuul/undercloud_install.log.txt.gz#_2018-05-16_10_35_0413:46
sshnaidm|roverquiquell, check is only that checks :)13:46
rlandypanda: ok - let's see what https://review.openstack.org/#/c/567060/ does13:47
rlandyrebased, added the deps - made some changes for comments13:47
*** jfrancoa has quit IRC13:48
quiquellsshnaidm|rover: Without +1v the patch does not go to gate ?13:49
myoung|ruckquiquell, sshnaidm|rover: I don't recall seeing "create_swift_temp_url_key" error with docker yet...have you?13:49
quiquellmyoung|ruck: Nope13:50
myoung|ruckhrm...this might be a new one.  thanks for heads up quiquell, /me investigates and might have another LP coming13:50
pandaquiquell: it's mandatory, the patch will need a +1v in checks before going to gates, but the gates will start automatically after the checks.13:50
*** jfrancoa has joined #oooq13:50
quiquellpanda: So we are stuck at check :-/13:50
pandaquiquell: patience, young padawan13:51
quiquellpanda: I have the worse or both worlds I am apadawan but I am not young :-)13:52
myoung|ruckrlandy, trown: I hate to interrupt during sprint, looking a a pile of stuff at once.  Could you please glance at https://bugs.launchpad.net/tripleo/+bug/1771563 to quickly sanity check my usage?  ovb reproducer is failing for me and I haven't had cycles to debug it yet.13:52
openstackLaunchpad bug 1771563 in tripleo "ovb reproducer fails with "AnsibleUndefinedVariable: 'dict object' has no attribute 'ansible_host'"" [High,Triaged]13:52
myoung|ruck^^ could be me doing things when I should be sleeping lol13:53
rlandymyoung|ruck: looking13:53
*** udesale_ has joined #oooq13:55
*** moguimar has joined #oooq13:55
rlandytrown: quiquell: just fyi ... we're testing the integration of the releases script into toci_quickstart now13:56
quiquellrlandy: It's working ?13:56
quiquellrlandy: Does it make sense to have a reproducer for the openstack-tox jobs ?13:58
rlandyquiquell: not quite - looking at the logs13:58
rlandyreproducer?13:58
*** udesale has quit IRC13:58
rlandytox13:58
rlandytox -e13:58
rlandydone13:58
quiquellrlandy: right13:58
quiquell:)13:58
rlandypanda: ok - we failed out13:59
quiquellrlandy, panda: Do you need two more eyes for the toci logs ?13:59
rlandybut hopefully with a log this time13:59
rlandyhttp://logs.openstack.org/60/567060/19/check/tripleo-ci-centos-7-containerized-undercloud-upgrades/289354e/logs/emit_releases_file.txt.gz13:59
rlandyquiquell: panda" ^^13:59
myoung|rucksshnaidm|rover, chandankumar: I noticed overnight (day for you) we've merged a patch for skiplist (https://bugs.launchpad.net/tripleo/+bug/1771508) for telemetry tests13:59
openstackLaunchpad bug 1771508 in tripleo "Telemetry tests fail in scenario-001 and 002 jobs" [Critical,Triaged] - Assigned to Pradeep Kilambi (pkilambi)13:59
rlandyok - at least we captured the error now13:59
myoung|ruckyesterday late in day (night for you) we think root cause was found/fixed: https://bugs.launchpad.net/tripleo/+bug/177143514:00
openstackLaunchpad bug 1771435 in tripleo "scenario001/002 failing on autoscaling with urllib3.exceptions.SSLError: [SSL: UNKNOWN_PROTOCOL] unknown protocol (_ssl.c:579)" [Critical,Fix released] - Assigned to Alex Schultz (alex-schultz)14:00
rlandynot copied yet14:00
quiquellIt's running before installing oooq ?14:00
rlandyNo such file or directory: '/home/zuul/workspace/.quickstart/config/general_config/featureset050.yml'14:00
rlandywe need to pull it from a diff place14:00
rlandywe can fix that14:01
pandaoooh14:01
pandawait14:02
quiquellAhhh that's easy :-) we are still not horribly broken14:02
pandarlandy: hhmm14:02
pandarlandy: I think we can take it from $TRIPLEO_ROOT/tripleo-quickstart/config/general_config/featureset050.yml14:03
pandarlandy: yep that's what the other release approach is doing14:04
pandarlandy: line 137 of toci_gate for example14:04
rlandypanda: yep - that is the right thing14:05
rlandycan we make the change on your patch14:05
rlandyand I will rebase mine?14:05
rlandy{I guess that is why I had a testing patch originally to iterate on}14:05
pandarlandy: yeah, change your patch, it's quicker14:06
rlandyfine - will do14:06
pandarlandy: I expect this will not be the last iteration14:06
rlandyyeah - me neither14:07
rlandyhmm - here is  the original def: FEATURESET_FILE="$LWD/config/general_config/$JOB_TYPE_PART.yml"14:09
pandarlandy: that works because this part is just preparing the argument for quickstart14:11
*** ykarel is now known as ykarel|away14:12
rlandyright I just need to switch out the dir14:12
pandayeah, it would be TRIPLEO_ROOT/tripleo-quickstart/config/general_config/$(basename $FEATURESET_FILE)14:12
myoung|ruckquiquell: looking at your check fail...do you have a sec to BJ?  want to confirm something with ya and it'll be faster14:12
myoung|ruck...than spamming here14:12
hubbotFAILING CHECK JOBS on stable/ocata: gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-ocata, tripleo-ci-centos-7-undercloud-upgrades @ https://review.openstack.org/564291, master: tripleo-ci-centos-7-3nodes-multinode, gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-queens-branch, gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master, gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001 (1 more message)14:13
quiquellmyoung|ruck: 200~https://bluejeans.com/789106523214:13
*** dtrainor has joined #oooq14:15
*** ykarel|away has quit IRC14:16
weshaymyoung|ruck, there is a skip list patch for the telemetry bugs?14:19
weshaythere should not be14:19
weshayre: "when this get's promoted we can (try to) revert skiplist patch..14:20
weshay"14:20
weshaysshnaidm|rover, should we just push this through? https://review.openstack.org/#/c/568701/14:23
weshayanyone else try this patch to resolve the libvirt repro?14:24
myoung|ruckweshay: i'm reconstructing what happened at 3am :)14:24
weshaymyoung|ruck, was a skip list patch posted?14:24
weshayI missed that14:24
myoung|ruckit was...was going thru LP updates this morning and flagged it...sec fetching link14:24
weshaymyoung|ruck, have a link?14:25
myoung|ruckweshay: https://bugs.launchpad.net/tripleo/+bug/177150814:25
openstackLaunchpad bug 1771508 in tripleo "Telemetry tests fail in scenario-001 and 002 jobs" [Critical,Triaged] - Assigned to Pradeep Kilambi (pkilambi)14:25
myoung|ruckweshay: https://review.openstack.org/#/c/568766/14:26
myoung|ruckbrb14:26
weshayhttps://review.openstack.org/#/c/568860/14:26
weshaysshnaidm|rover, Alex did not want to see a skip list for this14:27
weshayrlandy, trown need a core https://review.openstack.org/#/c/568860/14:28
sshnaidm|roverweshay, why not?14:29
*** akrivoka has joined #oooq14:29
sshnaidm|roverweshay, is it important enough to block gates?14:29
weshaysshnaidm|rover, go ask him.. but imho it's his call14:29
weshaysshnaidm|rover, pretty much14:29
sshnaidm|roverweshay, well14:29
weshaysshnaidm|rover, I asked him that upfront yesterday14:30
weshaysshnaidm|rover, should be fixed w/ the revert we put in14:30
myoung|ruckyeah...i had thought in this case we don't want to promote with this14:30
* weshay gets14:30
sshnaidm|roverweshay, but if you enable these tests, your patch will not pass checks :D14:30
myoung|ruckhttps://review.openstack.org/#/c/568700/14:31
weshaysshnaidm|rover, if that is the case.. he can clear the gates for the patch that fixes it vs.. skips it14:31
myoung|ruckhttps://review.openstack.org/#/c/568699/14:31
* myoung|ruck jumps on CIX call14:31
sshnaidm|roverweshay, why to do this so complex?14:31
sshnaidm|roverweshay, merging patch with fix, then reverting skip14:32
weshaysshnaidm|rover, https://review.openstack.org/#/c/568699/14:32
weshayafaik should have fixed it14:32
sshnaidm|roverweshay, oh, so it's fixed, then we can revert of course14:33
weshayat least the check jobs are catching the revert of the revert https://review.openstack.org/#/c/568736/14:33
weshaysshnaidm|rover, thank you14:33
weshaysshnaidm|rover, workflow that badboy14:33
weshaysshnaidm|rover, ^14:33
sshnaidm|roverweshay, let's see jobs14:34
quiquelltrown: Going to add PyYAML to "[WIP] Add CLI argument parser and YAML file parser"14:34
quiquellok ?14:34
quiquelltox-py27 is not passing14:34
trownquiquell: sure... all of the stuff marked WIP is very WIP...14:36
quiquelltrown: Cool pushin ps14:36
pandarlandy: mind if I push the change to your review ?14:38
rlandygot4it14:39
rlandywe got somewhere now ...14:40
rlandyhttp://logs.openstack.org/60/567060/20/check/tripleo-ci-centos-7-undercloud-upgrades/34d66ab/logs/emit_releases_file.txt.gz14:40
quiquelloutput release looks good for the releases14:42
rlandyyeah- we are getting there14:42
pandarlandy: you're still testing on 568717 ?14:42
rlandypanda: no - I will abandon that14:43
*** ykarel|away has joined #oooq14:43
pandarlandy: ah, ok14:43
rlandywe can work directly on https://review.openstack.org/#/c/567060/14:43
rlandypanda: ^^ you can make you changes there14:44
rlandyyou're fixing the current error?14:44
pandarlandy: ok, I didn't see new PS so I wondered if I should have pushed something, but we are good.14:44
rlandypanda: there is still an error14:44
rlandyare you fixing that no or should I?14:44
rlandyIOError: [Errno 2] No such file or directory: '/home/zuul/workspace/.quickstart/releases.sh'14:45
quiquellsome directory missing14:45
pandayeah the local working14:45
rlandywe can out in tripleo14:45
quiquellIt's not already created at this point ?14:45
rlandyor logs14:45
quiquellYep14:45
pandabecause it's created only after we launch quickstart14:45
rlandyit's fine14:46
rlandywe can do the same as we have with the other files14:46
pandarlandy: directly on logs dir then ?14:46
*** holser__ has quit IRC14:46
rlandypanda: ack - why not?14:46
quiquellrlandy, panda: tox zuul job has detect a defect :-) they are already giving us somet hing14:46
rlandyquiquell?14:46
rlandyoh - nvm, I understand14:47
rlandyhappy it helped14:47
*** holser__ has joined #oooq14:47
pandarlandy: you can put a patch, I would take longer me to do it at this point14:47
rlandypanda: sure - fixing14:47
*** ykarel|away is now known as ykarel14:49
rlandyand one more time14:49
*** links has quit IRC14:58
*** jfrancoa has quit IRC15:05
*** jfrancoa has joined #oooq15:08
rlandycloser ... one more patch15:09
pandarlandy: ?15:09
rlandypanda: I had to remove the copy line since we are placing the output in the logs dir anyways15:13
rlandypatch 22 testing now15:13
*** skramaja has quit IRC15:28
*** quiquell is now known as quiquell|off15:32
rlandyok people ... here we are ...15:41
pandarlandy: are we happy ?15:42
rlandyI can see queens passed to undercloud install15:42
rlandyand master elsewhere15:42
rlandylooking promising15:42
rlandyhttp://zuul.openstack.org/stream.html?uuid=3ccc18c8f9bc4655bb4f11cfb08feaf3&logfile=console.log15:43
rlandywe're in rpogress15:43
rlandyprogress15:43
rlandyI am updating the tox reviews in the mean time15:43
*** bogdando has quit IRC15:53
*** ykarel is now known as ykarel|away15:54
*** tcw has quit IRC15:55
weshayrlandy, trown, panda, quiquell|off sshnaidm|rover myoung|ruck arxcruz and finaly hrybacki  http://lists.openstack.org/pipermail/openstack-dev/2018-May/130560.html16:01
weshaynice work hrybacki and trown :)16:01
weshayplease comment in the thread if you have one16:02
*** udesale_ has quit IRC16:02
trownpretty cool16:02
*** marios has quit IRC16:04
arxcruzcool :)16:04
myoung|ruck\o/16:05
pandaToday openstack, tomorrow the linux foundation.16:06
*** saneax is now known as saneax-_-|AFK16:06
rascamyoung|ruck, can you give me a quick merge here: https://code.engineering.redhat.com/gerrit/#/c/138964/ this should fix OSP11 ospphase0 job16:08
* myoung|ruck looks16:08
myoung|ruckrasca: yay point releases.  merged.16:09
rascamyoung|ruck, thanks man, it is related to https://bugzilla.redhat.com/show_bug.cgi?id=156912216:11
openstackbugzilla.redhat.com bug 1569122 in instack-undercloud "Undercloud installation fails with "Execution of '/bin/getcert list' returned 1: Error org.freedesktop.DBus.Error.TimedOut"" [High,New] - Assigned to jslagle16:11
hubbotFAILING CHECK JOBS on stable/ocata: gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-ocata, tripleo-ci-centos-7-undercloud-upgrades @ https://review.openstack.org/564291, master: tripleo-ci-centos-7-3nodes-multinode, gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-queens-branch, gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master, gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001 (1 more message)16:13
*** Qubasa has joined #oooq16:18
*** Qubasa has quit IRC16:23
weshayrasca, myoung|ruck that reminds me.. did we change osp jobs to use the phase1 osp release yet?16:23
myoung|ruckweshay: no, but that's on deck for this sprint16:24
myoung|ruckI'll post a patch this afternoon...it's a quickie16:24
rlandyok we have some results ...16:24
rlandypanda: ^^16:24
rlandyhttp://logs.openstack.org/60/567060/22/check/tripleo-ci-centos-7-containerized-undercloud-upgrades/3ccc18c/logs/playbook_executions.txt.gz16:25
rlandyyou can see the mix of queens and master16:25
rlandyquiquell|off: ^^ trown:16:25
rlandywe are getting there16:25
rlandyhttp://logs.openstack.org/60/567060/22/check/tripleo-ci-centos-7-containerized-undercloud-upgrades/3ccc18c/logs/emit_releases_file.txt.gz16:26
pandammhh16:26
*** jfrancoa has quit IRC16:26
*** florianf has quit IRC16:26
rlandyhttp://logs.openstack.org/60/567060/22/check/tripleo-ci-centos-7-containerized-undercloud-upgrades/3ccc18c/logs/releases.sh16:27
rlandywe got our three files16:27
pandaah yeah this is undercloud only16:27
rlandycontainerized-undercloud-upgrades16:27
rlandyexport UNDERCLOUD_INSTALL_RELEASE="queens"16:27
rlandyexport UNDERCLOUD_INSTALL_HASH="current-tripleo"16:27
rlandyexport UNDERCLOUD_TARGET_RELEASE="master"16:27
rlandyexport UNDERCLOUD_TARGET_HASH="current-tripleo"16:27
rlandywe have a failed upgrade16:28
*** jfrancoa has joined #oooq16:28
rlandyfailed here as well ... http://logs.openstack.org/50/567550/8/check/tripleo-ci-centos-7-containerized-undercloud-upgrades/922512b/logs/undercloud/home/zuul/undercloud_upgrade.log.txt.gz16:30
rlandymaybe were good16:30
rlandyweshay: ^^ we have some initial test results16:31
*** holser__ has quit IRC16:31
rlandywill check other update/upgrage jobs16:32
rlandyupgrade16:32
pandarlandy: I think It's a starting point. It's testing time anyway16:32
rlandypanda: yep - waiting for the other update/upgrades tests to log stuff16:32
rlandythen we can analyze16:32
rlandybut we have a testing case now16:33
pandaperfect16:33
weshayrlandy, you refering to Emilien's bug?16:35
*** kopecmartin has quit IRC16:36
*** ccamacho has quit IRC16:37
*** panda is now known as panda|off16:37
panda|offrlandy: If you debug, write some status in one of the validation cards, I'll follow up tomorrow16:38
rlandypanda|off: sure16:38
rlandyweshay: we're getting there16:38
rlandywe will have to test with a change to queen/master16:38
rlandyqueens16:39
rlandynext up16:39
weshayrlandy, ah thanks16:39
rlandybut we have output of diff release files being passed16:39
hrybackiweshay: I like the idea of dedicating time at PTG for the docgen stuff. More of us will be there and we'll have more advance time to prep :)16:40
weshayhrybacki, thanks again for doing it16:41
*** tcw has joined #oooq16:42
weshaytrown, rlandy this blocks the libvirt repro https://review.openstack.org/#/c/568701/16:43
weshaywell.. that fixes the blocker on the libvirt repo16:43
weshayI should say16:43
myoung|ruckweshay, hrybacki: if we have a time/place to discuss to docgen topics, how we're doing it today, what opportunities for improvement we've got, trade-offs, etc, I'm interested.  I like what we have now, but have some ideas on how to improve it.16:43
weshaymyoung|ruck, it's going to happen at the summit16:43
weshayhrybacki, will report back :)16:43
myoung|ruckhrybacki: (not today) but if you have time thursday or friday for a quick chat, happy to communicate16:44
hrybackiweshay: It sounds like (from the email thread) that we don't enough folks attending summit ?16:44
weshayno comment16:44
rlandyweshay: https://review.openstack.org/#/c/568701/ has tw +216:45
rlandys already16:45
trownweshay: ya was going to say the same16:45
* myoung|ruck does not have the summit flag in his status bitfield set16:45
weshayrlandy, trown ya.. I just want to make sure you both are aware of how it broke.. just for your own info16:45
rlandyok16:45
trownweshay: looks mergeable... unless there is some reason to not +W things right now16:45
weshaymyoung|ruck, really?16:45
trownoh ok16:45
weshaymyoung|ruck, austin?16:45
myoung|ruckweshay: I was just saying that I'm not attending summit this year in a dorky/geeky way16:46
hrybackiweshay: was that no comment for me? :P too in-tandem convos here16:46
*** gkadam has quit IRC16:46
myoung|ruckhrybacki: the implication being, if you want a few ideas for docgen futures prior to summit...happy to chat16:46
weshayhrybacki, re: summit attendence16:46
hrybackiack16:46
myoung|ruckweshay: not making statements about have i been, or should i be there this year :)16:47
weshaytrown, rlandy I don't know how.. but it would really neat if we could figure out a way to trigger that workflow in ci.centos16:47
weshaymyoung|ruck, guess it's hard to figure out what you are saying :)16:47
trownweshay: those machines are not powerful enough16:47
myoung|ruckweshay, hrybacki: that's fair.  my bad.  is clear now?16:48
weshaytrown, we could take it through the undercloud16:48
weshaythat would validate the first setup part works..16:48
trownweshay: ya that would probably work... maybe just use the undercloud-only featureset16:49
weshaytrown, the rest is gated via the upstream16:49
weshayya16:49
weshaybut we'd have to do the whole bs.. of scp from the slave to the cico node16:49
weshayand kicking it remotely16:49
weshayI think16:49
weshaymyoung|ruck, my thought would be to respond to email first off16:51
*** links has joined #oooq16:54
*** dtantsur is now known as dtantsur|afk16:58
*** ykarel|away has quit IRC17:00
rascaweshay, by the way, the 7.5 thing was related just to osp10/11 baremetal deployments, the other (12 and 13) are already using 7.517:04
weshayrasca, not sure what you are referring to..17:05
rascaweshay, no worries, we're ok, it's enough :)17:05
weshayrasca, sorry buddy.. been busier than a one legged man in a butt kicking contest17:06
rascaweshay, ahaha17:06
weshaypanda|off, have you scheduled some time to cover your "how do dev's install" finding?17:11
*** trown is now known as trown|lunch17:11
*** links has quit IRC17:16
rascamyoung|ruck, undercloud installation for osp11 fixed: https://rhos-dev-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/oooq-rhos-11-puddle-bmu-hac01-lab-float_nic_with_vlans/37/console if it pass the rest we'll be green again, with just a one char patch :D17:20
myoung|ruckrasca: that's great17:22
*** amoralej is now known as amoralej|off17:25
*** tesseract has quit IRC17:27
*** sshnaidm|rover is now known as sshnaidm|off17:29
*** jfrancoa has quit IRC17:41
*** jfrancoa has joined #oooq17:42
weshayrasca, you have anything that is running 13?17:53
*** jfrancoa has quit IRC18:12
*** akrivoka has quit IRC18:12
hubbotFAILING CHECK JOBS on stable/ocata: gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-ocata, tripleo-ci-centos-7-undercloud-upgrades @ https://review.openstack.org/564291, master: tripleo-ci-centos-7-3nodes-multinode, gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-queens-branch, gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master, gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001 (1 more message)18:13
rlandyweshay: sorry, I know you are super busy but do you happen to have that bug you linked yesterday? I closed the window and I'd like to test the scripts with the exact change18:20
weshayhttps://bugs.launchpad.net/tripleo/+bug/176899718:22
openstackLaunchpad bug 1768997 in tripleo "containerized-undercloud-upgrades: dlrn tasks don't run" [Critical,Triaged]18:22
rlandyty18:22
*** atoth has quit IRC18:42
*** trown|lunch is now known as trown18:44
weshaymyoung|ruck, sshnaidm|off would be really nice if we could debug and resolv this for the weirdo guys, like yatin as they do for us in tripleo https://trello.com/c/DLL665U8/592-rdophase1masterpackstackscenario001-ansible-playbook-failing-in-weirdo-master-scenario0119:12
myoung|ruckweshay: will have a look19:13
myoung|ruckweshay: https://rhos-dev-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/oooq-rhos-13-puddle-bmu-had00-lab-float_nic_with_vlans/19:13
myoung|ruckwe still have some issues with virt jobs for 13 however..it's been starved thus far this sprint...but would like to have them running for gates19:14
myoung|ruckhttps://rhos-dev-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/osp-director-promote-13-puddle/19:14
weshaymyoung|ruck, well this is certainly good news19:17
weshaymyoung|ruck, this is rasca's job?19:17
myoung|ruckyes the BMU job19:18
weshaymyoung|ruck, ok.. so ovb should work19:18
myoung|ruckit needed 7.5 base19:18
weshayrasca, I don't see where your validation is logged19:18
myoung|ruckweshay: we also have 13 jobs for tq/tqe gates defined19:19
myoung|ruckhttps://rhos-dev-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/tripleo-ci-gates/job/tq-gate-rhos-13-ci-rhos-ovb-minimal-pacemaker-public-bond/19:19
myoung|ruckhttps://rhos-dev-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/tripleo-ci-gates/job/tqe-gate-rhos-13-ci-rhos-ovb-minimal-pacemaker-public-bond/19:19
myoung|ruckcurrent status is in sprint 13 etherpad19:19
myoung|ruckuntil container jobs blew up this afternoon my plan was to give them some love...19:20
myoung|ruckin theory FF the stable branches and we're done19:20
myoung|ruckor at least we'll get past what was the blocking issues (potentially) with hostnames19:20
*** tosky has quit IRC19:20
myoung|rucklooking at this packstack fail...will see if anything makes sense19:21
rlandyweshay: ping re: https://bugs.launchpad.net/tripleo/+bug/1768997 when you have a moment19:21
openstackLaunchpad bug 1768997 in tripleo "containerized-undercloud-upgrades: dlrn tasks don't run" [Critical,Triaged]19:21
rlandythe dlrn build still does not trigger19:21
rlandybecause the undercloud is installed with queens - still19:22
rlandythe change is on master19:22
rlandymaster only gets passed later19:22
rlandywhich seems correct19:24
*** tosky has joined #oooq19:24
weshayrlandy, right.. the upgrade playbook calls build-test so it should run then19:28
* weshay gets19:28
rlandyweshay: ok - well we will see when that runs19:28
rlandythe way the bug is written it looked like the expectation was for build to run on undercloud install19:29
weshayrlandy, hrm.. or maybe I'm confusing this w/ upgrade and we need a patch19:29
weshaynote the diff19:29
weshayhttps://github.com/openstack/tripleo-quickstart-extras/blob/master/playbooks/multinode-overcloud-update.yml19:29
weshayhttps://github.com/openstack/tripleo-quickstart-extras/blob/master/playbooks/multinode-overcloud-upgrade.yml#L1419:29
weshayI thought I saw it called and skipped though in the job I debugged for him19:30
weshayoh.. duh19:30
weshayit is an upgrade19:30
weshayI'm stupid19:30
weshayso ya19:30
weshayif we pass master and the upgrade role doesn't munge it.. I think it should just work19:30
weshaybased on the work you guys are doing19:30
rlandybasically the line linked is the undercloud install19:31
rlandythat will never run on a master change19:31
rlandyfor an undercloud upgrade/update19:31
rlandywe will have to make sure that role is called19:31
rlandythe bug is logged against undercloud upgrades19:32
weshayanyone :) https://review.openstack.org/#/c/568733/19:33
weshayrlandy, right.. but it calls the upgrade role19:33
weshayrlandy, right?19:33
weshayrlandy, OH I have a patch on       when: hostvars['undercloud']['compressed_gating_repo'] is defined and mixed_upgrade|default(false)|bool19:34
rlandyhttps://github.com/openstack/tripleo-quickstart-extras/blob/master/playbooks/multinode-undercloud-upgrade.yml#L1219:34
rlandythe role is called19:34
weshayah perfect19:35
weshaywait19:35
rlandylet's see what it does in the uundercloud-upgrade playbook19:35
weshayrepo-setup does not get it done19:35
rlandyyou can see what release files we pass19:35
weshaydo you have a patch?19:35
weshayrlandy, we need - role: install-built-repo19:35
rlandyrepo -setup vs dlrn-build19:35
weshaysay wha?19:35
rlandyI'm trying to un-confuse my brain about what role call what19:36
weshaycome on.. you got this19:36
rlandy- include: get-dlrn-hash.yml19:37
weshaythat's not it19:37
weshaywant to chat?19:37
rlandyright - so we are missing a call to the build role19:37
weshay\o/19:37
weshayalthough let me look at the job19:37
rlandyhttp://zuul.openstack.org/stream.html?uuid=d030255b70574b23bf65652a1d352ae7&logfile=console.log runs now19:38
rlandywhen the logs get printed, you will see the diff releases passed to diff playbooks19:38
weshaythat's good19:39
rlandythe jobs are busy doing undercloud install now19:39
rlandybut top to bottom we are still missing the call to the build role19:39
rlandyI'll put up another patch and test that19:40
rlandyinstall-built-repo is nowhere in upgrades/updates role19:41
rlandybuild-test-packages19:42
weshayrlandy, wes hayutin proposed openstack/tripleo-quickstart-extras master: execute the build / install of zuul changes in undercloud-upgrade  https://review.openstack.org/56894619:47
rlandyweshay: ok - adding deps to the test commit19:50
*** holser__ has joined #oooq19:53
rlandywatching tests19:53
rlandyhttp://zuul.openstack.org/stream.html?uuid=db9e4d3e13324683a9c531dd3041788f&logfile=console.log19:57
*** dougbtv_ has joined #oooq20:11
hubbotFAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-containers-multinode, tripleo-ci-centos-7-3nodes-multinode, gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-queens, gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-queens @ https://review.openstack.org/567224, stable/ocata: gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-ocata, tripleo-ci-centos-7-undercloud-upgrades @  (2 more messages)20:13
*** dougbtv_ has quit IRC20:17
*** holser__ has quit IRC20:51
*** trown is now known as trown|outtypewww21:05
*** Goneri has quit IRC21:14
rlandyweshay: we have the first results21:23
rlandyhttp://logs.openstack.org/31/568931/2/check/tripleo-ci-centos-7-containerized-undercloud-upgrades/db9e4d3/21:24
rlandyhttp://logs.openstack.org/31/568931/2/check/tripleo-ci-centos-7-containerized-undercloud-upgrades/db9e4d3/logs/playbook_executions.txt.gz21:24
rlandy^^ what got called with what release21:24
rlandyplaybooks/multinode-undercloud.yml gets called with master21:25
rlandybuild-test-packages failure21:26
weshay2018-05-16 20:37:55.770600 | primary | TASK [build-test-packages : Clone and build the specific DLRN projects] ********21:26
weshaywas skipped21:26
weshay2018-05-16 20:42:26.571288 | primary | TASK [build-test-packages : Compress the repo] *********************************21:26
weshayso that compressed_repo variable needs adjustment21:27
weshayit should not have skipped though21:29
weshayhttp://logs.openstack.org/31/568931/2/check/tripleo-ci-centos-7-containerized-undercloud-upgrades/db9e4d3/job-output.txt.gz#_2018-05-16_20_37_55_77060021:29
weshayrlandy, right?21:29
weshay- name: Clone and build the specific DLRN projects21:29
weshay  include: dlrn-build.yml artg_change={{ item }}21:29
weshay  with_items: '{{ artg_change_list|default([]) }}'21:29
weshay  when:21:29
weshay    - item.project not in artg_skipped_projects21:29
weshay    - release in item.branch21:29
weshay "artcl_txt_rename": true,21:30
weshay    "artg_change_list": [21:30
weshay        {21:30
weshay            "branch": "master",21:30
weshay            "host": "https://git.openstack.org",21:30
weshay            "project": "openstack/python-tripleoclient",21:30
weshay            "refspec": "refs/changes/31/568931/2"21:30
weshay        },21:30
weshay        {21:30
weshay            "branch": "master",21:30
weshay            "host": "https://git.openstack.org",21:30
weshay            "project": "openstack/tripleo-quickstart-extras",21:30
weshay            "refspec": "refs/changes/46/568946/1"21:31
weshay        },21:31
weshay        {21:31
weshay            "branch": "master",21:31
weshay            "host": "https://git.openstack.org",21:31
weshay            "project": "openstack-infra/tripleo-ci",21:31
weshay            "refspec": "refs/changes/60/567060/22"21:31
weshay        }21:31
weshayrlandy, ?21:37
*** agopi has joined #oooq21:38
rlandywell it errored21:39
rlandy"stderr": "tar (child): /home/stack/gating_repo.tar.gz: Cannot open: No such file or directory\ntar (child): Error is not recoverable: exiting now\ntar:21:39
*** d0ugal_ has joined #oooq21:40
*** d0ugal has quit IRC21:41
rlandyso yeah - it should not have skipped those tasks21:43
weshayya.. they never got cloned or built21:45
hubbotFAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-containers-multinode, tripleo-ci-centos-7-3nodes-multinode, gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-queens, gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-queens @ https://review.openstack.org/567224, stable/ocata: gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-ocata, tripleo-ci-centos-7-undercloud-upgrades @  (2 more messages)22:13
*** pliu has quit IRC22:14
*** rasca has quit IRC22:16
*** rnoriega has quit IRC22:16
*** jjoyce has quit IRC22:16
*** lhinds has quit IRC22:16
*** faceman has quit IRC22:16
*** jschlueter has quit IRC22:17
*** myoung|ruck has quit IRC22:17
*** weshay has quit IRC22:17
*** weshay has joined #oooq22:22
*** weshay has quit IRC22:26
rlandyhmmm ...22:28
*** rlandy is now known as rlandy|bbl22:28
*** jschlueter has joined #oooq22:34
*** weshay has joined #oooq22:34
*** faceman has joined #oooq22:35
*** myoung has joined #oooq22:35
*** pliu has joined #oooq22:35
*** lhinds has joined #oooq22:35
*** rasca has joined #oooq22:35
*** jjoyce has joined #oooq22:36
*** agopi has quit IRC22:37
*** rnoriega has joined #oooq22:37
*** agopi has joined #oooq22:38
*** d0ugal__ has joined #oooq22:40
*** d0ugal_ has quit IRC22:41
*** dougbtv_ has joined #oooq22:47
*** dougbtv_ has quit IRC23:00
*** lhinds has quit IRC23:16
*** pliu has quit IRC23:16
*** rnoriega has quit IRC23:16
*** weshay has quit IRC23:17
*** rasca has quit IRC23:17
*** myoung has quit IRC23:17
*** jschlueter has quit IRC23:18
*** faceman has quit IRC23:18
*** jjoyce has quit IRC23:18
*** rnoriega has joined #oooq23:21
*** weshay has joined #oooq23:21
*** rasca has joined #oooq23:21
*** pliu has joined #oooq23:21
*** myoung has joined #oooq23:23
*** faceman has joined #oooq23:23
*** jschlueter has joined #oooq23:23
*** lhinds has joined #oooq23:23
*** jjoyce has joined #oooq23:24
*** tosky has quit IRC23:34
*** myoung is now known as myoung|ruck23:56

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!