Monday, 2018-06-11

*** agopi has quit IRC00:01
hubbotFAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/56722400:37
*** agopi has joined #oooq01:45
*** agopi has quit IRC01:54
hubbotFAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/56722402:38
*** pgadiya has joined #oooq03:35
*** pgadiya has quit IRC03:35
*** yolanda has joined #oooq03:44
*** yolanda_ has quit IRC03:46
*** skramaja has joined #oooq04:08
*** rfolco has joined #oooq04:13
*** links has joined #oooq04:14
*** jbadiapa_ has joined #oooq04:20
hubbotFAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/56722404:38
*** jaganathan has joined #oooq04:51
*** jaganathan has quit IRC04:59
*** ykarel has joined #oooq05:05
*** udesale has joined #oooq05:09
*** saneax has joined #oooq05:26
*** quiquell has joined #oooq05:28
quiquellsshnaidm: Good morning05:36
*** udesale_ has joined #oooq05:40
*** udesale has quit IRC05:41
*** marios has quit IRC05:46
*** marios has joined #oooq05:47
*** udesale__ has joined #oooq06:01
*** udesale_ has quit IRC06:03
hubbotFAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/56722406:38
*** bogdando has joined #oooq06:43
*** ccamacho has joined #oooq06:57
*** ccamacho has quit IRC06:58
*** ccamacho has joined #oooq06:59
*** holser__ has joined #oooq07:08
*** kopecmartin has joined #oooq07:09
*** matbu has quit IRC07:09
*** jaosorior has joined #oooq07:10
*** florianf has joined #oooq07:14
*** matbu has joined #oooq07:14
*** jtomasek has joined #oooq07:14
bogdandoo/ folks. the reproduce quickstart script is missing in rdo ovb jobs. A known bug?07:16
bogdandordo ci07:16
*** panda|off is now known as panda07:17
quiquellbogdando: panda Good mornig07:19
quiquellbogdando: Do you have the logs ?07:19
bogdandoquiquell: https://review.openstack.org/572533 see the last 3-4 rdo ci runs in comments07:19
bogdandothen expand to the job logs link in jenkins logs07:20
bogdandothe repro quickstart sh is expected to be there07:20
ykarelbogdando, there is issue with collect logs:- https://logs.rdoproject.org/33/572533/5/openstack-check/gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master/Z659cc00387184293b4d2ee28ad68cb8c/quickstart_collect_logs.txt.gz07:25
*** jaganathan has joined #oooq07:25
ykarel'tempest_format' is undefined"07:26
quiquellbogdando, ykarel: Yep, collect logs is failing and the reproducer is not dumped07:26
bogdandoykarel: ack, thanks. so it is a known issue07:26
ykarelno idea which patch caused it and is there already a fix for it. Good to create a bug for it07:27
bogdandoykarel: the fix is make collect logs invokations tolerate failures prolly07:27
ykarelany way there are issues in rdo-cloud currently, so reproducer script will not work07:28
ykarelbogdando, ack, can you create bug, so someone can look at it07:28
bogdandoprolly like I do here https://review.openstack.org/57253307:28
*** amoralej|off is now known as amoralej07:28
bogdandoykarel: ok, and I'll push a patch07:28
ykarelbogdando, Thanks07:29
bogdandohttps://review.openstack.org/#/c/574152/07:37
sshnaidmbogdando, not sure, what is purpose of ^^ ?07:38
bogdandosshnaidm: there is a bug in the link07:39
sshnaidmbogdando, it doesn't explain though :)07:39
ykarelbogdando, doesn't seem it's fixing the issue:- 'tempest_format' is undefined, just ignoring07:40
bogdandosshnaidm: "10:26:38 AM GMT+3 - quiquell: bogdando, ykarel: Yep, collect logs is failing and the reproducer is not dumped"07:40
*** tosky has joined #oooq07:40
sshnaidmbogdando, yeah, I know, I'm asking how this patch is related07:40
bogdandoykarel: ok, please feel free to take it over :)07:40
ykarelbogdando, ok will check later if no one else takes it07:42
sshnaidmI'll fix07:45
ykarelsshnaidm, Thanks07:45
*** ykarel is now known as ykarel|afk07:46
*** links has quit IRC07:50
ykarel|afkso it's caused by:- https://review.openstack.org/#/c/566527/07:51
ykarel|afkupdated bug description07:52
quiquellpanda: You there ?07:55
pandaquiquell: more or less07:56
quiquellpanda: Have a zuul question with Depends-On, it's not doing what I was expecting07:57
quiquellpanda: https://review.openstack.org/#/c/571435/07:57
quiquellpanda: Job tripleo-ci-centos-7-containerized-undercloud-upgrades07:57
pandaquiquell: did you try to ask gozer first ? He's usualy nicer07:58
quiquellpanda: The Depends-On has trown's change as parent, but it's not working07:58
quiquellpanda: Who is gozer ?07:58
pandaquiquell: just kidding07:59
pandainternet here today is even slower than the other days. 2 seconds constant lag in IRC08:00
quiquellpanda: :-) excus me for my robotic sense of humor08:00
quiquellpanda: np, just help me debug the review08:00
quiquellpanda: Don't understand the zuul behaviour, maybe is because the change is in queens08:00
quiquellpanda: and zuul ignores the other changes ?08:00
pandaquiquell: that's probably it, I can be sure only when gertty fetches the change08:02
panda2 minutes to fetch a change ....08:02
pandaI'll try to reboot the router -_-08:02
quiquellpanda: Going to debug the logs08:04
*** skramaja_ has joined #oooq08:22
*** skramaja has quit IRC08:25
sshnaidmykarel|afk, bogdando https://review.openstack.org/#/c/574164/08:27
*** d0ugal has joined #oooq08:28
ykarel|afksshnaidm, containerized tempest available in release before master?08:29
arxcruz|ruckykarel|afk: dude, you're da man!08:30
sshnaidmykarel|afk, not sure08:30
bogdandosshnaidm: thank you!08:30
ykarel|afksshnaidm, if no setting default to containers here is wrong08:30
sshnaidmykarel|afk, why?08:30
ykarel|afksshnaidm, tempest_container would be included always whenever tempest_format is unset08:31
sshnaidmykarel|afk, ?? not sure I understand08:31
ykarel|afk"{% if run_tempest|bool and tempest_format|default('containers') == 'containers' -%}tempest_container{%- endif -%}"08:31
sshnaidmykarel|afk, it's a default for docs generation only08:31
ykarel|afkso if tempest_format is unset, ^^ condition would be true08:32
sshnaidmykarel|afk, I see your point now..08:33
ykarel|afkam not sure how this will go with docs08:33
ykarel|afkif we include tempest_format with older releases08:33
sshnaidmykarel|afk, I'll change the default to default in role08:34
pandaquiquell: It should worlk08:34
pandaquiquell: the devstack gate script tries to clone master if it doesn't find stable release for the repo08:34
quiquellpanda: Look like is not taking trown's change08:35
quiquellpanda: The --upgrade-from option is unknown08:35
sshnaidmchandankumar, arxcruz|ruck, any ideas why tempest_format is undefined in logs collection run?08:36
arxcruz|ruckthe problem is that tempest_format is in featureset001 to check and create the docs properly08:36
arxcruz|rucksshnaidm: however, it seems collect_logs doesn't call the validate-tempest, where this tempest_format is defined08:37
arxcruz|rucki'll add a defaults('packages')08:37
arxcruz|ruckdefault('packages')08:37
sshnaidmarxcruz|ruck, added already08:37
sshnaidmarxcruz|ruck, it shouldn't call validate-tempest08:38
sshnaidmarxcruz|ruck, it runs as separate playbook, but seems like it doesn't have this var cached08:38
hubbotFAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/56722408:38
arxcruz|rucksshnaidm: well, yeah, but when i wrote the patch i didn't knew it would make problems with the collect-logs08:38
arxcruz|rucksorry, i'm fixing08:38
pandaquiquell: where's john's change in the depends on ?08:38
quiquellpanda: It's the parent of the Depends-On08:39
ykarel|afksshnaidm, yes that would be better infact08:39
sshnaidmarxcruz|ruck, I have a patch already  https://review.openstack.org/574164 , but we need to investigate why tempest_format is not cached08:39
quiquellpanda: Look http://logs.openstack.org/35/571435/7/check/tripleo-ci-centos-7-containerized-undercloud-upgrades/0277101/job-output.txt.gz#_2018-06-11_06_13_27_32805008:39
*** dtantsur|afk is now known as dtantsur08:40
arxcruz|rucksshnaidm: should it cache? it's a separate ansible-playbook call08:41
pandaquiquell: I see not john's change up in the chain from the change you showed me08:41
quiquellpanda: The parent of the "Depends-On" change no the change itself08:42
pandaquiquell: from https://review.openstack.org/571435 I'm clicking on all the Depends-on I see, but none leads me to john's patch08:43
quiquellpanda: "Related Changes" are the parents08:44
pandaquiquell: ok, so it's not explicit depends on, it's a rebase08:46
quiquellpanda: Yep, but it appears at ZUUL_CHANGES in the logs, but later on is like tripleo-ci is not pointing to it08:52
sshnaidmarxcruz|ruck, maybe it's better to set tempest_format for every featureset file when we run tempest08:54
arxcruz|rucksshnaidm: sure, you're the ansible master :)08:55
* arxcruz|ruck didn't finish his ansible course yet :(08:55
pandaquiquell: found the problem08:55
pandaquiquell: you forgot a \08:55
pandaquiquell: commented the patch08:56
quiquellpanda: Arggg thanks !!!08:57
quiquell:-)08:57
pandaquiquell: yeah, it's like the missing ; in C code or unmatched brackets08:57
quiquellpanda: Ok, fixed, let's recheck08:59
arxcruz|ruckpanda: you know there's also a ; in greek that if you replace it on C code, will make your friends crazy looking for the error :P09:01
arxcruz|ruckhttp://www.fileformat.info/info/unicode/char/037e/index.htm09:01
quiquellarxcruz|ruck: That's sick !!!09:02
pandais this why you don't see many greek programmers around ?09:03
arxcruz|rucklol09:03
arxcruz|ruckno idea09:03
quiquellpanda, arxcruz|ruck: they only do python and the like :-)09:04
arxcruz|ruckwe should ask marios :)09:04
marioso/ arxcruz|ruck hi09:05
mariosarxcruz|ruck: wasnt me09:05
arxcruz|ruckmarios: the reason to not have greek programmers in C around, is because the ; greek question mark ?09:05
mariosoh yeah that is the question mark09:05
arxcruz|ruckmarios: so, instead of ? you use ; ?09:06
mariosarxcruz|ruck: heh i don't know actually man i'm pretty sure theres quite a few greek coders :) but i have absolutely nothing to backup that assertion with09:06
mariosarxcruz|ruck: right09:06
arxcruz|ruckwondering why portuguese didn't adopt that...09:06
arxcruz|ruckwe use ; in portuguese for a pause in a phrase that is more slower than a "," and faster than a "."09:07
arxcruz|ruckfor punctuation09:07
mariosarxcruz|ruck: yeah it is the same in english too09:07
marioslike pause in the sentence09:07
pandaarxcruz|ruck: because it's modern greek ?09:07
arxcruz|ruckdunno, i never study english grammar09:08
arxcruz|ruckmy english came from play world of warcraft09:08
pandaarxcruz|ruck: and I think protuguese exported more than it imported, for example the japanese "arigato"09:08
arxcruz|ruckis that a portuguese word? i don't think so09:09
mariosarxcruz|ruck: and in greek instead that pause is with 'upper dot' literally a full stop but in the middle of the line not on the bottom.09:09
arxcruz|ruckportuguese import a lot, specially from greek09:09
pandaarxcruz|ruck: it's not but it's supposed to come from the protuguese "obrigado"09:09
mariosarxcruz|ruck: https://en.wiktionary.org/wiki/%CE%AC%CE%BD%CF%89_%CE%BA%CE%B1%CE%B9_%CE%BA%CE%AC%CF%84%CF%89_%CF%84%CE%B5%CE%BB%CE%B5%CE%AF%CE%B109:09
arxcruz|rucklike all the "gias" from greek09:09
arxcruz|ruckbiologia, sociologia, pedagogia09:10
arxcruz|ruckpanda: yeah, arigato is obrigado in japanese09:10
pandawe should open #oooq-languages09:10
arxcruz|ruckwe also change a lot over the years09:10
arxcruz|ruckfor example, the word "you" in portuguese when we were colony, was Vossa Mercê09:11
arxcruz|ruckthen became vosmicê09:11
arxcruz|ruckand now is just você09:11
arxcruz|ruckon the internet language is just vc09:11
arxcruz|ruckhehehehe09:11
mariosarxcruz|ruck: and just fyi on that ; in english i'm not lieing ;) https://en.wikipedia.org/wiki/Semicolon09:11
arxcruz|ruckmarios: :)09:12
*** bogdando has quit IRC09:23
*** zoli is now known as zoli|lunch09:28
*** ykarel|afk is now known as ykarel09:30
quiquellsshnaidm: I am trying libvirt reproducer but get stuck at nodepool-setup : Create and enable swap on subnode09:33
sshnaidmquiquell, hmm.. just stuck?09:35
sshnaidmquiquell, could you ssh to node to see what happens? I didn't see it before09:36
quiquellsshnaidm: Let me clean up09:36
quiquellsshnaidm: The livirt reproducer can be used over a RHEL ? I don't have centos on the host09:37
sshnaidmquiquell, well, it *should*, but who knows..09:37
quiquellsshnaidm: Better if I use the beaker and install centos on it09:38
sshnaidmquiquell, yeah, that's definitely better09:39
*** udesale_ has joined #oooq09:45
*** udesale__ has quit IRC09:47
*** dtantsur is now known as dtantsur|bbl09:49
*** udesale_ has quit IRC09:50
quiquellsshnaidm: Can I use beaker to restore this machine ? rdo-ci-fx2-01-s4.v101.rdoci.lab.eng.rdu.redhat.com09:53
quiquellsshnaidm: Or how do we usually do ?09:53
sshnaidmquiquell, yeah, I use beaker to provision it to centos 7.3, and then manually run yum update to update it to 7.509:54
sshnaidmquiquell, because 7.3 is last version when provision works..09:54
quiquellsshnaidm: how do you add the machien to beaker it tell me that it's already there but cannot find it09:55
*** ykarel has quit IRC09:58
*** ykarel has joined #oooq10:01
*** holser__ has quit IRC10:13
*** holser__ has joined #oooq10:13
*** florianf has quit IRC10:23
pandaquiquell: your patch has a merge conflict10:33
quiquellpanda: Has just rebased and rechecked everything10:33
quiquellNow it's rolling10:34
quiquellpanda: Damn... there is a problem at trown's patch arg10:36
quiquellpanda: At least now we are testing it10:36
quiquellpanda: http://logs.openstack.org/35/571435/8/check/tripleo-ci-centos-7-containerized-undercloud-upgrades/0a57be3/job-output.txt.gz#_2018-06-11_09_53_29_22195410:36
pandaquiquell: yeah, one of my comment was about it10:37
*** florianf has joined #oooq10:38
quiquellpanda: bypass with --upgrade-from true will work ?10:38
pandaquiquell: I don't thing so, it must be treated as boolean10:38
hubbotFAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/56722410:38
pandaquiquell: byt the rest of the code10:38
quiquell--upgrade-from True mybe python interpert it as a boolean10:38
*** zoli|lunch is now known as zoli10:39
quiquellpanda: I can do changes at my DNM commit10:39
chandankumararxcruz|ruck: sprint 15 topics https://review.rdoproject.org/etherpad/p/tempest-sprint15 feel free to ++ or --10:40
pandaquiquell: ok, add action="store_true" in the argument parser10:42
pandaquiquell: also remove default=False, it's implicit10:43
quiquellpanda: Added it10:43
quiquellLet's test it10:43
quiquellpanda:  n-1 > n is working fine10:44
pandaquiquell: at least we are not introducing regressions10:45
quiquellpanda: That can allow as merge this without finishing n -> n + 1 ?10:46
pandaquiquell: if we really se there's no other choice10:47
*** ccamacho has quit IRC10:59
*** holser__ has quit IRC10:59
*** ykarel_ has joined #oooq11:00
*** ccamacho has joined #oooq11:01
*** holser__ has joined #oooq11:02
*** ykarel has quit IRC11:02
*** udesale has joined #oooq11:03
*** holser__ has quit IRC11:04
*** rfolco_ has joined #oooq11:07
*** rfolco has quit IRC11:08
*** udesale has quit IRC11:14
*** lucasagomes is now known as lucas-hungry11:15
*** lucas-hungry is now known as lucasagomes11:15
*** atoth has joined #oooq11:17
*** udesale has joined #oooq11:29
sshnaidmfolks, please review https://review.openstack.org/#/c/574164/ - it enabled reproducer on ovb jobs11:30
ykarel_sshnaidm, any idea if https://review.openstack.org/#/c/568701/ ever worked, this looks wrong to me11:31
*** ykarel_ is now known as ykarel11:31
ykarelsshnaidm, i am getting below with reproducer:-11:32
ykarel[zuul@subnode-0 ~]$ cat /etc/yum/vars/contentdir11:32
ykarel{"failed": false, "stat": {"exists": false}, "changed": false}11:32
*** udesale has quit IRC11:35
*** amoralej is now known as amoralej|lunch11:38
panda64 bytes from 8.8.8.8: icmp_seq=132 ttl=56 time=7013 ms11:42
* panda sighs11:42
panda64 bytes from 8.8.8.8: icmp_seq=2 ttl=56 time=33489 ms11:43
panda\o/ new record11:43
quiquellHow may bauds have your modem ? 1600 ?11:46
pandaquiquell: what's a modem ? I have an acoustic coupler here.11:47
quiquellpanda: You are always prepare for the futre11:48
quiquellfuture11:48
quiquellpanda: btw, maybe we can move a little this https://review.rdoproject.org/r/#/c/1360411:48
pandaevery time my daughter cries, the line drops11:48
quiquellpanda: Your daughter mood is a telco matter ?11:49
pandaquiquell: no, it disturbs the acoustic coupling11:51
pandaquiquell: I don't remember anything of that patch11:51
quiquellpanda: Related to this, https://bugs.launchpad.net/tripleo/+bug/176809011:53
openstackLaunchpad bug 1768090 in tripleo "promoter script is not comparing timestamps correctly when folding hashes" [High,In progress] - Assigned to Gabriele Cerami (gcerami)11:53
quiquellpanda: Maybe is no longer important11:53
*** holser__ has joined #oooq11:54
pandaquiquell: it is11:54
pandaquiquell: if it wasn't solved11:54
*** dtrainor has quit IRC11:54
quiquellpanda: So we need to merge that ?11:55
quiquellGoing for lunch, zuul is the bus that never arrives11:56
*** quiquell is now known as quiquell|lunch11:56
*** rfolco_ is now known as rfolco11:59
weshayneed a core https://review.openstack.org/#/c/573457/12:07
weshayhttps://review.openstack.org/#/c/573844/12:07
weshayykarel++12:10
hubbotweshay: ykarel's karma is now 312:10
*** quiquell|lunch is now known as quiquell12:12
weshayykarel, I don't see your escalation on rdo-cloud12:12
ykarelweshay, me too not seeing12:12
weshayykarel, were you hoping that email created it?12:12
ykarelmay be some issue with tooling12:12
*** trown has joined #oooq12:12
ykarelweshay, yes i was hoping email creates a card12:13
quiquellweshay: Do I need to abandon this ? https://review.openstack.org/#/c/574218/12:14
weshayykarel, For Upstream -12:14
weshay[CIX][<rel tag>][LP:<####>][<CIX phase tag>][<component>] <summary line>12:14
*** bogdando has joined #oooq12:14
*** panda is now known as panda|lunch12:17
*** holser__ has quit IRC12:17
ykarelweshay, i sent [rhos-dev][CIX][tripleoci][proa][rdo-cloud, so that's the reason card not created?12:17
*** rlandy has joined #oooq12:17
*** rlandy is now known as rlandy|rover12:18
*** dougbtv has quit IRC12:18
weshayykarel, I don't think it went through ykarel.. I don't see it12:18
*** dougbtv has joined #oooq12:18
weshayquiquell, not sure.. your patch looks fine, I'm happy to go w/ yours /me just trying to get to done12:19
ykarelsshnaidm, proposed https://review.openstack.org/574238 for the issue i mentioned about contentdir12:19
ykarelweshay, check ^^ i saw you also faced same issue with original patch12:19
weshayhttp://post-office.corp.redhat.com/archives/rh-openstack-dev/2018-June/date.html12:19
ykarelthis one:- http://post-office.corp.redhat.com/archives/rh-openstack-dev/2018-June/msg00118.html12:20
quiquellweshay: Don't know if it can be in the gates yet12:21
weshayykarel, so the script that picks those up is from lhh.. lon12:21
weshayykarel, you may just want to create the card directly on cix12:22
weshayand ping lhh12:22
weshaylon12:22
ykarelweshay, yup i tried to ping lon but he was not available, now he is available12:22
ykarelso will ping12:22
weshayarxcruz|ruck, rlandy|rover btw.. we owe ykarel another high five https://review.openstack.org/#/c/574145/12:23
arxcruz|ruckweshay: yeah, i have no idea how he figure that out12:23
arxcruz|ruckalready told him that he needs to write a blog about it12:23
rlandy|rovernice catch!12:25
weshaypanda|lunch, rlandy|rover trown I think I have everyone's comments addressed at this point for https://review.openstack.org/#/c/572217/ and deps12:27
ykarelweshay, created: https://trello.com/c/f1COW32i12:29
rlandy|roverweshay: https://review.openstack.org/#/c/572217/8/toci_gate_test-oooq.sh is going to have the same issue s as single node wrt taking just the rdocloud settings file12:29
rlandy|roverfixing your comments on that patch12:30
ykarelarxcruz|ruck, yes i remember, but not sure what to write there as everytime it's a new issue12:35
rlandy|roverarxcruz|ruck: hey - what's with the 14 hr periodic job?12:36
rlandy|roversf zuul12:36
rlandy|roverhave you checked into it?12:36
rlandy|roverotherwise - will look12:36
rlandy|rovercheck jobs also running at almost 15hrs12:37
ykarelrlandy|rover, ^^ because of rdocloud issue12:37
trownweshay: the standalone job is failing on https://review.openstack.org/#/c/572217/12:37
weshaybah of course12:38
hubbotFAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/56722412:38
*** udesale has joined #oooq12:38
quiquelltrown, panda|lunch, weshay: Adding release to naming gate looks good now https://review.openstack.org/#/c/572736/, it needs some reviewing12:39
quiquellAdding release to the gating repo I mean12:39
weshaytrown, hrm.. /me wonders if I need this if/else here https://review.openstack.org/#/c/572217/7..8/toci_gate_test-oooq.sh12:41
trownweshay: the line that is failing is actually in quickstart.sh... seems related to https://github.com/openstack/tripleo-quickstart/commit/b98f1bee5a1183e65040ce99b65fd6e6f7938bad12:43
rlandy|roverykarel: you get another gold star :)12:43
*** tcw has joined #oooq12:51
*** tcw has quit IRC12:52
*** tcw has joined #oooq12:52
*** amoralej|lunch is now known as amoralej12:58
arxcruz|ruckweshay: hey boss, pike is 4 days without promotion, i cannot find the problem, is that because there weren't any changes in pike ?13:00
arxcruz|ruckweshay: https://trunk.rdoproject.org/centos7-pike/report.html last package update was 4 days ago13:00
arxcruz|ruckjust want to double check before the report goes red tomorrow with 5 days without consistent being promoted13:01
sshnaidmweshay, 1:1?13:01
weshaysshnaidm, yes sir.. coming out of another mtg.. need 3min or so13:02
sshnaidmweshay, np13:02
*** udesale has quit IRC13:07
weshaysshnaidm, k.. ready13:14
*** udesale has joined #oooq13:15
ykarelarxcruz|ruck, regarding queens bug13:15
arxcruz|ruckykarel: yeah ?13:15
ykarelarxcruz|ruck, i can see it's affecting master as well13:16
arxcruz|ruckykarel: yeah, it doesn't13:16
arxcruz|ruckykarel: the problem right now is, why it's getting /v213:16
ykarelarxcruz|ruck, i think the issue is because of release of novaclient > 10-13:16
*** panda|lunch is now known as panda13:16
ykarelarxcruz|ruck, http://git.openstack.org/cgit/openstack/tripleo-quickstart/tree/roles/tripleo-inventory/templates/get-overcloud-nodes.py.j2#n24 logic is broken13:17
ykarelas if nc_version[0] returns 1 which is less that 613:17
*** Goneri has joined #oooq13:17
arxcruz|ruckykarel: LOL13:17
arxcruz|ruckykarel: you're right13:17
arxcruz|ruckholly molly13:17
arxcruz|ruckykarel: i'll create a reproduce env to check that13:18
ykarelarxcruz|ruck, just need to create a virtualenv, install python-novaclient==10.1.013:18
*** myoung|off is now known as myoung13:19
arxcruz|ruckykarel: yup, i think i have queens env up on my virthost13:19
ykarelarxcruz|ruck, ack13:19
ykarelthen it's fine, try there13:19
ykarelalso as newton is EOL: https://github.com/openstack/python-novaclient/tree/newton-eol13:20
ykarelso that workaround can be removed13:20
ykarelbut if "fetch overcloud node names and IPs" is used by any newton deployment that would see issue13:21
*** skramaja_ has quit IRC13:22
rlandy|roverarxcruz|ruck: running the cleanup script on downstream tenant13:23
rlandy|roversomething is blocked there13:23
arxcruz|ruckykarel: so, doesn't seems to be the same problem http://paste.openstack.org/show/723197/13:23
arxcruz|ruckat least i need to check, also, python-novaclient installed in the system is 9.013:24
quiquellpanda, trown: n -> n + 1 working https://review.openstack.org/#/c/571435/13:25
quiquellit integrates trown's work13:25
*** dtantsur|bbl is now known as dtantsur13:25
pandaquiquell: champagne!13:25
ykarelarxcruz|ruck, with queens 10.1.0 should be installed : https://trunk.rdoproject.org/centos7-queens/consistent/python2-novaclient-10.1.0-0.20180605113302.ed73aad.el7.noarch.rpm13:25
ykarelfor your test.py, have you sorced stackrc13:26
arxcruz|ruckykarel: yup...13:26
arxcruz|ruckhehehehe13:26
arxcruz|ruckfirst thing :P13:26
arxcruz|ruckykarel: but you're right, the nova client is python2-novaclient-10.1.0-0.20180605113302.ed73aad.el7.noarch13:27
arxcruz|ruckhttps://logs.rdoproject.org/openstack-periodic/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset002-queens-upload/e837432/undercloud/var/log/extra/rpm-list.txt.gz13:27
ykarelarxcruz|ruck, in venv you installed which version13:28
arxcruz|ruck10.1.013:28
ykarelok try 9.1.213:29
ykarelto see if there is some difference13:29
arxcruz|ruckykarel: also, one more thing...13:30
arxcruz|ruckif nc_version[0] <= "6":13:30
arxcruz|ruckthis would never work properly13:30
arxcruz|ruckbecause it's checking strings13:30
arxcruz|rucklol13:30
ykarelarxcruz|ruck, till now it was working as nc_version[0] used to return 913:31
ykarelagree this is not right way to check13:31
arxcruz|ruckykarel: i think we will have more problems with novaclient13:38
arxcruz|ruckwith version 9 is okay the code13:38
arxcruz|ruckbut 10 i got keystoneauth1.exceptions.connection.SSLError: SSL exception connecting to https://192.168.24.2:13000/v3/auth/tokens: HTTPSConnectionPool(host='192.168.24.2', port=13000): Max retries exceeded with url: /v3/auth/tokens (Caused by SSLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:579)'),))13:38
ykarelarxcruz|ruck, hmm seeing the same, but may be a virtualenv issue13:39
*** holser__ has joined #oooq13:39
ykarelas i can see same issue with 9 and 1013:40
*** matbu has quit IRC13:40
quiquellrfolco: fs037 on queens merged https://review.openstack.org/#/c/570902/13:42
*** matbu has joined #oooq13:42
rfolcoquiquell, :)13:43
rfolcoquiquell, this one failed on gate job - https://review.openstack.org/#/c/572306/13:44
quiquellrfolco: Now the per project commits13:44
quiquellrfolco: checking13:44
arxcruz|ruckykarel: CURL_CA_BUNDLE="" do the magic13:44
ykarelack, insecure=true could also be used i think13:45
quiquellrfolco: timed out... :-/13:45
quiquellarxcruz|ruck: Any issue with tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates ?13:45
ykarelarxcruz|ruck, so test.py works with 9.1.2 and fails with 10.1.0, right?13:45
quiquellarxcruz|ruck: It has timed out in the gates13:45
quiquellarxcruz|ruck: http://logs.openstack.org/06/572306/4/gate/tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates/d9124ed/job-output.txt.gz13:45
arxcruz|ruckykarel: yes, 9 pass, 10 fails without CURL_CA_BUNDLE13:46
ykareland same issue as we see in CI13:47
sshnaidm quiquell still online?13:47
quiquellsshnaidm: Yep13:47
quiquellpanda: could you +2 +1w this https://review.openstack.org/#/c/572308/ ?13:48
sshnaidmquiquell, need to sync about dashboard, but can also tomorrow if it's late13:48
quiquellsshnaidm: We have scrum in a few, let's sync tomorrow13:48
sshnaidmquiquell, sure13:48
quiquellsshnaidm: our mornings are quiet13:48
pandaquiquell: not sure I'allowed to +2 on that repo, maybe since it's CI stull I can do it13:50
quiquellpanda: You are right, let's ask at #tripleo13:51
trownquiquell: i got you :)13:51
quiquelltrown: sweet !!!13:54
ykarelarxcruz|ruck, with 10 you see same issue as CI or other13:54
quiquellrfolco: Going to recheck, it doesn't make sense13:55
arxcruz|ruckykarel: other13:57
quiquelltrown: The logs we save are specified in the jobs or in the featuresets ?13:57
*** matbu has quit IRC13:58
trownquiquell: they are specified via config ... it is not featureset though one sec14:01
*** matbu has joined #oooq14:04
trownquiquell: https://github.com/openstack-infra/tripleo-ci/blob/master/toci-quickstart/config/collect-logs.yml14:04
ykarelarxcruz|ruck, i can see the same issue as CI with 10.1.014:08
ykarelthough i have non tripleo setup14:08
ykareli mean:- keystoneauth1.exceptions.http.NotFound: (https://localhost:5000/v2.0/tokens): The resource could not be found. (HTTP 404)14:08
ykarelwith 9* it works fine14:09
arxcruz|ruckykarel: i hope this fix https://review.openstack.org/#/c/574270/14:12
ykarelarxcruz|ruck, can't we just remove the workaround14:12
arxcruz|ruckykarel: we still have newton jobs14:12
ykarelis newton jobs14:12
ykarelokk14:12
ykarelarxcruz|ruck, a small comment, rest looks good14:20
ykarelalso attach bug in commit message14:21
ykarelThanks, this should work now14:26
weshayrlandy|rover, bad news14:28
rlandy|roverweshay: which exact piece of bad news now?14:29
weshayrlandy|rover, undercloud is still busted on master14:30
rlandy|roverweshay: going nowhere now :(14:30
*** dtrainor has joined #oooq14:31
weshayrlandy|rover, OH wait..14:31
rlandy|roverfailure is not in undercloud14:32
weshayykarel, you mentioned for the kolla change to come in.. we need to move master to consistent?14:32
ykarelweshay, yes14:32
weshayhrm. k14:32
ykarelshould be consistent in few minutes14:32
weshayykarel++14:32
hubbotweshay: ykarel's karma is now 414:32
rfolcoquiquell, tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates on the previous changes.... and -upgrades on the last one.... what am I missing ?14:32
ykarelweshay, we also need arxcruz|ruck patch to promote master/queens:- https://review.openstack.org/#/c/574270/14:33
myoung*** CI squad: we're triaging tag:quickstart LP issues in half an hour, please kindly update any bugs actively being worked with status as appropriote, will help us whittle down the bug pile :)  Thanks!14:34
rlandy|roverarxcruz|ruck: what happened with the queens bug you had fs 2, 20, 35?14:34
rlandy|rover1 also failing now14:34
rlandy|roveris that still in flight?14:34
arxcruz|ruckrlandy|rover: https://review.openstack.org/#/c/574270/14:35
arxcruz|ruckneed this because meanwhile novaclient releases version 10, which makes the logic in the script fails14:35
arxcruz|ruckykarel++14:36
hubbotarxcruz|ruck: ykarel's karma is now 514:36
rlandy|roveroh gosh14:36
myoungarxcruz|ruck, ykarel, added reviewers in gerrit and "{patch} +=  https://trello.com/c/Ble66eRY/431-list-of-upstream-reviews"14:37
myoungtrown, rlandy|rover, sshnaidm, panda, weshay ^^  need eyes (promotion blocker)14:37
myoungoi...i'm tired, strike the += above and reverse it14:38
* myoung ducks14:38
hubbotFAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/56722414:38
quiquellrfolco: What do you mean ?14:38
ykarelmyoung, Thanks14:40
*** sanjayu_ has joined #oooq14:42
*** ykarel is now known as ykarel|away14:43
weshayarxcruz|ruck, not sure how my comment would relate exactly w/ available time and other priorities. It does seem to me that we've spent enough time in get-overcloud-nodes.py that some unit tests around it may helkp14:44
weshayhelp14:44
arxcruz|ruckweshay: well, the get-overcloud-nodes.py is a template in ansible, not sure how to keep unit tests there14:45
weshaymyoung, fyi.. noticing a number of incoming bugs w/o the appropriate tags that we may want to review fyi https://bugs.launchpad.net/tripleo/+bugs?orderby=-datecreated&start=014:45
*** saneax has quit IRC14:45
ykarel|awayweshay, can we remove alert:- https://bugs.launchpad.net/tripleo/+bug/177569814:46
openstackLaunchpad bug 1775698 in tripleo "tripleo nova containers restarting in a loop causing - 503 errors - starting container trunk.registry.rdoproject.org/tripleomaster/centos-binary-nova-compute-ironic" [Critical,Triaged] - Assigned to yatin (yatinkarel)14:46
weshayarxcruz|ruck, aye.. I see that too, however it's not really a template either14:46
arxcruz|ruckweshay: yeah, should be files14:46
weshayno jinja there it could be moved to facilitate14:46
quiquelltrown: Found the issue with the artifacts14:47
quiquell    repo_setup_script: "repo-setup-{{ lookup('env', 'UPGRADE_RELEASE') }}.sh"14:47
quiquell    repo_setup_log: "repo-setup-{{ lookup('env', 'UPGRADE_RELEASE') }}.log"14:47
quiquellIt's going to be dangerous if we use {{ release }} ?14:47
*** agopi has joined #oooq14:47
quiquellIs part of the refactoring14:47
rlandy|roverarxcruz|ruck: cannot be deleted directly via the port API: has device owner network:ha_router_replicated_interface ... that is killing downstream stack delete14:48
arxcruz|ruckrlandy|rover: shall i open a ticket ?14:48
rlandy|roverarxcruz|ruck: nobody will answer it ;( ... will try fix as much as we can from the tenant14:49
rlandy|roverjust fyi - in case you see it again14:49
rlandy|roverI think there is one job that adds that14:49
trownquiquell: hmm... ya14:51
trownquiquell: well actually... using release will be fine, because it is just the name14:52
quiquelltrown: Yep for undercloud we are good, we have to check it out for overcloud14:52
trownquiquell: and the other ones are named without any "-release" ... so worst case is just the reverse14:52
trownwhere some other job has misnamed artifacts14:52
*** holser___ has joined #oooq14:53
*** holser__ has quit IRC14:53
quiquelltrown: Also there is one missing artifact install_queens_gating_repo14:53
quiquellDon't know where do I configure those14:54
quiquellAhhno it's there14:54
quiquellsilly me14:54
myoungweshay: ack, let's address in triage, going thru the list now14:57
weshayarxcruz|ruck, rlandy|rover just need one of you for a bug triage15:00
weshayyou pick15:00
rlandy|roverweshay: arxcruz|ruck: joining15:00
arxcruz|ruckjoining15:00
*** bogdando has quit IRC15:03
*** agopi has quit IRC15:06
*** quiquell is now known as quiquell|off15:07
rlandy|roverugh ... just missed quiquell|off15:31
rlandy|roverquiquell|off: if you're not really off - your box is reprovisioned15:32
rlandy|roverwith the root password you sent15:32
rlandy|roverno centos user15:32
rlandy|roveryou can upgrade/add user15:32
myoungchandankumar, weshay, arxcruz|ruck (if you have time) - tempest squad scrum15:33
*** sanjay__u has quit IRC15:41
weshaykopecmartin, hey15:59
*** zoli is now known as zoli|gone16:09
*** zoli|gone is now known as zoli16:09
*** rnoriega has quit IRC16:13
*** rnoriega has joined #oooq16:20
weshaypanda, ping16:32
*** kopecmartin has quit IRC16:33
hubbotFAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/56722416:38
myoungweshay: rfolco, panda sorry16:39
*** trown is now known as trown|lunch16:45
*** udesale has quit IRC16:56
*** jfrancoa has joined #oooq17:29
*** trown|lunch has quit IRC17:35
*** dtantsur is now known as dtantsur|afk17:37
*** agopi has joined #oooq17:43
*** holser___ has quit IRC17:49
*** ykarel|away has quit IRC18:00
rlandy|roverpanda: sshnaidm: could you review pls ... https://review.openstack.org/#/c/573819/18:15
pandarlandy|rover: is this still "Do not Merge" ?18:16
rlandy|roverpanda: I left it as DNM over the weekend so nobody merged it w/o ci core review18:17
rlandy|roverthe DNM is removed now18:17
rlandy|roverweshay already left comments which I fixed18:18
pandarlandy|rover: ok18:18
rlandy|roverpanda: https://review.openstack.org/#/c/573820/ - also part of the fix18:18
rlandy|roverjust deciding what to do about bogdan's -118:18
rlandy|roverwe generally don;t double tag things18:18
*** amoralej is now known as amoralej|off18:23
rlandy|roverpanda: weshay: so this review is not a major deal either way ... https://review.openstack.org/#/c/573820/ - considering bogdan's comment, we could abandon. thoughts?18:25
rlandy|roverand here we go again on periodic18:25
*** agopi has quit IRC18:29
weshayrlandy|rover, def have a new dlrn hash this time18:35
weshayso we should see diff results18:35
weshay  "repo_url": "https://trunk.rdoproject.org/centos7-queens/87/66/87669a5034a80e2f04ce8d55d203e89d919f463d_15e3f3fe",18:36
rlandy|roverweshay: great - hoping for green pastures18:36
*** trown has joined #oooq18:38
hubbotFAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/56722418:38
trownlost power for a bit if anyone was trying to reach me, my bouncer was down18:39
*** agopi has joined #oooq18:49
rlandy|roverpanda: posted comments in https://bugs.launchpad.net/tripleo/+bug/1776186/comments/2 - pls see what you think for scheduling this18:50
openstackLaunchpad bug 1776186 in tripleo "Triple-O CI overrides jinja2 rendered network-environment.yaml" [High,Triaged] - Assigned to Ronelle Landy (rlandy)18:50
rlandy|roverprobably larger than a ruck/rivertaks - wrt testing all the options18:50
weshaysshnaidm++18:51
hubbotweshay: sshnaidm's karma is now 218:51
weshaysshnaidm, thanks for the fix on standalone18:51
*** myoung is now known as myoung|lunch18:52
weshayrlandy|rover, am I clear to help w/ bm.. https://rhos-dev-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/tripleo-quickstart-master-rdo_trunk-baremetal-hp_dl360_envE-single_nic_vlans/18:53
weshayrlandy|rover, re: moving configs over to other env?18:53
rlandy|roverweshay: ok -  ...18:54
rlandy|rovermy test on friday was a no-op as we still had the file passed in extras18:54
*** agopi has quit IRC18:54
rlandy|roverweshay: I think I can also get back to that now - had some downstream gate clear up to do this morning18:55
weshay%gatestatus18:55
weshay% gatestatus18:55
hubbotFAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/56722418:55
hubbotFAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/56722418:55
weshayrlandy|rover, k. k.. thank you!!18:55
rlandy|roverweshay: if we can stay with the plan we discussed, I'll set up the poc jobs now18:56
weshayya.. same plan afaict18:56
rlandy|roverso we actually test what we change18:56
rlandy|roverif you can work on the other env vars, that would be awesome18:56
weshayk18:56
weshayrlandy|rover, trown https://review.openstack.org/#/c/573457/18:57
trownweshay: it is failed in that patch :P18:58
weshaytrown, ya.. we know18:59
rlandy|roverweshay: what trown said - as long as we are ok with a failing job18:59
rlandy|roverit is non-voting18:59
* rlandy|rover thinks I asked this last time :(18:59
weshaytrown, maybe if the tripleo-upgrade guys see that.. they will help fix it :)19:00
rlandy|roverweshay: so we know why is was "successfully on experimental pipeline." but not here19:01
weshayrlandy|rover, it's old data19:01
weshayreally19:01
rlandy|roverI am against merging more failing jobs19:01
rlandy|roverbad precedent19:01
weshayrlandy|rover, trown imho.. not having check or gating jobs on a repo is worse precedent19:02
weshayI'll recheck it, it should actually work now after sofer's patch19:03
rlandy|roverweshay: thanks - after that I'll happily +2 this19:03
weshayrlandy|rover, note.. this change only runs the job on tripleo-upgrade changes19:03
rlandy|roverdoesn't change my objection to failing jobs19:04
rlandy|roverto stand on my soap box more about this ...19:05
rlandy|roveras ruck/rover, do you go through failing jobs to see that it's still the same failure?19:05
rlandy|roverin reality, we should, we don't19:06
weshayrlandy|rover, in reality ruck/rover do not watch check jobs19:06
weshayexcept for no-op changes19:06
rlandy|roverwe get pinged about them from people wanting to know why things failed19:07
weshayrlandy|rover, trown ok.. https://review.openstack.org/#/c/573844/19:07
weshaythat has sofer's patch19:07
rlandy|roverthat one has passing jobs - got my +219:10
rlandy|roverweshay: fixed your comments on https://review.openstack.org/#/c/573819/19:11
rlandy|roverwe should not merge that until we see test results19:11
rlandy|roverand make sure we didn't mess anything up19:11
weshayrlandy|rover, k https://review.openstack.org/#/c/573819/2/toci-quickstart/config/testenv/ovb-rdocloud.yml19:13
weshayso are the dns changes line 12-13 coming in from a rebase?19:13
* weshay looks at head19:13
rlandy|roverweshay: no - I removed them per your comment19:14
rlandy|roverhttps://review.openstack.org/#/c/573819/3/toci-quickstart/config/testenv/ovb-rdocloud.yml19:14
rlandy|roversee latest patch version19:14
weshayoh ya..19:14
weshaysorry was looking at the wrong ver..19:14
weshaythanks!19:14
weshaygood for another patch though19:14
rlandy|roverweshay: ack - one day19:15
weshayrlandy|rover, right19:15
weshaybusy busy19:15
rlandy|roverweshay: btw - I am taking the central-ci tenant for rhos-13 gate  - test19:15
rlandy|rovercentral-ci-stage19:15
weshayk19:15
rlandy|roverweshay: ^^ correction - stage tenant19:15
*** Goneri has quit IRC19:21
*** holser__ has joined #oooq19:22
*** atoth has quit IRC19:23
rlandy|roverhttps://rhos-dev-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/tq-gate-rhos-13-ci-rhos-ovb-minimal-pacemaker-public-bond/19:23
rlandy|roveron stage tenant - will see how it does19:23
rlandy|roverweshay: fyi ... I tested this a while back ... https://rhos-dev-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/poc-rlandy-tq-gate-rhos-12-ci-rhos-ovb-fs/configure - downstream ovb gate running fs00119:24
rlandy|roverwe got a pass once19:25
weshayk.. ya.. that should also run fs00119:25
weshayagree19:25
rlandy|roverdoing 13 test - then can test fs00119:25
rlandy|roverk - back to bm19:26
weshayfak19:35
weshaytrown, how close to ready are we for adding fs51 to use the new release tools.. http://logs.openstack.org/44/573844/3/check/tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades/b93bc99/job-output.txt.gz#_2018-06-11_12_16_37_58911019:37
*** holser___ has joined #oooq19:42
rlandy|roverugh19:44
rlandy|roverreally?19:44
rlandy|roverhttps://review.rdoproject.org/jenkins/job/periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master/299/console19:44
rlandy|roverhttps://review.rdoproject.org/jenkins/job/periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset016-master/567/console - looks better19:45
*** holser__ has quit IRC19:45
*** sanjayu_ has quit IRC19:46
weshaytrown, https://review.openstack.org/57441719:47
rlandy|roverweshay: ^^ looking at fs001 failure in promotion19:48
weshayrlandy|rover, hrm.. in prep-images19:48
rlandy|roverI know right - weird19:49
rlandy|roverwaiting for logs19:49
rlandy|roverfs002 look sok19:49
rlandy|rovermultinode is progressing19:49
*** holser__ has joined #oooq19:51
rlandy|roverb35fe4ba-d4c2-43f1-9573-7b62eb8d250c. Error: IPMI call failed: power status.'}, {u'result': u'Node 3c3fcde9-30a9-49ec-a631-2834582b0e4d did not reach state "manageable", the state is "enroll", error: Failed to get power state for node19:53
rlandy|roverseen that before19:53
rlandy|roverweshay: that may just be unlucky/sporadic19:54
rlandy|roverlet's see what fs035 does19:54
weshayk cool19:54
rlandy|roverfs002 is ok19:54
*** holser___ has quit IRC19:55
-openstackstatus- NOTICE: Zuul was restarted for a software upgrade; changes uploaded or approved between 19:30 and 19:50 will need to be rechecked19:55
*** myoung|lunch is now known as myoung19:59
*** holser__ has quit IRC20:00
*** holser__ has joined #oooq20:01
rlandy|roverweshay: https://review.rdoproject.org/jenkins/job/periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-master/351/console looks good - I'm going with sporadic on fs001 unless we see other evidence20:02
* rlandy|rover wishes for a retry button20:02
weshayrlandy|rover, depending on the other results and ack from tripelo leads, we could change the critieria20:03
weshaytrown, quiquell|off https://review.rdoproject.org/r/1418020:03
rlandy|roverweshay: eek - didn't know we did that upstream - I guess we're really behind in promotions, it makes sense in this case20:06
weshayrlandy|rover, we usually don't but we've done it a couple times20:07
weshayrlandy|rover, you have to be really careful w/ what you skip as to not impact upstream gate jobs20:07
rlandy|roverweshay: I'll kick a reproducer on my local tenant20:07
weshayrlandy|rover, rock20:07
*** ccamacho has quit IRC20:14
*** jfrancoa has quit IRC20:16
weshaytrown, looking for your reassurance / review on our new tooling and fs051. Thanks for the rdo review20:17
weshayguess we'll find out w/ https://review.openstack.org/#/c/574417/20:17
trownweshay: ya, not sure what will break tbh... might just work20:18
trownweshay: I am a bit surprised it doesnt work without using emit releases... if that is the case, we probably should just take out that featureset restriction20:18
trownI am 100% confident in emit releases when no upgrade is involved... so if upgrades dont work without it, not much to lose by using it20:19
trownmaybe they are broken in a different way... but maybe they also "just work"20:20
weshaytrown, ya..20:21
*** jtomasek has quit IRC20:23
weshayrlandy|rover, this needs a blocker bug https://logs.rdoproject.org/openstack-periodic/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-master/50abb54/undercloud/home/jenkins/tempest.log.gz20:31
weshayrlandy|rover, probably the same issue arx hit on get_overcloud_nodes.py20:31
weshayarxcruz|ruck, ^20:31
rlandy|roverlooking20:32
*** ccamacho has joined #oooq20:33
rlandy|roveryeah - tempest red all over20:33
rlandy|rovercould we what arxcruz|ruck logged before - checking20:34
hubbotFAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/56722420:39
rlandy|roverweshay: afaict, this is the bug arxcruz|ruck has open ... https://bugs.launchpad.net/tripleo/+bug/177536520:43
openstackLaunchpad bug 1775365 in tripleo "periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset002-queens-upload job is failing to fetch overcloud node names" [Critical,Fix released] - Assigned to Arx Cruz (arxcruz)20:43
rlandy|roverreview not yet merged20:43
rlandy|roverlooks different20:43
weshaywell.. his patch won't fix the problem in tempest.. but wondering if it's the same root20:43
rlandy|rovermaybe20:44
rlandy|rovercreating another bug in the meant time20:44
rlandy|roverhttps://review.openstack.org/#/c/574270/20:44
rlandy|roverbut bug refers to  https://review.openstack.org/57270920:45
rlandy|roverI lie - it has merged20:45
rlandy|roverthe /574270 has not merged20:46
*** holser__ has quit IRC20:46
*** Goneri has joined #oooq20:46
*** trown is now known as trown|outtypewww20:56
rlandy|roverweshay: https://bugs.launchpad.net/tripleo/+bug/1776301 [master promotion] Tempest is failing with " KeyError: 'resources' "errors - Connection refused20:58
openstackLaunchpad bug 1776301 in tripleo "[master promotion] Tempest is failing with " KeyError: 'resources' "errors - Connection refused" [Critical,Triaged]20:58
weshaythanks21:04
weshaybbl21:04
*** hamzy has quit IRC21:08
rlandy|roverwell, I guess there is one way to tell21:08
rlandy|roverreproducer - master hash with change21:09
*** hamzy has joined #oooq21:11
*** Goneri has quit IRC21:37
rlandy|roverhttps://github.com/openstack/python-tempestconf/commit/283de2057402ebe9a5c2eabef9e550359b40e05f21:45
rlandy|roverchandankumar: you around?21:46
rlandy|roverlong shot :)21:46
*** myoung is now known as myoung|off21:50
rlandy|roverweshay: back?21:50
rlandy|roverweshay: reproducer gets by perp-images on fs00121:57
rlandy|roverbut obviously we have other problems21:58
rlandy|rover%gatestatus22:31
hubbotFAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/56722422:31
*** florianf has quit IRC22:32
hubbotFAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/56722422:39
*** EmilienM|PTO is now known as EmilienM22:41
*** tosky has quit IRC23:32
rlandy|roverweshay: testing bm with fs001 ... https://rhos-dev-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/rlandy-poc-tripleo-quickstart-master-baremetal-hp_dl360_envE-single_nic_vlans23:51
*** rlandy|rover is now known as rlandy|rover|bbl23:55

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!