Monday, 2018-08-20

*** dtrainor_ has joined #oooq02:14
*** dtrainor_ has quit IRC02:20
*** skramaja has joined #oooq03:16
*** jaganathan has joined #oooq04:05
*** ratailor has joined #oooq04:23
*** jaosorior has joined #oooq04:31
*** sanjayu_ has joined #oooq04:41
*** apetrich has joined #oooq06:01
*** jtomasek has joined #oooq06:22
*** jfrancoa has joined #oooq06:26
*** amoralej|off is now known as amoralej06:44
*** ykarel has joined #oooq06:57
*** jaosorior has quit IRC07:17
*** tosky has joined #oooq07:30
*** jaosorior has joined #oooq07:37
*** bogdando has joined #oooq08:07
*** ratailor_ has joined #oooq08:21
*** ratailor has quit IRC08:24
*** dtantsur|afk is now known as dtantsur08:25
*** chem has joined #oooq08:36
*** sanjayu_ has quit IRC08:43
pandaykarel: about https://review.openstack.org/593383, were you able to run all the unit tests locally ? I see only half of them in the job http://logs.openstack.org/83/593383/3/check/openstack-tox-py27/4c601ab/job-output.txt.gz#_2018-08-19_15_18_09_08370509:04
ykarelpanda, i ran tox -epy27, i think all ran but can check again09:05
pandammhh, so why only some of them are running in the job ? :/09:05
ykarelpanda, how u got only few running?09:06
ykareli see it's 100 %09:06
pandaykarel: look at the link, only 6 tests are run09:07
pandaykarel: for example there's no test_periodic_noop_target_is_the_same09:07
ykarelpanda, all those tests are in test_release_name.py,09:08
ykarelpanda, so if u see last merge patch, stats are different there:- http://logs.openstack.org/65/592365/6/check/openstack-tox-py27/31b4bd2/job-output.txt.gz#_2018-08-18_12_42_05_48053409:10
ykareljob not printing tests which are running09:11
pandaykarel:  47 passed in 0.35 seconds09:11
ykarelso comparing tests yes, 47 and now 6309:11
pandaykarel: ok, I'll add a patch to use pytest -v09:12
ykarelpanda, ack,09:13
pandasilly pytest just shows the files that contain the test09:13
pandaykarel: ok, approving then09:13
ykarelack, Thanks09:13
ykarelso we just need fs020 to get rocky promotion now09:13
ykarelthere is a patch to fix it: https://review.openstack.org/#/c/591982/09:16
ssbarneasshnaidm: hi! can you please read https://review.openstack.org/#/c/587371/ and the bug it reffers to, this problem became more important now as people encounter it downstream.09:18
sshnaidmweshay|ruck, marios fyi https://bugs.launchpad.net/tripleo/+bug/178791009:26
openstackLaunchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b)09:26
jfrancoasshnaidm: hello o/, do you have +2 rights in rdoproject? could you please review https://review.rdoproject.org/r/#/c/15557/ ? It's supposed to fix container-to-container legacy jobs which is failing in deploy at the moment09:27
sshnaidmssbarnea, ok, but you need to run it with some dummy patch so that all jobs will run with it. Currently no jobs ran there if you can see09:27
ssbarneasshnaidm: sure, what file do I need to touch to build all jobs?09:28
sshnaidmjfrancoa, yeah, let's try09:28
jfrancoasshnaidm: thanks!09:28
sshnaidmssbarnea, you can change some role in quickstart-extras and depend on this patch, for example09:29
*** sshnaidm is now known as sshnaidm|afk09:29
*** saneax has joined #oooq09:52
*** ykarel is now known as ykarel|lunch09:54
mariosack sshnaidm|afk checking09:55
ssbarneamarios:  thanks for the hint regarding use of yaml anchors, i didn't know they were working in ansible. i would clearly try then for next patch, this one took too much time to get the testing and review votes and I would prefer not to start from zero again with it :D10:19
mariosssbarnea: ack np as i wrote it is outside of scope/pre-existing repetition there anyway thanks10:20
*** ratailor has joined #oooq10:24
*** ratailor_ has quit IRC10:25
*** dtantsur is now known as dtantsur|brb10:27
jfrancoaykarel|lunch: hey, is there any LP for this promotion blocking issue you commented in https://review.openstack.org/#/c/576520/8/docker/services/database/mysql.yaml@265 ? I was thinking about having a look at the issue10:30
*** ykarel|lunch is now known as ykarel10:35
ykareljfrancoa, i think https://bugs.launchpad.net/tripleo/+bug/178701010:35
openstackLaunchpad bug 1787010 in tripleo "The Parameter (UndercloudUpgrade) was not provided." [Critical,Triaged]10:35
jfrancoaykarel: great, thanks10:35
ykareli think marios is looking into it as per last comment10:36
jfrancoaykarel: yes, seems like he did a big analysis of the problem. marios do you need some more eyes on it? or do you have it under control?10:37
ykareljfrancoa, Emilien just proposed a patch10:38
ykarelhttps://review.openstack.org/#/c/59357210:38
jfrancoaykarel: thanks for the info10:40
mariosjfrancoa: sorry was in a call reading10:53
mariosjfrancoa: ykarel please check if you have time i couldn't root cause it just added some notes so i will revisit10:53
mariosykarel: thanks checking the patch10:54
jfrancoamarios: Emilien already proposed a patch to solve it https://review.openstack.org/#/c/593572 , it seems some task from rocky landed in queens during an upgrades refactor10:54
mariosjfrancoa: thanks10:54
*** ratailor has quit IRC11:00
*** ratailor has joined #oooq11:01
*** holser_ has joined #oooq11:01
*** ratailor has quit IRC11:02
*** ratailor has joined #oooq11:03
*** holser_ has quit IRC11:20
*** holser_ has joined #oooq11:20
ssbarneacan somoene help with this review? https://review.openstack.org/#/c/583965/11:22
ssbarneapanda: ^11:23
ssbarneawhat happens is very easy to check, https://seashells.io/v/7pRFCq3F - a known bug.11:24
*** amoralej is now known as amoralej|lunch12:01
*** weshay|ruck is now known as weshay12:03
*** panda is now known as panda|lunch12:03
*** sshnaidm|afk is now known as sshnaidm12:04
*** ratailor has quit IRC12:06
*** trown|outtypewww is now known as trown12:13
mariosweshay: o/ why did you tag https://trello.com/c/SXBPujPR/702-cixlp1787910tripleociproa-ovb-overcloud-deploy-fails-on-nova-placement-errors as upgrades12:16
mariosweshay: (we just  retagged)12:17
weshaymarios, sorry that was meant for the other card..12:17
mariosweshay: but checking12:17
weshayupgrade card12:17
mariosweshay: ack thanks12:17
weshaymarios, lack of coffee12:17
mariosweshay: np12:17
mariosweshay: sanity checking i thought it was mburns that changed it but he said it was you12:18
toskycan anyone help finding out why this change to a configuration file in tempestconf, which should not trigger in the CI, fails?12:19
toskyhttps://review.openstack.org/#/c/590298/12:19
weshayssbarnea, need to chat w/ you when ur back online12:21
ssbarneaweshay: i am back.12:21
weshayhttps://redhat.bluejeans.com/u/whayutin/12:21
weshayssbarnea, https://bluejeans.com/411356779812:25
weshaysorry.. had to change to the app12:25
*** ccamacho has joined #oooq12:33
*** rlandy has joined #oooq12:36
*** jbadiapa has joined #oooq12:37
*** bogdando has quit IRC12:44
*** dtantsur|brb is now known as dtantsur12:45
*** bogdando has joined #oooq12:48
rlandyweshay: are you working with any stacks in ci-rhos? can I delete baremetal_92749?12:48
rlandyweshay: job gets are far as overcloud deploy - nodes are stuck cleaning12:49
rlandyhow big a deal would it be it be if I disabled node cleaning?12:49
*** amoralej|lunch is now known as amoralej13:03
weshayrlandy, sshnaidm join the scrum13:03
*** dtrainor_ has joined #oooq13:03
weshaypanda|lunch, scrum13:03
*** skramaja has quit IRC13:13
* chandankumar headed home13:32
weshayssbarnea, http://dashboard-ci.tripleo.org/d/cEEjGFFmz/cockpit?orgId=113:35
weshayssbarnea, http://cistatus.tripleo.org/promotion/13:43
weshayssbarnea, http://rhos-release.virt.bos.redhat.com:3030/rhosp13:44
weshayssbarnea, marios https://docs.google.com/document/d/1ySHfOi6vP_KCexlIJXaodsnGiYXgy6GN0Bz2fZUSTSo/edit13:53
weshayssbarnea, marios https://specs.openstack.org/openstack/tripleo-specs/specs/policy/ci-team-structure.html13:55
sshnaidmweshay, do you join tls mtg?13:59
*** chandankumar has quit IRC14:12
rlandypanda|lunch: rf0lc0: what are we going to do now wrt design discussion and POC reviews from Friday?14:15
rlandyI need to determine what to do with the remaining reviews from the sprint 17 card14:15
rlandysshnaidm: pls review my response on https://review.openstack.org/#/c/587228/9/playbooks/tripleo-ci/templates/toci_gate_test.sh.j2 and let me know if you are ok with that14:17
rlandypanda|lunch: rf0lc0: also reviews that are needed for reproducer - older workflow ... https://review.openstack.org/#/c/58148814:18
rlandypanda|lunch: rf0lc0: weshay: sshnaidm:do we want to keep this with the new workflow ... https://review.openstack.org/#/c/591864?14:19
rf0lc0rlandy, I'll do a review on those asap... the last one looks odd... I am confused now on the motivation for add fs number to the file... might be missing something14:23
*** panda|lunch is now known as panda14:25
sshnaidmweshay, the external node in ovb actually exists, the problem is more with adapting freeipa role for it afaik14:26
sshnaidmrlandy, well, weshay workflowed this, so not sure if it's relevant now14:27
sshnaidmrlandy, I'm not sure this is a big problem to add this one line later when we have more than one, tbh, and it'll be less confusing so14:28
weshaysshnaidm, I didn't workflow it14:28
rlandysshnaidm: we didn't merge it14:28
weshaywell.. I took it back14:28
sshnaidmweshay, "Workflow+1 wes hayutin" :P14:28
weshayrather14:28
rlandythe desigb is up for discussion14:29
weshayit's not merged14:29
weshayif you -1 workflow a patch even in gate.. the gate stops14:29
rlandyI am trying to sort out all these reviews14:29
weshay$tricks to clean up the gate queue14:29
rlandywhich ones we want to keep and toss14:29
sshnaidmweshay, it'll be merged when you recheck14:29
weshayok.. now it won't be14:30
weshaysshnaidm, any questions on my featureset vs tempest config requirement?14:30
weshaydoes it make more sense now?14:30
*** myoung is now known as myoung|training14:31
sshnaidmweshay, you mean white/black list for features?14:31
weshaysshnaidm, well.. that is an implementation14:31
weshaywhich I'm not married to14:31
weshayhowever..14:31
bogdandoo/ PTAL https://review.openstack.org/#/c/57674614:31
bogdandoupdated with logs14:31
weshayvalidation config should be able to be overriden by zuul14:32
weshaybut featureset config should not14:32
bogdandoand the topic as well https://review.openstack.org/#/q/topic:localcon+(status:open+OR+status:merged)14:32
weshaybogdando, we need to see ci.centos quick-basic jobs pass to review that14:32
bogdandoweshay: sure14:33
bogdandojfyi, there is logs as sshnaidm asked for14:33
weshaybogdando, k :)14:33
weshaybogdando,  + /home/jenkins/workspace/tripleo-quickstart-gate-master-delorean-quick-basic/bin/cico node get --arch x86_64 --release 7 --count 1 --retry-count 6 --retry-interval 60 -f csv14:34
weshay2018-08-20 13:54:14.834 |  The requested operation failed as no inventory is available.14:34
weshaythat happens :( sorry14:34
sshnaidmweshay, it's clear, but I just think it's overhead and kind of overreaction, if somebody will want to override something, you can't stop him14:34
bogdandoweshay: um, that tells me nothing :(14:34
weshaybogdando, ci.centos ran out of nodes14:35
bogdandoah14:35
weshaynot your fault14:35
weshaysshnaidm, if the neutron team ran tripleo-ci-centos-7-containers-multinode14:35
weshayw/ and changed the heat templates used in the deployment14:35
weshaybut the job is still named tripleo-ci-centos-7-containers-multinode14:36
weshayisn't that going to suck for us.. when they say the job is broken?14:36
weshaywe can't have different types of deployments happening w/ jobs of the same name14:37
sshnaidmweshay, if somebody from neutron team will figure out how to override templates using jobs configuration, I think we need to invite him to our team14:37
weshaysshnaidm, you are basically opening a ci exploit if you allow the featureset to be overidden14:37
weshaysshnaidm, I'm not really looking for jokes at this point14:37
sshnaidmweshay, me too, the chance for this case is near the zero14:38
weshaysshnaidm, however how would we ever know14:38
weshay?14:38
weshayit would be done in the zuul config on the neutron repo14:38
sshnaidmweshay, to look at jobs config??14:38
weshaysshnaidm, I don't plan on monitoring neutron jobs14:38
weshaysshnaidm, rf0lc0 rlandy if you guys want to chat about this.. I'll open my blue14:39
weshayhttps://bluejeans.com/411356779814:39
weshaysshnaidm, I don't think it's fair to just assume no one will override a settings14:40
weshaysshnaidm, I don't think it's fair to just assume no one will override a setting14:40
weshayif they can.. I think we should assume they will14:41
weshaythat is a safer approach14:41
sshnaidmweshay, I prefer to rely on people, and when somebody overrides something I think he knows what he does14:41
weshaypanda, ^14:41
weshayfyi14:41
weshaysshnaidm, but when the job w/ the same name as ours.. starts failing for no reason14:42
weshayor for some reason they caused by overriding a fs option14:42
*** bogdando has quit IRC14:42
sshnaidmweshay, so what? you look at its config and discover the difference14:42
weshaysshnaidm, I have problems w/ that taking up our time14:42
weshaysshnaidm, I don't want to have to do that14:42
weshayI don't want to look at that many jobs14:42
weshaysshnaidm, you can join us on blue?14:43
sshnaidmweshay, well, I don't have so strict objections to it to continue to argue :)14:43
*** chandankumar has joined #oooq14:47
*** jfrancoa has quit IRC15:27
chandankumarmyoung|training: weshay arxcruz meeting time15:31
arxcruzchandankumar: joined15:32
*** saneax has quit IRC15:32
chandankumararxcruz: at this url na https://bluejeans.com/7050859455/ ?15:33
arxcruzchandankumar: yes15:33
rlandysshnaidm: https://review.openstack.org/#/c/587228/9/playbooks/tripleo-ci/templates/toci_gate_test.sh.j2 - ok with my response here?15:36
sshnaidmrlandy, well, not really, but it's not so critical, will remove -115:36
rlandysshnaidm: no need15:36
rlandylet's fix it15:37
sshnaidmrlandy, <sshnaidm> rlandy, I'm not sure this is a big problem to add this one line later when we have more than one, tbh, and it'll be less confusing so15:37
rlandysshnaidm: I'll change it back then if that helps15:37
rlandysshnaidm: updated https://review.openstack.org/#/c/58722815:43
weshaysshnaidm, rlandy rf0lc0 https://github.com/openstack/cinder-tempest-plugin/blob/master/.zuul.yaml#L2315:44
rlandyweshay: I am going to change https://review.openstack.org/#/c/581488/26/playbooks/baremetal-full-deploy.yml to switch the browbeat playbook based on enable_minial_browbeat not the fs number https://github.com/openstack/tripleo-quickstart/blob/master/config/general_config/featureset053.yml#L14515:45
rlandythen we can ditch the fs number review for the moment15:46
sshnaidmrlandy, thanks15:49
rlandysshnaidm: your opinion is important to us :) you should not remove you -1 easily16:16
sshnaidmrlandy, well, it's really not critical issue16:17
*** ykarel is now known as ykarel|away16:17
sshnaidmrlandy, I need to improve my review ratio, it's about 65% now... the lowest I think :D16:17
toskya low review ratio may mean different things, and it's not always a bad thing16:20
rlandysshnaidm: I think that is a badge of honor16:20
rlandyI think my approval ratio is too high16:20
* rlandy aims to be more like sshnaidm16:20
rlandyrf0lc0: panda: what do you want to do with this review - abandon? https://review.openstack.org/#/c/593063/16:27
rlandyalso, should I reparent my browbeat job on this^^ or rf0lc0's review?16:27
toskydoes anyone know why legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master failed here? https://review.openstack.org/#/c/590298/16:35
*** trown is now known as trown|lunch16:35
rlandypanda: weshay: corrected per sshnaidm's comments - pls can one of you vote on https://review.openstack.org/#/c/587228?16:37
sshnaidmpanda, needed to restart dlrn-promoter service, otherwise it still uses the old dlrn_promoter.sh and didn't try to promote rocky16:49
sshnaidmtosky,  ERROR - Failed creating OVB stack https://logs.rdoproject.org/98/590298/2/openstack-check/legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master/0312208/job-output.txt.gz#_2018-08-20_12_42_38_61126616:51
*** ccamacho has quit IRC16:51
sshnaidmtosky, (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') , long live rdo cloud..16:51
rf0lc0rlandy, why abandon ? your change is a good start, let's keep it.16:52
toskysshnaidm: so "recheck" is the solution?16:53
sshnaidmtosky, "check-rdo" will recheck rdo cloud jobs only16:56
*** sshnaidm is now known as sshnaidm|off16:58
toskysshnaidm: both from RDO Third Party CI (voting) and RedHat RDO CI (apparently non-voting)?16:58
toskyor is also RedHat RDO CI voting?16:58
sshnaidm|offtosky, third party only16:58
sshnaidm|offtosky, no, only 3party vote16:58
toskyoki, thanks16:58
rlandyhttps://review.openstack.org/#/c/581488 updated16:59
toskyI didn't want to use again tons of resources for no reason16:59
*** dtrainor_ has quit IRC17:01
weshaypanda, we need to age off the cards in the epic column17:02
weshaypanda, and remove what we can17:02
*** ykarel|away has quit IRC17:10
*** ykarel|away has joined #oooq17:10
*** dtrainor has joined #oooq17:15
*** dtantsur is now known as dtantsur|afk17:17
*** ykarel|away has quit IRC17:18
weshayrlandy, when you have a sec.. I need your eyeballs on a standalone review17:42
weshayI'm missing something17:42
rlandyweshay: one sec - rekicking ovb job17:43
*** trown|lunch is now known as trown17:48
rlandyweshay: ok - what should I be looking at?17:49
weshayrlandy, ok.. just noticed.. this.. unrelated https://review.openstack.org/#/c/588527/17:49
weshayfairly easy to review17:49
weshayrlandy, and then I share my tale17:50
weshayrlandy, I'd like to chat about https://review.openstack.org/#/c/593295/2/ci-scripts/include-gate-changes.sh for a minute17:50
rlandyok - looking at unrelated17:50
rlandydone17:51
rlandylooking at https://review.openstack.org/#/c/593295/2/ci-scripts/include-gate-changes.sh17:51
weshayrlandy, ya.. I can give you some context17:51
weshayhttps://bluejeans.com/4113567798  if you want.. or irc17:52
weshaydon't care17:52
rlandyk - joining bj17:52
weshayhttps://review.openstack.org/#/c/586843/18:01
rlandyhttps://review.openstack.org/#/c/593077/2/roles/undercloud-deploy/templates/hieradata-overrides-classic-undercloud.yaml.j218:24
rlandyhttps://review.openstack.org/#/c/579406/8/roles/overcloud-prep-flavors/templates/overcloud-prep-flavors.sh.j218:24
*** amoralej is now known as amoralej|off19:03
*** jtomasek has quit IRC19:36
*** jtomasek has joined #oooq20:21
*** jtomasek has quit IRC20:24
*** jtomasek has joined #oooq20:31
*** jtomasek has quit IRC20:34
*** holser_ has quit IRC20:45
rlandyweshay: ok - we're getting somewhere ... https://rhos-dev-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/tq-gate-reproducer-rhos-13-ci-rhos-ovb-featureset001/9/console20:49
weshayrlandy, what do we do w/ epel on subnode-0?20:56
weshaybut that is great news20:56
weshayur on the node.. and working w/ it20:57
weshay++20:57
rlandyweshay: I am just trying to extract zuul_changes from refspec20:57
rlandythen will deal with node setup20:57
rlandyI cheated that to start20:57
weshayk.. we don't install epel though right?20:57
rlandywell I am on rhel20:57
rlandyyou may be better off20:57
rlandyonce I get this somewhat working, we can consider the libvirt multinode case20:58
* rlandy hacks a bit more20:58
rlandythen we can evaluate20:58
rlandyweshay: to answer your question, yep _ I am on the undercloud node20:59
rlandyI can access it as zuul20:59
*** holser_ has joined #oooq21:01
*** trown is now known as trown|outtypewww21:02
weshaynice21:03
*** holser_ has quit IRC21:38
rlandyhmmm ... need to pick up dependent changes21:45
*** chem has quit IRC22:22
*** tosky has quit IRC23:08
rlandyweshay: I am not sure how we ever brought dependent changes into jenkins23:16
*** agopi_ has joined #oooq23:17
*** agopi has quit IRC23:20
*** agopi_ is now known as agopi23:39

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!