Wednesday, 2019-10-30

*** rlandy has quit IRC00:02
*** panda has quit IRC00:02
*** panda has joined #oooq00:05
*** ykarel|pto has joined #oooq04:01
*** ykarel|pto is now known as ykarel04:01
*** epoojad1 has joined #oooq04:08
*** epoojad1 has quit IRC04:32
*** udesale has joined #oooq04:35
*** ykarel is now known as ykarel|afk05:02
*** ykarel|afk has quit IRC05:02
*** soniya29 has joined #oooq05:05
*** skramaja has joined #oooq05:15
*** ykarel has joined #oooq05:25
*** ratailor has joined #oooq05:47
*** sanjayu_ has joined #oooq05:47
*** akahat has joined #oooq06:40
*** kopecmartin|off is now known as kopecmartin06:40
*** marios has joined #oooq06:54
*** marios is now known as marios|ruck06:54
*** jbadiapa has joined #oooq06:57
*** ykarel is now known as ykarel|afk06:59
*** sanjayu_ has quit IRC07:02
*** saneax has joined #oooq07:02
*** sanjayu_ has joined #oooq07:04
*** saneax has quit IRC07:05
*** sanjayu__ has joined #oooq07:07
*** sanjayu__ has quit IRC07:08
*** sanjayu_ has quit IRC07:09
*** saneax has joined #oooq07:10
*** sanjayu_ has joined #oooq07:12
*** akahat has quit IRC07:13
*** saneax has quit IRC07:13
*** akahat has joined #oooq07:16
*** rascasoft has quit IRC07:29
marios|ruckzbr: please see comment #8 @ https://bugs.launchpad.net/tripleo/+bug/1848512 when you have 5 mins thanks07:39
openstackLaunchpad bug 1848512 in tripleo "ansible-lint broken when called from pre-commit" [Critical,In progress] - Assigned to Sorin Sbarnea (ssbarnea)07:39
*** bogdando has joined #oooq07:56
*** ykarel|afk is now known as ykarel08:03
arxcruzmarios|ruck: good morning08:11
arxcruzmarios|ruck: https://review.opendev.org/#/c/689106/ please08:11
*** ratailor has quit IRC08:16
arxcruzzbr: https://review.rdoproject.org/r/#/c/23508/08:16
arxcruzkopecmartin: is the designate service already enabled in tempestconf jobs ?08:17
arxcruzkopecmartin: i see https://review.opendev.org/691893 but not sure if this actually enable this on the job08:18
*** ratailor has joined #oooq08:18
*** chandankumar has quit IRC08:19
*** chandankumar has joined #oooq08:20
kopecmartinarxcruz: it should be, why do you think it's not enough?08:21
kopecmartinmaybe i overlooked somehting08:22
arxcruzkopecmartin: i'm not thinking is not enough, just want to double check with you before approves08:22
arxcruzthat's all08:22
arxcruz:)08:22
arxcruzi don't want to go through the logs to check that, if you are saying this is working, i'm okay with that08:22
*** holser has joined #oooq08:22
kopecmartinwell, i think it is, I wrote to local.conf what openstack/designate doc proposes08:24
arxcruzok08:24
marios|ruckarxcruz: ack08:26
chandankumarkopecmartin: arxcruz standalone scenario03 has designate08:26
arxcruzchandankumar: https://review.opendev.org/689106 works08:26
chandankumararxcruz: yes, thanks for fixing that :-)08:27
chandankumar' and ` this combo seeing for first time08:27
chandankumarkopecmartin: just one question https://review.opendev.org/#/c/691893/2 do we also want to run few designate tests there?08:33
kopecmartinchandankumar: i don't know, do we? I guess we can , at least a few08:34
chandankumarkopecmartin: yes, it would be good to have some tests08:34
kopecmartinchandankumar: ok, i'll edit devstack-admin-plugins job so that we run a few designate tests08:35
*** jpena|off is now known as jpena08:37
arxcruzjpena: zbr https://review.rdoproject.org/r/#/c/23508/08:44
*** amoralej|off is now known as amoralej08:46
chandankumararxcruz: https://review.rdoproject.org/r/#/c/23508/ commented08:49
arxcruzchandankumar: fair enough08:51
*** dtantsur|afk is now known as dtantsur09:01
*** ykarel is now known as ykarel|lunch09:03
arxcruzchandankumar: can you explain me the pagure thing ?09:07
zbrarxcruz: lint before upload, it will not pass.09:12
zbrarxcruz: yeah, I visually-lint faster than the linters!09:13
arxcruzzbr: i did run linters before09:14
zbrand remember to smoke/drink/meditate before today's meeting09:14
*** sshnaidm|afk is now known as sshnaidm09:21
*** jfrancoa has joined #oooq09:23
*** jfrancoa has quit IRC09:28
chandankumararxcruz: good question09:38
*** ratailor_ has joined #oooq09:38
*** tesseract has joined #oooq09:38
chandankumararxcruz: the goal here is to replace https://github.com/rdo-infra/rdo-jobs/blob/master/playbooks/tripleo-ceph-integration/pre.yaml and https://github.com/rdo-infra/rdo-jobs/blob/master/playbooks/tripleo-ceph-integration/post.yaml with distro zuul jobs09:39
chandankumarhttps://pagure.io/zuul-distro-jobs/blob/master/f/roles09:39
chandankumararxcruz: in long run, we will be also removing build-test-packages role and reuse the same09:39
chandankumarinstead of maintaining our own stuff09:39
*** ratailor has quit IRC09:41
chandankumarzbr: \o/09:43
chandankumarzbr: regarding this card https://tree.taiga.io/project/tripleo-ci-board/task/1352?kanban-status=1447274 create zuul-jobs role to install go-lang for centos and ubuntu it means we are going to create a seperate role in zuul-jobs?09:43
chandankumarmarios|ruck: Do we have patch up for enabling rhel8 train promotion pipeline?09:47
chandankumararxcruz: please have a look at this user story https://tree.taiga.io/project/tripleo-ci-board/us/1342?milestone=242589 some of the ceph-ansible job work might needs coordination with rlandy09:48
marios|ruckchandankumar: https://review.rdoproject.org/r/#/c/2351009:48
marios|ruckchandankumar: sec gonna update it for th enit09:49
chandankumarmarios|ruck: thanks!09:49
arxcruzchandankumar: this is a mess. i'm working basically on the same thing...09:52
arxcruzzbr: jpena  https://review.rdoproject.org/r/#/c/23517/09:57
*** holser has quit IRC10:11
*** ykarel|lunch is now known as ykarel10:13
*** holser has joined #oooq10:13
*** holser has quit IRC10:13
*** slaweq has joined #oooq10:14
*** holser has joined #oooq10:14
slaweqmarios|ruck: hi, do You know maybe who with +2 privilege to ask for review https://review.opendev.org/#/c/655423/ ?10:15
zbrmarios|ruck: i reported https://review.rdoproject.org/zuul/builds?job_name=sova-tracking-jobs broken few days ago, any fixes?10:17
zbri am going to make it non-voting as i preventing other changes.10:18
*** Tengu has quit IRC10:18
chandankumararxcruz: zbr: marios|ruck https://review.opendev.org/#/c/689085/10:18
chandankumarplease have a look, thanks!10:18
*** holser has quit IRC10:19
*** holser__ has joined #oooq10:19
*** Tengu has joined #oooq10:19
zbrmarios|ruck: please give a kick on https://review.rdoproject.org/r/#/c/23518/10:24
chandankumarzbr: do you have the logs like what is broken there?10:24
chandankumarsorry got the logs10:24
zbrchandankumar: totally unrelated, bad design. until that is fixed, it must be nv10:25
chandankumarzbr: it needs some pr, not a hard fix10:25
chandankumarzbr: http://logs.rdoproject.org/openstack-periodic-24hr/review.rdoproject.org/rdo-infra/ci-config/master/sova-tracking-jobs/382c596/job-output.txt10:25
chandankumarneeds to just add the missing jobs in sova10:25
chandankumargive me some time I will fix it10:26
zbrchandankumar: you are wrong here: a the job is badly designed, i will disable it until someone is avoiding this kind of regression10:26
zbri don't care that some jobs are missing from sova, we have a periodic job that nobody watches, that has bad triggers (good ones should have prevented reaching that case)10:27
zbrwe are not talking about being broken for 1-2 days, is more than 30 days!10:28
zbrusually for few days, i would say yep, lets fix it, and go on.10:28
zbrdone, made nv10:29
zbrmarios|ruck: please add note to ruck etherpad about doing a proper fix for that job10:29
marios|ruckzbr: ack10:30
marios|ruckzbr: which job?10:30
zbrsova-tracking-jobs10:30
marios|ruckzbr: ack was for review request10:30
zbri hope everyone agrees here: we should not allow jobs to end-up like this by having proper triggers.10:31
zbrbeing too narrow with file-filters on triggers or missing to trigger a job on some repos that can break it, is likely to cause this type of problem.10:31
marios|ruckzbr: whats the deal with cherrypicked on the commit message tho?10:32
zbrahh, a side effect of reordering commints.10:32
marios|ruckzbr: maybe update it10:33
marios|ruckmight catch the merge yet10:33
marios|ruckunworkflow10:33
zbrmarios|ruck: done. revote10:33
marios|ruckzbr: gerrit kept them10:33
marios|ruckslaweq: sorry i missed your ping looking now10:38
marios|rucksshnaidm: panda chandankumar zbr arxcruz can you check that lets get it merged ! the added role there doesn't affect anything tripleo-ci it is exercised there https://review.opendev.org/#/c/651828/ (tobiko tobiko-tripleo-update-standalone )10:39
marios|rucksshnaidm: panda chandankumar zbr arxcruz that : https://review.opendev.org/#/c/655423/ Add playbook and role to run tobiko10:40
chandankumarAh I lost my vote, it is good to go10:40
zbrmarios|ruck: done. only +1 as i do not have more there.10:41
marios|ruckack thanks folks slaweq i think we need a recheck there am afraid its like > 2 weeks old in check10:42
marios|rucksshnaidm: beat me to it... was thinking of recheck... but i guess gate will cover it ;)10:44
sshnaidmmarios|ruck, should be fine10:44
marios|rucksshnaidm: i mean i'm only ruck for like ~5 hours ish so...10:44
* marios|ruck shrug 10:44
marios|ruck;)10:44
marios|rucksshnaidm: yeah it doesn't touch anything of tripleo-c10:44
marios|rucktripleo-c10:44
marios|ruckwow10:44
marios|ruckci10:44
sshnaidmmarios|ruck, we'll ask you to stay more, you're so good!10:44
marios|rucksshnaidm: sorry i have holidays booked in honolulu with rfolco|rover10:44
marios|ruckmojito sunsets10:44
*** jtomasek has quit IRC10:44
*** openstackstatus has quit IRC10:44
sshnaidmmarios|ruck, you'll continue to ruck-rover downstream in mojito sunsets10:45
marios|ruckhaha10:45
sshnaidmmarios|ruck, there a lot of problem in this jenkins, y'know10:45
marios|rucksshnaidm: i forgot to ask you & was out monday did you do the rdo cloud bmc update you were talking about last week10:45
sshnaidmmarios|ruck, I couldn't coz it's protected image and can be replaced by admins, so I sent a ticket to ops10:47
sshnaidmmarios|ruck, will ping them now to update it, while you're here10:47
marios|rucksshnaidm: fantastic10:47
* marios|ruck starts packing10:47
sshnaidmmarios|ruck, I'm sure rfolco will hold a mojito for you10:48
sshnaidmeven for a week10:48
* marios|ruck eye twitching10:48
*** jtomasek has joined #oooq10:49
*** rfolco|off has joined #oooq10:54
*** panda is now known as panda|pto11:00
arxcruzkopecmartin: chandankumar https://63b837b4ad9a4e528fa6-44886a430c48ba6939226a5d4d9ebe5b.ssl.cf5.rackcdn.com/691893/3/check/python-tempestconf-tempest-packstack-admin/e7e54e4/job-output.txt is it related to the change? I don't think so right?11:09
*** arxcruz is now known as arxcruz|lunch11:10
slaweqthx marios|ruck chandankumar and sshnaidm for reviews :)11:12
*** epoojad1 has joined #oooq11:13
*** sshnaidm has quit IRC11:23
zbrbtw, i discovered that hackmd has nice usernames and multiple documents, so I am ready to make the switch if anyone want it.11:27
zbrrfolco|off: can you give me admin on https://hackmd.io/IhMCTNMBSF6xtqiEd9Z0Kw ?11:34
*** rfolco|off has quit IRC11:35
*** sshnaidm has joined #oooq11:43
chandankumarweshay: marios|ruck Do we want to move retro call meeting just after prodchain call?11:45
marios|ruckchandankumar: not sure what was decided folco and weshay were gonna discuss. calendar says we have a call in 1 hour 15 mins but then we will have to drop after an hour for the other call11:46
kopecmartinarxcruz|lunch: i don't think so either .. this failed with the same error: https://review.opendev.org/#/c/691573/11:55
*** ratailor_ has quit IRC11:56
*** jpena is now known as jpena|lunch11:59
*** marios|ruck is now known as marios|call12:00
*** ykarel is now known as ykarel|afk12:02
chandankumarweshay: marios|call arxcruz|lunch zbr sshnaidm kopecmartin I have created a new label for sprint 17 retro https://trello.com/b/0VFswmht/rdo-infra-retrospective?menu=filter&filter=label:UniSprint17 please add the cards.12:05
*** rlandy has joined #oooq12:13
*** epoojad1 has quit IRC12:14
*** epoojad1 has joined #oooq12:15
*** arxcruz|lunch is now known as arxcruz12:18
weshayepoojad1, hello :)12:20
weshayepoojad1, you available to chat today?12:20
epoojad1weshay, hello12:22
epoojad1weshay, yes12:22
weshayepoojad1, k cool.. I'll be getting off a call soon and I'll ping you :)12:23
epoojad1weshay, sure :)12:24
*** udesale has quit IRC12:26
*** akahat has quit IRC12:30
chandankumarrlandy: Hey12:37
rlandychandankumar: hi12:37
chandankumarrlandy: needs eyes on following reviews https://review.rdoproject.org/r/#/c/23516/ https://review.opendev.org/687460 https://review.opendev.org/68908512:38
chandankumarThanks!12:38
rlandylooking in a sec12:38
*** aakarsh has quit IRC12:47
rlandychandankumar: wrt https://review.rdoproject.org/r/#/c/23516/ - don;t know much here - but can w+ this and try it12:48
rlandyseems like it won't break too much12:48
*** Goneri has joined #oooq12:49
rlandychandankumar: arxcruz: for https://review.opendev.org/#/c/689085 marking -1 due to failure in tempest on fs21 - if unrelated let me know and I'll revote12:52
arxcruzrlandy: just post a comment ;)12:52
rlandyarxcruz: ok - revoted12:54
rlandyat least there is a trace I actually looked at the error :)12:54
rlandyarxcruz: chandankumar: w?12:55
chandankumarrlandy: yes12:56
chandankumarReminder, please add the retro card https://trello.com/b/0VFswmht/rdo-infra-retrospective?menu=filter&filter=label:UniSprint1712:58
*** rfolco has joined #oooq13:01
chandankumarrlandy: weshay rfolco arxcruz sshnaidm zbr retro time https://meet.google.com/mce-cnhy-gmk13:02
*** jpena|lunch is now known as jpena13:02
arxcruzyay, retro time, best time ever13:02
*** soniya29 has quit IRC13:11
*** amoralej is now known as amoralej|lunch13:12
*** epoojad1 is now known as epoojad1|afk13:12
*** epoojad1|afk has quit IRC13:17
zbrwho can give me a hint about "No package openvswitch available." on http://logs.rdoproject.org/23/23423/3/check/tripleo-ci-centos-8-scenario000-standalone/f44a244/job-output.txt13:33
*** TrevorV has joined #oooq13:38
*** aakarsh has joined #oooq13:44
*** ykarel|afk is now known as ykarel13:45
*** aakarsh|2 has joined #oooq13:47
*** kopecmartin is now known as kopecmartin|scho13:48
*** aakarsh has quit IRC13:50
marios|callweshay: posted forgotten tht and updated https://tree.taiga.io/project/tripleo-ci-board/task/1345 https://review.opendev.org/692144 https://review.opendev.org/#/q/topic:train-test13:55
sshnaidmweshay, https://tree.taiga.io/project/tripleo-ci-board/task/1364?kanban-status=144727413:56
*** amoralej|lunch is now known as amoralej13:58
marios|calldocs review please when you next have some time thanks https://review.opendev.org/#/c/689530 build there https://3b42e78a11ed997b2368-81a8be848ef91b58aa974b4cb791a408.ssl.cf1.rackcdn.com/689530/3/check/openstack-tox-docs/01b917d/docs/ci/chasing_promotions.html13:59
weshaywes_prodchain13:59
weshaymarios|call, rfolco I guess we'll pick a ruck rover tomorrow morning, is that ok?14:00
rfolcok14:00
*** ykarel is now known as ykarel|meeting14:04
*** jaosorior has joined #oooq14:21
*** Goneri has quit IRC14:49
*** ykarel|meeting is now known as ykarel15:01
*** amoralej is now known as amoralej|off15:03
*** jpena is now known as jpena|off15:04
rlandychandankumar: need to borrow your tempest brain ... https://sf.hosted.upshift.rdu2.redhat.com/logs/periodic-hourly/code.engineering.redhat.com/openstack/tripleo-ci-internal-jobs/master/periodic-tripleo-ci-centos-7-bm_envA-3ctlr_1comp-featureset001-master/f9346b1/job-output.txt. Consistent failure on os_tempest : Ping router ip address - possible that bm is missing a setting?15:04
rlandymaster only15:04
*** marios|call is now known as marios15:04
chandankumarrlandy: checking15:04
chandankumarrlandy: please share the job definition15:05
chandankumarrlandy: sorry15:06
chandankumarone min15:06
*** jaosorior has quit IRC15:06
chandankumarrlandy: https://github.com/openstack/tripleo-quickstart/blob/master/config/general_config/featureset001.yml#L16615:06
chandankumarneed to fix this part15:06
chandankumarthe current address used is from ovb15:06
rlandyhttp://git.app.eng.bos.redhat.com/git/openstack/tripleo-ci-internal-jobs.git/tree/zuul.d/baremetal-jobs.yaml#n12715:06
chandankumarI donot know bm value there15:06
* rlandy looks15:07
chandankumarbasically that is undercloud cidr15:07
rlandyoh - thanks15:07
chandankumaryou get the correct cidr, it will work15:07
weshayrfolco, you can hang on as ruck w/ me right?15:08
*** weshay is now known as weshay|rover15:08
weshay|roverrfolco, until tomorrow morning?15:08
rfolcoweshay|rover, sure15:08
chandankumarrlandy: do you have node hold?15:08
*** rfolco is now known as rfolco|ruck15:08
rfolco|ruckwhy not15:09
chandankumarif yes, just change that part and rerun the playbook of os_tempest it will work15:09
weshay|roverthanks15:09
rlandyhttp://git.app.eng.bos.redhat.com/git/tripleo-environments.git/tree/hardware_environments/dell_fc430_envA/network_configs/single_nic_vlans/env_settings.yml settings overrides15:09
chandankumarif you are going to summit, please buy it15:09
chandankumarhttps://www.eventbrite.com/e/ansible-rdo-celebration-of-openstack-tickets-7918800044115:09
rlandywill add that setting there15:09
weshay|roverrfolco|ruck, chandankumar are we pre-planning today?15:09
weshay|roveror did we go through it enough yesterday?15:09
chandankumarweshay|rover: rfolco|ruck I think we had done that yesterday15:09
chandankumarrfolco|ruck: weshay|rover https://tree.taiga.io/project/tripleo-ci-board/epic/1363 needs review to be introduced in next sprint15:10
rfolco|ruckI can work on taiga board offline15:10
*** sanjayu_ has quit IRC15:10
chandankumarrlandy: undercloud_network_cidr: 10.9.120.128/2615:10
chandankumarit will work I think15:10
rlandychandankumar: thanks - will try thta15:11
rlandythat15:11
chandankumarzbr: we will be using hangout now for discussion?15:11
*** marios is now known as marios|ruck15:11
marios|ruckrfolco|ruck: fight!15:11
rfolco|ruckwait a minute boy15:12
rfolco|ruckwho you think you are15:12
marios|ruckrfolco|ruck: ground rules no pet scorpions allowed15:12
marios|ruck(hows that going btw)15:12
zbrchandankumar: i think that irc is not appropiate for that kind of discussions. but for day to day work, irc rules.15:13
*** rfolco|ruck is now known as rfolco|rucker15:13
chandankumarirc is much more simpler for me15:13
chandankumarrlandy: if still not work, hold the node and pass the creds to me throigh email will dig tomorrow15:14
rlandychandankumar: I don;t think it's a different setting but I am matching it from ovb15:15
rlandythanks for the hint15:15
chandankumarrlandy: -)15:15
chandankumarsshnaidm: panda|pto please +w it https://review.opendev.org/#/c/689085/15:16
chandankumarend the os_tempest stuff15:16
rlandyExternal (10.0.0.0/24) net15:16
sshnaidmchandankumar, you know it's failing in patch, right?15:17
rlandy10.9.120.192/2615:17
chandankumarsshnaidm: it is fs02115:18
weshay|roverrfolco|rucker, ok.. ready to clarify the rhel8 bit15:18
rfolco|ruckerweshay|rover, k15:18
chandankumarrlandy: undercloud_network_cidr: 10.9.120.128/26 is different there?15:18
sshnaidmchandankumar, yes, f012 is failing there15:18
weshay|roverrfolco|rucker, https://meet.google.com/hwb-ofjm-pax15:19
sshnaidmchandankumar, s/f012/f02115:19
chandankumarsshnaidm: fs021 is where we run skip list as whitelist15:19
chandankumarwhich is meant for failure15:19
chandankumarthe behaviour is expected15:19
chandankumarsshnaidm: http://logs.rdoproject.org/85/689085/3/openstack-check/tripleo-ci-centos-7-ovb-1ctlr_2comp-featureset021/e1ae695/logs/stestr_results.html15:19
chandankumaris the proof that it is working15:20
rfolco|ruckerweshay|rover, sorry bad timing15:20
rlandychandankumar: yep15:20
rfolco|ruckerweshay|rover, i gotta go take my son at school and have lunch15:20
rlandyit's the external cidr here15:20
sshnaidmchandankumar, ack15:21
rfolco|ruckerweshay|rover, still out of home, sorry for the inconvenience15:21
rlandythese networks are hard-wired15:21
sshnaidmchandankumar, do we have results in text somewhere?15:21
rfolco|ruckerweshay|rover, will be back asap15:21
rlandyoh shame - rfolco|rucker ... still rucking15:21
chandankumarrlandy: http://logs.rdoproject.org/85/689085/3/openstack-check/tripleo-ci-centos-7-ovb-1ctlr_2comp-featureset021/e1ae695/logs/undercloud/var/log/tempest/tempest_run.log.txt.gz15:21
chandankumarsshnaidm: ^^15:22
sshnaidmchandankumar, great, we need to add this to logstash and sova tracking15:22
chandankumarsshnaidm: I am going to write script to compare results from two jobs15:22
chandankumarsshnaidm: I have added it recently available for train and master15:23
sshnaidmchandankumar, you can integrate it in jcomparison15:23
chandankumarsshnaidm:yes, there itselfd15:23
sshnaidmchandankumar, but anyway, we need to index it in logstash15:23
chandankumarok will put a patch up then15:24
chandankumarsee ya tomorrow people, Have a nice day/night ahead :-)15:25
*** rfolco|rucker has quit IRC15:26
weshay|roversshnaidm, is this the sova result? https://storage.gra1.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_b49/678630/12/check/tripleo-ci-centos-7-containers-multinode/b49e2d2/logs/_No_failure_reason_found.log15:48
sshnaidmweshay|rover, yes15:48
*** Goneri has joined #oooq15:49
weshay|roverchandankumar, zbr rlandy marios|ruck arxcruz https://review.opendev.org/#/c/678630/ please don't make me ask again :)15:49
*** jaosorior has joined #oooq15:51
marios|ruckweshay|rover: i left some comments but they are unaddressed faics15:53
marios|ruckweshay|rover: https://review.opendev.org/#/c/678630/12/library/sova.py@77 proceeded_files15:53
weshay|roversshnaidm, ^15:53
marios|ruckweshay|rover: i have been checking since then but my comments weren't addressed so i assumed it was still being worked on15:53
marios|ruckweshay|rover: (esp cos sagi did respond to them)15:53
sshnaidmmarios|ruck, I'll look15:54
*** ykarel is now known as ykarel|away15:55
weshay|roversshnaidm, /me agrees w/ jesse btw.. if no failure is found lets skip writing the file if possible15:55
sshnaidmmarios|ruck, actually I responded to them15:55
marios|rucksshnaidm: yeah but i still see proceeded @ https://review.opendev.org/#/c/678630/12/library/sova.py@7715:56
marios|ruck      - Files which have been proceeded by module15:56
sshnaidmweshay|rover, I'd like to write a file in any case, because if we start to pull these files we'll need it anyway15:56
weshay|roversshnaidm, ok15:56
sshnaidmmarios|ruck, ack, will fix15:57
marios|rucksshnaidm: thanks will check again tomorrow15:57
sshnaidmweshay|rover, in case all jobs will have it we don't need sova server anymore, just pull it, parse and send to DB15:58
sshnaidmand display in grafana maybe15:58
sshnaidm"observability and metrics consolidation"!15:59
weshay|roversshnaidm, sounds good16:00
weshay|roversshnaidm, ya.. it would be nice to have a column that shows the sova error on failed jobs in the cockpit..16:01
weshay|roveragree16:01
marios|ruckci docs review please when you next have some time thanks https://review.opendev.org/#/c/689530 build there https://3b42e78a11ed997b2368-81a8be848ef91b58aa974b4cb791a408.ssl.cf1.rackcdn.com/689530/3/check/openstack-tox-docs/01b917d/docs/ci/chasing_promotions.html16:02
weshay|roversshnaidm, a nit would be the file name seems a little funny to me.  maybe something like automatic_debug_results16:03
sshnaidmweshay|rover, instead of failures_file? https://50b981921d0b42196f35-e24395b1f226ab7bf437be1dd808e069.ssl.cf5.rackcdn.com/678630/12/check/tripleo-ci-centos-7-scenario012-multinode-oooq-container/4567f5e/logs/16:04
weshay|roversshnaidm, marios|ruck btw.. I have the undercloud ansible module working..  :)16:04
weshay|roversshnaidm, aye16:04
sshnaidmweshay|rover, nice!16:04
weshay|roverthanks for the help!!!16:05
marios|ruckweshay|rover: nice i guess you worked out the params then16:05
weshay|roversshnaidm, figured out how to instantiate it16:05
sshnaidmweshay|rover, so you submit a patch?16:05
weshay|roversshnaidm, ha.. i have it atm in a personal gitlab16:06
weshay|rovernot ready to submit quite yet.. going to further socialize at the ptg16:06
* marios|ruck almost out weshay|rover need something before i go... things i know about are at https://etherpad.openstack.org/p/ruckroversprint17 as usual (no new bugs today that i know of at least)16:06
weshay|roverk16:07
weshay|roverthanks16:07
zbrweshay|rover: please wf https://review.opendev.org/#/c/691832/16:08
rlandyleft comments on sova review16:08
*** TrevorV has quit IRC16:17
sshnaidmzbr, functional and other tests will be in following up patches, I think will create a molecule job for it16:22
arxcruzsshnaidm: you said molecule, zbr put's a smile on his face lol16:22
zbrarxcruz: at least sshnaidm wrote a podman driver for it :)16:23
*** kopecmartin|scho is now known as kopecmartin|off16:24
zbri do not care if that testing is  just pytest, or molecule, as long is something I can run in mintues locally to validate that this module does not choke with various python versions for example16:24
zbri do not even have a min coverage requirement, just a single execution path would be infinitely better than none.16:25
sshnaidmzbr, no versions, only 3 :)16:25
*** marios|ruck is now known as marios|out16:25
*** TrevorV has joined #oooq16:32
chandankumarsshnaidm: zbr weshay|rover we have lots of python2 code in ci-config, any plans to do a sprint on that?16:40
sshnaidmchandankumar, hackaton!16:40
*** marios|out has quit IRC16:41
zbrnot sure if we need a sprint for that, but is clearly needed. i would add it to "chore" category, something we need to check for every sprint (like bumping linters)16:41
zbri may be a good idea to document what chores we need to do on each sprint16:42
zbrsome kind of implicit goal16:42
chandankumarsshnaidm: yes16:42
chandankumarzbr: that one also sounds good.16:42
chandankumarrlandy: sshnaidm: I have added you as a watcher new epic please have a look and have your feedback there on component gating stuff16:43
weshay|roverchandankumar, oh ya.. we need to16:44
chandankumarzbr: sshnaidm arxcruz rlandy https://tree.taiga.io/project/tripleo-ci-board/epic/1363 just chime in, have comments in the card, no planning16:45
chandankumarweshay|rover: will put a card then we can plan16:45
weshay|roverchandankumar, tuesday evening16:48
weshay|rover14hr flight16:48
chandankumarweshay|rover: ah, mine is around 8 hrs16:48
rlandyok16:48
chandankumararxcruz: once fs021 os_tempest merged, do we want to send an email to list about os_tempest job porting?16:52
*** TrevorV has quit IRC16:54
*** TrevorV has joined #oooq16:55
*** openstackstatus has joined #oooq17:02
*** ChanServ sets mode: +v openstackstatus17:02
*** bogdando has quit IRC17:04
*** rfolco has joined #oooq17:04
rfolcoweshay|rover, back, free to chat about rhel8 jobs17:06
weshay|roverrfolco, the rhel8 train jobs are defined, just remarked out.. for periodic17:07
weshay|roverrfolco, also line #67    should be a rhel8 job.. vs centos7     - tripleo-ci-rhel-8-ovb-3ctlr_1comp-featureset001:17:08
rfolcoweshay|rover, yes I re-enabled them in the periodic yesterday17:09
rfolcoweshay|rover, looking at fs00117:09
weshay|rovercool17:11
weshay|roverrfolco, there is a also a openstack-virtual-baremetal repo defined there.. running centos-7 fs00117:11
weshay|roverthat can also be rhel8 imho17:12
weshay|roverspeaking about tripleo.yaml17:12
weshay|roverline 51417:12
weshay|roverrlandy, you want to 1-1 or are we up 2 date17:28
rlandyweshay|rover: I think we're up to date unless you have something to discuss17:29
rlandyI guess we'll cover promotion pipeline tomorrow17:29
weshay|roverya17:29
rlandyok - so we'll skip?17:30
rlandyanything for the week of summit?17:30
rlandywe should know/do/cover?17:30
rlandyweshay|rover: ^^17:30
weshay|roverrlandy, it's a very informal ptg afaik17:31
weshay|roverno decisions shall be made there for tripleo17:31
rlandyok17:32
weshay|roverrlandy, help me out.. where is the tripleo-ovb template defined?17:37
weshay|roverconfig?17:37
rlandyweshay|rover: which piece of it?17:37
* rlandy gets17:37
weshay|roverthe def of the tempalte17:38
weshay|rovertemplate17:38
rlandyhttps://github.com/rdo-infra/rdo-jobs/blob/master/zuul.d/ovb-jobs.yaml - jobs themselves - base is in https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/zuul.d/tripleo-rdo-base.yaml#L2717:39
rlandyweshay|rover: ^^  think you want the second one17:40
rlandyhas t be in config due to secrets17:40
* weshay|rover looking for the templates17:41
rlandyweshay|rover: pls tell me what you mean by template17:41
weshay|roverovb-001-lite and tripleo-ovb17:41
rlandyoh17:41
weshay|rover:)17:41
weshay|rovertemplate is overloaded for sure17:41
weshay|roverbut that's what this is in fact called17:41
rlandyhttps://github.com/rdo-infra/rdo-jobs/blob/b75dbdf3d4be17c8b744fd260cd29af2a30b5cf7/zuul.d/project-templates.yaml#L26317:42
rlandyhttps://github.com/rdo-infra/rdo-jobs/blob/master/zuul.d/project-templates.yaml#L263 (w/o the commit)17:42
rlandyhttps://github.com/rdo-infra/rdo-jobs/blob/master/zuul.d/project-templates.yaml#L6917:42
rlandysorry - ^^ those two17:43
rlandyproject-template17:43
rfolcoweshay|rover, do you want to replace for specific projects... not all of them? like tripleo-ansible also has fs001 centos-717:44
weshay|roverrfolco, hrm.. I thought tripleo-ansible had rhel 817:44
* weshay|rover looks17:44
weshay|roverrfolco, it does have rhel8 https://review.opendev.org/#/c/691857/  but also centos-717:45
rfolcoweshay|rover, just above virtual-baremetal17:45
weshay|rovermakes sense to have both there for tripleo-ansible17:45
weshay|roverrfolco, ya.. when I was scrolling through it.. that is all that caught my eye.. what job to run where.. is always a WIP17:45
rfolcok17:46
chandankumarrlandy: does fs01 bm os_tempest changes worked?17:47
rlandychandankumar: still running17:47
chandankumarok17:47
rlandyI had to change nodeset as well on one job17:48
chandankumarrlandy: sshnaidm can we get a session on bm and ovb for whole team ?17:48
chandankumarfor debugging stuff17:48
rlandygladly17:48
*** dtantsur is now known as dtantsur|afk17:49
chandankumarrlandy: sshnaidm I will try to schedule a meeting may be after ptg for that, having an exercise would be great.17:49
rlandyjob is installing the undercloud17:50
chandankumarrlandy: is the bm job slow? comparing to rdocloud?17:52
rlandychandankumar: ack because the undercloud is virt on bm17:54
rlandyso you have to create the vm17:54
chandankumarah, ok17:54
rlandythen get images17:54
rlandythen modify images etc.17:54
rlandyrepo setup runs on bm, and undercloud vm17:54
rlandyin reality you can use the bm hardware as the undercloud but the requires reprovision each time17:55
chandankumarrlandy: is it possible to use the preprovisioned bm?17:56
rlandychandankumar: yes17:56
rlandybut this hw is not random17:56
rlandythese machines are networked in a particular way17:56
rlandyso that the overcloud machines can boot off the undercloud17:57
rlandythe easiest way to keep rerunning on the same hw is to use a vm17:57
rlandyand recreate it each time17:57
rlandyalso ,,,17:57
rlandyin theory17:57
chandankumarah ok, so many things to know, /me stops asking questions17:57
rlandyyou can spin a rhel vm n a centos bm box17:58
rlandythus repo setup needs two diff configs17:58
rlandychandankumar: wrt https://tree.taiga.io/project/tripleo-ci-board/epic/1363 - what did you mean by ' have comments in the card, no planning'17:59
rlandywe won;t discuss at the planning meeting?17:59
chandankumarrlandy: I mean go through the slides and doc have questions there17:59
chandankumarbecause during planning, we just jump on stuff how to solve the problem18:00
rlandythat might be our problem :)18:01
chandankumarhaving an understanding early helps us to plan better18:01
rlandychandankumar: hmm - hate to say this but I think it's my turn on ruck/rover this sprint18:05
chandankumarrlandy: is it? Do you want to pair once again if weshay|rover as I will be out for a week.18:05
rlandychandankumar: ack - can do18:06
chandankumar*wesley|rover allows18:06
rlandylet's see what shakes out at the planning meeting18:06
chandankumaryup18:06
weshay|roversec in mtg18:10
* chandankumar moved to f31, upgrade is faboulus18:11
*** dsneddon has joined #oooq18:12
*** holser__ has quit IRC18:14
*** chandankumar is now known as raukadah18:17
raukadahweshay|rover: arxcruz fs020 and fs021 now using os_tempest, we made it after an year18:29
*** ykarel|away has quit IRC18:31
rlandyrfolco: question about rhel-8 jobs18:41
rlandyhttps://review.rdoproject.org/r/#/c/23510/3/zuul.d/tripleo.yaml18:41
rlandycompare with master ...18:41
rlandy        - periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset002-master-upload:18:42
rlandy            dependencies:18:42
rlandy- periodic-tripleo-centos-7-master-containers-build-push18:42
*** Goneri has quit IRC18:46
weshay|roverraukadah, lolz.. congrats18:47
rfolcorlandy, hmm you mean missing fs00218:51
rfolcorlandy, not for rhel818:53
rlandyrfolco: so confusing to read this :(18:54
rlandytrying to compare to master18:54
rfolcorlandy, this was enabled previously... look, I just uncommented it out. We tried to enable it earlier, and the packages were not ready for train yet.18:56
*** tesseract has quit IRC19:03
rlandyweshay|rover: rfolco: ok - pls explain this:19:07
rlandyhttps://github.com/rdo-infra/review.rdoproject.org-config/blob/master/zuul.d/tripleo.yaml#L8819:07
rlandyhttps://github.com/rdo-infra/review.rdoproject.org-config/blob/master/zuul.d/tripleo.yaml#L16919:07
rlandy        - periodic-tripleo-rhel-8-buildimage-ironic-python-agent-master:19:08
rlandy            dependencies:19:08
rlandy- periodic-tripleo-centos-7-master-promote-consistent-to-tripleo-ci-testing19:08
rlandyvs ..19:08
rlandy        - periodic-tripleo-rhel-8-buildimage-ironic-python-agent-master:19:08
rlandy            dependencies:19:08
rlandy- periodic-tripleo-rhel-8-buildimage-overcloud-full-master19:08
rlandy^^ that pattern is coped in train19:08
rlandydefined twice two diff deps19:09
* rlandy wonders if I am crazy or this is wrong19:09
raukadahrlandy: weshay|rover rfolco I got confirmation from jpena|off today RHEl-8 train packages are available now19:09
rfolcoraukadah, https://review.rdoproject.org/r/#/c/23510/3/zuul.d/tripleo.yaml19:10
rlandyraukadah: that's great but this review is not19:10
raukadahrlandy: it is mine wrong doing19:10
raukadahrlandy: rfolco needs periodic-tripleo-centos-7-train-promote-consistent-to-tripleo-ci-testing to be removed19:10
raukadahit is added twice19:10
rlandyhttps://review.rdoproject.org/r/#/c/23510/19:10
rlandyraukadah: ^^ see comments there19:10
rlandyrfolco: ^^19:11
rlandytwo dupicates19:11
rlandybut we have the same problem in master19:11
rlandyand that pipeline is workin19:11
rlandysomehow19:11
rlandy**crazy**19:11
raukadahrlandy: yes two duplicates19:11
rlandyraukadah: how is master working then????19:12
* rlandy spends an hour checking myself19:12
raukadahrlandy: but the dependencies are different there19:14
raukadahrlandy: https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/zuul.d/tripleo.yaml#L169 and https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/zuul.d/tripleo.yaml#L8819:14
raukadahmight be that's why it is working19:14
rfolcorlandy, i think zuul considers just one and doesn't check dups there19:14
raukadahrlandy: may be zuul takes care of sorting the jobs19:14
rlandytwo depends should be added to one definiton19:15
rlandyso one of the deps is not being taken into account here19:15
rfolcoindeed19:15
rlandyguessing that the last one wins19:15
rlandywe need to be sure which one is correct19:15
rfolco periodic-tripleo-rhel-8-buildimage-overcloud-full-master does not exist19:20
rfolcooh wait19:20
rfolcoignore me19:20
*** jaosorior has quit IRC19:21
rlandyrfolco: ok if I w+ https://review.rdoproject.org/r/#/c/23523/?19:23
rlandyweshay|rover: ^^?19:23
rfolco+1 rlandy19:24
*** aakarsh|2 has quit IRC19:25
rfolcorlandy, raukadah last one defined is the one used. I verified that  periodic-tripleo-rhel-8-buildimage-overcloud-full-master exists, but never ran19:26
rfolcorlandy, raukadah: sorry again. One more false statement19:29
rfolcodelete. clear. reset.19:29
rlandyraukadah: you familiar with collect logs?19:31
rlandyhttps://review.opendev.org/#/c/691994/ - need some help on where the files are located at that point19:32
*** Goneri has joined #oooq19:36
rfolcorlandy, raukadah why rhel8 has buildimage-overcloud-full and centos7 doesn't ?19:41
rlandyfs002?19:42
weshay|roverrlandy, thanks19:42
weshay|roverrfolco, because centos-7 used fs00219:44
weshay|roverrfolco, we're migrating away from fs002 to the upstream check / build jobs19:45
rfolcoweshay|rover, ack19:45
weshay|roverthat kind of migration happens at release..19:45
rlandyrfolco: that part is correct - it's the duplicates that are wrong19:45
weshay|roverrfolco, so I would expect train to not have fs00219:45
rlandywe need to decide which dep to keep19:45
weshay|roverI think.. or just master19:45
* weshay|rover looks19:45
weshay|roverwhich review19:46
rfolcorlandy, git fetch ssh://rafaelfolco@review.rdoproject.org:29418/config refs/changes/10/23510/3 && git checkout FETCH_HEAD19:46
rfolcooops19:46
rfolcoweshay|rover, ^19:46
rlandyweshay|rover: we have some funny business going on with the pipelines19:46
rlandypls read back19:46
weshay|roverhrm19:47
weshay|roverya.. we only need periodic-tripleo-centos-7-master-promote-consistent-to-tripleo-ci-testing once and it does the work for rhel8 as well19:48
weshay|roveradditionally we should scrap periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset002-master-upload:19:49
weshay|roverand build centos-7 images now the same as rhel819:49
weshay|roverbut I'll leave that to the next ruck / rover19:49
weshay|roverok.. so two dupe lines.. ipa and tripleo-ci-testing19:50
weshay|roverrlandy, am I missing something?19:50
rlandyyep - in the train review19:50
rlandyonly one dup in master pipeline19:51
rlandythat is all19:51
weshay|roverok.. rfolco you good?19:51
rfolcoreading19:51
rfolcook deps should be consistent-to...19:52
rfolcok I got it19:53
rfolcothx rlandy weshay|rover19:53
rlandyweshay|rover: unrelated question about collect logs ...19:54
rlandyso ... https://sf.hosted.upshift.rdu2.redhat.com/logs/95/168695/13/check/tripleo-ci-rhel-8-standalone-rhos-16/9e85042/logs/quickstart_collect_logs.log19:54
rlandyshows that the sanitize tasks run but on the wrong locations19:54
rlandy            "paths": [19:55
rlandy                "/home/zuul/workspace/logs/undercloud/etc/yum.repos.d/"19:55
rlandy            ],19:55
rlandy "matched": 019:55
rlandy "path": "/home/zuul/workspace/logs/undercloud/home/zuul/containers-prepare-parameters.yaml.4667.2019-10-30@13:12:08~.gz",19:55
rlandy^^ and I clearly get the wrong path there19:55
rlandyso ...19:55
rlandyhttps://review.opendev.org/#/c/691994/4/tasks/publish.yml19:55
rlandyat the point that ^^ tasks runs, where are the logs?19:56
rlandy {{ artcl_collect_dir }}/?19:56
rlandyalso iiuc, they would not have .gz extension yet as that rename happens afterwards19:57
*** rfolco has quit IRC20:12
*** Goneri has quit IRC20:22
rlandysshnaidm: still around?20:23
* weshay|rover looks20:24
sshnaidmrlandy, ya20:24
rlandysshnaidm: oh I think weshay|rover is looking at my question now20:24
rlandyI am trying to sanitize the logs20:24
sshnaidmrlandy, renaming to .gz is done last20:25
weshay|roverya20:25
rlandyok - so is my location wrong?20:25
rlandythe task runs but does nothing20:25
rlandy"/home/zuul/workspace/logs/undercloud/etc/yum.repos.d/" matches nothing20:26
weshay|roverparams are here https://code.engineering.redhat.com/gerrit/#/c/184172/5/config/release/rhos-16.yml20:27
weshay|roverrlandy,  I wonder if slashes are getting you20:27
weshay|roverorig_string could be .. just download20:27
weshay|rovernot https://download20:27
rlandyweshay|rover: I was thinking of just removing redhat.com from everywhere20:27
weshay|roverthis seems right..20:28
weshay|roverdir_path: '{{ artcl_collect_dir }}/undercloud/etc/yum.repos.d/'20:28
weshay|roverrlandy, aye.. re:redhat.com20:28
rlandyweshay|rover; the whole dictionary things works20:28
rlandyif we wanted to add more strings20:28
rlandybut for now I think I will just try scan /var/log, /etc and home/zuul for redhat.com and be done with it20:29
weshay|roverrlandy, if that works, just do it.. I wouldn't spin on this too long20:29
rlandyweshay|rover: I didn;t intend to :(20:31
weshay|roverrlandy, it's fine.. hasn't been long20:31
*** holser has joined #oooq20:32
weshay|roverrlandy, qq.. for you.. if I want to pass a variable to one of the pre.yamls.. I can just do that in the zuul def right?20:33
rlandyweshay|rover: ack20:34
rlandyshould work20:34
rlandychanged the string to reg expression20:38
rlandywill see if that works20:39
weshay|roveris it still proper to when: var|bool20:48
weshay|roverrlandy, ^20:49
weshay|roverzbr, ^20:49
rlandyweshay|rover: in what context?20:49
rlandyidk that had changed - I still use it20:50
weshay|roverhttps://review.opendev.org/69222320:50
weshay|roverzbr, ^20:50
rlandyzbr's linters should stop you if you are wrong20:50
rlandy|bool is still in our code base ... https://github.com/openstack/tripleo-quickstart-extras/search?q=bool&unscoped_q=bool20:51
rlandywoohoo - first overcloud deploy pass on train - env C20:56
weshay|rovernice20:56
rlandyok - got some os_tempest vars to fix20:59
rlandy BadRequestException: 400: Client Error for url: https://10.9.121.218:13696/v2.0/subnets.json,21:00
rlandygetting there21:00
*** panda|pto has quit IRC21:14
*** panda has joined #oooq21:18
*** holser has quit IRC21:34
*** TrevorV has quit IRC21:36
rlandyhmm - ok that was the wrong os_tempest change21:41
*** holser has joined #oooq21:44
*** dtrainor has quit IRC22:29
*** holser has quit IRC22:42
*** rlandy has quit IRC23:00
*** slaweq has quit IRC23:05
*** slaweq has joined #oooq23:11
*** slaweq has quit IRC23:16
*** EmilienM has quit IRC23:47
*** EmilienM has joined #oooq23:49

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!