14:00:22 #startmeeting tripleo 14:00:23 ack thanks for the notice 14:00:23 Meeting started Tue Jan 30 14:00:22 2018 UTC and is due to finish in 60 minutes. The chair is EmilienM. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:00:24 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 14:00:26 The meeting name has been set to 'tripleo' 14:00:34 hello everyone 14:00:37 o/ 14:00:38 EmilienM: you may want to send something to the list as there are a lot more people that can approve patches than just that small group 14:00:38 o/ 14:00:39 o/ 14:00:41 o/ 14:00:41 o/ 14:00:43 o/ 14:00:47 hi 14:00:49 o/ 14:00:50 o/ 14:00:57 o/ 14:01:12 o/ 14:01:25 #topic agenda 14:01:32 o/ 14:01:51 o/ 14:02:03 * Review past action items 14:02:06 * One off agenda items 14:02:08 * Squad status 14:02:08 o/ 14:02:10 * Bugs & Blueprints 14:02:12 * Projects releases or stable backports 14:02:14 * Specs 14:02:16 * Open discussion 14:02:18 EmilienM: ok 14:02:19 o. 14:02:21 o./ 14:02:26 reminder: Anyone can use the #link, #action and #info commands, not just the moderator! 14:02:38 #topic review past action items 14:02:41 o/ 14:02:52 1) team to review https://review.openstack.org/#/q/topic:bug/1744763 so we can move forward with RH1 14:03:17 only https://review.openstack.org/#/c/536461/ remains, and a few patches on puppet-pacemaker 14:03:22 sshnaidm: we can call it done I guess 14:03:28 EmilienM, yeah 14:03:41 sshnaidm: thanks for the work (as usual) 14:03:43 2) team to review multiple arch patches https://review.openstack.org/#/c/528000/ and https://review.openstack.org/#/c/528060/ 14:03:59 patches are still under review 14:04:06 3) EmilienM to release tripleo queens milestone 3 this week 14:04:14 done ^ we released yesterday, wouhouu 14:04:17 o/ 14:04:21 4) EmilienM to release tripleo stable newton/ocata/pike this week 14:04:26 done ^ last week 14:04:35 5) EmilienM to reply to gfidente's FFE 14:04:52 gfidente: I was concerned about CI jobs but I think work has been done 14:05:10 I haven't had time yet this week to catchup on those 14:05:24 o/ 14:05:24 some release notes were missing, I think it was added 14:05:28 o/ 14:05:58 EmilienM well there is scenario004 deploying manila 14:06:03 but it doesn't enable ganesha yet 14:06:11 ok 14:06:21 gfidente: no big deal if it doesn't 14:06:26 at least it doesn't break anything in place 14:06:28 EmilienM well but it can 14:06:39 EmilienM: fyi, we also talked to seb about adding tripleo CI to ceph-ansible 14:06:39 tbarron ^^ what do you think about that? 14:06:48 #topic one off agenda items 14:07:33 fultonj: sounds awesome, you want to talk about it now? 14:07:39 anything we need to discuss? 14:07:47 EmilienM: basically, he agreed but we need to work out how 14:07:53 fultonj: imho you should also sync with pabelanger for the zuul thing 14:07:54 i will reply to your email on list 14:07:58 cool 14:08:08 EmilienM can we discuss that at the PTG 14:08:18 I think we need help with setting up zuul 14:08:19 gfidente: is it in the agenda? if not, add it! 14:08:24 #link https://etherpad.openstack.org/p/tripleo-ptg-rocky 14:08:24 great, thanks 14:08:58 gfidente: please add it 14:09:03 allright 14:09:07 sshnaidm: go ahead 14:09:45 We had jobs in experimental and non-voting pipelines for various projects, but after moving to zuulv3 all of them disappeared. Let's decide where do we want to check patches and how - voting, non-voting, experimental, etc 14:09:55 just reminder how it was before: https://docs.google.com/spreadsheets/d/1bLg-uEGrQXyRZ-FuR6pf1WT4XN0-6MrlfqEShI7xMxg/edit#gid=0 14:10:18 URGENT TRIPLEO TASKS NEED ATTENTION 14:10:18 https://bugs.launchpad.net/tripleo/+bug/1745686 14:10:19 https://bugs.launchpad.net/tripleo/+bug/1746120 14:10:20 Launchpad bug 1745686 in tripleo "[gnocchi-db-sync]: gnocchi-upgrade --config-file /etc/gnocchi/gnocchi.conf --skip-storage --skip-incoming returned 2 " [Critical,In progress] - Assigned to Pradeep Kilambi (pkilambi) 14:10:21 Launchpad bug 1746120 in tripleo "403 downloading puppet-concat" [Critical,Triaged] 14:10:34 after we moved OVB to 3d party I think we can consider adding jobs to nova, neutron and other projects to detect breakages asap 14:10:40 sshnaidm: do people used experimental ovb jobs to be honest? (except maybe jaosorior with the TLS everywhere work?) 14:11:00 sshnaidm: IMHO nova/neutron folks don't care at all about TripleO jobs 14:11:05 at least it's my experience 14:11:19 EmilienM, we have people in tripleo squads that could care about it 14:11:21 EmilienM: only tripleO folks care 14:11:28 so asking them to check experimental something is a loss of time, they won't do it 14:11:29 yeah 14:12:00 check experimental has been regularly used on the heat repo, fwiw 14:12:06 sshnaidm, well.. capacity will have to be considered there 14:12:10 heat, true 14:12:10 EmilienM, these jobs are mostly for tripleo folks of course 14:12:24 We are doing a lot of background work to expand rdo-cloud this year 14:12:26 weshay|ruck, not only ovb, also multinode jobs 14:12:34 sure 14:12:46 EmilienM, It's at least valuable when you fix an issue 14:13:03 EmilienM: I don't think the heat experimental job needs to be ovb though, multinode probably gives us similar coverage 14:13:03 imho this discussion should happen within the CI squad and you folks should come up with a proposal here eventually or on the ML 14:13:14 therve: I don't disagree 14:13:45 shardy: true. I think the most valuable use case where the heat convergence thing (which is now a default anyway in tripleo) 14:14:26 EmilienM: yeah, but I see it regularly used for various large/risky changes, which is great as it probably saves us time debugging promotions 14:14:40 I know not all projects do that, but it'd be good to keep that option available IMO 14:14:54 even if it becomes rdo check experimental or whatever 14:15:12 shardy: I agree 14:15:25 I think Heat, Mistral and Ironic are good candidates 14:15:34 +1 14:15:35 all the 3 projects collaborate a lot with tripleo (in a friendly way) 14:15:46 Heat: multinode is enough probably 14:15:55 but Mistral + Ironic: probably fs001 (ovb-ha) 14:16:13 I think it's a good start to have that... 14:16:16 sshnaidm: wdyt? 14:16:21 EmilienM, ok, and what about nova and neutron? 14:16:44 yeah I mean, you can add it 14:17:01 EmilienM, ok 14:17:03 anything we *add* w/ regards to rdo-cloud will need to be reviewed for capacity atm 14:17:13 yes, that ^ 14:17:14 we have a large gap atm w/ upgrades 14:17:15 of course 14:17:28 so priority is heat mistral ironic 14:17:31 the rest can wait imho 14:17:35 Carlos Camacho proposed openstack/tripleo-common master: Add extra environment files for the update https://review.openstack.org/526214 14:17:36 so if we could as a group, think about priorities.. that would be a good exercise 14:17:53 weshay|ruck, but experimental won't run a lot, and they have lower priority too 14:17:59 aye 14:18:03 sshnaidm: I think we can move on 14:18:08 sshnaidm: you have a second topic 14:18:29 EmilienM, yeah, I wanted to raise a question about voting in ovb jobs - do we want it or not 14:18:54 we'll consider it voting when it'll have an acceptable rate of success 14:18:55 but considering stability of rdo cloud I'd like not to raise it atm 14:18:58 which is I think not the case right now 14:19:09 sshnaidm: so let's move on? 14:19:10 so let's assume we do have stability 14:19:12 EmilienM, yep, exactly 14:19:17 and raise the question 14:19:22 sure, let's make it voting 14:19:22 I don't think third-party jobs can be voting? Or has that changed since zuulv3? 14:19:24 w/ stability do we vote in some way 14:19:39 shardy: they can 14:19:41 it can verify +1 / -1 14:19:53 but not to gate 14:19:55 right 14:19:57 yeah exactly 14:19:57 yeah by voting I mean part of the gate pipeline so they can block merges 14:20:13 shardy, no, it can't 14:20:20 a patch which wouldn't pass the 2 ovb jobs would never land 14:20:34 shardy, but +1/-1 would increase the visibility 14:20:39 functionally a voting job which isn't in the gate pipeline is only voting by convention, but +1 on paying attention to them as we always have done :) 14:20:44 ah ok so it wouldn't block? 14:21:12 well.. I think a core could change the -1 to a +1 14:21:14 that is not exactly true 14:21:14 I think our group has been a fairly good job at monitoring ovb jobs 14:21:16 sshnaidm: agreed, the ovb jobs have always been +1/-1 in the check pipeline, so keeping things that way sounds good to me 14:21:41 there is nothing to stop a job in the check pipeline from voting 14:21:57 EmilienM, I think it can block on check level, but not to run on gates 14:21:58 Carlos Camacho proposed openstack/tripleo-common master: Add extra environment files for the update https://review.openstack.org/526214 14:22:02 trown|rover, right? ^^ 14:22:05 ya 14:22:08 exactly 14:22:15 there are risks in doing that... 14:22:18 gates are upstream only 14:22:36 but there is nothing that prevents configuring third-party jobs to vote in check 14:22:38 Ok I'm confused, historically a core could +A a patch with failing OVB jobs, and it would gate and potentially merge 14:22:51 shardy: ya, they dont actually vote 14:22:54 I'm unclear if we're just saying lets still do that, or something different 14:23:16 shardy: the proposal is to actually make them vote in the same way that multinode jobs vote 14:23:30 ie, you cant +A a patch with failing multinode jobs and have it go to gate 14:23:34 trown|rover: OK thanks for clarifying 14:24:07 so that would prevent a ovb-failing patch to move in gate 14:24:13 the danger is that you can break the gates because you are not actually testing things in the same way in the gate queue... but I think that is less of an issue for us 14:24:30 it is more of an issue when coordinating changes between projects 14:25:39 so, I think we have an agreement to make them voting when they are stable, right? 14:25:41 I guess we can wait a bit more on that decision 14:25:49 sshnaidm: yes for me 14:26:05 ok, then that's it from me 14:26:06 Sounds good provided we can prove/improve the reliability and runtime 14:26:16 trown|rover: thanks for the 3rd party 101 :) 14:26:28 haha 14:26:31 shardy, runtime is pretty good actually, not longer that multinode jobs 14:26:34 sshnaidm: you good? 14:26:39 EmilienM, yep 14:26:47 moving on 14:26:54 someone wrote "TripleO-CI office hours directly after the TripleO meeting" but there is no name 14:27:01 EmilienM: we are kind of "3rd party" in name only... we dont need to follow the same rules because it is actually our project CI 14:27:06 sshnaidm: Ok I was thinking of the old ovb jobs which were hitting timeouts 14:27:20 we control the timeout in rdo sf 14:27:24 trown|rover: we're 1st party :P 14:27:39 who wrote about office hours? ^ 14:27:43 I did 14:27:44 ice cream party 14:28:04 weshay|ruck: go ahead 14:28:32 Just announcing the ci team is available to anyone immediately after this meeting for office hours 14:28:50 I thought you were available 24/7 14:28:55 ahahha 14:28:57 ahahah 14:29:04 I think you were talking to our answering machine 14:29:06 :( disappointed 14:29:15 i have been called worse 14:29:23 heh 14:29:30 ok moving on 14:29:32 thanks weshay|ruck 14:29:43 #topic Squad status 14:29:50 #link https://etherpad.openstack.org/p/tripleo-ci-squad-meeting 14:29:52 #link https://etherpad.openstack.org/p/tripleo-upgrade-squad-status 14:29:54 #link https://etherpad.openstack.org/p/tripleo-containers-squad-status 14:29:56 #link https://etherpad.openstack.org/p/tripleo-integration-squad-status 14:29:58 #link https://etherpad.openstack.org/p/tripleo-ui-cli-squad-status 14:30:00 #link https://etherpad.openstack.org/p/tripleo-validations-squad-status 14:30:02 #link https://etherpad.openstack.org/p/tripleo-networking-squad-status 14:30:04 #link https://etherpad.openstack.org/p/tripleo-workflows-squad-status 14:30:09 please update the etherpad, I'll use the notes today to write the next Weekly Owl 14:30:36 * EmilienM quickly checking etherpads 14:31:31 chem` / marios / matbu ^ please update the upgrade squads status if you can 14:31:57 gchamoul^ please update validations etherpad 14:32:20 otherwise looks good, thanks all 14:32:25 #topic bugs & blueprints 14:32:29 FYI.. the jobs running in rdo-cloud are captured in Graphana https://review.rdoproject.org/grafana/dashboard/db/tripleo-ci?orgId=1 14:32:33 have a look 14:32:43 more to come there 14:33:06 weshay|ruck: awesome! I would maybe send it over openstack-dev as well for those who can't see the meeting now 14:33:12 I will 14:33:23 maybe in the weekly owl 14:33:28 yeah I will 14:33:31 lolz 14:33:34 I mean 14:33:38 your fellow reporter will 14:33:39 whoot 14:33:41 are the numbers seconds? 14:33:53 do we have any particular bug or blueprint to discuss this week? 14:33:54 slagle, lightyears actually 14:34:03 note bugs moved to queens-rc1 milestone 14:34:22 EmilienM: ack sorry had call conflict 14:34:25 at this point of the release, we want to focus on High & Critical bugs only for RC1 and move Low and Medium to rocky-1 14:34:32 marios: no problem 14:35:07 there are still 11 blueprints open, I'll take a look again this week but any help on updating them is very welcome 14:35:15 mathieu bultel proposed openstack/python-tripleoclient master: Add major update implementation https://review.openstack.org/526213 14:35:29 marios: actually you're here, I have questions 14:35:42 marios: what's the status on https://blueprints.launchpad.net/tripleo/+spec/fast-forward-upgrades and https://blueprints.launchpad.net/tripleo/+spec/major-upgrade-workflow ? 14:35:55 do you think they'll make queens upstream GA release? 14:36:15 EmilienM: ffu not likely upgrades "maybe" but probably not (as usual ;) ) 14:36:15 or will they slip to rocky-1? 14:36:25 ok I see 14:36:35 I'll keep the blueprint for queens at maximum and at last time move them to rocky 14:36:36 Jose Luis Franco proposed openstack-infra/tripleo-ci master: WIP: Upgrade UC and OC using tripleo-upgrade role https://review.openstack.org/515643 14:36:40 EmilienM: yeah.. for q upgrades we have this wip for progress https://review.openstack.org/#/c/535859/ 14:36:51 EmilienM: (docs patch for queens cli) 14:36:51 marios: ack 14:36:57 EmilienM: i'll add to status for sqaud 14:37:25 EmilienM: for ffu chem has more updates i think (re the ci job for example) 14:37:28 EmilienM: we'll update 14:37:40 jaosorior, thrash: you still have open blueprints for Queens, please update them today (the one from Ryan but he's not online atm) 14:37:44 #link https://launchpad.net/tripleo/+milestone/queens-rc1 14:37:49 EmilienM: marios yeah, worknig on ffu job 14:38:13 marios, chem : no worries, things will probably get backported to stable/queens 14:38:21 and as usual, we'll wait for the very last time to branch 14:38:49 anything else about blueprints and bugs ? 14:39:05 Athlan-Guyot sofer proposed openstack/tripleo-quickstart master: [WIP] New featureset for fast forward upgrade testing. https://review.openstack.org/525686 14:39:17 mathieu bultel proposed openstack/python-tripleoclient master: Add major update implementation https://review.openstack.org/526213 14:39:20 #topic projects releases or stable backports 14:39:28 so like I said, we released queens milestone 3 14:39:35 I just wanted to thank everyone involved in that work 14:39:47 look my email, we exploded the numbers, we fixed more bugs than any other cycle in the past 14:39:58 and the release isn't done yet! 14:40:12 now, RC1 is coming 14:40:27 our target is Feb 26 - Mar 02 14:40:37 for a final GA on Mar 12 - Mar 16 14:40:40 #link https://releases.openstack.org/queens/schedule.html 14:41:09 so we have ~1 month to finish queens 14:41:35 focus is on finishing the blueprints with FFE granted, stabilize CI, and fix Critical / High bugs 14:41:53 also focus is on planning Rocky (please propose topics on https://etherpad.openstack.org/p/tripleo-ptg-rocky) 14:42:03 any question about release management? 14:42:14 Damien Ciabrini proposed openstack/puppet-tripleo master: Fix Redis TLS setup, including replication traffic https://review.openstack.org/527398 14:42:47 #topic specs 14:42:47 EmilienM: just want to know a general plan on all the abandoned patches 14:42:49 #link https://review.openstack.org/#/q/project:openstack/tripleo-specs+status:open 14:42:52 Carlos Camacho proposed openstack/tripleo-common master: Add extra environment files for the update https://review.openstack.org/526214 14:42:54 sheesh didn't even let me finish typing! 14:42:58 :) 14:43:01 jrist: which ones? 14:43:07 um 14:43:12 https://review.openstack.org/#/c/515045/ +2 needs +2 (passes CI) 14:43:12 https://review.openstack.org/#/c/528282/ +2 needs +2 (passes CI) 14:43:23 you said you'd take care of them 14:43:26 but just curious on the general plan 14:43:28 did you read my comment? 14:43:31 aye 14:43:35 well, gate is having a lot of resets 14:43:46 and we need https://review.openstack.org/538346 landed ASAP 14:43:49 anything we can do to help? 14:43:51 so until https://review.openstack.org/538346 isn't landed, I won't restore them 14:44:03 jrist: make jobs more stable and reliable 14:44:06 k 14:45:02 I haven't followed everything (still early for me) but it seems like infra had some issues 14:45:04 with Ubuntu nodes 14:45:13 also weshay|ruck told me we have some multinode timeouts 14:45:20 Jose Luis Franco proposed openstack/tripleo-heat-templates stable/pike: Add validation task in docker services [Mixed-1] https://review.openstack.org/516627 14:45:22 :( ya 14:45:24 so yeah, help is welcom ehere 14:45:30 Jose Luis Franco proposed openstack/tripleo-heat-templates stable/pike: Add validation task in docker services [Heat] https://review.openstack.org/516613 14:45:39 Jose Luis Franco proposed openstack/tripleo-heat-templates stable/pike: Add validation task in docker services [Nova] https://review.openstack.org/516628 14:45:42 ehere, for your ebiz needs 14:45:47 Jose Luis Franco proposed openstack/tripleo-heat-templates stable/pike: Add validation task in docker services [Octavia] https://review.openstack.org/517266 14:45:48 538346 is now first in line in the gate 14:45:53 hopefully it pass and will land 14:45:54 Jose Luis Franco proposed openstack/tripleo-heat-templates stable/pike: Add validation task in docker services [Sahara] https://review.openstack.org/517267 14:46:04 #topic open discussion 14:46:18 time for any question of any feedback 14:46:40 how does tripleo.org get updated? 14:46:49 seems out of date w/ the upstream openstack docs 14:46:51 if have some cycles please check https://review.openstack.org/#/q/status:open+topic:bp/undercloud-backup-restore 14:46:55 thanks! 14:47:09 #action team to review https://review.openstack.org/#/q/status:open+topic:bp/undercloud-backup-restore 14:47:36 http://tripleo.org/ != https://docs.openstack.org/tripleo-docs/latest/ 14:47:56 alright, I'll give 13 min back to everyone and close the meeting, have a good one 14:47:58 #endmeeting