14:00:54 #startmeeting tripleo 14:00:54 Meeting started Tue Oct 11 14:00:54 2016 UTC and is due to finish in 60 minutes. The chair is EmilienM. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:00:56 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 14:00:58 The meeting name has been set to 'tripleo' 14:00:59 hello 14:01:05 hi 14:01:08 o/ 14:01:08 hi 14:01:09 howdy 14:01:09 \m/ 14:01:10 o/ 14:01:12 o/ 14:01:12 hi 14:01:20 hey 14:01:22 o/ 14:01:25 o/ 14:01:29 o/ 14:01:31 can you guys tell dtrainor is from way out west 14:01:35 o/ 14:01:36 o/ 14:01:36 o/ 14:01:37 o/ 14:01:40 hi 14:01:41 * dtrainor tips his hat 14:01:43 o/ 14:01:44 hey! o/ 14:01:51 0/ 14:01:58 o/ 14:01:59 o/ 14:02:00 jrist: I thought howdy was a Texas thing 14:02:09 west/south 14:02:14 hi 14:02:19 better known as "southwest" 14:02:36 lol 14:02:58 welcome everyone! 14:03:03 #topic agenda 14:03:04 * CI 14:03:06 * bugs 14:03:08 * projects releases or stable backports 14:03:10 * specs 14:03:12 * one off agenda items 14:03:14 * open discussion 14:03:22 during the meeting, feel free to add more off items https://etherpad.openstack.org/p/tripleo-meeting-items 14:03:23 o/ 14:03:36 #topic CI 14:04:26 for CI i have 2 topics 14:04:28 it seems like derekh and panda worked on fixing an issue with OVB jobs, I saw some successful runs 14:04:58 hi \o 14:05:00 this morning, I also saw some transient issues on multinode jobs with something trying to reach EPEL servers (wtf?) 14:05:08 hi 14:05:12 it seems fixed now, (I have no idea why yet) 14:05:27 1/ looks like the mitaka periodic is broken for a while (september 30), idk how to solve, but i can help 14:05:36 so I guess you can use 'recheck' if some jobs were failing 14:05:57 matbu: yes, I started to investigate it last week and found 2 problems 14:06:14 matbu: one was networking-cisco package that were depending on python-neutron-tests, now fixed 14:06:26 2/ since https://review.openstack.org/383825 is landed i switch the upgrade from newton to master, i would like to propose 2 jobs one mitaka to newton and newton to master (but idk if mitaka to newton make sense actually ?) 14:06:28 another was unknown and requires more investigation. 14:06:54 EmilienM: those make the undercloud install failling ? 14:07:04 matbu: maybe could you sync with slagle who is doing similar work for undercloud AFIK 14:07:19 matbu: yes, eventually 14:07:22 EmilienM: ack 14:07:28 matbu: it was making neutron-db-manage failing 14:07:50 but I've been off 3 days, I might need to catch-up on this Mitaka failure. 14:08:14 matbu: i have a patch to make undercloud upgrades tested: https://review.openstack.org/381286 14:08:15 EmilienM: k, afaik it start to failing on OC deployment and then UC 14:08:28 slagle: thx /me looks 14:08:38 matbu: if you wanted to add periodic jobs to make sure it's tested daily, i think that's a good idea 14:08:52 so EmilienM slagle does it make sense to maintain a mitaka to newton job ? or should we switch to newton to master ? 14:09:04 +1 for a periodic job, reporting results to some ML maybe 14:09:13 matbu: we need both imo 14:09:27 my patch adds support for both (or fixes, rather) 14:09:27 slagle: EmilienM ack 14:10:14 do we have any question or useful information about CI ? 14:10:59 ok, let's go ahead then 14:11:05 #topic bugs 14:11:24 so we still have 3 bugs in progress for RC3 https://launchpad.net/tripleo/+milestone/newton-rc3 14:12:03 EmilienM: there is a new one 14:12:05 https://bugs.launchpad.net/tripleo/+bug/1632330 14:12:05 Launchpad bug 1632330 in tripleo "Undercloud upgrade fails on the yum update we run before undercloud is re installed" [High,In progress] - Assigned to Marios Andreou (marios-b) 14:12:06 hi i filed this today https://bugs.launchpad.net/tripleo/+bug/1632330 with proposed https://review.openstack.org/#/c/385012/1 - especially wanted to hear from bnemec about landing this as workaround until we get the proper undercloud upgrade moved to instack-undercloud as per the spec 14:12:40 matbu: did you add info on the openstack-heat issue you saw there /me refresh 14:12:58 EmilienM: and another https://bugs.launchpad.net/tripleo/+bug/1632232 14:12:59 Launchpad bug 1632232 in tripleo "M/N upgrade: Cinder upgrade fails to converge." [Undecided,New] 14:12:59 marios: not yet, i'm collecting info before running the UC upgrade 14:13:06 matbu: ack 14:13:24 chem: could you self-triage it when filing a bug? 14:13:32 chem: High - newton-rc3 14:13:40 EmilienM: ack, oki 14:13:46 marios: I thought we agreed to document this workaround 14:13:50 marios: why do we need the workaround? 14:13:58 slagle: do you remember this discussion wrt https://review.openstack.org/#/c/385012/ ? 14:14:03 slagle: hanging service on yum update 14:14:09 yes, we documented it 14:14:27 so if you're hitting that issue, the docs aren't being followed 14:15:06 http://tripleo.org/installation/installation.html#updating-undercloud-components 14:15:17 we can add the support to instack-undercloud if we want to remove the documentation, but i dont think a workaround in tripleoclient is the right thing 14:15:21 I agree we need to find a longterm solution 14:15:45 slagle: OK... there was a docs review from bnemec as part of the spec i thought you were referring to that 14:15:48 slagle: yes, +1 for i-u 14:16:04 Yeah, this should eventually live in i-u, but the docs look sane right now. 14:16:10 slagle: EmilienM if it is known that we should stop services before running the undercloud upgrade (docs are fine i missed that) then works for us 14:16:29 marios: yea, i'd suggest just using steps 3 thru 5 from http://docs.openstack.org/developer/tripleo-docs/installation/installation.html#updating-undercloud-components 14:16:35 chem: triage done. 14:16:42 slagle: EmilienM we just have the same in osp10 docs for upgrade 14:16:50 then we dont need any workarounds, or any last minutes fixes as of now 14:17:03 slagle: ack wfm matbu chem... so we can rely on docs 14:17:14 chem: also please use newton-backport-potential tag in launchpad 14:17:46 re: bugs, We've seen this creep up when using the ui https://bugs.launchpad.net/tripleo/+bug/1632007 , to quote jtomasek "when GUI shows the parameter value, it displays it inside input and when this happens the string gets mangled". he picked it up but jistr mentioned that this might be a bigger issue being that we don't preserve data types properly 14:17:46 Launchpad bug 1632007 in tripleo "Failed deployment due to Hiera resolution of keystone::wsgi::apache::workers" [High,Triaged] - Assigned to Jiri Tomasek (jtomasek) 14:18:16 EmilienM: oki, (couldn't set the milestone, neither the importance), adding the tag 14:18:34 Bug I saw yesterday when updating: https://bugs.launchpad.net/tripleo/+bug/1631976 14:18:35 Launchpad bug 1631976 in tripleo "Updating fails with heatclient auth url attribute error" [Undecided,New] 14:18:37 I am on it now, I'll mark that bug with newton-backport-potential tag 14:19:35 I'd like to bring up https://bugs.launchpad.net/tripleo/+bug/1626128 as the fix we are delivering is changing command options eg api for the newton release. I think change is ok if we document it. 14:19:35 Launchpad bug 1626128 in tripleo "openstack overcloud update stack is broken" [Critical,In progress] - Assigned to Brad P. Crochet (brad-9) 14:19:35 rhallisey: same, could you please make self triage of bugs 14:19:48 it makes me think I might need to send a reminder again, about self triage in launchpad :( 14:19:49 EmilienM, I don't have perms 14:20:04 rhallisey: ask for it :) 14:20:07 can you add me as a driver pls 14:20:10 I'll see if I'm adming 14:20:14 admin* 14:20:30 #action EmilienM to add rhallisey and chem into launchpad tripleo group 14:20:31 EmilienM: can you add me as a bug supervisor pls 14:20:43 chem: i'll try :) 14:20:49 hehe 14:22:10 social_: what is your question? 14:22:31 EmilienM: are we ok with changing the overcloud update the way that --templates and -e will be ignored? 14:23:04 social_: if it's documented, why not 14:23:28 and it looks like it is, looking at the patch 14:23:42 EmilienM: yeah got updated today, for me this is now OK 14:23:44 https://review.openstack.org/#/c/381899/12/tripleoclient/v1/overcloud_update.py 14:23:46 ok 14:23:59 anything else about bugs? 14:24:32 please target new bugs to ocata-1 and use the newton-backport-potential tag if needed 14:25:14 #topic Projects releases or stable backports 14:25:24 I have some updates about release management 14:25:31 I sent an email on Friday but let me repeat it here 14:25:48 dhellmann sent an email about projects that use trailing release 14:26:12 so it seems that we have more time to produce a last RC (was scheduled on Friday) 14:27:01 I think we are in a good shape to propose RC3 by next week and tag final release just before the Summit 14:27:34 so I'll propose RC3 early next week 14:27:53 and the final release will be 20 Oct 14:28:14 so if you have patches that we need in Newton, please use the gerrit topic: 14:28:16 https://review.openstack.org/#/q/topic:tripleo/rc3+status:open 14:28:27 also, please take care of your backports 14:28:40 if a patch merges in master, and we need the fix in Newton, please backport it to stable/newton 14:28:59 I saw a lot of patches merged in master, targeted for newton but not backported 14:29:07 sorry to repeat but it doesn't seem clear to everyone 14:29:44 today I'll go through the patches and do the backports myself if I still see them not backported 14:30:23 everything that is related to upgrades *needs* to be backported 14:30:29 chem, marios, bandini ^ 14:30:35 EmilienM: ack 14:30:41 EmilienM: ack 14:30:46 probably would be a good idea to come up with rules around who cherry-picks 14:31:03 mwhahaha: well, my hope is that patches owners would do it 14:31:03 like it should be the core's responsibility or the submitters (probably the core's) 14:31:07 I have no outstanding patches that need cherry-picking atm 14:31:10 this is a bad time to have connection issues, sorry 14:31:42 i think the cores would be better because they would hopefully have a better understanding if it should be in the previous release 14:31:43 I would think submitters responsibility as well 14:31:48 but we should codify it 14:31:48 mwhahaha: it would be easier if submitters would do it, it's hard to catchup with all patches 14:31:50 mwhahaha: i feel like it should be the review author but anyone can propose a cherrypick really i don't think we need rules... important thing is not to land into earlier branch (stable/newton before master) 14:32:07 well since it's not happening, this is why i say come up with a standard rule 14:32:13 mwhahaha: i.e. you can't stop people proposing reviews, like cherrypicks... the rules are for when we land them 14:32:22 ya, +1 to having a standard 14:32:42 (i don't actually care who it is) we just need to make sure that expectation is clear 14:32:53 mwhahaha: it means the cores would have to follow all merged patches and backport them? 14:32:55 marios, not so much a rule about who CAN submit them... more who is RESPONSIBLE for submitting them 14:32:55 other projects it's the cores 14:33:06 I would prefer to let authors to do it and cores to vote 14:33:08 EmilienM: the one who +A's could do it 14:33:27 +1 for having the authors submit it 14:33:28 indeed, that's an option 14:33:47 i'm generally not seeing a whole bunch of people paying attention to reviews so to me i think it encourages more thought into the patches 14:34:06 but that's my opinion it it, we can take it off line 14:34:09 we do already have the rule that if a core submits a backport it only requires 1 +2, so in that sense it could make sense to have cores responsible 14:34:33 I'm fine either way 14:34:50 i feel it's the author's responsibility tbh 14:34:52 yeah I would also back a general 'rule' for patch authors to also be responsible for proposing to stable... but again, we can't enforce that absolutely... if author doesn't care if her patch makes it to stable, just wants in on master for whatever reason 14:35:06 just as it is is the author's responsibility to get the backport to pass CI 14:35:25 how will they even know it's been backported if someone does it for them 14:35:46 slagle: they have gerrit notif 14:35:56 slagle: it is on the review as a cherrypick or should be (I mean gerrit ui)... personally i also notify the person if i've done that, e.g. is somethign urgent 14:35:58 when someone backports a patch where you are author, Gerrit adds you as reviewer 14:36:07 (in case you read Gerrit emails :-)) 14:36:14 ok, so they know when the master patch merges then, so why does someone else need to propose the backport? 14:36:36 slagle: maybe timezone/still early for them and you can propose and merge it now 14:36:40 you have to understand people may fix bugs but not know it needs a backport 14:36:58 to be more community friendly i think the cores need to be more proactive 14:37:11 rather than assume the reviewer is going to know 14:37:25 mwhahaha: +1 only takes a second to ping someone and say you just backported foo 14:37:28 maybe we can find a middleground here: cores to propose the backports and teach authors to do it themselves by communicating (IRC, etc) 14:37:52 so longterm authors will do it themselves 14:37:58 EmilienM: also can we bear in mind there is literally a button you click on the review and it will stable/branch review for you assuming no merge conflict 14:38:01 just sayin 14:38:37 just something to keep in mind, i think we need to be more proactive on the bugs/backports/etc 14:38:43 it's honestly everyones responsibility 14:38:47 * shardy is late, apologies 14:38:54 so don't just assume it's someone elses problem 14:38:58 mwhahaha: you ok to start documenting it in tripleo-specs? we have a doc for that, i'll chat with you after meeting 14:39:01 meeting conflict 14:39:15 k 14:39:16 shardy: np 14:39:45 #action mwhahaha & EmilienM to improve backport doc in tripleo-specs and get feedback from other cores 14:40:16 #action EmilienM to cherry-pick missing backports for newton rc3 14:40:23 anyone can help ^ 14:40:32 I have a last update about release management 14:40:53 this week-end I updated https://launchpad.net/tripleo/ocata with the deadlines for Ocata: a good FYI if you wonder about scheduling 14:41:05 basically one milestone per month 14:41:20 starting from november, then december, then january and February is final :-) 14:41:26 so really short cycle this time. 14:42:05 any question or feedback about release management and backports? 14:42:39 EmilienM: I can help with backports 14:42:49 shardy: cool, let's sync post meeting then, thanks 14:43:08 #topic specs 14:43:15 we're running short time 14:43:17 #link https://review.openstack.org/#/q/project:openstack/tripleo-specs+status:open 14:44:10 we have some specs open, most of them are for ocata, I would ask authors to make sure specs are ready for review, so discussion can happen prior Summit 14:44:49 do we have anything about specs this week? 14:45:02 #topic off items 14:45:05 #link https://etherpad.openstack.org/p/tripleo-meeting-items 14:45:19 dtrainor: you want to start? 14:45:27 Sure, thank you. I'm in the process of writing a spec which should be done today, but the jest of it is, make an Undercloud SSL-enabled whenever enable_ui = true. This enables haproxy, which will then be used to proxy connections for the API endpoint services that UI needs to talk with. These endpoints would listen on an external, routable IP address/interface as defined by a new undercloud.conf variable. 14:45:54 This is something we've been trying to get working in a better manner for UI for a while and based on discussion on the list and testing various setups, this seems to be the most viable and least intrusive option 14:46:22 #link https://blueprints.launchpad.net/tripleo/+spec/enable-communication-ui-undercloud 14:46:32 (can I use link?) 14:46:35 dtrainor: isn't that functionality already available if you set the undercloud_public_vip variable from the undercloud.conf? You could set that to a public interface already and that's what HAProxy will use. 14:46:53 dtrainor: yes you can use link 14:47:34 jaosorior, I'm not familiar with the finer details of undercloud_public_vip but i would love to look in to this. If undercloud_public_vip is configured, does this have any impact on any Overcloud components that expect to access it from the ctrlplane? 14:47:40 I'm not sure "Essential" is the right priority 14:47:52 maybe "High" ? 14:48:09 without something like this, the ui will not work. jrist re: priority? 14:48:12 dtrainor: the overcloud should be using the internal/admin endpoints, so it should be fine 14:48:22 jaosorior, i'll test that, thank you 14:49:08 dtrainor: not a big deal, we can rediscuss it offline 14:49:30 dtrainor: is it something we can do during ocata-1 ? 14:49:40 I set the milestone, but feel free to propose a new one 14:50:12 we would still need a server in haproxy to proxy traffic for ui to enable ssl, even if undercloud_public_vip opens these services up as we need them to be opened, i'd still like the option of making this work if that doesn't achieve what we're looking for (i'll of cours edo more testing) 14:50:27 i'm certain it is yes 14:50:33 ok 14:51:05 dtrainor: does it requires a design change? if yes, maybe a tripleo-specs would be useful 14:51:17 dtrainor: alright, lets see if that does the trick. Anyway, feel free to add me as a reviewer if you come up with some patches. Even if they're WIP 14:51:35 EmilienM, I'm in the process of writing a spec 14:51:41 dtrainor: nice. Thanks 14:51:45 ok, let's move on 14:51:49 shadower: go ahead 14:51:51 if undercloud_public_vip solves this then i'd love for it to be in newton 14:51:55 (sorry, wanted to add that) 14:52:03 We've added the tripleo-validations repo in Newton 14:52:08 thanks for the time 14:52:14 right now it's just me and mandre who know it well 14:52:16 dtrainor: np, sorry we're running out of time 14:52:22 completely understood, thank you 14:52:26 but we need other 8cores to review (as in +2/A) to get things merged 14:52:27 dtrainor: undercloud_public_vip is already in newton :) 14:52:32 shadower: I was wondering, would it be useful to put this doc in the repo itself? 14:52:42 like other python projects, in docs/ dir 14:52:49 EmilienM: I think so, will do taht 14:52:53 so the doc in question: 14:52:57 jaosorior, yes it is, but i want to test this and make sure it will achieve what i was looking to do as defined in the blueprint (and whatever the spec ends up being) 14:52:58 that would be documented on docs.openstack.org/developer/ 14:53:05 #link http://lists.openstack.org/pipermail/openstack-dev/2016-October/105364.html 14:53:22 shadower: or in tripleo-docs 14:53:24 this ^ is a list of things you can look for when reviewing a tripleo-validations patch 14:53:46 would appreciate people having a look and reviewing tripleo-validations more 14:53:52 or in http://docs.openstack.org/developer/tripleo-validations/ 14:53:59 actually, http://docs.openstack.org/developer/tripleo-validations/ would be the perfect place I think 14:54:00 shadower: you can also add the repo here: http://tripleo.org/reviews.html 14:54:13 slagle: ah, will do thanks 14:54:16 shadower: via scripts/website/tripleo-reviewday.yaml in tripleo-ci 14:54:21 EmilienM: I'd like to send a few patches for tripleo-docs shortly to talk about validations etc 14:54:42 good 14:54:46 more docs! 14:54:46 and yeah add more specific stuff to the tripleo-validations docs 14:54:49 YEP 14:54:53 next week or so 14:55:13 in the meantime, people can get more familiar with the reviews by checking out the email 14:55:18 that's it from me 14:55:42 excellent, thanks 14:55:49 ok so my turn, I have 2 infos: 14:55:57 #link https://etherpad.openstack.org/p/ocata-tripleo 14:56:33 this is another reminder for our group to review the topics and prepare the agenda 14:57:08 I sent emails to topic chairs about it, and make sure we have an outcome for each session 14:57:40 for each session, we want to define the problem to solve and propose / discuss eventual solutions 14:57:59 that's what we need to prepare before the Summit 14:58:26 the other topic is PTG 14:58:29 EmilienM: I missed that email for "Growing the Team", or is that one good since we went through it together? 14:58:42 #link https://www.openstack.org/ptg/ 14:58:45 trown: it's good :) 14:58:52 cool 14:58:56 trown: iirc, I just sent an email on containers & upgrades 14:59:08 just a reminder to marios / dprince etc to update etherpad 14:59:20 ack thanks EmilienM already had first pass 14:59:27 CI topic still needs a chair as well 14:59:29 so I need to tell OpenStack foundation if tripleo needs PTG sessions 14:59:45 oh nevermind, I see slagle volunteered 14:59:49 ok we're running overtime 14:59:49 i can chair or co-chair the CI one if anyone else is eager 14:59:55 slagle: ++ 15:00:00 which i gather no one is since it was TBD :) 15:00:01 I need to close the meeting 15:00:15 open discussion on #tripleo, I'll stay available if anything 15:00:24 thanks everyone ! 15:00:29 thanks! 15:00:33 #endmeeting