14:00:22 <shardy> #startmeeting tripleo
14:00:28 <openstack> Meeting started Tue Aug  9 14:00:22 2016 UTC and is due to finish in 60 minutes.  The chair is shardy. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:00:29 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:00:31 <openstack> The meeting name has been set to 'tripleo'
14:00:34 <shardy> #topic rollcall
14:00:38 <shardy> Hi all, who's around?
14:00:40 <adarazs> o/
14:00:40 <_coolsvap_> o/
14:00:41 <rhallisey> hello
14:00:41 <karthiks> HI
14:00:44 <jtomasek> o/
14:00:47 <beagles> o/
14:00:47 <SumitNaiksatam> hi
14:00:48 <hrybacki> o/
14:00:49 <shadower> heya
14:00:50 <akrivoka> hi \o
14:00:52 <cdearborn> o/
14:00:54 <mandre> hi
14:00:59 <panda> o/
14:01:00 <marios> o/
14:01:04 <sshnaidm> o/
14:01:17 <ccamacho> HEllo!
14:01:21 <weshay> o/
14:01:34 <jpich> o/
14:01:58 <shardy> #topic agenda
14:01:58 <shardy> * one off agenda items
14:01:58 <shardy> * bugs
14:01:58 <shardy> * Projects releases or stable backports
14:01:58 <shardy> * CI
14:02:01 <shardy> * Specs
14:02:03 <shardy> * open discussion
14:02:05 <bnemec> o/
14:02:07 <EmilienM> o/
14:02:09 <shardy> #link https://etherpad.openstack.org/p/tripleo-meeting-items
14:02:24 <shardy> Seems we've already got several one-off items, please add any remaining ones to the end of the list
14:03:00 <shardy> #topic one-off agenda items
14:03:03 <dprince> hi
14:03:30 <shardy> rkolli added one but doesn't seem to be here?
14:03:34 <SumitNaiksatam> shardy: hi, i am proxing for rkolli
14:03:41 <shardy> SumitNaiksatam: Aha, hi!
14:03:42 <SumitNaiksatam> he was planning to be here
14:03:50 <SumitNaiksatam> meanwhile i will try to fill in
14:04:10 <SumitNaiksatam> so we were looking for guidance on how to make progress with: #link https://review.openstack.org/#/c/352661
14:04:10 <shardy> So, I'm wondering what the related patch is, which requires this new hieradata?
14:04:19 <shardy> #link https://review.openstack.org/#/c/352661
14:04:26 <SumitNaiksatam> shardy: right
14:04:29 <shardy> #link https://review.openstack.org/#/c/351358
14:04:50 <shardy> SumitNaiksatam: basically the issue is, we don't allow feature backports, and we're trying to land everything in the new composable services format to master
14:05:23 <SumitNaiksatam> shardy: so 352661 is essentially a backport of 351358
14:05:30 <shardy> what you're proposing seems low-risk, but for master it's a poor fit for our new architecture, and for mitaka I'm wondering if it implies a follow-up patch that backports a feature?
14:05:57 <SumitNaiksatam> shardy: i believe there are no follow up patches to be upstreamed
14:06:12 <SumitNaiksatam> ah rkolli is here
14:06:16 <shardy> SumitNaiksatam: Ok, so this is to enable an out-of-tree integration via ControllerExtraConfigPre?
14:06:16 <d0ugal> o/
14:06:22 <SumitNaiksatam> shardy: right
14:07:03 <SumitNaiksatam> shardy: we wondering if #link https://review.openstack.org/#/c/352661 can be treated as perhaps a bug and accepted?
14:07:13 <SumitNaiksatam> in stable/mitaka
14:07:21 <shardy> SumitNaiksatam: does the out of tree template contain logic to calculate hieradata, or is it just writing some static hiera keys?
14:07:54 <SumitNaiksatam> shardy: good question, i havent seen that part of the code, so I will have to defer to rkolli
14:08:26 <shardy> SumitNaiksatam: Ok, well I'll add some comments to the review, if it's just some key/value pairs there's a way to do it without this backport
14:08:29 <SumitNaiksatam> shardy: i believe rkolli is having some logistical difficulties with connecting
14:08:40 <SumitNaiksatam> shardy: ah okay
14:08:42 <shardy> SumitNaiksatam: lets chat about it later in #tripleo when I've added those comments
14:08:47 <rkolli> Steven, the out of tree template writes the out the values provided by env file to the hiera data
14:09:14 <shardy> rkolli: Ok, I think you may be able to use the ControllerExtraConfig and ComputeExtraConfig parameters instead
14:09:23 <shardy> it's hard to comment when we can't see the out of tree code
14:09:34 <shardy> if you can post it somewhere it would help provide more specific advice
14:09:54 <rkolli> I will share the out of tree code after this meeting
14:10:13 <shardy> rkolli: Ok, thanks - let's follow up on the review and/or in #tripleo
14:10:16 <SumitNaiksatam> shardy: and as far as #link https://review.openstack.org/#/c/351358 is concerned its a firm no go?
14:10:43 <shardy> SumitNaiksatam: If we have to we probably can do that, but I'd like to discuss the other options first, if that's OK?
14:10:56 <SumitNaiksatam> shardy: good point, thanks!
14:11:47 <shardy> The next item is re SR-IOV and DPDK, who wants to give more details on that one?
14:11:51 <karthiks> i would like to give an update on the SR-IOV and DPDK features.
14:11:52 <karthiks> on SR-IOV, we have uploaded all the patches, and it is pending for reviewer.
14:12:12 <karthiks> we have done the integration of all patches, tested and demoed to QE team on the DPDK automation, execept 2 WIP (depends on openvswitch patch), all the patches (including os-net-config) has been uploaded and it is in review state.
14:13:05 <shardy> karthiks: Ok, thanks for the update, we can push on reviewing those
14:13:18 <shardy> karthiks: was there a dependency for SR-IOV on custom roles?
14:13:19 <karthiks> Thanks....
14:13:43 <shardy> I was wondering how you worked around that, did you just enable it for all compute nodes?
14:14:08 <karthiks> as of now we are considering uniform cluster ....
14:14:08 <skramaja> shardy: yes.. we are enabling for all compute nodes with
14:14:40 <skramaja> assumption as uniform compute nodes..
14:14:47 <shardy> beagles: can you help with reviewing these please?  IIRC you had a lot of good feedback on the specs
14:14:56 <beagles> shardy, absolutely
14:15:01 <shardy> skramaja, karthiks: OK, sounds like a good first step, thanks for the update
14:15:09 <shardy> let's see if we can get it landed for newton-3
14:15:25 <shardy> anything else on this before we move on?
14:15:27 <karthiks> :)
14:15:43 <karthiks> Nothing shardy
14:15:48 <shardy> ok, thanks!
14:16:08 <shardy> jaosorior: you're next re the network range
14:16:21 <shardy> #link https://review.openstack.org/#/c/343443/
14:16:42 <jaosorior> shardy: Right, so we finally set up a deprecation message for the 192.0.2..0/24 range in the undercloud
14:16:51 <jaosorior> but would be nice to come up with some other default.
14:17:07 <jaosorior> so I went ahead and pushed a commit in tripleo-quickstart. So at least an alternative is there as reference.
14:17:10 <jaosorior> What do you guys think?
14:17:36 <shardy> jaosorior: is there any way we can switch the instack-undercloud default, but have it look at the local box so existing deployments aren't modified on update?
14:17:58 <shardy> I'm not opposed to the quickstart fix, but there are a lot of non-quickstart users, so it feels like fixing it in the wrong place?
14:18:39 <bnemec> The problem is we can't know whether a configuration was using the old default.
14:18:42 <pabelanger> o/
14:18:52 <panda> I too prefer to wait for a converged solution
14:19:01 <jaosorior> so I thought that at least new users (using quickstart) could start using the new default. So that would be a good start IMO
14:19:09 <jaosorior> but yeah, bnemec pretty much pointed out the problem
14:19:28 <bnemec> There's a deprecation message telling people not to do it, next cycle we can change it in instack-undercloud.
14:19:37 <shardy> bnemec: couldn't the undercloud install look at br-ctlplane and have a conditional based on that?
14:19:44 <shardy> a special-case for backwards compatibility?
14:20:15 <shardy> bnemec: just changing it is still going to break folks on upgrade isn't it, even if we warned them?
14:20:24 <bnemec> shardy: Maybe
14:20:32 <jaosorior> AFAIK yeah (I tried)
14:20:42 <jaosorior> but it's been a while
14:20:42 <bnemec> shardy: The warning tells them to explicitly configure 192.0.2.0 if they need to keep using it.
14:20:45 <ayoung> 192.0 is not a legal range to use
14:20:54 <ayoung> its for documentation only.
14:20:56 <bnemec> And yet it works fine.
14:21:00 <bnemec> In most cases.
14:21:02 <ayoung> bnemec, no it does not
14:21:07 <shardy> ayoung: Yeah, we know that, the problem is folks are already using it
14:21:09 <ayoung> bnemec, we would not be discussing it if it did
14:21:22 <ayoung> it works in *some cases.
14:21:34 <ayoung> THe issue is that we don't know what will or won't work
14:21:34 <bnemec> ayoung: If it didn't work we would not be discussing it.  Because we wouldn't have been able to use it for so long.
14:21:37 <ayoung> we need to ask the user
14:21:52 <bnemec> That's not the point anyway.
14:21:57 <bnemec> We're discussing how to change it.
14:22:11 <ayoung> can we go IPv6?
14:22:20 <ayoung> dumb question, but lets throw it out there
14:22:26 <ayoung> it makes the clash a lot less likely
14:22:31 <bnemec> shardy: I can maybe look into the br-ctlplane thing.  It's possible that would work.
14:22:51 <jaosorior> Anyway, here's the proposed range in reference of tripleo-quickstart https://review.openstack.org/#/c/343443/ . If people are OK with that, it would be a good start, and then we can start figuring out how to fix the undercloud to have a different default and respect previous configurations.
14:22:54 <shardy> bnemec: ack, sounds good - we could have the workaround trigger a very noisy warning too I guess
14:23:09 <bnemec> It's probably going to be a little hacky, but that's probably okay.
14:23:26 <bnemec> I don't _think_ a lot of people are going to be using this in production anyway, but I've seen crazier things. :-)
14:23:34 <jaosorior> lol
14:23:45 <ayoung> can we try and get Tripleo to use IPv6 for all purely internal communication?  Is that a well enough tested path that we can at least pursue it?
14:23:53 <gfidente> I think best result would be to make undercloud not change pre-existing deployments and or on upgrade, but switch to a new default for new installs
14:24:05 <shardy> Ok, sounds like we have a plan, folks can comment on the quickstart patch and bnemec will look into an instack-undercloud fix/workaround
14:24:15 <jaosorior> great!
14:24:36 <marios> gfidente: not switching on upgrade implies having a n upgrade specific env file (or similar) to have /specify the old defaults
14:24:39 <panda> awesome
14:24:45 <shardy> ayoung: it's not a well tested path AFAIK
14:24:55 <gfidente> marios, I was thinking about undercloud just *not* applying the changes if pre-configured
14:25:17 <marios> gfidente: k, lets talk more offline then thanks
14:25:31 <ayoung> shardy, so, I think that changing the defaults, while they will work for us, we are basing them on what we have in our organization, whcih is, I suspect, the same organization fro most developers here
14:25:42 <bnemec> Hmm, good point.  Does this affect the net-iso template defaults too?
14:25:53 <bnemec> That's a little scarier.
14:25:56 <shardy> ayoung: it's just a default, but we chose a bad default
14:26:05 <ayoung> shardy, and anything we propose as a default is going to trip someone up.  I would like to make the "query for network ranges" be part of the workflow
14:26:39 <shardy> ayoung: sure, there's probably a few ways we could be smarter about this during the undercloud configuration phase
14:26:51 <shardy> ayoung: want to switch to your Federation update?
14:26:55 <ayoung> shardy, sure
14:26:57 <shardy> (ayoung) Federation PoC Update and lessons learned
14:26:58 <ayoung> this is good news
14:27:18 <ayoung> OK,  so Federation means that we have an external applicaiton (KeyCloak in mycase) that has the user database
14:27:59 <ayoung> we have to configure Keystone to accept redirects from Horizon, and redirect to Keyscloak, user signs in, and then the reverse the redirecets (keycloak to keystone, keystone to horizon)
14:28:11 <ayoung> this has been a journey, and I got a proof of concept to work last night.
14:28:31 <ayoung> I am sure there are things I need to do better, and probably some changes coming to puppet modules
14:28:41 <ayoung> the big lessons learned:
14:29:14 <ayoung> 1.  All of the external facing interfaces need to use TLS.  This includes the Auth URL that Horizon uses to talk to Keystone.  It was not expoesed in the past, but will be with Fedration
14:30:06 <ayoung> 2.  We need to use FQDNs as the normal way to deploy.  A Keystone server, for example, should not thinkg of its ServerName as controller-0  but rather opensack (or whatever the  cloudname is)
14:30:31 <jaosorior> #2 is coming soon :D
14:30:40 <ayoung> Also, huge thanks to many people that helpd out... shardy EmilienM dprince all provided huge degrees of help
14:30:55 <ayoung> a big lesson learned is that developers should work with things in HA configuration
14:31:10 <shardy> ayoung: thanks for the update, good that you got a postive outcome :)
14:31:10 <ayoung> since that is the expected deployemnt, you get something working without HA, it might be meaningless
14:31:19 <ayoung> shardy, blog post with details.
14:31:29 <ayoung> shardy, Oh, and I also got dynamic policy to work to.
14:31:37 <ayoung> It was a good week for Tripleo/Keystone
14:31:48 <shardy> ayoung: Ah, I'm drafting a blog post for you about the DeployArtifacts interface
14:31:50 <jaosorior> nice!
14:31:56 <shardy> did you get that working anyway?
14:32:11 <ayoung> shardy, I did.  I only did Keystone, but it would work for all of the services.
14:32:29 <shardy> ayoung: Ok, well that's good news - I'll post the blog anyway as it's nearly done
14:32:35 <ayoung> shardy, I might want to use the same approach for dpleoying the metadata for Fedraion
14:33:06 <ayoung> its a cluster of files.  The one thing I need to figure out is how to force a yum install of a module required first, or everything will reak
14:33:06 <shardy> Ok, anything else on this before we move on?
14:33:07 <ayoung> brek
14:33:11 * ayoung done
14:33:31 <ccamacho> shardy just added a blueprint draft to check what people think about that idea, we can discuss it later on #tripleo (not getting time from meeting)
14:33:36 <shardy> ccamacho: You're next - the naked overcloud! :)
14:33:49 <ccamacho> not sure if we have enough time
14:33:53 <bnemec> Scandalous!
14:33:54 <shardy> ccamacho: so, this is a followup from last week, where we discussed enabling less services by default?
14:34:03 <ccamacho> yeahp
14:34:24 <shardy> I'm +1 on that, but as we discussed last week, we do need a CI matrix that provides coverage of services not enabled by default
14:34:28 <ccamacho> its the idea in a blueprint
14:34:31 <shardy> or they'll just be constantly broken
14:35:01 <ccamacho> ack, Ill keep elaborating on that
14:35:28 <shardy> ccamacho: re actually enabling them, which I see you mention in the blueprint, see https://review.openstack.org/#/c/330414/
14:35:44 <ccamacho> The next part of the blueprint should be defining this coverage matrix
14:35:50 <shardy> ramishra has heat patches for that, so we'll avoid having to map stuff to OS::Heat::None
14:36:13 <shardy> folks can help by pulling and testing his patches (it's also on my todo list)
14:36:22 <EmilienM> ccamacho: as an FYI, there is this kind of matrix in puppet CI https://github.com/openstack/puppet-openstack-integration#description
14:36:28 <marios> shardy: have a link to how that works (not using OS::Heat::None)
14:36:33 <ccamacho> Ill add a note to my todo
14:36:40 <shardy> https://review.openstack.org/#/c/351092/
14:36:57 <shardy> marios: it's the spec link I just posted, and ^^ is the main patch implementing it
14:37:01 <marios> thanks shardy
14:37:08 <ccamacho> thanks
14:37:19 <shardy> Ok, I think that's all of the one-off items
14:37:30 <shardy> #topic bugs
14:37:50 <bnemec> Can somebody drop a link to ccamacho's spec?
14:37:58 <shardy> https://bugs.launchpad.net/tripleo/?orderby=-id&start=0
14:38:23 <ccamacho> https://blueprints.launchpad.net/tripleo/+spec/deploying-a-naked-overcloud
14:38:35 <shardy> #link https://bugs.launchpad.net/tripleo/+milestone/newton-3
14:38:46 <shardy> Does anyone have specific bugs to raise this week?
14:39:01 <EmilienM> ccamacho: excellent proposal
14:39:09 <shardy> I think other than the sahara issues EmilienM mentioned earlier in #tripleo, the bug backlog looks OK
14:39:32 <shardy> we still have a lot of stuff to land for newton-3 though, so please prioritize reviews to help burn down that list
14:39:33 <EmilienM> shardy: right, and jaosorior has a fix
14:39:52 <EmilienM> shardy: https://review.openstack.org/#/c/352746/
14:39:57 <bnemec> ccamacho: Is there an actual spec?  blueprints are really awkward to review (thus the spec process).
14:40:04 <shardy> EmilienM: Ok, thanks for the update
14:40:06 * bnemec apologizes for being stuck on the previous topic
14:40:20 <ccamacho> bnemec, I'll change it then to a spec
14:40:38 <EmilienM> well, it's a spec-lite
14:41:02 <shardy> #topic Projects releases or stable backports
14:41:24 <shardy> So I didn't have time to propose stable branch releases last week, but I'll do that later today
14:41:42 <shardy> and as mentioned, we're running out of time for newton-3, so please review all-the-things :)
14:41:57 <shardy> anyone have anything else to raise re releases or backports?
14:42:48 <shardy> #topic CI
14:43:01 <EmilienM> shardy: matbu and I are working on upgrade / update jobs
14:43:07 <shardy> So, who can give an update re CI seeing as derekh isn't around?
14:43:21 <shardy> EmilienM: excellent, I saw the ML discussion, sounds good
14:43:28 <pabelanger> last I checked, everything was working in tripleo-test-cloud-rh1
14:43:29 <EmilienM> FYI, we renamed upgrades -> updades jobs
14:43:35 <shardy> the main question I have is how do we do it within the infra timeout
14:43:38 <pabelanger> I also created: http://grafana.openstack.org/dashboard/db/nodepool-tripleo-test-cloud
14:43:49 <pabelanger> so people can visualize what the cloud is doing
14:43:52 <marios> shardy: +1
14:44:03 <shardy> What's the status re RH1, do we have full capacity there now?
14:44:08 <EmilienM> shardy: right now, we're still testing on multinode-nonha (aio overcloud), so i don't think we'll hit it
14:44:14 <shardy> pabelanger: nice! :)
14:44:27 <pabelanger> right now it is setup to launch 50 nodes
14:44:31 <bnemec> shardy: Not completely.  I think we're still being limited to 50 envs on the infra side.
14:44:34 <bnemec> Yeah, that. :-)
14:44:36 <pabelanger> not sure what the actually hardware is
14:44:48 <bnemec> The cloud is configured for up to 80 right now.
14:45:07 <bnemec> That said, I'm a little concerned by the amount of CPU usage on the controller right now.
14:45:09 <pabelanger> I also raised a discussion again about the future of tripleo-test-cloud-rh1, if people would like to get up to speed on it: http://lists.openstack.org/pipermail/openstack-dev/2016-August/101043.html
14:45:29 <pabelanger> generate reaction is in favor of both, moving to 3rd party CI and running more openstack jobs on the hardware
14:46:35 <shardy> pabelanger: thanks for that, I remember there was some heated debate on that topic last time ;)
14:47:05 <pabelanger> shardy: indeed
14:47:08 <shardy> Ok, anything else re CI before we move on?
14:47:45 <pabelanger> I'll continue my topic on the ML
14:47:57 <shardy> pabelanger: ack, sounds good, thanks!
14:48:02 <shardy> #topic Specs
14:48:08 <shardy> #link https://review.openstack.org/#/q/project:openstack/tripleo-specs+status:open
14:48:33 <shardy> getting near to the end of the cycle and there's still some stuff to land there, pls review if you have any free cycles
14:49:10 <dprince> shardy: I posted a new spec for the undercloud heat idea as well
14:49:16 <dprince> shardy: https://review.openstack.org/#/c/352113/
14:49:20 <shardy> dprince: nice, just spotted that
14:49:30 <shardy> I liked the demo, will review the spec
14:49:33 <dprince> perhaps more of an 'O' thing
14:49:45 <shardy> dprince: Yeah, I think so, but good to start the discussion now
14:50:27 <shardy> Related to that, I've already started deferring anything not started in launchpad to Ocata
14:50:59 <shardy> please update the implementation status of your blueprints that are targetted to newton-3, so we can get a clear view of what's going to land and what might slip
14:51:26 <shardy> Anything else re specs to raise?
14:52:10 <shardy> #topic Open Discussion
14:52:47 <shardy> I had one thing - a few folks have been asking me for RFEs related to tripleo, particularly heat features we need or would like
14:52:51 <bswartz> do you guys also own the triple-quickstart project?
14:53:09 <shardy> I'll probably start a ML thread with an etherpad linked so we can track the wishlist there
14:53:09 <bswartz> tripleo-quickstart*
14:53:27 <shardy> bswartz: There are a subset of TripleO folks who maintain it, yes
14:53:41 <bswartz> but it's supposed to be stable, right?
14:54:02 <EmilienM> define: stable
14:54:02 <bswartz> I tried it and had poor luck -- I didn't know if I should try something else or hack away at it
14:54:03 <panda> bswartz: it should be stable. Any issue ?
14:54:24 <panda> bswartz: ask us on #tripleo at the end of the meeting
14:54:34 <bswartz> panda: it fails nondeterministically
14:54:37 <bswartz> okay thanks
14:54:52 <shardy> bswartz: It is supposed to be stable, but it isn't what we run in upstream CI right now
14:55:08 <shardy> yup, lets discuss in #tripleo to figure out the issues, thanks panda!
14:55:24 <shardy> Anyone else have anything to mention?
14:56:29 <shardy> Ok then, if that is all, we can wrap up early and head back to #tripleo
14:56:33 <shardy> thanks everyone!
14:56:34 <EmilienM> thx shardy !
14:56:37 <ccamacho> thanks guys!!!
14:56:41 <shardy> #endmeeting