16:01:25 <adrian_otto> #startmeeting containers
16:01:26 <openstack> Meeting started Tue Mar 15 16:01:25 2016 UTC and is due to finish in 60 minutes.  The chair is adrian_otto. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:01:27 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:01:29 <openstack> The meeting name has been set to 'containers'
16:01:35 <adrian_otto> #link https://wiki.openstack.org/wiki/Meetings/Containers#Agenda_for_2016-03-15_1600_UTC Our Agenda
16:01:40 <adrian_otto> #topic Roll Call
16:01:42 <adrian_otto> Adrian Otto
16:01:45 <strigazi> o/ Spyros Trigazis
16:01:50 <Kennan> o/
16:01:51 <yolanda> o/
16:01:56 <hongbin> o/
16:02:02 <rpothier> o/
16:02:04 <Tango> Ton Ngo
16:02:08 <dane_leblanc> o/
16:02:08 <muralia_> murali allada
16:02:10 <jward> John Ward
16:02:10 <eghobo> o/
16:02:14 <rochaporto> o/
16:02:35 <adrian_otto> Hello strigazi Kennan yolanda hongbin rpothier Tango dane_leblanc muralia_ jward eghobo and rochaporto
16:02:38 <wznoinsk> Waldemar Znoinski
16:02:45 <muralia_> hi everyone
16:02:48 <adrian_otto> hello wznoinsk
16:02:55 <levi_b> Levi Blackstone
16:03:28 <adrian_otto> okay, let's begin
16:03:30 <suro-patz> o/
16:03:35 <adrian_otto> #topic Announcements
16:03:41 <adrian_otto> 1) We are now in Feature Freeze and String Freeze
16:04:18 <adrian_otto> this means that today I will branch for mitaka, and any code that needs to land for mitaka from this point forward needs an FFE (feature freeze exception).
16:04:26 <adrian_otto> I can grant FFE's.
16:04:42 <adrian_otto> bugs may be merged against the branch without an FFE
16:05:06 <thomasem> o/
16:05:17 <Tango> What if the bug involves some string changes?
16:05:18 <adrian_otto> if you submit a review against the stable/mitaka branch, and it merges, you'll need to submit the same patch to master as well
16:05:48 <adrian_otto> Tango, ideally we would file a tech debt ticket to handle the string changes in the next release tag
16:05:57 <Tango> ok
16:06:07 <adrian_otto> but if adjusting strings is central to solving the bug, then we can use an FFE for that.
16:06:23 <adrian_otto> the rules are there to help us, but if it makes sense to bend them, we will.
16:06:45 <adrian_otto> any other questions on this announcement?
16:07:05 <adrian_otto> ok, next...
16:07:07 <adrian_otto> 2) Welcome Shu Muto to magnum-ui-core!
16:07:27 <adrian_otto> Shu Moto was added to the group this morning. Welcome!!
16:07:46 <Kennan> welcome to Shu Moto
16:07:47 <muralia_> congrats
16:08:00 <Tango> Welcome Shu!
16:08:08 <suro-patz> Welcome Shu!
16:08:12 <adrian_otto> Dims asked to be dropped from the group, which I will do today as well.
16:08:34 <adrian_otto> looks like that's done now
16:08:57 <adrian_otto> any other announcements from team members?
16:09:31 <adrian_otto> #topic Review Action Items
16:09:39 <adrian_otto> (referring to my notes)
16:10:04 <adrian_otto> 1) adrian_otto to start an ML thread about Magnum functional gate tests taking forever to execute in OpenStack CI.
16:10:16 <adrian_otto> this may be something I can touch base with yolanda about
16:10:24 <adrian_otto> I did not start the ML thread, sorry.
16:10:27 <yolanda> adrian_otto, seems i joined in the right day :)
16:10:28 <adrian_otto> #action adrian_otto to start an ML thread about Magnum functional gate tests taking forever to execute in OpenStack CI.
16:10:36 <juggler> o/
16:10:45 <adrian_otto> so yolanda, real quick while we are on the subject
16:10:52 <adrian_otto> the OVH vms are super slow
16:11:04 <adrian_otto> and gates that used to complete in 20 minutes now take 2 hours
16:11:09 <adrian_otto> and frequently time out
16:11:09 <yolanda> IO slowness?
16:11:16 <adrian_otto> yes, exacty.
16:11:23 <yolanda> ok i can report it
16:11:30 <adrian_otto> the other cloud providers work okay
16:11:43 <adrian_otto> part of the problem is that we have added more func tests
16:11:51 <adrian_otto> but part of the problem is slow hardware
16:12:11 <adrian_otto> we have been landing patches to eliminate extra work from the gate tests
16:12:39 <yolanda> adrian_otto, do you have some links to the ovh failures?
16:12:41 <adrian_otto> to try to keep things working, but it's been a real disappointment since the HPE servers were lost from the mix.
16:12:54 <adrian_otto> we can get that to you, but I don't have it at my fingertips
16:13:34 <yolanda> we got complains about IO slowness for vexxhost last week as well
16:13:42 <adrian_otto> ok, let's advance to the next agenda item
16:13:51 <adrian_otto> thanks yolanda!!
16:14:20 <adrian_otto> #topic Bay node OS support in Magnum
16:14:40 <adrian_otto> so, before we get too deep here, I want to thank everyone who has been participating on the ML on this topic
16:15:12 <adrian_otto> this is a complex subject, and it's been tricky to get everyone to gel on the compromise for the approach to balance choice with complexity.
16:15:43 <adrian_otto> and as of last night, I think we are in a good place. I will do my best to characterize the approach, and consider input if this neeeds to be tweaked
16:16:04 <adrian_otto> we want to make Magnum more modular by moving COE specific code into drivers
16:16:38 <adrian_otto> each driver can support a combination of bay node os + dependencies (docker, k8s, etc), templates, and parameter logic
16:17:01 <adrian_otto> the detail that each _driver_ only supports one OS was an initial friction point
16:17:25 <adrian_otto> once I explained that we can have multiple drivers per COE type, that seemed to resolve the friction point
16:17:35 <adrian_otto> we will have a single driver identified as the default
16:17:52 <adrian_otto> alternates can be proposed, and initially placed into a contrib directory
16:18:25 <adrian_otto> we can have in-tree alternates once they meet criteria that we agree on (adequate testing, and an active maintainer, for example)
16:18:32 <adrian_otto> make sense so far?
16:18:37 <adrian_otto> any concerns with this approach?
16:18:54 <hongbin> I think it sounds good
16:18:59 <suro-patz> +1
16:19:02 <Kennan> looks ok
16:19:14 <Tango> sounds good
16:19:17 <Kennan> better with high-level bp about them
16:19:19 <eghobo> +1
16:19:32 <adrian_otto> keep in mind that the default driver (os) for a given COE should be chosen based on which one works best with that COE
16:20:07 <adrian_otto> so say at come point we have a "CoreOS" COE, it would obviously make sense to use CoreOS in that driver.
16:20:21 <adrian_otto> the default driver for Mesos/Marathon woudl be Ubuntu, you get the idea.
16:21:13 <Kennan> yes, we need some spec or clear bp to specify them. @adrian_otto
16:21:16 <adrian_otto> ok, to Kennan's point, we may have someone working ona  BP now
16:21:20 <muralia_> is anyone working on this blueprint currently?
16:21:25 <adrian_otto> muralia_: do you remember who will be drafting that?
16:21:38 <adrian_otto> oh, that answers that question
16:21:39 <muralia_> jamie from rackspace agreed to work on it
16:21:45 <adrian_otto> ok.
16:21:50 <muralia_> but i was wondering if anyone got started already
16:22:09 <muralia_> looks like its a no
16:22:11 <adrian_otto> I have not seen anyone tag me for approval on one
16:22:17 <adrian_otto> so that may still be forthcoming
16:22:27 <adrian_otto> this is something we will definitely vote on an spec for
16:22:43 <Kennan> I did not think anyone start it now
16:22:51 <adrian_otto> so reviewers will have lots of opportunity to shape the plan for implementation
16:23:07 <muralia_> yes
16:23:51 <adrian_otto> thanks everyone for your attention and collaboration on this topic.
16:24:27 <adrian_otto> #topic Blueprint Review
16:24:36 <dims> adrian_otto : thanks!
16:24:37 <adrian_otto> since we are in FF, I'm tempted to skip this
16:24:47 <adrian_otto> dims: :-)
16:25:02 <yolanda> adrian_otto, i added a topic in the agenda for fedora dib images, but i did at last minute
16:25:09 <adrian_otto> the OpenStack summit in Austin is in 6 weeks
16:25:42 <adrian_otto> #topic Fedora atomic image generation: how to validate
16:25:48 <adrian_otto> #link https://blueprints.launchpad.net/magnum/+spec/fedora-atomic-image-build
16:25:53 <adrian_otto> #link https://review.openstack.org/287167
16:25:57 <adrian_otto> thanks yolanda
16:25:59 <adrian_otto> proceed
16:26:14 <yolanda> so i've been working on that blueprint for some time, and i have a working image for fedora atomic, on f23
16:26:30 <yolanda> i was wondering what should be the best way to validate it, and ensure that it will be good for testing
16:26:46 <yolanda> so far i tried to run devstack, and use that as magnum_guest_image_url
16:27:02 <hongbin> yolanda: I would suggest to run it againest the quickstart guide, to see if everything is working
16:27:19 <yolanda> hongbin well, things boot, i can spin up bays, etc
16:27:25 <Tango> One problem is that we don't have enough functional tests to cover all the use cases.
16:27:38 <hongbin> yolanda: Also, I think the next step is to submit a patch and run it in gate
16:27:40 <yolanda> when i tried deploying kubernetes i get an error about an atomic.io key not existing
16:27:44 <Tango> Often the cluster comes up but is not functional
16:27:52 <yolanda> yep, that's my fear
16:28:08 <Kennan> Tango also know lots of built images
16:28:16 <yolanda> hongbin, changes for diskimage-builder are still on review, so i even cannot create a job for publishing
16:28:17 <Tango> I can help with checking out the image
16:28:29 <yolanda> but i'd like to check that the process is ok, and the image is valid, before merging that
16:28:53 <hongbin> yolanda: I think you can do it in two steps
16:28:59 <yolanda> Tango, thx.. so https://review.openstack.org/287167 is the link for the diskimage-builder change
16:29:18 <Tango> I will take a look and try it out
16:29:21 <yolanda> and i also have generated qcow that i can upload somewhere
16:29:26 <hongbin> yolanda: 1. Build an image and upload it to fedora people, then submit a patch to verify that image works.
16:29:44 <hongbin> yolanda: 2. Introduce the dib in the patch
16:30:18 <yolanda> hongbin, so submit a patch for magnum , to use my generated fedora image?
16:30:31 <hongbin> yolanda: yes
16:30:44 <yolanda> ok i can do it, send some experimental job?
16:30:49 <Kennan> yolanda: is your image is same as Fedora official atomic image ? Or can be customized with any installed software
16:31:09 <yolanda> Kennan it's based on fedora official ostree. It could be customized if we had the need
16:31:10 <hongbin> yolanda: You can use the existing job to test it
16:31:19 <Tango> I think before #1, we should do a fair amount of local testing, otherwise debugging at the gate is difficult.
16:31:38 <hongbin> Tango: +1
16:31:42 <adrian_otto> +1
16:31:46 <Kennan> +1
16:32:04 <Kennan> yolanda: what's the difference between your image and official atmic image ?
16:32:21 <yolanda> so it's based on https://dl.fedoraproject.org/pub/fedora/linux/atomic/23/
16:32:36 <yolanda> the difference is that is generated with diskimage-builder, so we can generate it from infra regularly
16:32:59 <Kennan> OK. Thanks
16:33:01 <yolanda> it can be uploaded on a regular basis, and we can host in our mirrors
16:33:28 <Tango> I guess we can also customize the version of particular software, like Kubernetes?
16:33:28 <yolanda> also if there was some need to use a custom tree, it could be done
16:33:38 <yolanda> yes, there  can be that possibility
16:33:40 <adrian_otto> should we expect that to accelleterate our gate test times?
16:33:54 <yolanda> adrian_otto, that is the idea, and reduce failures
16:34:03 <yolanda> downloading content from an external source is a point of failure
16:34:08 <adrian_otto> that would be awesome
16:34:26 <Kennan> yolanda: one point
16:34:43 <Kennan> most time functional test is image itself, not speed(download time)
16:34:52 <yolanda> i started that to kick the integration with shade. I noticed that i had to increase shade timeout because of that external download, so that was not well received for shade people
16:35:44 <yolanda> yep, that won't help with that. Only to reduce the download time and, to have own images
16:36:14 <adrian_otto> sorry, but what is shade?
16:36:41 <yolanda> adrian_otto, so http://git.openstack.org/cgit/openstack-infra/shade
16:36:58 <yolanda> is a wrapper for openstack calls mostly... we use that in nodepool
16:37:06 <adrian_otto> oh, thanks
16:37:07 <yolanda> and we use that on ansible for openstack as well
16:37:25 <yolanda> so enabling that integration with magnum, can open some doors to start using ansible + magnum, or nodepool + magnum
16:38:23 <adrian_otto> yolanda: can you explain some more about the nodepool + magnum concept?
16:38:23 <Tango> +1
16:39:00 <yolanda> adrian_otto, this was some topic discussed on Tokyo, but at the moment there is no progress on the short time
16:39:19 <yolanda> the idea was to allow nodepool to work with containers at some point
16:39:44 <yolanda> the thing that we agreed is to start first with that shade+magnum integration, the same as there is integration with nova, or with ironic
16:40:15 <adrian_otto> ok, I do have a memory of this
16:40:17 <yolanda> once it's there, i hope that the idea of nodepool can be moved forward again, but that was the first tep needed
16:40:39 <yolanda> yep, we got some conversations with team about that
16:40:44 <adrian_otto> I see. How can the magnum team help?
16:41:27 <yolanda> so the main problem i had this time is the lack of experience with fedora atomic, see the better way of creating this integration with diskimage-builder
16:41:48 <yolanda> at the moment i'd say that helping in validation of that concept, to test that the images we create are valid, will be awesome
16:42:29 <adrian_otto> not to speak for all of us, but I think that helping with that is in our mutual interest
16:42:56 <adrian_otto> as we make more efficient use of the openstack CI infra, and we get faster gate tests as a result.
16:42:58 <yolanda> i'll be glad to have some collaboration, that will be fantastic. I had the blueprint a bit stopped but i resumed in the last month
16:44:10 <yolanda> so i can start uploading the image to fedora people and share link here to help on validation?
16:44:14 <adrian_otto> ok, what woudl be good next steps to take?
16:44:34 <adrian_otto> that sounds good
16:44:48 <yolanda> also if i can get reviews on the diskimage-builder patch, to be sure that i'm not missing anything, will be good
16:45:01 <adrian_otto> let me know if you'd like to schedule a breakout meeting on irc to collaborate on that with interested magnum contributors
16:45:04 <imtiaz> Hi all. I am joining this meeting for the very first time. I met adrian and couple of other core members at a recent meetup organized by Lida.
16:45:08 <imtiaz> Lisa*
16:45:18 <yolanda> so far i was talking with some people in RH, and the guidance i got is to start running ostree, tweak grub and tweak docker storage
16:45:25 <adrian_otto> thanks yolanda
16:45:31 <adrian_otto> #topic Open Discussion
16:45:35 <imtiaz> @yolanda - what type of image validation do you have in mind?
16:45:38 <adrian_otto> imtiaz: hi there
16:45:38 <yolanda> adrian_otto, thanks for your attention
16:46:01 <yolanda> imtiaz, i'd like to be sure that it passes all magnum tests,and that this is a valid image for the platform
16:46:40 <yolanda> i have tested it, it boots, it has the required packages , but i don't have the expertise to dig deeper
16:46:41 <imtiaz> Do we have a framework to launch a VM with that image?
16:47:01 <adrian_otto> imtiaz: yes, devstack.
16:47:38 <adrian_otto> it's the behavior of the Magnum bay that's in question
16:48:03 <strigazi> adrian_otto: I have some updates on magnum installation guide
16:48:11 <adrian_otto> possible configuration settings of Docker (my guess is related to image sign verification)
16:48:22 <adrian_otto> strigazi: great! tell us more
16:48:42 <strigazi> Magnum installation guide is ready for rdo. What about other distributions? There are magnum-packages *only* for rdo. Should I add a note saying, install from source?
16:49:07 <imtiaz> Adrian: o.k. I was thinking about an automated gate that could validate the image.
16:49:43 <adrian_otto> yes, if we have a guide for istalling from source (not devstack) that would help with other OpenStack environments
16:49:56 <strigazi> ok, 2: Magnum installation guide is ready for rdo. What about other distributions? There are magnum-packages *only* for rdo. Should I add a note saying, install from source?
16:50:14 <strigazi> ok, 2. Can this be for mitaka? https://blueprints.launchpad.net/openstack-manuals/+spec/add-magnum-to-the-installation-guide , its the bp for the installation-guide under openstack-manuals
16:50:41 <hongbin> yolanda: ping me in IRC if you need help for the image validation
16:51:10 <yolanda> hongbin thanks. I'll try uploading to public fedorapeople, and share the link
16:51:23 <Tango> strigazi: Would the guide be applicable to Ubuntu?
16:51:41 <strigazi> Tango: from source, yes
16:51:58 <Tango> strigazi: thanks
16:52:02 <adrian_otto> we do get continued interest in Ubuntu setup, so that would definitely be valuable.
16:52:36 <strigazi> I'm an  ubuntu user at home, so that interests me as well
16:53:03 <suro-patz> my sincere request to conclude the review for the spec on async mode of container operations - https://review.openstack.org/#/c/275003/  - it has gone through multiple rounds of iteration, review and rework. IMHO, it would be good to merge at this point and take it forward from there: adrian_otto
16:53:21 <adrian_otto> strigazi: the BP you referenced above is not a magnum one. It's an openstack-manuals one, so we'll need to find that PTL and ask there. It might be too deep into FF to get it into Mitaka.
16:53:41 <strigazi> finally, this https://review.openstack.org/#/c/289994/ needs review, its the spec file for magnum-installation-guide and need to be link to the bp
16:53:43 <adrian_otto> I know that thingee expressed an interest in it thought
16:54:43 <adrian_otto> wow, you made a spec for that. Nice, strigazi. Usually for things of that complexity we skip that step.
16:54:54 <adrian_otto> but it's welcome nonetheless.
16:55:30 <strigazi> A core member of openstack-manuals asked about it
16:55:42 <adrian_otto> Thanks strigazi for all your work on that important item. We really appreciate it.
16:56:01 <Tango> +1
16:56:07 <strigazi> :)
16:56:11 <adrian_otto> strigazi: oh, I see, they have a spec process in place for manuals
16:56:16 <adrian_otto> I'll vote ont aht
16:56:20 <adrian_otto> thanks for the pointer.
16:56:36 <adrian_otto> ok, we have just a few minutes remaining.
16:56:44 <adrian_otto> any other topics before we wrap up?
16:56:44 <suro-patz> my sincere request to conclude the review for the spec on async mode of container operations - https://review.openstack.org/#/c/275003/  - it has gone through multiple rounds of iteration, review and rework. IMHO, it would be good to merge at this point and take it forward from there: adrian_otto
16:57:26 <adrian_otto> suro-patz: acknowledged. Let's vote on that review, and identify any remaining concerns.
16:57:36 <suro-patz> thanks adrian_otto!
16:57:38 <adrian_otto> I'm ready to move forward on it
16:58:17 <strigazi> I'd like to draw your attention to this bp: https://blueprints.launchpad.net/magnum/+spec/allow-user-softwareconfig
16:58:30 <strigazi> Will it be approved?
16:58:51 <adrian_otto> strigazi: I will review and comment on it
16:59:03 <strigazi> adrian_otto: thanks
16:59:55 <adrian_otto> Thanks everyone for attending today. Our next meeting will be Tuesday 2016-03-22 at 1600 UTC. See you then!
17:00:05 <adrian_otto> #endmeeting