16:00:44 <vkozhukalov> #startmeeting Fuel
16:00:45 <openstack> Meeting started Thu Jun  5 16:00:44 2014 UTC and is due to finish in 60 minutes.  The chair is vkozhukalov. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:46 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:00:48 <openstack> The meeting name has been set to 'fuel'
16:00:58 <agordeev> o/
16:00:58 <vkozhukalov> greeting everyone
16:01:04 <mihgen> hi folks
16:01:08 <vkozhukalov> who is here?
16:01:20 <vkozhukalov> and as always here is agenda
16:01:32 <mihgen> I hear xarses, but don't see him here
16:01:33 <vkozhukalov> #link https://etherpad.openstack.org/p/fuel-weekly-meeting-agenda
16:01:35 <mihgen> let me ping him)
16:02:16 <vkozhukalov> are we actually ready? if yes topic is
16:02:21 <vkozhukalov> #topic current initiatives in Fuel team
16:02:33 <mihgen> ok let me debrief on it
16:02:45 <mihgen> after 5.0 release, we are working on multiple things now
16:02:50 <vkozhukalov> we are letting -)
16:03:14 <mihgen> the first priority is actually to nail down remaining issues, angdraug is responsible for it, will let him speak later
16:03:25 <mihgen> remaining issues for 4.1.1
16:03:46 <mihgen> it's based on havana, and users need a stable release with fixed HA and many more other bug fixes
16:04:27 <mihgen> from other initiatives, the largest part of Fuel is working on 5.1 design.
16:04:52 <mihgen> small group on enabling upgrades for 5.0 -> 5.0.1, evgeniyl is the lead
16:05:16 <mihgen> Folks, I'm in a big worry for bugs in master, let me send couple of links
16:05:38 <mihgen> #link http://goo.gl/RE6fbm
16:05:49 <mihgen> this is high and critical bugs for 5.1
16:06:16 <aglarendil> mihgen: we have planned bug squashing tomorrow
16:06:20 <mihgen> 105 of them, and #link goo.gl/qPK4XY 306 in total..
16:06:30 <mihgen> aglarendil: ok, good to hear that
16:06:48 <mihgen> I was about to propose bug squashing day on Monday or some other day next week
16:07:13 <vkozhukalov> #chair vkozhukalov
16:07:14 <openstack> Current chairs: vkozhukalov
16:07:19 <aglarendil> we are gonna do this for an hour and do at least some bug sorting
16:07:24 <vkozhukalov> fogot -)
16:07:29 <mihgen> should you first go with the folks triaging first and then decide on whether we need squashing or not
16:07:48 <aglarendil> let's consider it as triaging & squashing
16:08:21 <mihgen> Ok. The only assigned to bugs is angdraug now as far as I know, after he finishes 4.1.1 activities
16:08:36 <mihgen> (in fuel-library) do we have anyone else to start working on it?
16:08:44 <mihgen> fixing bugs I mena
16:09:05 <aglarendil> let's triage them today and understand which resources we need
16:09:57 <aglarendil> *tomorrow
16:09:59 <mihgen> aglarendil: ok. sounds good. will wait an update on it. Don't forget to mark bugs as low-hanging-fruit, we may have someone helping out
16:10:15 <mihgen> aglarendil: Firday is today in CA ;)
16:10:30 <aglarendil> :) you don't say)
16:10:51 <mihgen> do we have any other more or less large activities besides?
16:11:24 <aglarendil> let's look into agenda, I guess
16:11:30 <mihgen> vkozhukalov: I'm ready for switching into more detailed discussion
16:11:36 <mihgen> on design and etc.
16:11:37 <vkozhukalov> ok
16:12:02 <vkozhukalov> #topic Design for 5.1
16:12:18 <vkozhukalov> #link https://review.openstack.org/#/q/project:stackforge/fuel-specs,n,z
16:12:23 <mihgen> I don't see too many specs there....
16:12:29 <vkozhukalov> here is a link to specs for 5.1
16:12:48 <vkozhukalov> design week is almost over and only 4 specs are on review
16:13:15 <vkozhukalov> ok, let's move one by one and discuss what we need to add
16:13:19 <aglarendil> vkozhukalov: mihgen: we need to convert a lot of stuff into rst format and some stuff still needs polishing
16:13:33 <aglarendil> mihgen: it will take some time for specs to be completed.
16:13:39 <mihgen> let's start proposing changes
16:13:39 <vkozhukalov> #topic image based provisioning
16:13:43 <mihgen> if it doesn't pass CI
16:13:52 <vkozhukalov> #link https://review.openstack.org/#/c/95575/
16:14:05 <mihgen> even* if doesn't pass CI
16:14:11 <vkozhukalov> there are four people who plused this spec
16:14:28 <vkozhukalov> frankly we already started to work on implementation
16:14:53 <mihgen> vkozhukalov: nice. we need +2th there too. I don't see anyone from QA there ..
16:15:04 <mihgen> angdraug: aglarendil we need to take a look there
16:15:05 <vkozhukalov> if it looks ok, i ask nurla set her plus
16:15:15 <mihgen> vkozhukalov: yep, please
16:15:40 <aglarendil> mihgen: yep, we will do spec reviewing for this feature
16:15:45 <vkozhukalov> next one is
16:15:46 <angdraug> aye
16:15:50 <vkozhukalov> #link docker building improvements
16:16:05 <vkozhukalov> it is about speeding up docker building process
16:16:16 <mihgen> mattymo: do we have any progress here?
16:16:18 <vkozhukalov> mattymo: do you have any comments&
16:16:49 <vkozhukalov> #link https://review.openstack.org/#/c/97204/
16:17:10 <vkozhukalov> mattymo: around?
16:17:17 <mihgen> looks like nope..
16:17:20 <mihgen> let's move on
16:17:59 <vkozhukalov> ok, as far as i remember his suggestion is to try to use amazon service for speeding up building docker images
16:18:08 <aglarendil> mihgen:
16:18:16 <aglarendil> I can provide some update
16:18:21 <mihgen> wow amazon service??
16:18:26 <mihgen> aglarendil: please go ahead
16:18:45 <vkozhukalov> mihgen: or amazon tool
16:18:46 <aglarendil> research of docker registry showed that it saves only 45 seconds
16:18:46 <angdraug> no mention of amazon in the spec
16:19:05 <aglarendil> mihgen: in comparison with consequent loading
16:19:21 <vkozhukalov> angdraug: maybe i am wrong
16:19:21 <mihgen> aglarendil: did we identify where is the bottleneck?
16:19:22 <aglarendil> so we are looking only towards shrinking the base image for docker centos
16:19:43 <mihgen> ok. wondering if new kernel can help a lot..
16:19:45 <aglarendil> mihgen: we thought it was in the consequent loading of images
16:20:09 <aglarendil> mihgen: looks like no. also, mattymo crashed his machine several times
16:20:18 <aglarendil> mihgen: so I am not sure we can use docker registry
16:20:33 <aglarendil> mihgen: thus we are thinking of making docker base image as thin as possible
16:20:40 <mihgen> interesting..
16:20:43 <aglarendil> mihgen: and thus save more than 45 seconds on unpacking
16:20:53 <mihgen> ok. how much then? 60 sec ?)
16:21:01 <xarses> mihgen: mattmo was saying that the new kernel should help performance
16:21:10 <aglarendil> we need to resarch this
16:21:34 <aglarendil> actually it is only a big deal for virtual environments
16:21:54 <vkozhukalov> ok, as far as nobody has enough detailed view my suggestion is to move on
16:21:55 <aglarendil> modern hardware servers unpack our docker archive easily
16:21:59 <mihgen> ok. where is the bottleneck again?
16:22:03 <mihgen> cpu, disk io?
16:22:20 <aglarendil> obviously, it is cpu in case of virtual env.
16:22:24 <aglarendil> the bigger the image is
16:22:30 <mihgen> vkozhukalov: yep.. .let's move on and ask mattymo later, we really need to fix it to speed up dev process
16:22:32 <aglarendil> the slower the unpacking on venvs
16:22:40 <mihgen> aglarendil: ok, I see
16:22:57 <vkozhukalov> #topic access control on master node
16:23:07 <aglarendil> I do not see salmon here
16:23:11 <vkozhukalov> #link https://review.openstack.org/#/c/96429/
16:23:13 <aglarendil> let's poke him
16:23:28 <vkozhukalov> i saw him several minutes ago
16:23:36 <vkozhukalov> in fuel-dev channel
16:23:41 <salmon_> i'm here
16:23:58 <vkozhukalov> salmon_: any comments?
16:24:33 <salmon_> so, I think blueprint is in good shape now. I got many comments from UI guy and from fuel-library guy
16:24:37 <mihgen> salmon_: how that comes that you have security feature, and security impact is none in your spec?)
16:24:38 <salmon_> and also from QA
16:24:55 <vkozhukalov> please if are you going to type a long message put ? sign first
16:24:58 <mattymo> mihgen, sorry I should have left the office several minutes ago. Need to catch a flight
16:24:59 <mihgen> salmon_: once I get a chance, I have a few important things to say too..
16:25:05 <salmon_> vkozhukalov: ok
16:25:15 <mattymo> mihgen, I have 3 items to report on improving docker progress quickly:
16:25:19 <salmon_> mihgen: yeah, I'm waiting for your comments
16:25:35 <mattymo> 1 - using CentOS's new base minimal image reduces the base docker container size from 300mb to 98mb.
16:25:49 <mattymo> 2 - I reduced the # of layers in the containers by combining many of the bash commands
16:25:58 <mihgen> salmon_: not now. i'll do it in design doc or in ML. Thanks for working on this feature
16:26:12 <mihgen> salmon_: can you provide a short status update on it?
16:26:17 <mihgen> where wea re now with it ?
16:26:27 <mattymo> 3 - docker registry is not speeding things up with docker-registry-0.6.6. The difference in load time is negligible. Fedora just built test packages for 0.7.1. I will test those next week.
16:26:34 <aglarendil> let's give a word to mattymo as he is writing on mobile, I guess
16:26:53 <aglarendil> then switch back to authentication
16:27:00 <mattymo> I reduced build ISO time from 24m to 19m, 201mb all docker containers combined, and deploy time of just lrzuntar + docker load reduced from 13mins to 8mins
16:27:13 <mattymo> we will get much better performance if we can get traction on Fuel Master on 3.10 kernel
16:27:28 <vkozhukalov> aglarendil: he has a word
16:27:29 <mattymo> ok I'm leaving.
16:27:30 <mihgen> mattymo: sounds good
16:27:38 <aglarendil> mattymo: awesome
16:27:48 <mihgen> thanks dude. looks like kernel 3.10 is still in the picture
16:28:14 <mihgen> mattymo: did you test 3.10 on master?
16:28:38 <angdraug> any thoughts about upgrading fuel master to ubuntu 14.04?
16:28:44 <xarses> mattymo: so, a ubuntu master?
16:29:01 <mihgen> angdraug: not now please
16:29:03 <vkozhukalov> salmon_: are you going to give a short status on your spec or you are done?
16:29:09 <mihgen> it's gonna kill us ;)
16:29:10 <salmon_> ?
16:29:54 <salmon_> all reported concerns are resolved now. I saw mail on ML from mihgen that he  have some comments and I'm waiting for them.
16:30:09 <mihgen> salmon_: did you start any implementation?
16:30:16 <aglarendil> salmon_: I have a question on basic Nginx auth
16:30:26 <mihgen> vkramskikh: are you aware of what salmon_ is working on, any help from UI team ?
16:30:41 <vkramskikh> mihgen: there is working UI part: https://review.openstack.org/#/c/97555/
16:30:41 <salmon_> mihgen: no, but vkramskikh already have implementation for UI part
16:30:49 <vkramskikh> it is not finished yet but it works
16:30:57 <salmon_> aglarendil: what question ?
16:31:03 <mihgen> salmon_: actually my biggest thing I wonna say about it is that we basically have only 2 iterations in 5.1
16:31:15 <mihgen> so I want to see something simple first, and then we can extend
16:31:21 <aglarendil> salmon_: I saw some details on it in spec. but it looks like we gonna completely implement authentication using keystone
16:31:36 <aglarendil> salmon_: there is no nginx authentication for keystone, I guess
16:31:39 <mihgen> if we start with complex, you won't be able to land code before FF
16:31:41 <vkramskikh> there won't be any basic auth
16:31:42 <salmon_> mihgen: this is why we added Stages. Stage I is really simple
16:31:58 <mihgen> #link https://wiki.openstack.org/wiki/Fuel/5.1_Release_Schedule this is proposed release schedule
16:32:07 <aglarendil> vkramskikh: thx, that's what I wanted to clarify
16:32:23 <salmon_> aglarendil: no nginx support, bassic auth is really really ugly
16:32:27 <mihgen> salmon_: ok, I'll get back to you with more details in ML
16:32:34 <vkozhukalov> salmon_: i agree
16:32:34 <salmon_> mihgen: cool
16:32:46 <vkozhukalov> moving on
16:32:46 <aglarendil> salmon_: ok, thx
16:33:01 <vkozhukalov> #topic  galera improvements
16:33:14 <vkozhukalov> #link https://review.openstack.org/#/c/97191/
16:33:27 <mihgen> vkramskikh: thanks, looking forward to see it live :)
16:33:31 <vkozhukalov> holser around?
16:33:49 <vkozhukalov> seems like nope
16:34:07 <vkozhukalov> aglarendil: do you have any comments here?
16:34:18 <aglarendil> holser was going to participate
16:34:26 <aglarendil> gimme a minute
16:34:39 <mihgen> vkozhukalov: let's move on and come back to it later
16:34:49 <vkozhukalov> mihgen: ok
16:35:04 <vkozhukalov> #topic Nailgun plugins
16:35:12 <vkozhukalov> that is a great topic
16:35:16 <mihgen> that's hot topic
16:35:21 <meow-nofer__> =)
16:35:33 <meow-nofer__> by the way, we had a demo yesterday
16:35:40 <vkozhukalov> looks like meow-nofer_ implemented a great POC
16:35:49 <meow-nofer__> everything almost works and is already on gerrit
16:36:01 <vkozhukalov> yes we had demo, what about spec?
16:36:06 <mihgen> we don't expect to have fully pluggable arch in 5.1, but I envision that we will have some pieces aleady in 5.1
16:36:06 <meow-nofer__> just doesn't pass any tests because of old package versions
16:36:32 <meow-nofer__> vkozhukalov, I guess I'll write a spec tomorrow
16:36:36 <mihgen> approach was here to start with POC, criticize it, and then create a spec
16:36:38 <meow-nofer__> at least, start doing it
16:36:44 <vkozhukalov> meow-nofer_ great
16:36:47 <mihgen> meow-nofer_: yep, cool
16:36:57 <vkozhukalov> moving on?
16:37:00 <odyssey4me> blueprint/bug url for the topic?
16:37:06 <mihgen> meow-nofer_: do you think we can land some pieces in master before FF
16:37:47 <meow-nofer__> mihgen, maybe, these little hooks doesn't change any logic right now
16:37:48 <mihgen> odyssey4me: #link https://review.openstack.org/#/c/97827/ Nailgun Ceph plugin
16:38:20 <vkozhukalov> #link https://review.openstack.org/#/c/97827/
16:38:31 <mihgen> meow-nofer__: I think you should land it if possible
16:38:41 <mihgen> it's gonna be impossible to review it otherwise later
16:38:44 <vkozhukalov> i believe robot understands links only if they are on a separate line
16:38:51 <mihgen> we need to go into the direction piece by piece..
16:38:53 <vkozhukalov> maybe i am wrong
16:39:04 <mihgen> vkozhukalov: ok, let's check it after the meeting )
16:39:16 <meow-nofer__> mihgen, yeah, I know, I continue working on it
16:39:50 <vkozhukalov> not too much time, moving on
16:39:57 <vkozhukalov> #topic openstack patching and fuel upgrade design status
16:40:01 <mihgen> #link https://docs.google.com/a/mirantis.com/document/d/1H-CkqUQEVinFLxpnTSKyF265pD_HthBrLbpUUFRlgks/edit Fuel plugin framework design discussion
16:40:17 <mihgen> this is overall direction where we would like to go with plugins.
16:40:30 <mihgen> I'm not sure.. but I think we will need to convert it into RST as well
16:40:34 <aglarendil> also, I wrote an email to ML today
16:40:57 <aglarendil> regarding how we gonna integrate OpenStack patching and FUEL Upgrade features
16:41:07 <mihgen> vkozhukalov: sorry, switching to patching now :)
16:41:29 <mihgen> aglarendil: I'll take a look.. anything urgent there?
16:41:32 <vkozhukalov> mihgen already there
16:41:46 <aglarendil> we gonna implement separate versioning for the product components and version the serializers
16:42:15 <odyssey4me> aglarendil - yeah, responded to that, but let's stick to topic now :)
16:42:32 <mihgen> aglarendil: will we be able to patch old env to new maintenance version of openstack?
16:42:32 <aglarendil> puppet part of openstack patching is gonna be started by QA guys
16:42:33 <evgeniyl> Yeah, we had several discussion about openstack patching, it looks more complicated, than we thought.
16:42:53 <aglarendil> mihgen: yep, but we will need to use
16:42:53 <mihgen> evgeniyl: hmm
16:43:17 <aglarendil> mihgen: additional maintenance release of FUEL manifests with all the features ripped off
16:43:21 <vkozhukalov> evgeniyl: any details?
16:43:29 <aglarendil> mihgen: and only bugfixes and newer OpenStack in place
16:43:44 <mihgen> aglarendil: ok. is it all in the email?
16:43:45 <evgeniyl> mihgen: yes, we will be able to patch openstack from one minor version to another, without new features.
16:44:03 <aglarendil> mihgen: not all of it. but it will be in the blueprint, obviously
16:44:27 <mihgen> ok..
16:44:31 <evgeniyl> vkozhukalov: details are in email, the point is we need to have separate versions for nailgun/astute/etc and fuel-library.
16:45:05 <aglarendil> we will keep several fuel library manifests and snippets versions in docker container and rsync corresponding
16:45:09 <vkozhukalov> evgeniyl: thx
16:45:23 <evgeniyl> I can add details regarding to nailgun/upgrade script part of this feature.
16:45:24 <aglarendil> ones to the slave nodes based on which fuel library version was used for this particular env
16:45:38 <evgeniyl> We have several patches from Alexandr (nailgun) and Igor (upgrade script), they are almost ready and I think we will merge them soon, I almost finished after upgrade health checker
16:45:46 <mihgen> I see odyssey4me was fast to reply on aglarendil's email .. I'll take a look too
16:46:16 <evgeniyl> Also we are waiting for make system from Dmitry P. to build tar ball with upgrade script, rpms, and manifests, when it’s ready we will be able to start integration.
16:46:37 <mihgen> evgeniyl: ok. did dpyzhov give any estimations on that?
16:46:58 <mihgen> as I know he is gonna take short vacation.. wondering if it's gonna be ready before it.
16:47:10 <evgeniyl> mihgen: he told that it should be easy and tomorrow we will have some working code.
16:47:39 <mihgen> evgeniyl: ok, thanks
16:47:42 <evgeniyl> mihgen: do we have other engineer who know how make system works?
16:47:58 <vkozhukalov> mihgen: if not, we have plenty of people who wants to deal with make stuff
16:48:01 <vkozhukalov> -)
16:48:11 <evgeniyl> vkozhukalov: ok, cool)
16:48:13 <mihgen> vkozhukalov: you are first candidate ;)
16:48:23 <vkozhukalov> moving on
16:48:25 <mihgen> I still remember that day when you forced me to use make )
16:48:45 <vkozhukalov> #topic some questions to @xarses on ml2 support if he is on the meeting
16:48:53 <xarses> here
16:49:18 <aglarendil> xarses: so, my question is whether you started working on this and what is your estimate to finish it
16:49:40 <aglarendil> xarses: it intersects a little bit with merge-openstack-puppet-modules blueprint
16:49:53 <aglarendil> #link https://blueprints.launchpad.net/fuel/+spec/merge-openstack-puppet-modules
16:49:54 <xarses> I've looked it over, I wont fully start untill this afternoon.
16:50:06 <mihgen> aglarendil: we have heard that ideally we need to consider linux bridges plugin
16:50:07 <aglarendil> as it is already implemented in upstream manifests
16:50:17 <mihgen> not sure if we can do it in 5.1
16:50:21 <aglarendil> mihgen: who was that ?
16:50:33 <xarses> aglarendil: yes, it does, i need to spend some time to understand how ml2 is configured and how to adapt it to fuel, and then pull in upstream neutron module
16:50:50 <aglarendil> xarses: but we have huge modifications in neutron module
16:50:52 <mihgen> aglarendil: one of the guys from fields )
16:51:16 <aglarendil> xarses: it would be awesome if you got a consensus with xenolog on the amount of merge
16:51:21 <aglarendil> xarses: we can afford for neutron module
16:51:36 <mihgen> aglarendil: +1 on sync with xenolog
16:52:01 <mihgen> please chat on daily basis folks...
16:52:23 <mihgen> xarses: any more info on this?
16:52:34 <aglarendil> anyway, ML2 implementation is feasible, I guess, but the best option is to use upstream modules
16:52:49 <mihgen> we need to hear about 4.1.1 so I'm trying to speed up the meeting
16:53:10 <vkozhukalov> ok
16:53:18 <xarses> mihgen: not yet, I haven't dived very far into it yet
16:53:26 <mihgen> ok. thanks for update
16:53:30 <aglarendil> btw, holser connected.
16:53:30 <vkozhukalov> #topic  4.1.1 status from @angdraug
16:53:46 <vkozhukalov> angdraug: here?
16:53:49 <angdraug> here
16:54:22 <angdraug> #1316210 is still not addressed
16:54:38 <angdraug> we have new ceph packages for #1325765
16:54:55 <angdraug> anyone from OSCI around to comment on heartbleed fix for CentOS?
16:55:04 <mihgen> that's crap
16:55:13 <vkozhukalov> #link https://bugs.launchpad.net/fuel/+bug/1316210
16:55:29 <angdraug> fixes for #1300027 and #1283035 are not merged
16:55:32 <vkozhukalov> #link https://bugs.launchpad.net/fuel/4.1.x/+bug/1325765
16:55:33 <uvirtbot> Launchpad bug 1325765 in fuel/4.1.x "Ceph v0.67.9 for 4.1.1" [High,In progress]
16:55:45 <vkozhukalov> 5 minutes
16:56:18 <angdraug> 3 new bugs were targeted for 4.1.1 this morning, I'll need to triage these, unless anyone speaks up I intend to push them to 4.1.2
16:56:41 <angdraug> #link https://launchpad.net/fuel/+milestone/4.1.1
16:57:10 <mihgen> Folks 4.1.1 is critical priority now
16:57:21 <mihgen> we need just a little bit of resources to make RC for it
16:57:30 <mihgen> let's concentrate and react quickly on it
16:57:38 <angdraug> the most critical is heartbleed
16:57:45 <angdraug> it's something we can't do in US
16:57:57 <mihgen> aglarendil: nurla: I need your help here in the first order
16:58:37 <mihgen> angdraug: I'll push folks to make it happen
16:58:54 <mihgen> thanks for status angdraug
16:59:00 <mihgen> vkozhukalov: anything left?
16:59:27 <vkozhukalov> if anyone has suggestions about design
16:59:37 <vkozhukalov> we have 30 seconds
16:59:52 <xarses> asdfgfgeG!!!!
16:59:54 <aglarendil> let's move to fuel-dev
16:59:58 <odyssey4me> not quite enough time for a design conversation :p
17:00:02 <vkozhukalov> ok looks like we are closing
17:00:03 <xarses> ^ lest implement it
17:00:11 <vkozhukalov> #endmeeting Fuel