16:03:10 <kozhukalov> #startmeeting Fuel
16:03:11 <openstack> Meeting started Thu Apr  9 16:03:10 2015 UTC and is due to finish in 60 minutes.  The chair is kozhukalov. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:03:13 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:03:15 <openstack> The meeting name has been set to 'fuel'
16:03:17 <kozhukalov> #chair kozhukalov
16:03:18 <openstack> Current chairs: kozhukalov
16:03:19 <mkwiek> hello
16:03:25 <kozhukalov> hey guys
16:03:30 <sambork> hi
16:03:32 <kozhukalov> agenda as usual
16:03:32 <daniel3_> hello
16:03:32 <sc68cal> hi
16:03:33 <barthalion> o/
16:03:37 <agordeev> o/
16:03:41 <kozhukalov> #link https://etherpad.openstack.org/p/fuel-weekly-meeting-agenda
16:03:46 <prmtl> o/
16:03:47 <mihgen> hi folks
16:04:00 <kozhukalov> any announcements before we get started?
16:04:22 <mihgen> the only announcement is that we are still working on bugs
16:04:27 <mihgen> and have not yet reached SCF
16:04:59 <kozhukalov> mihgen: thanx
16:05:10 <kozhukalov> ok just couple topics for today and both are about IBP
16:05:27 <kozhukalov> #topic IBP python build image script (kozhukalov)
16:05:56 <kozhukalov> like I wrote script work just fine
16:06:06 <kozhukalov> but we are still working on tests
16:06:16 <mihgen> in python?
16:06:17 <kozhukalov> review is welocome
16:06:19 <mihgen> instead of bash
16:06:25 <kozhukalov> mihgen: yes
16:06:28 <kozhukalov> in python
16:06:49 <kozhukalov> #link https://review.openstack.org/#/c/169413/
16:06:53 <salmon_> hi
16:07:00 <kozhukalov> there are three patches in fuel-web
16:07:01 <mihgen> I hope you tested it in different scenarious, including failures, etc…
16:07:55 <kozhukalov> mihgen: actually, it does exactly the same that our shell version
16:08:24 <kozhukalov> and agordeev is going to implement retry-s for debootstrap and apt-get
16:08:34 <mihgen> ok, cool
16:08:45 <agordeev> i'd like to notice that besides tests, we're in a need of more compehensive logging, retry logic and clean-up.
16:09:17 <kozhukalov> we are still waiting until warpc finishes his part about progress bar
16:09:36 <mihgen> kozhukalov: progress bar for what stage?
16:09:51 <kozhukalov> #link https://review.openstack.org/#/c/171137/
16:09:56 <mihgen> I tried deployment, and now UX is not very good when we are building image on master node
16:10:11 <kozhukalov> for both: building image and installing os (for IBP)
16:10:14 <mihgen> basically it says "provisioning", 0% complete, and nothing happens for next 10-15min;
16:10:34 <kozhukalov> yes, you are right
16:10:51 <kozhukalov> and this patch from warpc is going to make it better
16:11:15 <kozhukalov> i mean progress bar is going to be moving while image is being build
16:11:23 <kozhukalov> at least
16:11:55 <mihgen> we would ideally say "building image", not "provisioning"
16:12:11 <mihgen> not sure if it can be easily accomplished though
16:12:27 <kozhukalov> mihgen: will ask warpc if it is possible
16:13:20 <kozhukalov> mihgen:  but it requires to split stages, and it looks like we likely won't be able to do this by 6.1
16:13:31 <mihgen> ok, thx
16:14:11 <kozhukalov> another point here is that many QA report 1200 secs is not enough for building image
16:14:35 <kozhukalov> the reason for that is rather IO performance
16:14:46 <kozhukalov> low performance
16:14:51 <mihgen> it can be slow Internet too
16:15:12 <kozhukalov> i've made a patch to increase timeout up to 3600 secs
16:15:31 <mihgen> ok..
16:15:41 <kozhukalov> mihgen: it is sure to be connected with io
16:16:09 <kozhukalov> because if host does not ignore sync calls from guest
16:16:29 <kozhukalov> it could even take more time
16:16:56 <kozhukalov> for example for qemu we need to use nosafe cache mode for disk driver
16:17:16 <kozhukalov> that is it on this topic
16:17:23 <kozhukalov> moving on
16:17:24 <mihgen> thx. let's move on
16:17:38 <kozhukalov> #topic IBP getting rid of using md for /boot file system (kozhukalov)
16:18:08 <kozhukalov> some people reported that we need to support more that 20 and even 30 disks
16:18:29 <kozhukalov> currently we have 28 disks limitation
16:18:45 <kozhukalov> because we spread /boot over all disks
16:19:02 <maximov> but we found a way how two fix it in 6.1, right?
16:19:27 <kozhukalov> and legacy grub (which is on centos) does not support md > 0.9
16:19:33 <kozhukalov> maximov: yes
16:19:51 <maximov> yes, but for ubuntu there is no such problem
16:19:56 <kozhukalov> #link  https://review.openstack.org/#/c/171668/
16:20:11 <maximov> and second way was just to keep 1 /boot for 1 disk
16:20:16 <kozhukalov> this patch avoids using md for /boot totally
16:20:39 <maximov> this is sort of hack but will work for 6.1 until we implement advanced volume manager
16:20:49 <kozhukalov> yes, this patch implements this idea
16:21:33 <mihgen> related bug to the area
16:21:34 <kozhukalov> it is not a hack, because the whole data driver is intended to deal with data from nailgun
16:21:35 <mihgen> #link https://bugs.launchpad.net/fuel/+bug/1258347
16:21:36 <openstack> Launchpad bug 1258347 in Fuel for OpenStack "[volumes-refactoring] Can't exclude a volume from the boot md (raid 1)" [High,Confirmed] - Assigned to Fuel Python Team (fuel-python)
16:21:58 <mihgen> can you please comment on this one - any possibility to do something with this one in 6.1?
16:22:10 <mihgen> or it's actually duplicate?
16:22:44 <kozhukalov> mihgen: thanx, yes, this is related, but this particular bug should be addressed by volume manager refactoring
16:23:09 <kozhukalov> because this bug is actully about fuel flexibility
16:23:30 <mihgen> can't we just skip the devices which user marked as "don't touch"?
16:23:37 <mihgen> I believe this bug is about that
16:23:54 <kozhukalov> it is more about the situation when a user just does not want us to touch this particular disk
16:23:58 <mihgen> I'm concerned that we've got such a large heat on this bug, os many users hit it
16:24:14 <mihgen> yeah, that's what I'm talking about
16:24:41 <kozhukalov> yes, we'll do that when we implement new volume manager
16:25:32 <kozhukalov> my idea is to deal with all volumes as with a flat set of objects
16:25:44 <kozhukalov> like it is done in cinder for example
16:25:55 <kozhukalov> but will see
16:26:34 <mihgen> I was thinking if we could make a workaround to fix this particular thing easily without refactoring
16:26:36 <kozhukalov> at least 6.1 + IBP is gonna support unlimited number of disks
16:26:44 <kozhukalov> in a reason of course
16:26:52 <mihgen> it's actually "forget that you have a disk", so not to touch it
16:27:36 <kozhukalov> mihgen: ok, then i'll take a look at this bug carefully and will comment
16:28:26 <mihgen> thanks!
16:28:32 <kozhukalov> if there are no other questions, let's move on
16:28:49 <kozhukalov> #topic Fuel VXLAN support (sc68cal)
16:29:10 <sc68cal> howdy
16:29:22 <kozhukalov> sc68cal: hi
16:30:01 <sc68cal> There's been a bit of discussion about encapsulation technologies to support in fuel, and I'm just trying to track down fuel support for vxlan
16:30:25 <aglarendil> sc68cal: currently VXLAN as badily supported by OVS
16:30:45 <aglarendil> sc68cal: OVS cannot use multicast address as an endpoint address
16:31:12 <aglarendil> sc68cal: this means that VXLAN is not better than GRE encapsulation except for ability to offload it on hardware
16:31:35 <aglarendil> sc68cal: there is good support in Linux kernel, but unfortunately Neutron does not have a plugin for it
16:31:55 <aglarendil> sc68cal: so we decided to hang on for a while until there is a good support in upstream
16:32:18 <aglarendil> I have finished
16:32:27 <sc68cal> aglarendil: ability to offload to hardware is a big selling point. I understand the multicast issue, but I don't think that should be a blocker
16:32:46 <sc68cal> I'm willing to work on adding support, while we wait for upstream
16:33:10 <sc68cal> basically resume the work that is linked from https://blueprints.launchpad.net/fuel/+spec/neutron-vxlan-support
16:33:16 <aglarendil> sc68cal: okay, let's do it
16:33:22 <aglarendil> sc68cal: but I would rather make it a plugin
16:33:23 <mihgen> I would say that we started to explore different options now, which would suite efficient multirack deployment support
16:33:41 <aglarendil> sc68cal: emphasizing that it is not so good right now to be inside FUEL core
16:34:01 <mihgen> sc68cal: so your ideas / input would be great, can you start discussion in openstack-dev ML?
16:34:04 <sc68cal> aglarendil: I'd have to do a little more research on fuel plugins before I would commit
16:34:12 <aglarendil> I would better look towards something Open and created by well-known vendors
16:34:13 <sc68cal> but yes I'll start a thread on the ML
16:34:19 <aglarendil> like OpenDayLight or Juniper
16:34:26 <mihgen> fuel plugins would be the best wherever psosible of course
16:34:56 <mihgen> also we will extend it in the next timeframe (7.0) in order to support roles creation out of plugin
16:35:58 <mihgen> as I undertand, it should be failrly easy to support vxlan in fuel?
16:36:23 <sc68cal> unknown, just starting. Would assume so
16:36:41 <mihgen> aglarendil: do we need any nailgun changes in order to support vxlan?
16:36:59 <mihgen> I thought it's gonna be just some puppet stuff to configure neutron, etc.
16:37:14 <aglarendil> mihgen: not very much, I guess for unicast
16:37:32 <mihgen> kk
16:37:53 <xenolog_> mihgen: yep, it's a new segmentation type
16:38:46 <mihgen> cool. ok, then sc68cal - waiting for your email in openstack-dev
16:38:51 <kozhukalov> xenolog_: so, it is just gonna be a little radio on the cluster setting tab? right?
16:38:53 <sc68cal> mihgen: thanks. will do
16:38:53 <mihgen> let's move on if we are done here
16:39:13 <kozhukalov> ok, moving on
16:39:30 <kozhukalov> #topic Demo env on Vbox: connectivity issues when installing Ubuntu (mihgen)
16:40:10 <ikalnitsky> mihgen, what's wrong with Vbox?
16:40:17 <mihgen> ikalnitsky: if you do macos install
16:40:23 <mihgen> provisioning just fine
16:40:33 <mihgen> then l23network reconfigures your default gw
16:40:44 <mihgen> and you don't have access to Internet anymore
16:40:52 <ikalnitsky> oh. got it, there's no connectivity via public network.
16:41:06 <mihgen> you actually can ping default gateway, which is your vboxnet1 by default on your host
16:41:15 <mihgen> we use host-only nets in vbox
16:41:45 <ikalnitsky> should we change all of them to nat interface?
16:41:49 <mihgen> the question is how to configure vbox or host, so traffic can be forwarded further and not being dropped on vboxnet1
16:41:50 <ikalnitsky> just like we do for vboxnet0 ?
16:42:09 <mihgen> do we do it for vboxnet0??
16:42:20 <ikalnitsky> i'm not sure, but looks like it
16:42:36 <mihgen> I think we use second net adapter NAT in vbox for master node
16:42:39 <mihgen> but not for any other nodes
16:42:54 <mihgen> and when we use NAT, then it's dhcp, vbox provides IP
16:42:58 <ikalnitsky> Ok.. i see vboxnet0 is host-only. vboxnet1 is host-only and the third one is nat
16:43:01 <mihgen> and we want to use static IPs for our slaves
16:43:09 <ikalnitsky> i've just checked on my vbox installation
16:44:01 <mihgen> hm. ikalnitsky - it would be great if you can try to play a bit in order to fix it. If it doesn't work right away, we would find someone less busy to work on a solution
16:44:20 <mihgen> currently it's serious vbox demo issue - install fails
16:44:34 <ikalnitsky> mihgen, i'll try to do it tomorrow.
16:44:44 <ikalnitsky> mihgen, did you file a bug?
16:44:49 <mihgen> and even bigger problem is that it fails with just like something failed, go to logs
16:45:17 <mihgen> there are ton of logs of course.. we would need to see if we can do any improvement here for troubleshooting as well
16:45:40 <mihgen> ikalnitsky: pls try without bug, but I'll try to get time today to reproduce and file bug
16:46:41 <kozhukalov> #action ikalnitsky does research about how to fix internet connectivity for vbox demo
16:46:46 <mihgen> let's move on ..
16:47:02 <kozhukalov> #topic customer-found bugs moved to 7.0: http://bit.ly/1ClWmBd
16:47:34 <mihgen> I'd like to bring attention to these bugs, especially those which > medium priority
16:47:47 <mihgen> let's see if every of them has a very good reason to be moved out of current 6.1 milestone.
16:48:24 <mihgen> some of them are pretty important to get fixed sooner than later, and we might rather want to delay release, than to release with defects which hit a lot of users
16:48:59 <mihgen> overall we need to pay special attention to customer-found and other community reported bugs, taking them with care
16:49:19 <mihgen> #link https://bugs.launchpad.net/fuel/+bug/1278964
16:49:20 <openstack> Launchpad bug 1278964 in Fuel for OpenStack "[library] DB grew large within 1 week, filled root filesystem" [High,Confirmed] - Assigned to Vladimir Kozhukalov (kozhukalov)
16:49:33 <mihgen> I thougth we have a workaround for this one
16:50:35 <kozhukalov> it is all about volume manager
16:50:49 <mihgen> didn't we? original issue seemed to be that DB was growing just enormously and killed root partition. Can't it be easily solved by creating an lvm for db?
16:51:59 <mihgen> ok I don't want to go over all bugs during this meeting, just let's provide comment to every what would it take to fix it
16:52:08 <kozhukalov> but, yes for IBP we potentially can make workarounds for many cases which appear due to lack of flexibiltiy in volume manager
16:52:45 <kozhukalov> mihgen: will take a look
16:53:21 <mihgen> ok, thanks
16:53:23 <mihgen> let's move on
16:53:39 <_1_chopi4567> hii
16:53:49 <kozhukalov> yes, bugs are bugs, let's pay more attention to those which hit many people
16:54:00 <_1_chopi4567> ok
16:54:10 <kozhukalov> #topic NetApp plugin without maintainer (prmtl)
16:54:17 <prmtl> hey all
16:54:22 <_1_chopi4567> ok
16:54:23 <_1_chopi4567> hii
16:54:23 <kozhukalov> hey
16:54:34 <prmtl> the problem is that we still have one plugin left without a maintainer: https://github.com/stackforge/fuel-plugin-cinder-netapp
16:55:02 <prmtl> and there are bugs reported for it https://bugs.launchpad.net/fuel/+bug/1441022 https://bugs.launchpad.net/fuel/+bug/1405186
16:55:04 <openstack> Launchpad bug 1441022 in Fuel for OpenStack "typo in fuel cinder netap plugin" [Undecided,In progress] - Assigned to sbartel (samuel-bartel)
16:55:05 <openstack> Launchpad bug 1405186 in Fuel for OpenStack "fuel cinder_netapp plugin miss 7mode and Eseries storage familly support" [Medium,Triaged] - Assigned to Fuel Partner Integration Team (fuel-partner)
16:55:24 <prmtl> and there is no one that can take care of them
16:55:43 <mihgen> prmtl: well, we need to find someone to work on it, or mark it as experimental ..
16:56:15 <prmtl> original author do not have time, PI will not work on it since it's not ceritfied
16:56:38 <mihgen> prmtl: it would be cool to get someone from community to work on those..
16:57:01 <prmtl> Samuel Bartel, who reported that bug is probably good candidate
16:57:26 <prmtl> but still, there should be someone from company that will take care of the plugins that was created by us (even if they are not certified)
16:58:10 <mihgen> true. let me work on that and ensure that we'll have it in responsibilty matrix
16:58:20 <prmtl> great
16:58:20 <mihgen> sbartel actually proposed patch: https://review.openstack.org/#/c/171186/
16:58:46 <prmtl> yes, and probably he will aslo fix the second bug
16:58:49 <prmtl> also*
16:58:50 <mihgen> let's help, review / merge or even give him merge rights, if he will keep working on the plugin
16:59:16 <kozhukalov> 1 minute
16:59:27 <kozhukalov> ok, looks like we are done
16:59:40 <kozhukalov> thanx everyone for attending
16:59:41 <mihgen> ok thanks all
16:59:43 <prmtl> thanks
16:59:50 <kozhukalov> ending
16:59:58 <kozhukalov> #endmeeting