16:01:55 <kozhukalov> #startmeeting Fuel
16:01:56 <openstack> Meeting started Thu Nov 13 16:01:55 2014 UTC and is due to finish in 60 minutes.  The chair is kozhukalov. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:01:57 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:02:00 <openstack> The meeting name has been set to 'fuel'
16:02:06 <agordeev> o/
16:02:07 <kozhukalov> #chair kozhukalov
16:02:08 <openstack> Current chairs: kozhukalov
16:02:16 <alex_didenko> hi
16:02:23 <kozhukalov> agenda as usual
16:02:32 <kozhukalov> #link https://etherpad.openstack.org/p/fuel-weekly-meeting-agenda
16:02:55 <mihgen> #link https://wiki.openstack.org/wiki/Fuel/6.0_Release_Schedule
16:03:01 <mihgen> here is the schedule
16:03:04 <mihgen> for 6.0
16:03:11 <kozhukalov> #topic Announcements (mihgen)
16:03:34 <mihgen> folks we are doing dev in both 5.1.1 (stable/5.1) & 6.0 (master)
16:03:49 <mihgen> priorities and focus should be 5.1.1 first, and then 6.0
16:04:14 <mihgen> we wanted to get code freeze for 5.1.1 today, though it seems to be we still have a bunch of bugs there
16:04:31 <mihgen> also, today is the plan to call for SCF for 6.0
16:04:49 <mihgen> I don't see angdraug here to talk more about 5.1.1 status
16:05:06 <mihgen> for 6.0, in my opinion we are good to go with SCF
16:05:13 <mihgen> unless there are any objections
16:05:29 <mihgen> objections?
16:05:33 <kozhukalov> he was complaining that it is too early for US people
16:05:34 <msemenov> 3.13 kernel can be merged after SCF?
16:05:55 <kozhukalov> i mean angdraug ^^^
16:06:04 <mihgen> msemenov: for 6.0, we can consider it as exception..
16:06:13 <mihgen> when can you be ready with it?
16:06:33 <mihgen> if you need day or two, then it's fine. But no more I would say
16:06:43 <msemenov> we are ready, but have problems with this patch https://review.openstack.org/#/c/133925
16:06:50 <mihgen> as we will need to test a lot of things, and see stable bahavior for weeks
16:06:56 <msemenov> I think, day or two is ok for us
16:07:06 <akasatkin> SCF or HCF for 5.1.1?
16:07:28 <mihgen> akasatkin: there is no SCF/HCF for 5.1.1, just code freeze
16:07:33 <mihgen> though it's like HCF
16:07:33 <kozhukalov> msemenov: this patch needs to be a bit improved, i'am sure it works, but...
16:07:46 <mihgen> because we basically stop accepting anything to stable/5.1
16:08:02 <msemenov> ok, I hope we will come to some conclusion on it
16:08:06 <vkramskikh> do we have plans to make code freeze separate for repos? soft or hard
16:08:19 <kozhukalov> msemenov: I'm sure we will
16:08:22 <mihgen> vkramskikh: good question, it's in agenda
16:08:37 <akasatkin> https://launchpad.net/bugs/1387345 - for 5.1.1. How much is it important?
16:08:39 <mihgen> kozhukalov: I'm done with general announcement
16:08:40 <uvirtbot> Launchpad bug 1387345 in mos/5.1.x "Heat deployment in HA mode: different value for auth_encryption_key parameter on each controller node" [High,In progress]
16:09:14 <kozhukalov> ok, guys, let's use this topic for announcements only
16:09:15 <mihgen> akasatkin: hard to say quickly
16:09:19 <kozhukalov> not for discussions
16:09:33 <mihgen> kozhukalov: you have announcement regarding changing the time of the meeting?
16:10:09 <kozhukalov> yes, for US people it seems to be to early
16:10:28 <kozhukalov> my suggestion is to move our meeting to 1700 utc
16:10:38 <mihgen> 8pm msk?
16:10:45 <mihgen> same day?
16:11:02 <barthalion> o/
16:11:03 <kozhukalov> there is a free time slot on Thursdays 17-18 utc but it is in #openstack-meeting
16:11:11 <barthalion> won't it be a bit late?
16:11:21 <kozhukalov> yes, 20 msk, Thursdays
16:11:43 <kozhukalov> guys, is it ok?
16:11:49 <mihgen> what time is it in CA?
16:11:54 <mihgen> 9am?
16:12:13 <kozhukalov> no idea
16:12:23 <mihgen> yes, it's gonna be 9am
16:12:30 <mihgen> Now we start at 8am PST
16:12:37 <kozhukalov> ca -8 utc
16:12:38 <mihgen> it's not too early, seems to me fine
16:12:50 <kozhukalov> so 9am pst
16:12:58 <mihgen> now is 8.12am PST, so I'm against moving the time to +1 hour
16:13:10 <mihgen> it's gonna be too late for MSK timezone
16:13:13 <kozhukalov> 9am is ok
16:13:27 <mihgen> you see even late for barthalion , who is in Poland?
16:13:30 <kozhukalov> 8am is not for CA guys
16:13:45 <salmon_> mihgen:
16:13:57 <mihgen> they wake up at 6am to get intersection with all EU countries anyway
16:14:08 <kozhukalov> mihgen: poland time is +1 utc
16:14:17 <kozhukalov> or maybe +2 utc
16:14:21 <ntrueblood> 8am is fine for CA team.   The sun is up.  ;)
16:14:22 <salmon_> we have 5:14PM now
16:14:41 <kozhukalov> ntrueblood: ))
16:14:51 <kozhukalov> ok, let's make a desicion
16:14:54 <mihgen> let's keep it as is for now. If there are objections from other - we will do voting
16:15:02 <mihgen> and can discuss it in openstack-dev ML
16:15:17 <kozhukalov> ok, moving on then
16:15:18 <mihgen> ok guys let's move on
16:15:37 <kozhukalov> #topic Image based provisioning (agordeev)
16:15:41 <agordeev> hi
16:15:45 <kozhukalov> hi
16:15:56 <agordeev> image based provisioning
16:15:58 <agordeev> mdadm issues are still in progress, but closer to fixing. https://bugs.launchpad.net/fuel/+bug/1390492
16:16:01 <agordeev> hanging mount points issue is in progress too. https://bugs.launchpad.net/fuel/+bug/1391896
16:16:02 <uvirtbot> Launchpad bug 1390492 in fuel "Provisioning failed if inactive md device had been found" [High,Fix committed]
16:16:02 <agordeev> progress reporting is in progress too. https://bugs.launchpad.net/fuel/+bug/1383748
16:16:03 <uvirtbot> Launchpad bug 1391896 in fuel "hanging mount points when building iso" [High,In progress]
16:16:04 <uvirtbot> Launchpad bug 1383748 in fuel "Provisioning progress bar doesn't work if image based provisioning" [High,In progress]
16:16:04 <agordeev> hope to finish them on this week.
16:16:06 <agordeev> and i'm glad to tell that new bug haven't appeared so far
16:16:08 <agordeev> that's all
16:16:38 <mihgen> agordeev: thx. mdadm bug is closed actually now
16:16:46 <mihgen> you say it's not yet done - status wrong?
16:17:13 <agordeev> mihgen: it's not fixed yes. so it's re-opened
16:17:14 <mihgen> question - we are getting images build during master ISO build now?
16:17:30 <mihgen> and those are getting into 6.0 ISO now?
16:17:51 <kozhukalov> mihgen: exactly
16:17:53 <mihgen> also, are those getting into upgrade tarball too (6.0)?
16:18:04 <kozhukalov> besides there are two other patches
16:18:20 <lpabon> hi
16:18:21 <kozhukalov> #link https://review.openstack.org/#/c/133481/
16:18:23 <lpabon> sorry i'm late
16:18:50 <kozhukalov> #link https://review.openstack.org/#/c/133731/
16:19:01 <mihgen> looks like our ISO build now takes +20 more min build time (and becomes 1h build) because of this. Any When do you plan to make it as separate run so we don't rebuild them when not needed?
16:19:06 <kozhukalov> they are about placing images into upgrade tarball
16:19:10 <agordeev> mihgen: images building is turned on by default. but the feature turned off for the nailgun, you need to swtich it into experimental mode manually
16:20:12 <mihgen> kozhukalov: ok, I hope we can land those really soon - as we need QA to verify evrything when all pieces are in
16:20:18 <kozhukalov> mihgen: when devops guys will be ready
16:20:32 <mihgen> devops guys… ? ^^
16:20:45 <kozhukalov> i mean separate build
16:20:51 <kozhukalov> for images
16:21:09 <mihgen> yeah, that's my question to bookwar monester
16:21:23 <mihgen> don't see monester here, bookwar -please poke him
16:21:35 <kozhukalov> afaik dpyzhov is going to prioritize artifact based building next release
16:22:09 <mihgen> we are sufferting more and more without it  - as to see a little patch we need to rebuild too many things
16:22:17 <mihgen> just non-effective
16:22:35 <kozhukalov> yes
16:22:36 <mihgen> ok let's move on, I'll talk to devops later
16:22:39 <kozhukalov> moving on?
16:22:40 <bookwar> we don't have versioning and metadata for different pieces
16:22:54 <mihgen> bookwar: what's that about?
16:22:57 <bookwar> we can'not split the build unless it is resolved
16:23:05 <mihgen> kozhukalov: ?
16:23:17 <kozhukalov> bookwar: version.yaml ?
16:23:26 <kozhukalov> at least version.yaml
16:23:50 <mihgen> ok let's get it out of the meeting. we need to resolve it, we can't simply increase and increase build time … . (
16:23:51 <kozhukalov> what we need is switching to job builder
16:24:00 <kozhukalov> ok
16:24:13 <kozhukalov> #topic ironic integration (kozhukalov)
16:24:22 <kozhukalov> here there are some good news
16:24:32 <kozhukalov> ironic is in the core at the moment
16:24:51 <kozhukalov> so ironic people became much more open for new use cases and feature
16:25:28 <kozhukalov> some fuel guys took part in a series of beer meetings during summit
16:25:40 <mihgen> yeah I met Devananda at the summit, he said we need to provide our use cases
16:25:40 <kozhukalov> and now we have an agreement
16:26:01 <kozhukalov> we can implements everything we need as Ironic driver
16:26:08 <mihgen> I missed the beer part with him unfortunately, but anyway )
16:26:24 <kozhukalov> we already have working agent side
16:26:27 <mihgen> kozhukalov: did we start any mailng thread on this already?
16:26:37 <mihgen> or any other communication?
16:26:55 <kozhukalov> so it looks like 7.0 release is suitable for delivering ironic as a part of fuel
16:27:20 <kozhukalov> yesterday we had a discussion with one of ironic cores
16:27:47 <mihgen> I think we should a) monitor ML for [Ironic], b) start actively discussing alignment of what we need and what's there in ML, not only in private channels
16:27:57 <kozhukalov> we agreed that we need start discussing this in ML and start design efforts
16:28:04 <mihgen> kozhukalov: cool
16:28:14 <mihgen> sounds very good, really
16:28:25 <kozhukalov> moving on
16:28:38 <kozhukalov> #topic collect information on all the changes in the packages (dburmistrov)
16:28:44 <dburmistrov> hi all
16:28:59 <dburmistrov> We implemented global changelog for all packages included to the iso
16:29:07 <dburmistrov> Script gathers lastest changelog entry (if changelog exists) from each package and place it to global changelog file in the iso root
16:29:16 <dburmistrov> For deb packages script gathers changelog info only from changelog.Debian.gz file on order to prevent junk into global changelog file.
16:29:22 <dburmistrov> According to Debian policy another changelog* files contains upstream changelog.
16:29:31 <dburmistrov> If script can't find changelog.Debian.gz file, such package will be skipped from processing global changelog.
16:29:54 <dburmistrov> that's all
16:30:12 <mihgen> dburmistrov: this is excellent improvement in my opinion. People now can see what fixes in packages are in the ISO
16:30:24 <mihgen> dburmistrov: I have question about bugs raised on this
16:30:34 <mihgen> like it's inconsistent with reality
16:30:42 <kozhukalov> afaik there were some problems with changelog during iso building
16:30:50 <kozhukalov> bookwar: am i right?
16:30:52 <mihgen> did you resolve those or what's the status over those?
16:31:11 <mihgen> kozhukalov: that's probably a different issue you are talking about, mine is about content
16:31:15 <dburmistrov> yes, some packages haven't changelog or it placed twith incorrect name. we're working on it
16:31:30 <mihgen> dburmistrov: ok
16:31:48 <bookwar> kozhukalov: it already works for 5.1.1, we are dealing with 6.0 atm
16:32:06 <kozhukalov> bookwar: cool
16:32:17 <kozhukalov> moving on?
16:32:20 <mihgen> kozhukalov: bookwar loop device issue during the iso build?
16:32:39 <bookwar> mihgen: agordeev proposed a fix for it
16:32:47 <kozhukalov> mihgen: fix is ready for loop device
16:33:00 <mihgen> good, thanks
16:33:06 <mihgen> let's move on
16:33:11 <kozhukalov> #link https://review.openstack.org/#/c/134029/
16:33:27 <kozhukalov> #topic granting FFE to sahara-create-default-templates (dmitryme)
16:33:50 <kozhukalov> dmitryme: around?
16:33:55 <dmitryme> Hello people, so basically I wanted to ask for feature freeze exception for the following feature: https://blueprints.launchpad.net/mos/+spec/sahara-create-default-templates
16:34:03 <dmitryme> It requires changes in Fuel and the CR is already proposed here https://review.openstack.org/#/c/132196/
16:34:17 <dmitryme> Basically it is a really nit change, so should be rather safe to merge it now.
16:34:35 <dmitryme> can we discuss it right now?
16:34:52 <mihgen> dmitryme: how large is the feature?.. blueprint misses our standard header with responsible people
16:35:10 <mihgen> dmitryme: and design spec. See good example - https://blueprints.launchpad.net/fuel/+spec/send-anon-usage
16:35:27 <dmitryme> mihgen: it is a single CR you can see above ^^
16:35:48 <mihgen> dmitryme: patch looks more simple than complex, and probably affects only sahara
16:36:27 <mihgen> dmitryme: but I think we should get a closer look into it. Could you send email in openstack-dev so we can give a day to look into.. ?
16:37:04 <dmitryme> mihgen: I’ve already sent an email http://lists.openstack.org/pipermail/openstack-dev/2014-November/050283.html :-)
16:37:04 <mihgen> The only worry I have right away is that qa has to know what's up there to ensure production quality
16:37:34 <dmitryme> mihgen: that should be tested by MOS QA
16:37:48 <mihgen> dmitryme: oh cool, I'll ping folks to take a closer look then
16:37:57 <dmitryme> mihgen: thank you
16:38:12 <mihgen> kozhukalov: let's move on
16:38:26 <kozhukalov> #topic separate code freeze for repos (vkramskikh)
16:38:35 <vkramskikh> hi
16:38:40 <vkramskikh> There was an idea to make a separate code freeze for repos, but we decided not to do it. Do we plan to try it this time? It is really painful to maintain multi-level tree of dependent review requests and wait for a few weeks until we can merge new stuff in master.
16:38:40 <kozhukalov> hi
16:38:51 <vkramskikh> though 2 weeks before HCF are ok for me, unless we postpone HCF :)
16:39:42 <mihgen> so for everyone the issue is that UI usually have no bugs for weeks, while others still struggle with bugs
16:39:57 <mihgen> and basically UI team can't land any fixes in master as it's closed..
16:40:05 <vkramskikh> exactly
16:40:22 <ikalnitsky> how we gonnda decide for fuel-web? we have there many projects, not only ui part?
16:40:26 <mihgen> I'm +1 for changing the flow and defining other criteria for branching
16:40:36 <ikalnitsky> i mean ui may be stable, but what about fuel-upgrade or nailgun?
16:40:53 <mihgen> ikalnitsky: that's a good question. Probably we can do it only if both stable
16:41:06 <mihgen> unless we separate the code into different repos
16:41:13 <mihgen> (not in 6.0 obviously)
16:41:31 <mihgen> vkramskikh: do we have a ML thread for this issue?
16:41:47 <kozhukalov> +1 for separating projects
16:41:57 <vkramskikh> mihgen: yes, the topic is "[Fuel] Separate code freeze for repos"
16:42:06 <ikalnitsky> +1 for different repos
16:42:07 <vkramskikh> (without replies)
16:42:28 <mihgen> I remember there were objections to separate date for creating stable branch for separate projects… are there any now?
16:42:35 <vkramskikh> as for separating projects, it needs a separate discussion as it is not trivial
16:42:54 <vkramskikh> as we need both UI and nailgun to run UI functional or CLI tests
16:43:00 <kozhukalov> mihgen: the objection was mostly about testing
16:43:14 <kozhukalov> mihgen: currently ui needs nailgun for its tests
16:43:17 <ikalnitsky> vkramskikh: git pull nailgun is ok for tests. it's not a problem.
16:43:32 <kozhukalov> fuelclient also needs nailgun for testing
16:43:41 <kozhukalov> that is not normal
16:43:45 <mihgen> ok let's try to do one by one first of all, so fuel-web as a whole
16:43:54 <mihgen> if it's stable enough, we could do branching
16:44:02 <vkramskikh> +1
16:44:04 <mihgen> while fuel-lib is still struggling with bugs for instance
16:44:38 <ikalnitsky> yep, so we need to decide how can we know that fuel-web (as a whole) is stable. let's discuss that in ML
16:44:39 <mihgen> vkramskikh: ok, I'll reply on your email. Other, especially qa and release mgrs, this is imprortant to take a look into
16:44:55 <mihgen> ikalnitsky: yeah I have some thoughts in mind on this
16:45:02 <mihgen> thx folks, kozhukalov - let's move on
16:45:08 <kozhukalov> ok
16:45:21 <kozhukalov> #topic diff upgrade tarball (ikalnitsky)
16:45:31 <ikalnitsky> this monday we have merged all latest fixes in both 5.1.1 and 6.0, so now you can download a fully-working tarball from our ci.
16:45:37 <ikalnitsky> last two days qa have performed some manual tests, and today some automation tests are successful passed. so i can say that at this moment all looks good.
16:45:49 <ikalnitsky> some info about tarball size:
16:45:49 <ikalnitsky> upgrade tarball for 5.1.1 = ~1.1 Gb (brings 5.1.1-5.1 diff)
16:45:50 <ikalnitsky> upgrade tarball for 6.0 = ~1.6 Gb (brings 5.1.1-5.1 diff and 6.0-5.1 diff)
16:45:56 <ikalnitsky> that's all for now
16:46:04 <mihgen> ikalnitsky: excellent! it was like 2.5gb before for 5.1
16:46:13 <mihgen> what about system tests?
16:46:21 <mihgen> is it all connected now to systests?
16:46:41 <ikalnitsky> mihgen: yep, but not all jobs are completed
16:46:52 <mihgen> bookwar: devops part - did we configure all required attention to jobs, like notifications, etc.
16:46:58 <ikalnitsky> some of them have not been started
16:46:59 <mihgen> ikalnitsky: what's left?
16:47:03 <bookwar> we test 5.1 to 5.1.1 upgrade tarball as a part of the swarm run
16:48:00 <bookwar> mihgen: the 5.1.1 upgrade tarball is build together with 5.1.1 iso with same notificattions, the 6.0 upgrade tarball is not yet
16:48:32 <ikalnitsky> bookwar: when you're going to connect 6.0 tarball to swarm?
16:48:44 <mihgen> bookwar: we need to treat these jobs in the same way as we treat 6.0 iso build..
16:49:21 <bookwar> ikalnitsky: we'll move from 6.0.iso to 6.0.all job tomorrow probably and then it go through the whole testing process
16:49:32 <ikalnitsky> bookwar: nice. thank you.
16:50:29 <kozhukalov> ok, looks we have nothing to discuss in our agenda
16:50:37 <mihgen> kozhukalov: open discussion..
16:50:37 <kozhukalov> #topic open discussion
16:51:07 <mihgen> we've got bvt centos failure https://bugs.launchpad.net/fuel/+bug/1392386
16:51:08 <uvirtbot> Launchpad bug 1392386 in mos "Neutron network deletion fails: "DBDeadlock: (OperationalError)"" [High,New]
16:51:22 <mihgen> I'm wondering why it's High, not Critical - if it blocks our bvt
16:51:28 <mihgen> from being green
16:51:42 <mihgen> anyone took a look into it?
16:52:24 <mihgen> dmitryme: targets neutron too
16:52:38 <bookwar> mihgen: artem_panchenko and mattymo are looking into it
16:52:52 <mihgen> kk
16:52:58 <bookwar> it has not been reproduced on reverted environment
16:53:08 <bookwar> so it is some floating issue probably
16:53:20 <mihgen> worst issue to have..
16:53:23 <dmitryme> mihgen: just received a message from aignatov: he noticed the bug
16:54:53 <kozhukalov> any other q?
16:55:23 <kozhukalov> looks like we are done
16:55:46 <kozhukalov> closing then
16:55:53 <mihgen> thx all
16:55:55 <kozhukalov> thanx everyone
16:56:06 <kozhukalov> #endmeeting