16:01:21 <xarses> #startmeeting fuel
16:01:22 <openstack> Meeting started Thu Jul  2 16:01:21 2015 UTC and is due to finish in 60 minutes.  The chair is xarses. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:01:23 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:01:25 <openstack> The meeting name has been set to 'fuel'
16:01:34 <xarses> hi all
16:01:36 <kozhukalov_> hi
16:01:37 <ikalnitsky> o/
16:01:38 <mwhahaha> hola
16:01:41 <bookwar> hi
16:01:44 <federico3> o/
16:01:45 <mattymo> good day
16:01:52 <mihgen> hi
16:01:55 <kozhukalov_> #chair xarses
16:02:05 <vkramskikh> hi
16:02:33 <xenolog> hi!
16:02:34 <rmoe> hi
16:02:38 <xarses> #topic Node role as a plugin (ikalnitsky)
16:02:44 <kozhukalov_> xarses: give chair command
16:02:44 <ikalnitsky> hi guys!
16:02:59 <ikalnitsky> we have a bunch of patches on review:
16:03:04 <ikalnitsky> #link https://review.openstack.org/#/c/195678/
16:03:10 <ikalnitsky> #link https://review.openstack.org/#/c/197151/
16:03:11 <ikalnitsky> #link https://review.openstack.org/#/c/195731/
16:03:17 <dpyzhov> hi
16:03:17 <ikalnitsky> #link https://review.openstack.org/#/c/198029/
16:03:24 <ikalnitsky> so please feel free to review, especially the first patch - it's huge but very important for further feature development.
16:03:27 <mattymo> \o/
16:03:29 <xarses> #chair xarses
16:03:29 <openstack> Current chairs: xarses
16:03:33 <ikalnitsky> next week we're planning to start mixing plugin things with core ones.
16:03:39 <ikalnitsky> it should be easy for node roles..
16:03:43 <ikalnitsky> it shouldn't be a challenge for deployment task..
16:03:48 <ikalnitsky> but I'm worry about volumes..
16:03:52 <ikalnitsky> you know, we have this volume manager refactoring in 7.0 and it might affect this feature.
16:03:59 <ikalnitsky> that's it :)
16:04:14 <ikalnitsky> and yeah, we are on schedule. all looks good.
16:04:39 <xarses> ikalnitsky: does it work with the patches in a custom ISO yet? When can you demo it?
16:05:16 <ikalnitsky> xarses, i asked QA to test it with some of them
16:05:16 <kozhukalov_> ikalnitsky: what do you mean volume manager is gonna affect your feature?
16:05:20 <ikalnitsky> there were no issue
16:05:28 <kozhukalov_> ikalnitsky: plugin conflicts?
16:05:54 <ikalnitsky> kozhukalov_, i mean that we will be able to export custom volumes <-> node roles mapping, and new volumes just like we do in openstack.yaml, but in plugins
16:06:10 <ikalnitsky> if you gonna change the format
16:06:15 <ikalnitsky> of the way it works
16:06:19 <ikalnitsky> it may affect me
16:06:31 <ikalnitsky> so please add me to review
16:06:37 <ikalnitsky> so i'll track your changes
16:07:08 <kozhukalov_> ahh, ok, i'll ask sebastian to keep you feature in mind
16:07:12 <mihgen> ikalnitsky: I'm worried to see so large patchsets which have not been reviewed yet.. when did you plan to land those?
16:07:26 <mihgen> can't we break those into pieces to simplify review / landing ?
16:07:34 <ikalnitsky> mihgen, you mean the first one? it was reviewed by evgeniy li.
16:07:39 <ikalnitsky> no, we can't.
16:07:50 <ikalnitsky> the most code in there is tests and migrations
16:08:28 <mihgen> ok. when we do we plan to land it?
16:08:44 <mihgen> is there anything demo-able?
16:09:14 <ikalnitsky> i will try to do it tomorrow, but i don't think it's possible. say, it will be landed at the beginning of next week.
16:09:41 <mihgen> ikalnitsky: ok.. what about demo?
16:09:45 <ikalnitsky> as for demo, we can mix roles. i mean core ones and plugins ones. it should be easy after this refactoring
16:09:50 <ikalnitsky> so we can show demo
16:09:54 <xarses> its seems like https://review.openstack.org/#/c/195678/ is the most important and needs more reviewers
16:10:08 <ikalnitsky> yeah, i think by the end of next week i'll try to show demo
16:11:11 <mihgen> ikalnitsky: ok. the sooner is the better.. we need to cross check that we are moving in the right direction
16:11:45 <ikalnitsky> yeah, but it's hard to show something now..
16:11:57 <ikalnitsky> before we mix things from plugins
16:13:34 <ikalnitsky> more questions?
16:13:53 <mattymo> ikalnitsky, is skipping a task via plugin still in scope for your feature?
16:14:02 <ikalnitsky> yes, it's in scope
16:14:09 <mattymo> great
16:14:22 <mihgen> folks we have many items in agenda, let's keep focused
16:14:26 <mihgen> can we move on?
16:14:27 <xarses> moving on
16:14:33 <xarses> #topic SSL status (sbog)
16:15:11 <mihgen> sbog: around?
16:15:33 <xarses> lets come back
16:15:33 <kozhukalov_> the same as last time, let's skip this, looks like he is not here
16:15:36 <mihgen> xarses: let's move on, we can't wait long today.
16:15:39 <xarses> #topic Granular deployment (mattymo)
16:15:46 <mattymo> yay my turn
16:16:07 <mattymo> on the plus side, the Python side issues are all more or less taken care of. We have public IP assignment for custom roles merged. Plugin format v3 is ready and we are ready to merge the patch for custom plugin scripts on install/uninstall
16:16:22 <mattymo> Unrestricted VIP names patch is ready for merge as well.
16:16:29 <mattymo> OSTF adaptation for endpoints separate from management_vip is now being developed
16:16:35 <mattymo> VIP definition from a plugin is also underway, which is a huge feature for our project
16:16:49 <mattymo> consumer-driven DB creation (such as from nova or glance task) is on review and should get through today. Client driven keystone endpoint creation was put up for review today.
16:17:42 <mattymo> https://review.openstack.org/#/c/194226/ <- db changes
16:17:49 <mattymo> https://review.openstack.org/#/c/197981/ <- keystone changes
16:18:01 <mattymo> We still need some effort for enabling separation of neutron task, but it's a stretch goal for the spec.
16:18:12 <mihgen> who is doing OSTF & VIP?
16:18:29 <mattymo> the project was in warning status because of python side dependencies, but once those are more or less making progress, we should go to "on track"
16:18:51 <mattymo> OSTF - apparently dshulyak. VIP alexander saprykin
16:19:01 <mihgen> VIP via API?
16:19:06 <mihgen> or thru a plugin meta?
16:19:09 <ikalnitsky> no, vip via plugins
16:19:10 <sbog_> Hi all. Sorry for being late
16:19:13 <ikalnitsky> via network roles
16:19:16 <mattymo> now it's going to be through plugin, not API
16:19:23 <mihgen> oh that sounds wonderful
16:19:24 <akasatkin> yep
16:19:24 <mattymo> and that is a lot more convenient for a plugin dev
16:19:55 <mihgen> the only thing I'm then worried is that if you guys are all gonna be on track with main features, like new role as a plugin / flexible net
16:20:01 <xarses> so are we getting help to fix OSFT?
16:20:11 <mihgen> since we seem to take more work, but no resources really added
16:20:28 <mihgen> if we are about to find python engineers to help, where those could help?
16:20:34 <mattymo> xarses, https://review.openstack.org/#/c/197945/
16:20:51 <xarses> sweet
16:21:24 <mattymo> we'll verify core openstack services functionality if we have these custom roles added, which is effectively a compromise given the level of effort needed to analyze status of corosync or mysql or rabbitmq if we don't know where the hosts are
16:21:52 <mattymo> so mihgen you probably want to know about demoability
16:22:38 <xarses> moving on
16:22:46 <mihgen> mattymo: yep
16:22:55 <mihgen> I hope we can do it next week?
16:23:18 <mattymo> first demo will be a plugin that defines a new role for databgase only. It will set a database_vip field in hiera, but needs 2 hacks that are from expected features: disable database task on controller (waiting on role-as-plugin) and hacking env metadata to add a "database" VIP
16:23:26 <mattymo> yes next week will be 100% good
16:23:40 <mihgen> ok cool, thanks
16:23:42 <mattymo> and if progress is good, I should also be able to demo separate keystone
16:23:50 <mihgen> excellent
16:24:02 <mattymo> ok xarses please continue
16:24:04 <xarses> #topic SSL status (sbog)
16:24:18 <sbog_> UI part done, rebased onto new control, tests written today, ready to merge
16:24:31 <sbog_> Python part mostly done, some tests needed
16:24:51 <sbog_> Library part done, need to write tests also
16:25:21 <mihgen> autogenerated certs?
16:25:26 <sbog_> done
16:25:34 <mihgen> or you'd have to upload from UI?
16:25:39 <sbog_> also done
16:25:55 <mihgen> very good
16:26:11 <mihgen> will you be able to do demo early next week?
16:26:19 <sbog_> I think so
16:26:26 <mihgen> this is the feature where most of holleywar happens
16:26:34 <mihgen> about how it should and should not be done
16:26:43 <mihgen> so we certainly need demo very soon to sort it out
16:26:53 <mihgen> thanks sbog_
16:26:54 <sbog_> Ok, I will prepare the demo
16:27:11 <mihgen> moving on?
16:27:11 <sbog_> next thuesday or so
16:27:22 <mihgen> sbog_: what about earlier?)
16:27:25 <sbog_> *tuesday
16:27:31 <mihgen> oh it's better :)
16:27:55 <mihgen> xarses: move on!
16:27:59 <xarses> #topic kilo� and sync of upstream manifests
16:28:05 <xarses> mihigen you asked for this but we didn't get any topic leaders do you want to ping some one?
16:28:24 <mihgen> we didn't, but I hope Marek is here
16:28:34 <mihgen> and could provide latest update
16:28:48 <mihgen> or someone else from the team
16:29:12 <xarses> ok, lets revisit it if we have time
16:29:18 <xarses> #topic templates for networking (akasatkin)
16:29:23 <mihgen> adidenko, holser not here too :(
16:29:25 <akasatkin> Hi
16:29:34 <akasatkin> Spec https://review.openstack.org/#/c/195109/ is merged recently
16:29:34 <akasatkin> Development is in good progress.
16:29:40 <akasatkin> Template engine and API in Nailgun are ~80% ready and should be tested with deployment today-tomorrow.
16:29:40 <akasatkin> Will hopefully be merged on Mon.
16:29:40 <akasatkin> Cinder and Swift services separation in library is going to be ready in one-two days.
16:29:40 <akasatkin> Network manager for 7.0 is separated. Network roles are introduced with some hardcode.
16:29:47 <akasatkin> We hope we can show first working template version on Demo (on Mon),
16:29:47 <akasatkin> and Cinder / Swift separated may be too.
16:29:55 <akasatkin> There are several tasks left, like: VIPs rework, API for networks creation,
16:29:55 <akasatkin> IP allocation for hosts according to template, CLI,
16:29:55 <akasatkin> templates validation, net-checker.
16:30:03 <akasatkin> Overall status is good. Some tasks from original scope could be postponed though.
16:30:03 <akasatkin> It depends on nova-network support requirements inter alia.
16:31:47 <mihgen> thanks akasatkin
16:31:49 <xarses> are we coordinating the vip rework with the guys working on vip's for plugin so we don't create too many conflicts on each other?
16:32:02 <akasatkin> yes, we are
16:32:05 <mihgen> can you please clearify on nova-network?
16:32:07 <mzawadzki_mirant> (sorry for interrupting: regarding kilo and sync of upstream manifests I can update when it's time)
16:32:38 <akasatkin> we'll need additional time to support nova-network if it is required
16:32:40 <mihgen> xarses: I think we would need to combine CI for Kilo & general Kilo support in agenda
16:33:04 <xarses> ok
16:33:09 <mihgen> akasatkin: but we don't break standard support of nova-net, right?
16:33:21 <mihgen> so you are talking about getting templates for nova-net now
16:33:23 <mihgen> ?
16:33:36 <akasatkin> to support it in 7.0 we need some work anyway
16:34:07 <akasatkin> it's because of network roles which are added regardless of templates
16:34:19 <mihgen> hmm… let's keep it second priority for now, and I expect from you guys estimated costs for nova-network, I really want to get rid of it..
16:34:48 <mihgen> question about IP ranges, UX - where user suppose to set it with templated nets?
16:35:08 <mihgen> and whether we could have range, not subnet, for management, storage, etc. nets
16:35:21 <mihgen> is it in scope?
16:35:21 <akasatkin> it will be possible to add/remove networks via API
16:35:25 <xarses> mihgen: rmoe is planning to fix the apicode so it can be updated in the network yaml
16:35:32 <akasatkin> but not with UI
16:35:44 <mihgen> network yaml ?
16:35:52 <mihgen> do we have a sample of how it's gonna look like?
16:36:10 <xarses> the move from old keys, to the new network roles will also remove the last blocker that prevents the UI from using ranges for every network if we want
16:36:15 <akasatkin> we have one in spec
16:36:23 <mihgen> ok thanks I'll take a look
16:36:48 <akasatkin> ranges will be possible via API
16:37:03 <mihgen> thanks akasatkin & team - excellent work, it's great that we are introducing flexibility finally, and moving fast
16:37:13 <akasatkin> ranges' modification
16:37:17 <xarses> moving on?
16:37:42 <xarses> #topic slow image build on qemu (what are the recommendations?) (kozhukalov)
16:37:59 <kozhukalov_> not very pleasant topic
16:37:59 <kozhukalov_> qemu uses cache=writethrough by default (in new versions writeback, but drivers can change it to writethrough)
16:37:59 <kozhukalov_> it slows down the process and sometimes 3600 secs is not enough to finish build process
16:37:59 <kozhukalov_> not sure it is enough to mention this in docs
16:37:59 <kozhukalov_> users tend to try 6.1 before they read docs
16:37:59 <kozhukalov_> another thing is what could we recommend as a best option to avoid this issue
16:38:00 <kozhukalov_> 1) cache=unsafe (too risky for production)
16:38:00 <kozhukalov_> 2) cache=writeback (could be helpful, but depends)
16:38:01 <kozhukalov_> 3) use vbox instead (which is almost the same as cache=writeback)
16:38:01 <kozhukalov_> 4) use hardware instead
16:38:02 <kozhukalov_> 5) use classic provisioning instead
16:38:02 <kozhukalov_> of course we need to improve this, as far as we want fuel to work on qemu out of the box
16:38:34 <mihgen> is there a way to understand this by code, from fuel master node?
16:38:45 <mihgen> like measure something and understand it's so slow?
16:39:00 <mihgen> we need some check like this, and just fail early with the message to the user about this
16:39:10 <kozhukalov_> debootstrap and apt-get
16:39:15 <mihgen> we could push this as updated package even
16:39:22 <kozhukalov_> they produce loads of syncs
16:40:08 <kozhukalov_> ok, your suggestion is before starting to build make sure that the performance is high enough
16:40:09 <mihgen> kozhukalov_: do we have this info mentioned in release notes?
16:40:12 <kozhukalov_> right?
16:40:14 <mihgen> yes
16:40:22 <xarses> would it help if we zero some of the image before starting?
16:40:31 <mihgen> if we don't have it in release notes, then we should add it there
16:40:37 <kozhukalov_> any ideas how to do this?
16:40:42 <xarses> my guess is the high sync is due to it attempting to allocate more space
16:41:10 <xarses> its also possible that the underlaying filesystem may have some impact so maybe we want to test options
16:41:13 <kozhukalov_> there is doc bug
16:41:34 <kozhukalov_> but it wasn't added into docs before yesterday
16:42:20 <mihgen> kozhukalov_: it has to be in release notes. Let's see if we can run what I suggested as well…
16:42:30 <kozhukalov_> xarses: it is not true about allocating
16:43:06 <kozhukalov_> mihgen: yes, good idea, let's try to find out how to implement this
16:43:17 <mihgen> guys we are pressed for time, can we move on?
16:43:33 <xarses> yep
16:43:35 <kozhukalov_> moving
16:43:37 <mihgen> kozhukalov_: thanks for bringing this up, this is important, I'll follow up on this ..
16:43:52 <xarses> lets add this to the ML
16:43:56 <xarses> #topic removing classic provisioning (agordeev)
16:43:59 <agordeev> hi
16:44:00 <agordeev> finally, the scope has been confirmed and spec was merged.
16:44:02 <agordeev> It was decided just to disallow to use classic provisioning for environments created in 7.0.
16:44:07 <agordeev> Will be implemented as a small change in nailgun.
16:44:13 <agordeev> It was also decided to keep classic provisioning for older environments.
16:44:15 <agordeev> In other words, after upgrade to 7.0, older environments provisioned with 5.x/6.x will be able to continue to use classic provisioning if they were provisioned with classic.
16:44:54 <mihgen> agordeev: when do we plan to remove the code completely?
16:45:07 <xarses> #action mihgen will follow up about proposing a package to push update for qemu
16:45:10 <mihgen> or it's ok as we don't need to support it anyway?
16:45:11 <agordeev> mihgen: nor earlier than in 8.0
16:45:25 <mihgen> do you need to hack kickstart now to make it work?
16:45:31 <mihgen> for 7.0 I mean?
16:45:49 <agordeev> mihgen: no hack
16:45:52 <mihgen> I remember we had some if else clauses there just to check version, so then to use right kernel version..
16:46:08 <mihgen> agordeev: ok. did QA agreed on the approach?
16:46:09 <agordeev> mihgen: check will be implemented in API handler in nailgun
16:46:30 <agordeev> mihgen: yeah, qa agreed on it
16:46:45 <mihgen> agordeev: excellent, let's do it quick, it should be tiny change
16:48:09 <mihgen> xarses: ?
16:48:23 <mihgen> let's move on guys
16:48:25 <xarses> #topic Kilo and Juno packages on master node (rvyalov) Ci for Kilo (afedorova)
16:48:29 <rvyalov> For 7.0 on master node we need openstack keystone from juno and openstack-clients (for OSTF) from kilo.
16:48:30 <rvyalov> But  they have crossed python dependency.
16:48:31 <rvyalov> So the mos-packaging team will build a one keystone packages with dependencies and will upgrade openstack-clients to Kilo version.
16:48:33 <rvyalov> related bug: https://bugs.launchpad.net/mos/+bug/1469552
16:48:33 <openstack> Launchpad bug 1469552 in Mirantis OpenStack "update python-clients pacakges for master node Fuel 7.0" [High,Confirmed] - Assigned to MOS Packaging Team (mos-packaging)
16:48:49 <xarses> sorry mihgen
16:49:12 <angdraug> is this the same bug: https://bugs.launchpad.net/fuel/+bug/1470844
16:49:12 <openstack> Launchpad bug 1470844 in Fuel for OpenStack "MOS 7.0 can't be deployed on CentOS because of python-openstackclient" [Critical,Confirmed] - Assigned to Ivan Berezovskiy (iberezovskiy)
16:49:17 <xarses> rvyalov: do we have the horizon package fixed yet?
16:49:26 <mzawadzki_mirant> no
16:49:30 <mzawadzki_mirant> fix in development
16:49:34 <rvyalov> no
16:49:41 <mihgen> guys do we actually update keystone on master or not?
16:49:44 <xarses> CI status
16:49:57 <rvyalov> no, we dont update keystone
16:50:02 <mzawadzki_mirant> Horizon/JS fix in progress: https://review.fuel-infra.org/#/c/8945/
16:50:21 <mihgen> so what's the CI readiness now?
16:50:29 <bookwar> mihgen: keystone will be old and it will be installed into virtual environment, while all clients will be updated
16:50:31 <mihgen> do we want to run both Juno & Kilo, right?
16:50:46 <mihgen> bookwar: why virtual env???
16:50:53 <xarses> lets add manifests status so we can discuss them all after hearing it
16:51:00 <angdraug> bookwar: in a meeting with a linux team, we discussed an imho better option
16:51:00 <mihgen> why can't we update all clients without keeping old keystone?
16:51:13 <bookwar> mihgen: because of dependencies
16:51:13 <angdraug> put Kilo clients in a docker container, we only need them for ostf right?
16:51:28 <mihgen> what dependencies?
16:51:30 <rvyalov> it is will be 1 package include all dependencies
16:51:37 <rvyalov> python dependencies
16:51:38 <mihgen> clients requre newer keystone?
16:51:50 <mihgen> or 3rd party python deps
16:51:54 <rvyalov> no, client require new dependencies
16:52:01 <angdraug> 3rd party python
16:52:28 <mihgen> and there is no backward compatibilty? what exact pkgs?
16:52:44 <mihgen> why keystone won't work with some newer 3rd party libs?
16:53:02 <mihgen> I just don't want to build complexity with venv
16:53:06 <bookwar> question goes to igor yozhikov and his team
16:53:08 <mihgen> more over, we already have docker
16:53:25 <mihgen> why the hell one more virt layer is needed?
16:53:38 <angdraug> zigo: around? can you comment on python-openstackclient dependencies?
16:54:04 <mihgen> ok let's keep it as open question, we need to follow up in ML
16:54:14 <angdraug> and once again, can anyone confirm if we need the kilo client for anythong other than ostf???
16:54:14 <mihgen> let's move to CI question then
16:54:39 <bookwar> about CI, there is no point in creating Kilo jobs to test kilo patches on Fuel CI, because to test patches we'd need Kilo ISO and the only good Kilo iso we have is exactly the ISO which contains all those Kilo patches already
16:55:04 <IvanBerezovskiy> angdraug, Kilo python-openstackclient is needed for Kilo puppet modules
16:55:17 <angdraug> bookwar: yet holser is saying we did that for juno and before that for icehouse
16:55:17 <mihgen> wow there is a point in CI
16:55:29 <mihgen> as you'll at least start landing changes on top immediately
16:55:57 <mihgen> angdraug: also, I don't quite get it: we can have ISO with Kilo packages
16:56:05 <mihgen> fuel-lib is actually replaced each time, right?
16:56:18 <angdraug> IvanBerezovskiy: do we run Kilo puppet modules *on Fuel node*?
16:56:28 <mihgen> so it's in fact having CI which uses kilo pkgs instead of juno, and just have manifests under testing
16:56:42 <mihgen> angdraug: we do for keystone
16:56:56 <bookwar> mihgen: iso with kilo packages but without fuel-library patches doesn't pass CI
16:56:58 <mihgen> but question is still open, what exactly is ran there from openstack clients perspective
16:57:07 <mihgen> and whether it could be replaced to smth else
16:57:27 <bookwar> ..so it can not be used as a base for tests
16:57:34 <mihgen> bookwar: that's the idea!
16:57:38 <mihgen> it doesn't pass now
16:57:49 <angdraug> we also need to answer the question of why and where kilo branch of puppet-keystone breaks compatibility with juno
16:57:49 <xarses> so I was thinking that we should take custom ISO with packages and make it so the fuel library commits for kilo manifests so that we can ensure that the kilo manifests are well tested before merging them. I think it's a mess that if we merge may take a week to unwind if we are not careful
16:57:51 <mihgen> but once all fuel-lib changes are there, it will start passing
16:58:00 <mihgen> angdraug: +1
16:59:07 <angdraug> mihgen: do we have this transition process documented anywhere?
16:59:12 <mihgen> so bookwar again - I understand with old puppet modules and kilo pkgs  it's gonna be -1
16:59:24 <angdraug> if we've already done it twice, surely we've got a transition plan somewhere that we could reuse?
16:59:28 <mihgen> but if you chain patches needed to make it green, I expect it to become +1
17:00:07 <angdraug> mihgen: I am very worried about merging a long chain of patches where only the tip of the branch has +1 from CI
17:00:17 <xarses> ok guys, we are out of time
17:00:17 <mihgen> let's move to #fuel-dev to discuss it for a bit more
17:00:25 <xarses> #endmeeting