16:00:40 <xarses> #startmeeting fuel
16:00:40 <openstack> Meeting started Thu Jan 21 16:00:40 2016 UTC and is due to finish in 60 minutes.  The chair is xarses. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:40 <xarses> #chair xarses
16:00:40 <xarses> Todays Agenda:
16:00:40 <xarses> #link https://etherpad.openstack.org/p/fuel-weekly-meeting-agenda
16:00:40 <xarses> Who's here?
16:00:41 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:00:43 <openstack> The meeting name has been set to 'fuel'
16:00:45 <openstack> Current chairs: xarses
16:00:50 <mattymo> hi!
16:00:52 <jaranovich> hi!
16:00:53 <yottatsa> o/
16:00:54 <alex_didenko> hey
16:00:56 <romcheg> o/
16:00:56 <asvechnikov_> hi
16:00:58 <monester> hi
16:00:58 <SheenaG> o?
16:01:01 <SheenaG> o/
16:01:06 <dpyzhov> hi
16:01:11 <dbilunov> o/
16:01:11 <maximov> hi
16:01:18 <holser_> o/
16:01:24 <ashtokolov> o/
16:01:26 <aglarendil> \o/
16:01:37 <xarses> good morning // afternoon // evening // night folks
16:01:38 <mihgen> hi
16:01:43 <mwhahaha> hi
16:01:48 <zynzel> hello
16:01:48 <holser_> +1
16:01:52 <angdraug> o/
16:02:12 <rvyalov> hi
16:02:17 <fzhadaev> Hi!
16:02:36 <xarses> #topic Action items from last meeting
16:02:37 <gouthamr> helllo!
16:02:45 <xarses> pyzhov will announce changes in swarm-blocker tag - done
16:02:53 <xarses> vkramskikh will raise topic in ML about https://review.openstack.org/#/c/261229/ - done
16:02:59 <xenolog13> \~/
16:03:15 <warpc> hi!
16:03:29 <xarses> thanks for being on top of the action items
16:03:33 <xarses> #topic Telco Team Status (fzhadaev)
16:03:39 <fzhadaev> Fuel Telco Team Status:
16:03:39 <fzhadaev> 1. Primary activity is fixing of bugs
16:03:39 <fzhadaev> In progress: 6
16:03:39 <fzhadaev> Done: 7
16:03:39 <fzhadaev> 2. According to 9.0 we've started scoping a few NFV-related features. For now we have only started the work on the features so don't have blueprints yet.
16:03:39 <fzhadaev> That's all. Any questions?
16:04:37 <maximov> what NFV features did you start to scope?
16:04:45 <xarses> so you dont even know which you are working on?
16:05:03 <yottatsa> NFV + Workload optimization initiative is currently driven by dklenov. It provides huge pages, CPU pinning, SR-IOV for Fuel 9.0 on Mitaka and Kilo, and DPDK on Mitaka.
16:05:04 <fzhadaev> maximov: NUMA and CPU Pinning, Huge Pages, ect...
16:05:21 <yottatsa> xenolog and me are working on gathering requirements for UI and fuel-python teams. Estimates and specs will be ready by the end of this week.
16:05:38 <yottatsa> We have some limitations for it
16:05:39 <maximov> fzhadaev do you work with yottatsa on this?
16:06:09 <SheenaG> yottatsa: for NFV features, there should be no UI work in Mitaka
16:06:24 <fzhadaev> maximov: I'm working with dklenov :)
16:06:41 <maximov> SheenaG: sure? because Greg wanted to see everything in UI
16:07:07 <xarses> SheenaG: isn't that about being able to put attributes on nodes?
16:07:21 <SheenaG> maximov: I'll take it up with Greg - it's not a hard requirement AFAIK and I think it's unlikely we'll get UI done since scoping hasn't even started for API/CLI
16:07:36 <SheenaG> xarses: wat
16:07:57 <xarses> we need node attributes to target nodes for NFV features
16:08:18 <xarses> like this node nic 0-3 is SR-IOV dedicated
16:08:22 <angdraug> xarses: some nodes or all nodes in an env?
16:08:31 <SheenaG> angdraug: some nodes I believe
16:08:32 <xarses> angdraug: portion of nodes
16:08:48 <maximov> SheenaG: ok, if we can remove UI requirements from NFV features for mitake it will simplify our implementation of course
16:08:50 <SheenaG> xarses: yes, that sounds right - but my understanding was that CLI was the initial plan of attack
16:09:03 <xarses> ok, lets follow up offline
16:09:05 <SheenaG> maximov: I'll argue with Greg and get back to you
16:09:14 <maximov> SheenaG: yes, I agree with, we can start with CLI
16:09:24 <yottatsa> angdraug xarses: we assume that HW acceleration like SR-IOV or DPDK will be NIC attribute in interface configuration
16:09:53 <xarses> yes, I assume that's what we need from the UI
16:10:10 <xarses> lets follow up offline and move to the next topic
16:10:18 <angdraug> lets move on and leave an action for SheenaG to report back next week
16:10:23 <xarses> #topic Mixed Team Status (zynzel / bkupidura)
16:10:34 <SheenaG> angdraug: thanks
16:11:17 <xarses> #action SheenaG will follow up with GregE regarding desired UI elements from NFV features work in 9
16:11:23 <zynzel> Mixed team was working mostly on bug fixing and review. We fixed a few bug connected to features delivered in 7.0 and 8.0 (reduced footprint, advanced config). For now we have
16:11:23 <zynzel> only two high bugs. https://bugs.launchpad.net/fuel/+bug1524978 which can be tricky because it is connected to upstream qemu-kvm package. MOS-linux (Dmitry Teselkin) team already working on it. https://bugs.launchpad.net/fuuel/+bug/1535754 Alex Zvyagintsev and Viacheslav Valyavskiy is working on it, probably connected to https://bugs.launchpad.net/fuel/+bug/1518306 but still under investigat
16:11:25 <openstack> Launchpad bug 1535754 in Fuel for OpenStack 8.0.x "[Reduced footprint] Provisioning on KVM fail with "MCollective agents 'uploadfile' '9' didn't respond within the allotted time."" [High,Confirmed]
16:11:27 <openstack> Launchpad bug 1518306 in Fuel for OpenStack "Mcollective can be restarted after the deployment was started" [High,Fix committed] - Assigned to Artem Roma (aroma-x)
16:11:38 <zynzel> ion.
16:11:38 <zynzel> questions?
16:12:52 <zynzel> https://bugs.launchpad.net/fuel/+bug/1524978
16:12:54 <zynzel> missing '/' from link, sorry.
16:12:54 <openstack> Launchpad bug 1524978 in Fuel for OpenStack mitaka "/usr/bin/generate_vms.sh: Could not access KVM kernel module: Permission denied" [High,Confirmed] - Assigned to Bartosz Kupidura (zynzel)
16:13:05 <angdraug> thanks for the update zynzel
16:13:17 <angdraug> lets move on
16:13:36 <xarses> #topic Bugs team status (dpyzhov) https://etherpad.openstack.org/p/fuel-bugs-status
16:13:41 <dpyzhov> hi
16:13:51 <dpyzhov> we've moved all medium/low bugs to 9.0
16:14:04 <dpyzhov> And actually started to work on them
16:14:14 <dpyzhov> But most of the team is focused on 8.0 bugs
16:14:22 <dpyzhov> We have several blockers that are in progress
16:15:01 <holser_> What about high and critical?
16:15:10 <angdraug> dpyzhov: the etherpad shows combined 8.0/9.0 numbers right?
16:15:23 <dpyzhov> There are about 50 bugs in 8.0 that are high/crit
16:15:31 <dpyzhov> no, etherpad is only about 8.0
16:15:38 <dpyzhov> This is our main focus right now
16:16:09 <mihgen> are we converging by HCF?
16:16:10 <xarses> how is the burn down rate for 8.0?
16:16:10 <angdraug> hm, if it includes only 8.0, what are the 12 Medium/Low bugs in Python & Library?
16:16:15 <dpyzhov> There are 100 medium/low bugs in python for 9.0
16:16:17 <mihgen> do we expect lots of bugs to come .. ?
16:16:31 <dpyzhov> and 50 in library
16:17:13 <dpyzhov> angdraug: infra run autotargeting script today
16:17:15 <angdraug> we're down 16 high/critical bugs since last week, that gives us 4 weeks before we come down to zero at that rate
16:17:24 <dpyzhov> I've already moved that medium out of 8.0
16:17:36 <dpyzhov> we still expect new bugs
16:17:58 <dpyzhov> our acceptance testing will be finished at the HCF
16:18:08 <dpyzhov> so before that time we will get new bugs
16:18:21 <dpyzhov> But bugs income is less than our expectation
16:18:23 <angdraug> and we have 3 weeks left until HCF, so looks like we're still at risk of missing it, though not by much
16:19:06 <angdraug> dpyzhov monester: is autotargeting script still mis-labeling bugs?
16:19:11 <dpyzhov> well, about 50% of reports are really not bugs
16:19:53 <dpyzhov> maybe 40% ) it's my feeling, I didn't get real numbers
16:20:12 <angdraug> ok, many more topics to cover, lets move on?
16:20:46 <dpyzhov> ok for me
16:21:13 <mattymo> a lot of duplicates coming through as well
16:21:57 <dpyzhov> I guess by HCF we'll take every bug and decide if we really need it to be in 8.0
16:22:17 <dpyzhov> at the moment we are trying to fix as much as we can
16:22:18 * angdraug prods xarses
16:22:27 <xarses> yep, lets move along
16:22:42 <xarses> #topic UI team status (jaranovich)
16:22:47 <jaranovich> Hi! This week we've fixed 2 High bugs for 8.0 (last bugs, I hope), resolved some tech debt and are starting our work on 9.0 features. Here is the list of tasks we're going to deliver in 9.0:
16:22:47 <jaranovich> 1) https://blueprints.launchpad.net/fuel/+spec/converge-to-eslint-config-openstack - we're community project and we have to switch to OpenStack style of coding for JavaScript. This week we were mostly busy with this blueprint and are going to finish it next week.
16:22:47 <jaranovich> 2) Role panel redesign (no blueprint yet) - we're going to beautify our role panel. Currently it takes 1 screen on my laptop just to display the core list of roles.
16:22:47 <jaranovich> 3) UI for advanced settings feature (no blueprint yet) - advansed settings feature is going to be delivered in 8.0, but it will be available via CLI only. We're going to implement UI for it.
16:22:47 <jaranovich> We expect some more features to be delivered in 9.0, but we cannot start the work on it now:
16:22:48 <jaranovich> 1) https://blueprints.launchpad.net/fuel/+spec/remove-vendor-code - waiting for infra
16:22:48 <jaranovich> 2) https://blueprints.launchpad.net/fuel/+spec/separate-fuel-ui-repo - waiting for implementation of cross-repo tests and merge of https://review.openstack.org/#/c/260367/
16:22:49 <jaranovich> 3) NFV features - waiting for breaking down of the stories - it's not clear yet what needs to be done in UI.
16:22:49 <jaranovich> Questions?
16:23:55 <mihgen> great report jaranovich, very clear, thanks!
16:24:04 <SheenaG> Is there anyone here from infra who can comment on the status of the new repos?
16:24:20 <jaranovich> mihgen: thank you)
16:24:26 <xarses> jaranovich: please add me to the role's blueprint when it's ready
16:24:28 <angdraug> rvyalov: bookwar_ ^
16:24:42 <jaranovich> xarses: sure!
16:25:00 <kozhukalov> SheenaG: which repos are you talking about?
16:25:19 <SheenaG> kozhukalov: the downstream repos that unblock the ability to remove vendor code from Fuel UI codebase
16:25:21 <kozhukalov> do you mean separating UI from fuel-web?
16:25:50 <bookwar_> repos created, and we sync code there
16:26:14 <bookwar_> but we don't have a usage scenario for them yet
16:26:24 <bookwar_> no policy defined
16:26:33 <SheenaG> bookwar_: can teams commit vendor specific code there, or this is waiting on the usage scenario/policy?
16:26:59 <bookwar_> they can commit the code, but we need to understand who should be responsible to merge it
16:27:09 <angdraug> commiting code without policy? that would be anarchy :P
16:27:36 <bookwar_> and how ci should work for it
16:27:36 <xarses> lets follow up on the ML then
16:27:48 <mihgen> I would not imply any restrictions, I'd suggest to just discuss in ML once questions appears
16:28:14 <angdraug> mihgen: by any restrictions you mean, same permissions as for current fuel-web repo?
16:28:24 <angdraug> s/permissions/ACLs/
16:28:43 <xarses> #action SheenaG bookwar will follow up on creating a scenario/policy for vendor specific code
16:28:44 <angdraug> I think that keeping the same ACLs would be a sane default
16:29:02 <SheenaG> 2 action items!  Sweet!
16:29:15 <xarses> winner winner chicken dinner!
16:29:19 <xarses> #topic UCA Mitaka status (mattymo)
16:29:21 <mattymo> Hi all!
16:29:33 <mattymo> With https://github.com/mwhahaha/fuel-plugin-upstream and https://review.openstack.org/#/q/topic:puppet-mitaka , we are able to deploy Mitaka using UCA and upstream manifests. There are some minor adaptations to top level puppet code, but every change is compatible back to Liberty.
16:29:33 <mattymo> The only noteworthy change is the removal of EC2 Nova API, which means a bit of haproxy service renaming.
16:29:33 <mattymo> I was strongly hoping to get a passed BVT ready for you all today, but yesterday we were blocked by merge conflicts and rebasing, and today we are blocked by a critical bug blocking ISO build, https://bugs.launchpad.net/fuel/+bug/1536608
16:29:35 <openstack> Launchpad bug 1536608 in Fuel for OpenStack "Community ISO 9.0 build fails with errors about nginx container" [Critical,In progress] - Assigned to Stanislaw Bogatkin (sbogatkin)
16:29:35 <ogelbukh> hi mattymo
16:30:23 <xarses> mattymo: so, no update on the ML front yet?
16:30:24 <mattymo> I should note that I was able to deploy start-to-finish by hand yesterday with a custom ISO (plus 1 patch applied by hand) without any issues, plus pass OSTF
16:30:37 <mattymo> I can't technically write an email until I can get a BVT passed....
16:30:44 <xarses> ok
16:30:45 <angdraug> https://bugs.launchpad.net/fuel/+bug/1536608 has a fix on review: https://review.openstack.org/270859
16:30:46 <openstack> Launchpad bug 1536608 in Fuel for OpenStack "Community ISO 9.0 build fails with errors about nginx container" [Critical,In progress] - Assigned to Stanislaw Bogatkin (sbogatkin)
16:30:54 <mattymo> unless you want me to be untruthy
16:31:36 <mihgen> let's push this one forward!
16:31:37 <mattymo> angdraug, we're aggressively testing the fixes
16:31:37 <xarses> cool, glad to see this making progress
16:31:44 <xarses> #topic Upgrades team status (ogelbukh)
16:31:46 <mihgen> I'm looking forward for good news :)
16:32:26 <ogelbukh> we're working on this spec https://review.openstack.org/#/c/252376/
16:32:36 <ogelbukh> both backup and restore functions are in review
16:32:42 <ogelbukh> https://review.openstack.org/#/c/267086/
16:32:49 <ogelbukh> https://review.openstack.org/#/c/267702/
16:33:09 <ogelbukh> expect backup in QA tomorrow, and restore on Monday
16:33:28 <angdraug> hm, can you test backup without restore?
16:33:35 <ogelbukh> we also run some tests in scale lab for upgrade to 7.0
16:33:54 <ogelbukh> angdraug: yes, the procedure and contents of archive
16:34:41 <ogelbukh> results from scale lab I will try to get as a whitepaper of a kind
16:34:56 <ogelbukh> and we also work on configdb
16:35:16 <ogelbukh> now it has basic put/get and we integrate it with nailgun and deployment
16:35:37 <ogelbukh> to get a PoC for the external node classifier use case
16:35:57 <ogelbukh> guess that's all from my side
16:36:21 <angdraug> great, thank!
16:36:24 <xarses> thanks, do we have the configdb code up on review?
16:36:26 <angdraug> thanks, even :)
16:37:15 <ogelbukh> xarses: in Mirantis Gerrit yet
16:37:28 <angdraug> ogelbukh: why?
16:37:40 <ogelbukh> angdraug: it's still early PoC
16:37:51 <angdraug> not a good reason :/
16:38:20 <xarses> ok lets move on
16:38:22 <ogelbukh> ah, sorry
16:38:24 <ogelbukh> mirantis github
16:38:27 <ogelbukh> https://github.com/Mirantis/tuning-box
16:38:28 <angdraug> ogelbukh: please move it review.openstack.org
16:38:36 <ogelbukh> angdraug: k, will start moving
16:38:42 <xarses> #topic Team network status (alex_didenko)
16:38:58 <alex_didenko> Network team is working on bugfixing mostly. All needed system tests are completed and are running under swarm runner (thread_7).
16:38:58 <alex_didenko> We're also involved in NFV + Workload optimization story research.
16:38:58 <alex_didenko> We're still discussing problem of VIPs autoallocation for multi-rack environments, so if you're interested in this subject please feel free to participate:
16:38:58 <alex_didenko> http://lists.openstack.org/pipermail/openstack-dev/2016-January/084182.html
16:38:59 <alex_didenko> We're also working on external LB support, it's going to be implemented as Fuel plugin. I'm finalizing the work on plugin. Right now I'm able to deploy multirack with external LB and controllers in different racks with Fuel-8.0 + external-loadbalancer plugin + manual fix for LP#1524320.
16:39:03 <xarses> #action ogelbukh will start moving configdb code over to openstack
16:39:35 <alex_didenko> That's it as for out status
16:39:39 <alex_didenko> *our
16:39:57 <mihgen> alex_didenko: any blueprints for 9.0 .. ?
16:40:08 <angdraug> alex_didenko: is manual fix for the external LP bug going to get automated?
16:40:18 <hyamauchi> quit
16:40:18 <angdraug> err, external LB...
16:40:20 <hyamauchi> exit
16:40:21 * angdraug can't type
16:40:45 <alex_didenko> Moved from 8.0 to 9.0
16:40:45 <alex_didenko> https://blueprints.launchpad.net/fuel/+spec/allow-any-vip
16:40:45 <alex_didenko> https://blueprints.launchpad.net/fuel/+spec/dhcp-vips
16:40:45 <alex_didenko> https://blueprints.launchpad.net/fuel/+spec/selective-default-gateway-net
16:40:46 <alex_didenko> Planned for 9.0
16:40:46 <alex_didenko> (drafting, driven by sbogatkin) Selective SSL support in UI.
16:40:47 <alex_didenko> https://blueprints.launchpad.net/fuel/+spec/multi-rack-with-dynamic-routing
16:41:39 <alex_didenko> angdraug: that fix is not for external LB, it's about 500 internal server from nailgun error when controller are moved to different racks
16:42:30 <xarses> ok, moving?
16:42:30 <mihgen> ok thanks
16:42:46 <angdraug> yup, lets move
16:42:53 <xarses> #topic Enhancements Team status (ashtokolov)
16:43:10 <ashtokolov> We are working on Basic LifeCycle Management for 9.0
16:43:17 <ashtokolov> Main goals:
16:43:24 <ashtokolov> 1. Enable task-based deployment engine as a default
16:43:32 <ashtokolov> 2. Allow user to gracefully stop Deployment and further restart it
16:43:39 <ashtokolov> 3. Ensure that deployment Tasks in Fuel are idempotent
16:43:45 <ashtokolov> 4. Allow users to use "Settings" tab and rerun deployment Tasks on already deployed clusters
16:43:59 <ashtokolov> 5. Improve UX of Fuel Plugins in post-deployment stage
16:45:31 <ashtokolov> Questions?
16:45:44 <xarses> can we also fix the single deployment_tasks.yaml requirement and switch it back to what we support for library, tasks.yaml anywhere in the deployment folder
16:46:15 <angdraug> xarses: bug #?
16:46:27 <ashtokolov> xarses we are going to remove tasks.yaml from plugins v5 for 9.0
16:46:37 <xarses> I have to create one
16:46:40 <xarses> tasks.yaml?
16:46:47 <xarses> or deployment_tasks.yaml?
16:46:54 <ashtokolov> yes tasks.yaml
16:47:04 <mattymo> ashtokolov, +100. We needed that for a while already
16:47:09 <ashtokolov> this file contains old format of tasks
16:47:48 <xarses> ok, that will enable us so we go back to the fuel-library way of parsing then
16:47:49 <ashtokolov> And only deployment_tasks.yaml contains granular tasks
16:48:17 <xarses> ok, lets move
16:48:23 <xarses> #topic automation of MAINTAINERS file: current status and feedback (monester)
16:48:36 <monester> We lauched auto-add-reviews job 2 weeks ago. This job adding reviewers automatically based on content of maintainers file. Job is working perfectly well (already >3000 times) if there is no errors in maintainers file. Do you have any feedback for the script? Maybe you have faced problems with the script?
16:49:14 <angdraug> any feedback for monester anyone? mihgen?
16:49:18 <xarses> none, that I have noticed
16:49:21 <mihgen> ikalnitsky: holser_ Core reviewers - do you see a decrease in load?
16:49:35 <holser_> well
16:49:39 <holser_> kind of
16:49:40 <ikalnitsky> mihgen: yeah, i think i see.
16:49:41 <xarses> it looks awesome for it to go off automatically
16:50:07 <ikalnitsky> probably because of holidays.. but this days i see less stale reviews
16:50:46 <mihgen> holser_: kind of.. ?) let's sync offline tomorrow on this, I hope script will help to significantly reduce a load to core reviewers :)
16:50:53 <mihgen> if it doesn't, then may be we don't have enough maintainers
16:51:07 <holser_> ok
16:51:08 <mihgen> thanks monester for implementation / run of a script!
16:51:15 <holser_> let’s follow up tomorrow
16:51:25 <monester> thx
16:51:29 <angdraug> moving on?
16:51:30 <xarses> #topic NFV status (veremin aka yottatsa)
16:51:47 <angdraug> hm, deja vu
16:51:57 <yottatsa> as I've told ealier estimates and specs will be ready by the end of this week.
16:52:00 <xarses> erm ya
16:52:10 <xarses> anything we didn't already talk about?
16:52:18 <yottatsa> Right now we run into next limitations: for HUGE we will use 2048KB preallocated pages, for CPU pinning we are planning to provide mechanisms to reserve cores for system and dpdk, SR-IOV and DPDK interfaces will be limited to VLAN segmentation, and interfaces will be used for private traffic only.
16:52:22 <yottatsa> Those features requires fresh and modified packages (qemu-kvm, libvirt and openvswitch) to be provided, it would be yes as soon as all of it will be builded by dteselkin.
16:52:31 <angdraug> summit is coming
16:53:14 <xarses> yottatsa: are we going to be able to change the amount of reserved cores based on the role? i.e. ceph needs some cores if it's an OSD role
16:53:15 <angdraug> please start thinking about and proposing topics to discuss there
16:53:31 <angdraug> oops thought we're already into open discussion )
16:53:55 <yottatsa> xarses right now we're going to reserve cores for at least DPDK EPM
16:54:18 <angdraug> dteselkin: around? do you have an ETA for updated qemu, libvirt, and ovs?
16:54:18 <xarses> ok, keep in mind we will want to extend it
16:54:25 <yottatsa> seems that "accelerated" nodes will be not recommend to place another kinds of workloads
16:54:43 <xarses> #topic open discuss
16:54:46 <yottatsa> ovs and qemu is already built
16:55:00 <yottatsa> libvirt is in progress AFAIK
16:55:09 <mihgen> about summit - I'd suggest to discuss HA / reference architectures, joined talks with puppet-openstack team
16:55:41 <mihgen> multirack deployment in particular
16:55:47 <angdraug> I think configdb is important to bring up, too
16:56:22 <angdraug> and running fuel-qa on nodepool
16:57:19 <xarses> there is a proposal out for multirack
16:57:24 <angdraug> well don't all speak at once :)
16:57:56 <xarses> and for working on the reference arch
16:58:31 <xarses> apparently we are all afraid of public speaking
16:58:57 <angdraug> I know I am, but I am learning to control that fear :)
16:59:12 <angdraug> ok, time to close
16:59:20 <xarses> thanks all
16:59:25 <angdraug> thanks
16:59:30 <xarses> #endmeeting