16:00:13 <xarses> #startmeeting fuel
16:00:14 <xarses> #chair xarses
16:00:14 <xarses> Todays Agenda:
16:00:14 <xarses> #link https://etherpad.openstack.org/p/fuel-weekly-meeting-agenda
16:00:14 <xarses> Who's here?
16:00:15 <openstack> Meeting started Thu Feb 11 16:00:13 2016 UTC and is due to finish in 60 minutes.  The chair is xarses. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:16 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:00:17 <mwhahaha> hi
16:00:17 <vkramskikh> hi
16:00:18 <openstack> The meeting name has been set to 'fuel'
16:00:20 <openstack> Current chairs: xarses
16:00:20 <rlu> hi
16:00:21 <rmoe_> hi
16:00:22 <mattymo> hi
16:00:54 <akasatkin> hi
16:00:56 <kozhukal`> hi
16:01:00 <dguryanov> hi
16:01:06 <mihgen> hi
16:01:15 <fzhadaev_> Hi!
16:01:36 <xarses> #topic action items from last meeting
16:01:39 <SheenaG> o/
16:01:42 <xarses> maximov to coordinate with other teams and find a volunteer for puppet-openstack noop job
16:02:46 <xarses> xarses will bring up viability of version information (was version.yaml) for environment triage
16:02:57 <angdraug> \o
16:03:03 <xarses> this is not done, I will try to get to it shortly
16:03:05 <ogelbukh> o/
16:03:08 <xarses> xarses to create thread over QEMU/KVM on ML
16:03:11 <xarses> same =(
16:03:15 * xarses feels the shame
16:03:43 <xarses> back to our regularly scheduled program
16:03:54 <xarses> #topic Fuel CI for puppet-openstack (igorbelikov)
16:04:00 <igorbelikov> hey folks
16:04:09 <igorbelikov> 1) Deployment jobs for puppet-openstack modules testing
16:04:09 <igorbelikov> #link https://ci.fuel-infra.org/view/puppet-openstack/
16:04:09 <igorbelikov> Test cases copy existing ones for fuel-library and share the same product ISO, which is still Liberty for the moment
16:04:25 <igorbelikov> Can be switched to Mitaka ISO (which doesn't pass BVT yet), but I'd really love to use Community ISO for this
16:04:25 <igorbelikov> If there are any reasons not to use Community ISO for puppet-openstack CI - I'd like to hear them
16:04:52 <igorbelikov> 2) Custom test jobs to reproduce failures and get access to debug environments
16:04:52 <igorbelikov> Custom test jobs are still WIP, expected to launch in the next couple of days
16:04:52 <igorbelikov> Reproducing CI failures for puppet-openstack tests will be as easy as just pointing the job to Fuel CI build artifacts
16:05:05 <angdraug> can't be any good reason not to use community iso
16:05:33 <igorbelikov> angdraug: that's my thoughts, and everyone will benefit from more test coverage of community iso
16:05:52 <igorbelikov> 3) Plan for the further actions:
16:05:52 <igorbelikov> a) Switch to Mitaka-based ISO
16:05:52 <igorbelikov> b) Sync fuel-library to HEADs of puppet-openstack modules, fix existing issues and get green builds
16:05:52 <igorbelikov> degorenko, IvanBerezovskiy - any comments on this? ^
16:05:52 <igorbelikov> c) Enable Fuel CI in non-voting mode for every commit to puppet-openstack modules
16:06:44 <igorbelikov> that's the end of my paste, questions?
16:06:45 <xarses> igorbelikov: just the ones we use right?
16:07:03 <igorbelikov> xarses: yeah, you're right
16:07:07 <xarses> we will take account to configure the module as part of the test, or just turn everything on?
16:07:19 <xarses> like murano or similar
16:07:56 <angdraug> xarses: please explain
16:08:08 <xarses> murano is a setting option that is off by default
16:08:20 <xarses> should we test changes to puppet-murano?
16:08:22 <igorbelikov> xarses: can be specific test cases for each module, but initial plan is use the same ones we use for fuel-library
16:08:32 <xarses> do we turn it on for that test? or have it on for all
16:08:44 <angdraug> good question
16:08:51 <xarses> do we even have functional tests for murano
16:09:02 <xarses> in a job to use
16:09:10 <angdraug> ostf tests murano, but the test is heavy, that's why it's off by default
16:09:33 <angdraug> which leads me to conclude that we should only enable it when testing puppet-murano
16:10:37 <igorbelikov> I'll draft a spec with a section about module-testcase mapping, I suppose it will be the best way to settle this
16:11:07 <xarses> ok, for now I think we only vote on the core modules that are easily left on
16:11:16 <xarses> and follow up with the outliers
16:11:53 <xarses> thanks igorbelikov this will be great to have
16:11:58 <xarses> anything else?
16:12:10 <igorbelikov> that's all from my side
16:12:22 <xarses> #topic UI Team status (vkramskikh)
16:12:26 <vkramskikh> Hi! Here is our status for 9.0 features:
16:12:26 <vkramskikh> 1) https://blueprints.launchpad.net/fuel/+spec/remove-vendor-code - the removal request is ready: https://review.openstack.org/#/c/276279/ ; but we agreed not to merge it until we got first workong ISO built from downstream
16:12:26 <vkramskikh> 2) https://blueprints.launchpad.net/fuel/+spec/redesign-of-node-roles-panel - mostly merged, some minor fixes are going to land soon
16:12:26 <vkramskikh> 3) https://blueprints.launchpad.net/fuel/+spec/network-requirements-popup - implementation described in spec is merged, but there are some discussion in the spec
16:12:28 <vkramskikh> 4) https://blueprints.launchpad.net/fuel/+spec/node-display-ip-address - implemented
16:12:30 <vkramskikh> 5) Ability to show all network groups - part of https://blueprints.launchpad.net/fuel/+spec/multirack-in-fuel-ui which wasn't done in 8.0 - good progress, going to land this or the next week
16:12:33 <vkramskikh> 6) https://blueprints.launchpad.net/fuel/+spec/allow-choosing-nodes-for-provisioning-and-deployment - design is in progress
16:12:36 <vkramskikh> 7) https://blueprints.launchpad.net/fuel/+spec/separate-fuel-ui-repo - slow progress, currently trying to make a separate test runner using test runner tools: https://review.openstack.org/#/c/276814/
16:12:39 <vkramskikh> 8) NFV stuff - design is finalized, implementation has been strted
16:12:43 <vkramskikh> Questions?
16:13:44 <mihgen> vkramskikh: thanks, can you update status of blueprints pls?
16:13:51 <vkramskikh> ok
16:13:53 <mihgen> those which are implemented especially :)
16:14:45 <vkramskikh> that's all from my side
16:14:51 <kozhukalov> regarding to 7) we are waiting for postgres access on fuel-web ci slaves
16:15:06 <kozhukalov> gonna be done on Monday
16:16:16 <angdraug> thanks vkramskikh kozhukalov, lets move on?
16:16:23 <vkramskikh> yep
16:16:33 <xarses> #topic Telco Team Status (fzhadaev)
16:17:17 <fzhadaev_> Hi all.
16:17:18 <fzhadaev_> For now, Telco team has three main activities for 9.0:
16:17:18 <fzhadaev_> 1 Support NFV features:
16:17:18 <fzhadaev_> 1.1 Huge pages [1] - spec is on review
16:17:18 <fzhadaev_> 1.2 NUMA/CPU pinning [2] - spec is on review
16:17:18 <fzhadaev_> At the moment main concerns were resolved and we hope to merge them soon.
16:17:18 <fzhadaev_> 2 Daemon Resource Allocation Control [3]
16:17:19 <fzhadaev_> Spec is in progress
16:17:19 <fzhadaev_> 3 Removing Mirantis-specific code from fuel code
16:17:20 <fzhadaev_> Work in progress some patches are on review. Waiting for working ISO from downstream.
16:17:20 <fzhadaev_> [1] https://blueprints.launchpad.net/fuel/+spec/support-hugepages
16:17:21 <fzhadaev_> [2] https://blueprints.launchpad.net/fuel/+spec/support-numa-cpu-pinning
16:17:21 <fzhadaev_> [3] https://blueprints.launchpad.net/fuel/+spec/cgroups
16:17:22 <fzhadaev_> Do you have any questions?
16:18:13 <xarses> fzhadaev_: have we taken into account that we may want to use some of the cgroups stuff to controll processes we install on the nodes?
16:18:20 <xarses> at a later point
16:18:30 <xarses> ie isolate ceph/ceph-mon/mongo...
16:19:16 <fzhadaev_> xarses, design is in progress. you may get more info from spec :)
16:19:30 <xarses> k, thanks
16:19:52 <xarses> fzhadaev_: can you set the spec link in the bp?
16:20:26 <fzhadaev_> Oh, i'll do it. Sure
16:20:31 <xarses> thanks
16:20:56 <fzhadaev_> xarses, https://review.openstack.org/#/c/278426
16:21:07 <xarses> thanks
16:21:12 <xarses> #topic Enhancements Team status (ashtokolov)
16:21:34 <ashtokolov> 1. Task Based Deployment with Astute v1 (production ready) - https://blueprints.launchpad.net/fuel/+spec/enable-task-based-deployment in progress
16:21:42 <ashtokolov> 2. Unlock "Setting" tab and rerun deployment Tasks in post-deployment - https://blueprints.launchpad.net/fuel/+spec/unlock-settings-tab in design
16:21:52 <ashtokolov> 3. Stop/Restart deployment w/o environment reset - https://blueprints.launchpad.net/fuel/+spec/graceful-stop-restart-deployment on review
16:21:59 <ashtokolov> 4. Deployment Tasks idempotence with Fuel Mixed Team https://blueprints.launchpad.net/fuel/+spec/granular-task-idempotency https://blueprints.launchpad.net/fuel/+spec/granular-task-ensurability WIP
16:22:06 <ashtokolov> 5. Fixing backend bugs to unlock Separated deployment/provisioning of nodes on UI https://blueprints.launchpad.net/fuel/+spec/specify-nodes-for-provisioning-and-deployment-in-ui on review
16:22:50 <angdraug> any blockers?
16:23:24 <angdraug> if not, lets move on
16:23:26 <ashtokolov> Unlock "Setting" tab is probably blocked by ConfigDB
16:24:02 <ashtokolov> we schedeled a meeting with Yuri Taraday to discuss it
16:24:51 <ashtokolov> move on?
16:25:03 <angdraug> yup
16:25:06 <xarses> #topic Mixed Team Status (rlu/mrelewicz)
16:25:14 <rlu> Mixed team is working on two features
16:25:14 <rlu> first: multipath support for FC for MOS nodes
16:25:15 <rlu> More info you can find in blueprint: https://blueprints.launchpad.net/fuel/+spec/multipath-disks-support
16:25:24 <rlu> second: granular task idempotency, We have already selected problematic granular tasks, and we are trying to fix it, about 14 tasks has patches in review. already
16:25:35 <rlu> Do you have any questions?
16:25:49 <mihgen> is there any CI in plans to ensure idempotency.. ?
16:26:06 <mihgen> so that if we fix it, we don't regress later..
16:26:17 <xarses> multipatch for FC?
16:26:28 <mihgen> xarses: for idempotancy
16:26:41 <rlu> yes, we talk about ci, we have plan to do it
16:26:52 <mihgen> xarses: sorry misread
16:27:09 <xarses> are we not dealing with multipatch for iSCSI ? just the FC HBA?
16:27:12 <mihgen> rlu: whom do you guys sync with on this matter?
16:27:12 <mwhahaha> currently there are two patches to astute to start reporting information that can be used in a CI
16:27:15 <mwhahaha> for idempotency
16:27:41 <mwhahaha> i'm looking into a way to have something in the qa test framework to be able to report on this perhaps in swarm
16:27:45 <rlu> we start with FC, because is eisuer ti implement
16:28:07 <mihgen> mwhahaha: why not on gating?
16:28:12 <mwhahaha> takes too long
16:28:17 <mwhahaha> we have to double run deployments
16:28:22 <mwhahaha> to capture this information
16:28:52 <mwhahaha> in the long term if we can refactor our tasks to be normal puppet classes then we could use beaker or something
16:28:54 <mihgen> qa is gonna be busy till FF, so may be we would need to help them with writing such tests..
16:28:56 <rlu> about ci - we were talk on our sync-up, on Monday about
16:29:20 <mwhahaha> but currently we are relying on the deployment to capture the change information so to find out if something is truely idempotent we need to double run the deployments
16:29:31 <mwhahaha> which can take 2+ hours
16:29:38 <mihgen> got you
16:29:42 <mwhahaha> and we don't check sahara,etc by default
16:30:06 <rlu> mwhahaha: tomorrow we have sync too, I can invite you, we can talk about it
16:30:25 <mwhahaha> sure
16:30:39 <xarses> moving?
16:31:07 <xarses> #topic Bugs team status (dpyzhov)
16:31:11 <dpyzhov> hi guys
16:31:27 <dpyzhov> right now we have small income of bugs for 8.0
16:31:35 <dpyzhov> so team is focused on 9.0
16:31:38 <mattymo> +1 low bugs :D
16:31:51 <dpyzhov> Right now we working on bugs retriaging
16:32:03 <dpyzhov> we have only 54 bugs in library
16:32:09 <dpyzhov> and 137 bugs in python
16:32:30 <dpyzhov> I'm not sure how much time will take to fix them
16:32:37 <dpyzhov> we need to complete retriage first
16:32:49 <dpyzhov> some bugs will disappear with new features
16:32:59 <dpyzhov> some bugs are duplicates
16:33:12 <dpyzhov> I'd say that we are green in library
16:33:20 <dpyzhov> and in yellow state for python
16:34:12 <dpyzhov> for high priority we are have almost the same number of bugs in python and library
16:34:23 <dpyzhov> 24 in python and 17 in library
16:34:40 <dpyzhov> that's all from my side
16:34:43 <dpyzhov> any questions?
16:34:52 <angdraug> sounds good for 3 weeks before 9.0 FF :D
16:35:17 <dpyzhov> yep, it looks like most of medium bugs in python will be moved to 10.0
16:36:02 <xarses> moving?
16:36:41 <xarses> #topic Build team status (kozhukalov)
16:36:48 <kozhukalov> 1) Docker removal:
16:36:48 <kozhukalov> all patches have been merged
16:36:48 <kozhukalov> still there some patches on review that backport
16:36:48 <kozhukalov> those changes that had been made in nailgun module
16:36:51 <kozhukalov> by the time of merge party on Monday
16:36:54 <kozhukalov> 2) Centos bootstrap removal:
16:36:59 <kozhukalov> all patches have been merged
16:37:01 <kozhukalov> ISO build now takes about 7 minutes (make -j10)
16:37:05 <kozhukalov> gonna become even faster
16:37:08 <kozhukalov> Product ISO job takes about 13 minutes (single thread)
16:37:11 <kozhukalov> Now we don't have any late packages and we are ready for the next huge step
16:37:14 <kozhukalov> We are getting rid of building rpm/deb packages together with ISO
16:37:18 <kozhukalov> and instead we will download them from packaging CI using packetary
16:37:21 <kozhukalov> That is going to make build process fully data driven and
16:37:24 <kozhukalov> more flexible (e.g. mix custom repos).
16:37:24 <kozhukalov> 3) Separating of the master node provisioning/deployment
16:37:29 <kozhukalov> Please help to review this spec
16:37:32 <kozhukalov> #link https://review.openstack.org/#/c/254270/
16:37:35 <kozhukalov> it is absolutely necessary to merge this feature in 9.0
16:37:39 <kozhukalov> There is POC and Vitary Parakhin is working on making it happened.
16:37:42 <kozhukalov> 
16:37:45 <kozhukalov> that is it
16:37:54 <angdraug> 7 minutes!!!
16:38:24 <kozhukalov> -j10
16:38:31 <angdraug> should we use -j10 everywhere?
16:38:37 <kozhukalov> i mean make -j10
16:38:54 <kozhukalov> there is no need to use it everwhere
16:39:07 <kozhukalov> 15 minutes is ok
16:39:09 <xarses> angdraug: yes lets make -j10 fuel --env 1 deploy-changes =)
16:39:13 <mattymo> It's bad on envs with small memory or cpu (like small vms)
16:39:26 <mihgen> kozhukalov: great progress guys. I assume there will be large portion of work on QA side if we get rid of ISO.. ?
16:40:04 <kozhukalov> mihgen: maybe
16:40:12 <kozhukalov> yes, it is our dream
16:40:16 <kozhukalov> to get rid of iso
16:40:36 <xarses> ++
16:40:48 <xarses> moving?
16:40:52 <kozhukalov> yes
16:40:55 <xarses> #topic UCA Mitaka status (mattymo)
16:41:25 <mattymo> UCA mitaka is currently stalled a bit due to https://bugs.launchpad.net/fuel/+bug/1544505
16:41:26 <openstack> Launchpad bug 1544505 in Fuel for OpenStack mitaka "plugin metadata is broken" [Critical,Confirmed] - Assigned to Ilya Kutukov (ikutukov)
16:41:26 <mattymo> There are Fuel mitaka packages on the way, but I'm not fully apprised of that situation. There's some naming concerns, but I think mos-packaging team will handle it.
16:41:30 <mattymo> I've got a rewrite of fuel-plugin-upstream ready to submit as soon as LP#1544505 is fixed and I can do a quick test.
16:42:00 <mattymo> The next stage is to get UCA support fully integrated into Fuel for 9.0 release. I hope I can get that done around Monday.
16:42:43 <mattymo> One other item I should mention is zigo requested support for debian trusty-backports. It requires no extra effort (except for another item choice and URL field) - so I've also got that working in my tests
16:42:55 <mattymo> and it means 1 more possible deployment option
16:43:02 <mihgen> what about extending current list of releases with UCA one, so users who take Fuel can just choose UCA in dropdown?
16:43:20 <ikutukov> working on 1544505, ETA - tomorrow (12/02/2016)
16:43:44 <mattymo> mihgen, we _can_ do that, but it's just a lot of duplication of the existing fuel release
16:44:07 <mattymo> it's exactly the same fuel with 1 repo added, 3 apt package pins, and changing the os_package_type fact
16:44:08 <mihgen> it's yaml, you should not duplicate the whole thing
16:44:54 <xarses> mattymo: I thought all we needed to do was use the anchor and change what we want
16:44:58 <mihgen> can we do magic in there and refer to the original release, with override of a just a few things in there which you've mentioned?
16:45:09 <mattymo> it's 150 lines of yaml
16:45:14 <mwhahaha_> shouldn't need to duplicate the release, we should just be able to add in a configuration item for the package type & repos
16:45:29 <mattymo> yeah just a simple UI addition to change where openstack packages come from
16:45:44 <zigo> I really would like this to happen.
16:45:45 <mihgen> not a UI. yaml please
16:45:50 <angdraug> debian trusty backports work just like that? wow, nice )
16:46:01 <mattymo> mihgen, why?
16:46:05 <mattymo> the ui is yaml too
16:46:16 <zigo> The issue is that Fuel 9.0 is still using liberty, right?
16:46:17 <mattymo> it's actually going to go in the _same_ yaml file
16:46:23 <mihgen> if so, then I'm fine. I'm against of hacking JS if we can change yanml
16:46:29 <mattymo> 150 lines duplicated in openstack yaml or 15 lines of UI elements defined in openstack.yaml
16:46:40 <zigo> The Liberty puppet stuff will *not* handle my package correctly, they do not have the necessary patches.
16:46:56 <mihgen> I don't care, but get me another release with UCA please! ;)
16:47:00 <zigo> Though I could patch them, and include that in the packages I did for puppet-openstack.
16:47:05 <angdraug> zigo: that's what we're trying to fix here
16:47:20 <zigo> It's a simple backport of like 3 or 4 patches.
16:47:23 <angdraug> zigo: I mean fix using mitaka, not liberty
16:47:44 <zigo> angdraug: Mitaka puppet-openstack is already fully working with my packages normally.
16:47:48 <zigo> Out of the box...
16:47:55 <mattymo> so please keep this on your radar, mihgen angdraug  "<ikutukov> working on 1544505, ETA - tomorrow (12/02/2016)"
16:48:02 <mattymo> ^ all plugins can't serialize data at the moment
16:48:23 <zigo> When is it planned to have Fuel 9 to start using Mitaka packages? It becomes urgent to start doing it.
16:48:33 <angdraug> #link https://bugs.launchpad.net/fuel/+bug/1544505
16:48:34 <openstack> Launchpad bug 1544505 in Fuel for OpenStack mitaka "plugin metadata is broken" [Critical,Confirmed] - Assigned to Ilya Kutukov (ikutukov)
16:48:37 <angdraug> ...for the record
16:48:44 <mattymo> zigo, very soon
16:49:03 <mattymo> that's it for me, by the way
16:49:04 <angdraug> based on the bug ^ remaining the only blocker, could be Monday
16:49:31 <zigo> Cool.
16:49:42 <xarses> moving?
16:50:05 <xarses> #topic Octane team status (ogelbukh)
16:50:47 <ogelbukh> we're in QA/bugfix stage for backup/restore in fuel-octane
16:51:17 <ogelbukh> the upgrade feature spec finally landed
16:51:27 <angdraug> how many bugs in it have you found so far?
16:51:42 <ogelbukh> about 10
16:52:15 <ogelbukh> some broken manifests with no idempotency
16:52:21 <ogelbukh> had to work around it
16:52:38 <ogelbukh> one minor issue with cluster metadata (ui part0
16:52:56 <angdraug> doesn't sound too bad
16:53:03 <ogelbukh> we also have a regression/bug for 7.0 version
16:53:23 <ogelbukh> network downtime for workloads during upgrade of primary controller
16:53:35 <ogelbukh> investigating that
16:53:46 <ogelbukh> most likely some corosync preparations needed
16:53:58 <ogelbukh> before bringing down a node for reinstall
16:54:05 <ogelbukh> more or less, that's it
16:54:28 <angdraug> thanks
16:54:36 <angdraug> lets move on to the last topic in the agenda
16:54:42 <xarses> #topic https://bugs.launchpad.net/fuel/+bug/1544109 - how do we fix in stable/8.0 (mihgen)
16:54:45 <openstack> Launchpad bug 1544109 in Fuel for OpenStack "Horizon ssl support error. Unable to connect after deploy." [Critical,In progress] - Assigned to Fuel UI Team (fuel-ui)
16:55:02 <mihgen> just wanted to hear your opinion folks on how do we fix this critical in 8.0
16:55:25 <mihgen> https://review.openstack.org/279063 seems to be too risky for 8.0
16:55:47 <mihgen> so my proposal is just to lock UI to have SSL configured for both REST API & Horizon at the same time
16:56:02 <mihgen> (bug is about if Horizon SSL only is enabled, deploy fails)
16:56:08 <xarses> why do both have to be configured?
16:56:24 <xarses> it seems we just forgot to configure haproxy for horizon correctly
16:56:55 <angdraug> sbog: around? can you comment?
16:57:03 <mihgen> holser?
16:57:43 <mihgen> vkramskikh: would UI change be complicated if we take this route?
16:57:55 <xarses> but yea, We can drop the priority, if we lockout/remove the granular ssl selection
16:57:56 <mihgen> I hope it's just yaml config
16:58:23 <xarses> has anyone confirmed that if both are on, it's setup correctly?
16:58:35 <mihgen> nurla says so
16:59:18 <xarses> we should be able make horizon ssl require the other to be enabled in config then
16:59:39 <xarses> ok, we are out of time
17:00:01 <xarses> thanks everyone
17:00:03 <xarses> #endmeeting