16:01:25 <kozhukalov> #startmeeting Fuel
16:01:26 <openstack> Meeting started Thu Jan 29 16:01:25 2015 UTC and is due to finish in 60 minutes.  The chair is kozhukalov. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:01:27 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:01:30 <openstack> The meeting name has been set to 'fuel'
16:01:31 <kozhukalov> hey guys
16:01:36 <mihgen> hi
16:01:38 <docaedo> hello
16:01:43 <kozhukalov> agenda as usual
16:01:47 <brain461> hello
16:01:54 <kozhukalov> #link https://etherpad.openstack.org/p/fuel-weekly-meeting-agenda
16:02:12 <prmtl> hi
16:02:18 <ikalnitsky> o/
16:02:18 <kozhukalov> #chair kozhukalov
16:02:19 <openstack> Current chairs: kozhukalov
16:02:20 <akislitsky> hi
16:02:32 <kozhukalov> we have just a few topics to discuss
16:03:02 <kozhukalov> so if someone have any short announcements please go ahead
16:03:16 <kozhukalov> right now before actual topics
16:03:26 <agordeev> hi
16:03:52 <kozhukalov> does anybody have any announcements?
16:03:57 <kozhukalov> nobody?
16:04:01 <kozhukalov> really? ))
16:04:13 <kozhukalov> ok, let's start
16:04:31 <kozhukalov> #topic separate MOS packages from upstream linux (vparakhin)
16:04:40 <brain461> hi
16:04:48 <brain461> Blueprint for the feature: https://blueprints.launchpad.net/fuel/+spec/separate-mos-from-linux
16:04:58 <brain461> Spec is prepared, there are few last-minute changes from latest discussions, going to apply them in a few hrs from now, so the spec will be ready for review.
16:05:10 <brain461> Spec is here https://review.openstack.org/#/c/148279/
16:05:19 <brain461> I have a PoC for separating vanilla Ubuntu packages that are not present on upstream ISO: https://review.openstack.org/#/c/150520/
16:05:24 <kozhukalov> everyone is welcome to review
16:05:29 <brain461> I expect to have working PoC for creating split repos this Monday.
16:06:14 <kozhukalov> we've just finished discussing last points about this task
16:06:16 <brain461> questions?
16:06:17 <kozhukalov> great
16:06:55 <kozhukalov> another important point here is making sure Fuel (nailgun) is able to deal with several separate repos
16:07:18 <kozhukalov> and afaiu we have not put this part into this spec
16:07:24 <kozhukalov> am i right?
16:07:31 <kozhukalov> brain461: ^
16:07:33 <brain461> no detailed description yet
16:08:00 <kozhukalov> ok, i'll add this nailgun related stuff in this spec
16:08:18 <kozhukalov> as i am working on this currently
16:08:46 <kozhukalov> it is necessary to note
16:09:03 <kozhukalov> that there was pinning problem
16:09:15 <angdraug> is this about separate mirrors on master node only?
16:09:23 <kozhukalov> preseed and kickstart are not able to prioritize repos
16:09:33 <angdraug> or will it allow to specify external repos too?
16:09:35 <kozhukalov> but it is solvable
16:09:43 <brain461> not only on aster node, on fuel mirrors as well
16:09:56 <agordeev> kozhukalov: that hurts builduing target images for IBP then
16:10:05 <kozhukalov> external repos to
16:10:13 <kozhukalov> agordeev: no, it does not
16:10:22 <agordeev> kozhukalov: ok
16:10:46 <kozhukalov> because we are planning to build image in two stages
16:11:05 <kozhukalov> first is just basic OS from one basic repo
16:11:28 <kozhukalov> and then all other packages are to be installed with apt-get and yum
16:11:32 <angdraug> not really obvious from the spec, proposed change section only mentions local mirrors
16:11:41 <kozhukalov> and those tools support pinning
16:12:16 <kozhukalov> angdraug: it is supposed that we put repo_metadata into cluster attributes
16:12:30 <brain461> angdraug: yes, this will be clarified, thanks
16:12:46 <kozhukalov> and potentially user will be able to add repos for a particular cluster
16:13:01 <kozhukalov> i'll add this info to the spec tomorrow
16:13:54 <kozhukalov> angdraug: are you ok with that? did i answered you q?
16:14:31 <kozhukalov> ok, if there are no other q, let's move on then
16:14:51 <kozhukalov> #topic image based provisioning (agordeev)
16:15:11 <agordeev> hi
16:15:16 <agordeev> 3 blueprints were created as fuel folks have suggested at the previous meeting.
16:15:18 <agordeev> https://blueprints.launchpad.net/fuel/+spec/ibp-build-ubuntu-images
16:15:20 <agordeev> https://blueprints.launchpad.net/fuel/+spec/ibp-image-checksums
16:15:22 <agordeev> https://blueprints.launchpad.net/fuel/+spec/ibp-reconnect
16:15:24 <agordeev> Specs are about 80% ready and will be populated with more precised details of implementation on the next week.
16:15:26 <agordeev> About bugs: no new bugs so far. Still a couple of fixes are on the review now.
16:15:28 <agordeev> Also i think i've figured out the root case of the most epic bug of Fuel-agent. "Device or resource is busy" which appears on the partitioning step of provisioning.
16:15:30 <agordeev> I had put some thought here https://etherpad.openstack.org/p/Fuel-Agent-EPIC-BUG-device-is-busy
16:15:32 <agordeev> ETA of its fix is 1-2days.
16:15:34 <agordeev> So, everything is proceeding according to the plan. That's all.
16:16:07 <dpyzhov> agordeev: excellent =)
16:16:19 <agordeev> So my short version of the root cause: udev has got scaling issues when processing a lot of events generated by `parted`.
16:17:28 <kozhukalov> agordeev: great research, we really need fuel agent to be reliable
16:18:10 <kozhukalov> does anybody have any questions?
16:18:43 <kozhukalov> ok, looking forward for the fix of this epic bug
16:19:23 <kozhukalov> are you planning to rework partition utils so as to run parted as few times as possible&
16:19:27 <kozhukalov> ?
16:20:30 <kozhukalov> ok, looks like we can move on
16:20:43 <kozhukalov> #topic fuel-library modularization status (adidenko)
16:21:05 <agordeev> kozhukalov: nope, making less calls of parted doesn't improve the situation.
16:21:12 <alex_didenko> hi guys
16:21:18 <kozhukalov> alex_didenko: hi
16:21:20 <alex_didenko> BP: https://blueprints.launchpad.net/fuel/+spec/fuel-library-modularization
16:21:20 <alex_didenko> Latest spec in HTML format: http://docs-draft.openstack.org/91/147591/8/check/gate-fuel-specs-docs/763fc7e//doc/build/html/specs/6.1/fuel-library-modularization.html
16:21:32 <alex_didenko> As you may know, we're using granular task based deployment in master already.
16:21:32 <alex_didenko> Currently we have a set of separate manifests that are applied by puppet as separate tasks. Here they are: https://github.com/stackforge/fuel-library/tree/master/deployment/puppet/osnailyfacter/modular
16:21:32 <alex_didenko> As the final puppet task we run 'legacy.pp' which is almost exact copy of old 'site.pp' so the deployment process is almost the same as before.
16:21:32 <alex_didenko> As the next stage of fuel-library granularization, we're going to move node roles (controller, compute, cinder, ceph-osd, mongo, zabbix-server) into separate tasks. So every role is going to be deployed by its own separate puppet manifest applied as Fuel task.
16:21:33 <alex_didenko> This code is already on review, tested with BVTs and ready to merge. We're waiting for some CI updates (switching centos from 'multinode' to 'HA') in order to pass CI.
16:21:34 <alex_didenko> So when we merge those patches with roles as separate tasks, we'll be able to remove "osnailyfacter/manifests/cluster_ha.pp" as we no longer need it.
16:21:35 <alex_didenko> Also we're switching from $::fuel_settings to Hiera. All the configuration data will be pulled into manifests via hiera() only.
16:21:44 <alex_didenko> We also have some changes in fuel-library CI related to granular deployment. Since our deployment process depends on tasks, which are shipped with fuel-library repo, we've added new CI test to make sure we have a working tasks graph in every commit. Here is an example: https://fuel-jenkins.mirantis.com/job/fuellib_tasks_graph_check/39/console
16:22:17 <alex_didenko> So atm we're following our implementation plan and everything looks OK.
16:23:36 <kozhukalov> afaiu we hiera data format will differ from what we have in astute.yaml
16:23:41 <kozhukalov> am I right?
16:24:00 <alex_didenko> not quite, hiera allows to pull any YAML compatible data
16:24:15 <izinovik> alex_didenko, am i understanding correctly that $::fuel_settings will completely dissapear eventually?
16:24:19 <kozhukalov> ohh, great
16:24:39 <alex_didenko> izinovik: yes, ritgh with those roles tasks patch merge
16:24:45 <alex_didenko> we don't use fuel_settings there
16:24:51 <alex_didenko> only hiera
16:24:54 <izinovik> thanks, got it
16:25:33 <angdraug> #link http://lists.openstack.org/pipermail/openstack-dev/2015-January/055299.html
16:25:45 <angdraug> can we explore this topic in a bit more detail?
16:26:09 <kozhukalov> and afaiu hiera allows you to use a little bit more secure way to store sensitive data
16:26:54 <alex_didenko> hiera provides flexibility for operators or developers
16:27:14 <alex_didenko> not sure about scurity, since it's going to use the same source - astute.yaml in our case
16:27:36 <alex_didenko> angdraug: we're going to remove simple mode from CI and QA
16:27:48 <angdraug> alex_didenko: that's not what this was about
16:27:54 <angdraug> I'm fine with removing simple mode
16:28:24 <angdraug> but I want to understand what is the value of fuel modularization if it makes it _harder_ to maintain features like simple mode instead of making it _easier_
16:29:10 <alex_didenko> it does not make them harder
16:29:43 <alex_didenko> but since it's a huge refactoring, it will just require more resource to migrate two modes from monolithic to granular deployment
16:30:34 <alex_didenko> so having the flexibility of granular deployment you can easily develop your own "mode" - adjust tasks, prepare manifests and import them into Nailgun
16:31:39 <angdraug> so "doubling the effort" was an overstatement?
16:32:30 <angdraug> I'm just worried that in current fuel-library, difference between simple and HA is down to a single manifest, cluster_ha vs cluster_simple
16:32:43 <alex_didenko> maybe a little bit... but I really see no point to do this since we were going to drop simple mode anyway
16:33:11 <angdraug> that's a fair point, just trying to understand how much effort it would be to add an alternative reference HA architecture
16:33:16 <alex_didenko> yes, but there is a lot of code duplication
16:33:19 <alex_didenko> currently
16:33:29 <alex_didenko> for HA and simple
16:33:48 <izinovik> some modules in fuel-library/ also have to adapt and know what deployment mode was chosen
16:33:52 <alex_didenko> we declare the same classes but with different parameters in HA and Simple
16:34:30 <alex_didenko> izinovik: why? they will see that deployment_mode is "ha_compact" and run as usually
16:34:41 <izinovik> not always
16:35:02 <izinovik> in simple mode you have to install you own custom init.d scripts to run multiple service instances
16:35:19 <izinovik> in HA it is not, you just install OCF script and you are done
16:35:41 <angdraug> it would be nice to get rid of all the if(deployment_mode) conditionals
16:35:43 <izinovik> IMO simple mode is just adds complexity to fuel-library/
16:36:13 <alex_didenko> angdraug: we will get rid of them
16:36:23 <angdraug> as long as we don't end up having to re-add half of them later to support something like qpid as a plugin
16:36:24 <kozhukalov> izinovik: if you are enforced to install custom init.d script then maybe the better place for that is package spec
16:36:45 <angdraug> kozhukalov: +!
16:36:47 <angdraug> kozhukalov: +1
16:37:39 <kozhukalov> ok, guys, resume is, granular deployment has much more pros than cons
16:37:45 <kozhukalov> moving on?
16:37:50 <angdraug> yes, my question is addressed
16:38:08 <kozhukalov> #topic python-fuelclient status update(romcheg)
16:38:16 <romcheg> Hi folks!
16:38:23 <kozhukalov> romcheg: hi
16:38:30 <barthalion> what's up, doc
16:38:39 <romcheg> ISO build scripts have been switched to stackforge/python-fuelclient as the main source of Fuel Client. fuel-web/fuelclient directory was removed finally \o/
16:38:54 <romcheg> akasatkin, dpyzhov, evgeniyl___ and ikalnitsky were added to python-fuelclient-core group
16:39:16 <romcheg> Because they were cores in the original gerrit project
16:39:31 <romcheg> Guys, please keep your activity high enough because that group is managed according to Approved Governance process.
16:39:55 <romcheg> That basically means that we are going to add/remove core members according to the number of useful reviews
16:40:13 <romcheg> ATM I’m working on releasing python-fuelclient-6.0.0 to PyPi. That requires to fix issues with packaging and configuration files
16:40:22 <romcheg> #link https://review.openstack.org/#/c/149661/
16:40:26 <romcheg> #link https://review.openstack.org/#/c/151263
16:41:02 <romcheg> As soon as 6.0.0 is released QA dept. and other guys will be able to start testing fuel client in their projects
16:41:28 <romcheg> For the fuel client team making a release will allow to start breaking backwards compatibility.
16:42:11 <romcheg> We are going to switch iso-make scripts from master branch of the client to 6.0.* as soon as it's released
16:43:14 <kozhukalov> romcheg: are there any testing issues about fuelclient according to the case that it was previously tested together with nailgun?
16:43:19 <romcheg> Some of the folks also asked about job parameters for building custom ISOs with different revisions. That request is atm in progress https://bugs.launchpad.net/fuel/+bug/1414069
16:43:36 <zigo> romcheg: Is there now a valid version number for fuelclient?
16:43:38 <romcheg> kozhukalov: No, looks like no one was using fuelclient :)
16:44:23 <romcheg> zigo: That version is 6.0.0, but it is in the master branch
16:44:24 <barthalion> like it was easily possible…
16:44:41 <kozhukalov> romcheg: but fuelclient used nailgun for its "unit" -) tests
16:44:58 <zigo> One big issue of separating all components from the main fuel-web repo is that most components don't really have a valid version number. They all report 0.1
16:45:14 <romcheg> kozhukalov: The job for that was launched last week. I think there was an update in the mailing list
16:45:56 <romcheg> zigo: I'll keep track of version number in both python and rpm packages
16:45:57 <kozhukalov> romcheg: how do you think we will keep nailgun rest api synced with what we have in fuelclient since now?
16:46:32 <zigo> romcheg: I have Debian packages for all Fuel components, I could easilly upload fuelclient to Sid now if needed.
16:46:37 <zigo> romcheg: Should I proceed?
16:46:45 <xarses> kozhukalov: we have to test against it
16:47:02 <romcheg> kozhukalov: Pretty easy — x.y.z version of the client will be compatible with x.y/y-1.z versions of Nailgun
16:47:45 <romcheg> zigo: Probably it's better upload packages when they are released on PyPi.
16:47:53 <kozhukalov> zigo, that would be great
16:48:06 <zigo> romcheg: I don't care PyPi, I do git tag based packaging.
16:48:17 <kozhukalov> zigo: to put at least fuelclient in sid
16:48:29 <romcheg> zigo: In our case a tag === a pypi release actually
16:48:38 <zigo> About nailgun, I haven't seen any dependency of it in fuelclient.
16:48:50 <romcheg> Nothing should be linked to master except for testing reasons
16:49:14 <zigo> Ah, right, no tags at all for fuelclient.
16:49:19 <zigo> Could someone make one?
16:49:40 <zigo> Even 0.0.1 is fine, so that I at least have a tag to play with ...
16:49:41 <barthalion> it will be there once ready
16:49:41 <romcheg> zigo: I'm working on that. As I've said ^ two problems have to be resolved first
16:49:47 <zigo> Ok.
16:49:48 <barthalion> romcheg is still working on that
16:49:55 <zigo> romcheg: Please ping me when it's tagged.
16:50:15 <romcheg> #action romcheg Ping zigo when fuelclient is tagged
16:50:35 <romcheg> Will do
16:50:38 <zigo> Also, someone should write a valid short and long description for fuelclient.
16:50:41 <zigo> Thanks ! :)
16:51:36 <zigo> romcheg: Could you work out a better long description, telling what it does? Like listing nodes, etc.
16:51:44 <kozhukalov> yes, having  classic old man page would be great
16:51:46 <zigo> I don't think I know fuelclient enough to be able to tell...
16:51:53 <romcheg> zigo: I'll try.
16:52:12 <romcheg> kozhukalov: A manpage will be great but I believe someone will have to work on that
16:52:14 <zigo> kozhukalov: Not having a man page isn't a blocker for uploading to Debian. Not having a long desc is.
16:52:44 <kozhukalov> ok, great
16:52:51 <romcheg> I think we should look on how docs are done in other os projects
16:53:08 <romcheg> iirc that scheme allows generating manpages
16:53:28 <kozhukalov> romcheg: you just need to ping one of fuel tech writers
16:53:52 <romcheg> kozhukalov: nice idea
16:54:03 <romcheg> #action romcheg Poke fuel tech writers
16:54:45 <kozhukalov> ok, if no other q, then open discussion
16:54:54 <kozhukalov> #topic open discussion
16:55:30 <kozhukalov> does anybody have something?
16:55:32 <romcheg> Everyone went for coffee? :-P
16:55:33 <barthalion> we have 6 minutes left, and I need to leave sson
16:55:43 <barthalion> so I'll quickly complain about quality of our shell scripts
16:56:00 <kozhukalov> barthalion: for example
16:56:05 <barthalion> we should decide whether we stick to POSIX standards or use bash
16:56:13 <barthalion> any run_tests is full of bashisms
16:56:19 <barthalion> not using all bash features
16:56:22 <barthalion> or almost none
16:56:29 <kozhukalov> we definitely would like to stick to POSIX
16:56:50 <angdraug> should we use dash to enforce that?
16:57:08 <barthalion> I'd rather use busybox
16:57:12 * romcheg just ported run_test.sh to bash….
16:57:13 <kozhukalov> barthalion: ok, please bring this up in ML
16:57:17 <barthalion> dash is far from perfect from my experience
16:57:27 <barthalion> kozhukalov: will do, thanks
16:57:33 <zigo> I have something else to discuss ...
16:57:42 <kozhukalov> 2 minutes
16:57:48 <zigo> Has anyone tested my bootstrap ISO image?
16:57:48 <barthalion> #action barthalion bring quality of shell scripts to the ML
16:58:02 <zigo> I'd like some fuel expert to do so.
16:58:09 <kozhukalov> zigo, if it is longer than this let's move our discussion to #fuel-dev
16:58:14 <angdraug> zigo: link?
16:58:22 <kozhukalov> zigo, i did
16:58:23 <angdraug> (for the record)
16:58:49 <zigo> Hum...
16:59:10 <angdraug> 1 minute
16:59:17 <zigo> http://juno-wheezy.pkgs.mirantis.com/debian/fuel-debian-bootstrap.iso <--- There ! :)
16:59:23 <kozhukalov> zigo: your idea to move root fs on nfs is great
16:59:23 <zigo> kozhukalov: Did it work as expected?
16:59:28 <angdraug> #link http://juno-wheezy.pkgs.mirantis.com/debian/fuel-debian-bootstrap.iso
16:59:36 <zigo> kozhukalov: The idea isn't to move root fs on NFS !!!
16:59:40 <kozhukalov> yes i managed to run it
16:59:50 <kozhukalov> zigo it is one of
16:59:54 <zigo> It's to use NFS on the PXE boot to serve the .iso, as that's the way it works.
17:00:01 <kozhukalov> ok we have no time any more
17:00:13 <kozhukalov> closing, and welcome to #fuel-dev
17:00:15 <zigo> Let's move to #fuel-dev then ! :)
17:00:24 <kozhukalov> thanx everyone
17:00:27 <kozhukalov> #endmeeting