16:04:00 <mihgen> #startmeeting #fuel
16:04:00 <openstack> Meeting started Thu Feb  5 16:04:00 2015 UTC and is due to finish in 60 minutes.  The chair is mihgen. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:04:02 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:04:04 <openstack> The meeting name has been set to '_fuel'
16:04:25 <mihgen> let's start folks
16:04:39 <mihgen> #topic New upstream approach to Ubuntu and impact on other features
16:04:56 <mihgen> tzn: is holser around?
16:05:14 <tzn> checking
16:06:24 <tzn> let’s go with second point
16:06:31 <tzn> Ok, he is here :)
16:06:45 <holser> Hey
16:07:06 <evgeniyl___> hi
16:07:50 <holser> Finally, we finalized our approach how we consume packages
16:08:35 <holser> Fuel will stay very close to community and upstream packages
16:09:31 <holser> Basically, we’ll consume packages from Ubuntu upstream (main, universe, security …) and we’ll have own repository with own packages
16:10:20 <holser> It’s a big challange as we don’t control Ubuntu packages, so we are going to test our product everytime Ubuntu changes repository
16:11:14 <xarses> We should keep in mind that it would help with CentOS if we did something similar. We should keep an eye out for any low cost things we can also get in place for it.
16:11:22 <holser> If some packages introduce regression or breaks the product completely we’ll put the package to our repository. Meanwhile we start working on most permanent resolution
16:11:36 <holser> xarses, CentOS is on the way also
16:11:59 <mihgen> holser: what is the delta time we are going to have in order to fix broken package in our repo?
16:12:00 <tzn> But not in 6.1
16:12:20 <xarses> holser: ok, just hoping we leave constructs open so centos is easy to plum into the fixtures for ubuntu
16:12:24 <holser> We’ll have 24 hours SLA
16:12:25 <tzn> We will think about it, but intermediate solution should be in repo in 24h
16:12:54 <holser> and we are going to test even before mirror updates
16:13:22 <tzn> We will inspect pending updates queue
16:13:30 <holser> barthalion is doing some research on CLI and UI level
16:13:36 <tzn> So we should be able to catch any problems before they are published
16:13:38 <holser> to inform user …
16:13:50 <mihgen> tzn: yeah, and my question is how much time do we have?
16:13:55 <tzn> We will use hook for apt-get update
16:13:55 <holser> tzn, yeah, that’s what we are going to do
16:13:58 <mihgen> after we see a package in queue
16:14:17 <holser> we’ll catch problems, so we’ll have 24h to fix
16:14:23 <mihgen> ok
16:14:29 <tzn> Will investigate
16:14:33 <holser> after 24h packages will appear in mirrors officially
16:14:47 <holser> our repo should be updated before that
16:15:01 <tzn> https://wiki.ubuntu.com/QATeam/PerformingSRUVerification
16:15:06 <mattymo> there is a testing repo of ubuntu as well we could get a further advance notice on
16:15:32 <holser> yeah
16:15:40 <mihgen> thx guys
16:16:06 <tzn> minimum 7 days
16:16:07 <mattymo> but clearly there may be hot packages that race right through testing and get to live updates in <24hrs
16:16:13 <holser> I’ve started a draft of spec today
16:16:19 <tzn> No
16:16:39 <tzn> Minimum time is 7 days
16:16:43 <angdraug> what about embargoed security updates?
16:16:51 <mattymo> think of heartbleed 2.0
16:16:55 <holser> https://wiki.ubuntu.com/StableReleaseUpdates
16:17:02 <holser> move the package into -updates after it has passed a minimum aging period of 7 days.
16:17:30 <mihgen> so for most of the packages we gonna have about a week, not 24h
16:17:32 <xarses> angdraug: we should hopefully be able to get on the security group for embargoed updates
16:18:19 <mihgen> moving on?
16:18:34 <mihgen> #topic image based provisioning (agordeev)
16:18:36 <agordeev> hi
16:18:56 <agordeev> my update is a quite short for today. We're still being almost concentrated on bug fixing.
16:18:58 <agordeev> We were working hard and being busy till the late evening every day of that week.
16:19:00 <agordeev> 2 high bugs fresh fixes is not reviewed and merged yet. Kindly asking python-team to help with reviewing them
16:19:02 <agordeev> https://review.openstack.org/152568
16:19:04 <agordeev> https://review.openstack.org/152609
16:19:06 <agordeev> https://review.openstack.org/152610
16:19:08 <agordeev> https://review.openstack.org/152560
16:19:10 <agordeev> Meanwhile, IBP Specs are still untouched. Hope to address all coments and update them ASAP
16:19:12 <agordeev> Implementing isn't started too. Hope to start on next week.
16:19:43 <mihgen> agordeev: thx. I hope you nailed down all critical issues?
16:19:55 <mihgen> agordeev: including those found on scale lab
16:19:55 <agordeev> mihgen: yes, indeed.
16:20:12 <tzn> cool
16:20:36 <mihgen> agordeev: thx. about implementation - do you plan it in python?
16:20:38 <agordeev> mihgen: wait. i'm not informed about issues on scale lab
16:21:07 <mihgen> hmm tzn: do you know if scale lab runs on ibp?
16:21:08 <agordeev> mihgen: IBP specs - are all about improving fuel-agent, which's written in python
16:21:20 <tzn> No, they still test 6.1
16:21:24 <tzn> sorry, 6.0
16:21:25 <mihgen> agordeev: I'm talking about building an image on master node
16:21:46 <agordeev> mihgen: ah, that one. Nope. It will be bash script.
16:21:46 <mihgen> tzn: that's bad. IlyaE - when do you guys plan to start on 6.1?
16:21:57 <mihgen> IlyaE: scale testing on 6.1 I mean
16:22:33 <mihgen> agordeev: I have some worries about bash here. Let's catch up on this topic later
16:23:06 <agordeev> mihgen: on previous meeting, afair V Kozhukalov said, that 100 nodes scale test was passed without flaws
16:23:15 <barthalion> it doesn't sound like a good task for bash
16:23:57 <mihgen> yeah I like bash but it's quite terrible for all error handling procedures )
16:23:59 <agordeev> barthalion: ok, let's disscuss that on #fuel-dev later
16:24:04 <mihgen> all right, let's move on
16:24:06 <barthalion> k
16:24:07 <xarses> agordeev: you noted specs are un-touched, can you link?
16:24:09 <mihgen> if no other questions
16:24:47 <mihgen> #topic granular deployment status (dshulyak)
16:24:57 <agordeev> xarses: sure. https://review.openstack.org/149314 https://review.openstack.org/149568 https://review.openstack.org/148962
16:24:57 <dshulyak> hi guys one more time
16:25:12 <dshulyak> we are almost finished with items that was in scope of granular deployment for 6.1
16:25:37 <dshulyak> almost all pre/post tasks that was in astute - already defined by configuration and on review
16:25:58 <dshulyak> after this is merged - our deployment will be completely data-driven
16:26:20 <dshulyak> also prmtl is working on visualization of deployment graph, and in my opinion it is very helpfull
16:26:28 <dshulyak> if will be embedded right into fuel client
16:26:34 <dshulyak> s/if/it
16:26:50 <mihgen> dshulyak: sounds cool
16:27:08 <mihgen> almost finished - what exactly is left and ETA?
16:27:17 <dshulyak> and i am going to push letter with detailed explanation of granular api
16:27:32 <mihgen> dshulyak: and one more question about UX of fuel client to run particular tasks, where I can look for examples of use?
16:28:14 <dshulyak> mihgen: i can show the  patch, or wait for my letter
16:28:18 <dshulyak> it is actually merged
16:28:27 <dshulyak> i just need to inform everyone :)
16:28:36 <mihgen> dshulyak: I can wait for email, if you plan to send it to openstack-dev :)
16:29:00 <mihgen> so I'd like to know how I can run certain task, remove one from whole role run
16:29:03 <dshulyak> left: review/merge pre/post tasks
16:29:08 <mihgen> and other possible manipulations
16:29:09 <dshulyak> visualization
16:29:36 <mihgen> dshulyak: ok, thanks.
16:29:40 <dshulyak> and couple of fixes and improvements
16:29:42 <xarses> Guys, we need documentation about how to develop / troubleshoot the granular tasks
16:29:54 <xarses> probably including a KT demo so we can discuss
16:29:55 <mihgen> dshulyak: and currently we still use MCollective ssh - to run remote task, right?
16:30:22 <dshulyak> mihgen: not ssh, communication is done with rabbitmq queus
16:30:47 <mihgen> dshulyak: wait, who dispatches what has to be done on slave node?
16:31:16 <mihgen> sorry I meant remote shell call
16:31:19 <mihgen> not ssh
16:31:26 <mihgen> remote shell call by MCollective agent
16:31:43 <dshulyak> than everything like you said)
16:31:57 <dshulyak> i hope to start promotion of mistral after we are done with 6.1 scope
16:32:33 <xarses> dshulyak: mike ^
16:32:35 <mihgen> ok, understood. thanks.
16:32:56 <mihgen> +1 on xarses's Guys, we need documentation about how to develop / troubleshoot the granular tasks
16:33:10 <mihgen> any more questions on granular tasks?
16:33:24 <xarses> soon, otherwise we cant help very well to troubleshoot issues
16:33:28 <xarses> We have code being merged before the spec
16:33:28 <xarses> #link https://review.openstack.org/#/c/113491/
16:33:28 <xarses> #link https://review.openstack.org/#/c/147591/
16:33:28 <xarses> #link https://review.openstack.org/#/c/147249/
16:33:28 <xarses> furthermore we have confusion about the purpose of the spec, is it approval, documentation, a living document to iterate constantly.
16:34:11 <dshulyak> xarses: first one is about orchestration of granular api
16:34:14 <evgeniyl___> we have a mailing thread in openstack-dev about that
16:34:29 <mihgen> xarses: let's discuss actually this thing once we are here and have time - added to agenda
16:34:37 <mihgen> #link https://etherpad.openstack.org/p/fuel-weekly-meeting-agenda
16:35:00 <mihgen> let's move on and discuss procedure as separate topic
16:35:12 <mihgen> if there are no more concrete things on granular
16:35:20 <mihgen> moving..
16:35:21 <dshulyak> i am done
16:35:28 <mihgen> #topic upload ubuntu ISO? (mihgen/tzn/holser)
16:35:43 <mihgen> I'd like to clarify if we drop this initiative, tzn, holser?
16:35:50 <tzn> So according to proposal, we should drop it
16:36:03 <tzn> It doesn.t make any sense now
16:36:08 <mihgen> tzn: ok.
16:36:24 <tzn> igor could help with rewriting current iso creation scipts (make)
16:36:27 <tzn> to use upstream
16:36:50 <tzn> Also, as a note, It would be good if any feature lead take a look at new approach and check if it impacts feature
16:36:56 <mihgen> tzn: we will need resources to handle some things related to new approach
16:37:07 <tzn> yes
16:37:14 <xarses> ?? please explain the context of the 'upload ubuntu iso' here and what's deprecating it?
16:37:20 <mihgen> for instance, if you run env, and then add node - and your proxy with packages is not available
16:37:53 <tzn> we have notes on many challanges and will publish them
16:38:04 <tzn> xarxes: we go for online installation
16:38:09 <mihgen> also, whether slave nodes should have direct connection to pkgs proxy or via fuel master
16:38:14 <tzn> xarses: ^
16:38:15 <xarses> so we are dropping copying the ISO into the fuel node for online install?
16:38:19 <tzn> yes
16:38:32 <xarses> guys, we know that the proxy sucks from before 3.1
16:38:52 <tzn> Yes, we know
16:38:54 <xarses> its not reliable for everyone. What happens when someone wants to use fuel isolated?
16:38:56 <mihgen> xarses: let's get more info on this one.
16:39:10 <tzn> We haven’t decided yet on the full approach
16:39:22 <mihgen> xarses: we can ask users to do mirror first locally
16:39:27 <mihgen> and then fuel would use it
16:39:43 <xarses> mihgen: we cant ask them to, we can give them a button to do that
16:39:59 <barthalion> how much space does a mirror take?
16:40:02 <tzn> this is enterprise appraoch
16:40:02 <mihgen> xarses: yes.
16:40:11 <tzn> aroung 7 gigs per ubuntu
16:40:17 <tzn> full mirror
16:40:25 <mihgen> barthalion: but we don't do mirroring to master node, right?
16:40:29 <tzn> no
16:40:31 <mihgen> so it's on user's shoulders
16:40:36 <angdraug> 7gigs is just main, not universe, right?
16:40:41 <barthalion> yes, was just wondering
16:40:45 <xarses> the user experience would be terrible if we don't validate that we have all the packages we need first
16:40:48 <barthalion> 7GB is not much
16:40:50 <barthalion> angdraug: yes
16:40:58 <mihgen> +1 to xarses
16:41:03 <tzn> sorry, wron paste
16:41:09 <tzn> full mirror is 642GB
16:41:24 <tzn> but we don;t want full, we need only one release
16:41:30 <xarses> barthalion: 7GB it is to our test environment
16:41:53 <angdraug> creating mirror on master doesn't have to be mandatory
16:42:15 <angdraug> providing a link to pre-cooked local mirror (that most large ubuntu users already have) has to be an option
16:42:36 <tzn> +1 angdraug
16:43:00 <barthalion> not mandatory, but sane default I guess?
16:43:09 <tzn> user will have a choice to select mirror
16:43:17 <angdraug> barthalion: can't have a default here
16:43:26 <tzn> we wil not have default
16:43:28 <angdraug> user has to perform an action one way or the other
16:43:30 <barthalion> ah
16:43:39 <angdraug> either setup a mirror or provide a link to one they've already got
16:43:40 <tzn> to run Fuel it;s much less than even trusty mirror
16:43:47 <tzn> we don;t want installation to take ages
16:43:57 <barthalion> I see
16:44:38 <mihgen> ok. let's move on
16:44:51 <mihgen> #topic merge code before spec is merged? (mihgen)
16:45:13 <mihgen> guys pls find corresponding discussion in openstack-dev and link here for ref
16:45:15 <xarses> +1 to point to local mirror
16:45:35 <evgeniyl___> #link http://www.mail-archive.com/openstack-dev@lists.openstack.org/msg45085.html
16:45:57 <mihgen> my opinion here is the following. For this exact particular case, if don't like patch merged for real reason, we can always revert and fix what is needed
16:46:10 <mihgen> I don't want here to be too religios and restrictive
16:46:20 <mihgen> then, in general -
16:46:54 <angdraug> wrong link, thread starts here:
16:46:55 <angdraug> #link http://lists.openstack.org/pipermail/openstack-dev/2015-January/054752.html
16:47:00 <mihgen> I think core reviewers / mandatory design reviewers should see that spec is  around 90% ready, and only minor things are needed to finish, then we just agree and merge
16:47:11 <dshulyak> but where people will comment after spec if merged?
16:47:22 <mihgen> and all remaining low priority questions / comments are collected separately, and then addressed as new patchset
16:47:47 <angdraug> 90% is a bad criteria
16:47:58 <evgeniyl___> mihgen: if spec is merged, you cannot track the completeness of spec
16:48:01 <mihgen> dshulyak: true as well…
16:48:02 <izinovik> and if spec was merged, but developer found major problem while implementing merged and approved spec?
16:48:16 <izinovik> how this situation is resolved?
16:48:23 <xarses> evgeniyl___: you aren't supposed to track spec progress on the review card
16:48:30 <xarses> thats what launchpad is for
16:48:31 <angdraug> izinovik: as mihgen just said, with a new patchset
16:48:58 <evgeniyl___> xarses: I'm supposed to know if my comments were fixed in the next patches, without keeping all of this in my head
16:48:59 <mihgen> angdraug: good question from dshulyak, how he can comment if it was merged?
16:49:10 <xarses> review is not the place for a living document
16:49:21 <dshulyak> xarses: than we need google doc?
16:49:24 <xarses> if we need a living document for our spec, it needs to not be in review
16:49:25 <evgeniyl___> xarses: why not?
16:49:46 <izinovik> or maybe it should be merged after all work is done (code, tests) ?
16:49:51 <xarses> review is for approvals, review workflow
16:49:57 <mihgen> evgeniyl___: not addressed questions can be incorporated in patch and addressed later as separate patch
16:49:59 <angdraug> instead of 90%, I think a better criteria would be that there is consensus among mandatory reviewers about the direction of the implementation, and remaining concerns are limited to informative parts
16:50:10 <xarses> izinovik: that points to iterating on the spec, with multiple patch sets
16:50:19 <xarses> one should be merged (approved)
16:50:24 <evgeniyl___> mihgen: in this case you have to remember about all of the comments which you left
16:50:30 <mihgen> angdraug: that's what my 90% mean actually, main direction
16:50:32 <evgeniyl___> mihgen: and ping leads to fix them
16:50:42 <xarses> and then iterations of what was final in the following review sets
16:50:46 <mihgen> evgeniyl___: no need to remember, they will be in your spec
16:50:51 <angdraug> can't be 90%, has to be 100% consensus
16:50:55 <mihgen> you would put them there before merging
16:51:07 <dshulyak> angdraug: +1 i was typing almost same thing :)
16:51:12 <mihgen> consensus 100%, but not all minor things addressed
16:51:15 <angdraug> actually you can add comments to review after it's merged
16:51:18 <evgeniyl___> angdraug: +1 it should be 100% ready
16:51:30 <xarses> angdraug: +1
16:51:31 <evgeniyl___> angdraug: nobody looks at merged patches
16:51:35 <angdraug> if your comments are substantial enough, just start a new patchset
16:51:46 <mihgen> guys are +1 angdraug for absolute and full spec?
16:51:56 <mihgen> then it's not gonna happen
16:52:01 <angdraug> evgeniyl___: I do all the time
16:52:07 <xarses> mihgen: the details need to be 100% consensus
16:52:15 <angdraug> besides, as I said, if it's substantial start a new patchset
16:52:15 <dshulyak> i am saying that it is responsibility of reviewers to ask questions on spec before merging code
16:52:17 <evgeniyl___> mihgen: yes we are, typos are ok, but detailed design should be ready
16:52:20 <xarses> the spec might not be
16:52:41 <xarses> we are also reviewing/ working on specs too late
16:52:45 <angdraug> what's the point of appointing mandatory reviewers if in the end some of them still have non-cosmetic concerns?
16:53:53 <xarses> we should right now (and more so in code freeze) be spending ~1-2hr a week for specs for 7.0 so that they are ready to work on
16:53:57 <angdraug> dshulyak: it's responsibility of reviewers to -1 the code change until spec is merged
16:54:11 <angdraug> xarses: +1
16:54:32 <dshulyak> angdraug: not blindly -1
16:54:51 <angdraug> not blindly, but inevitably )
16:54:54 <mihgen> so guys what are agree on? that spec should be fully ready, absolutely clean and perfect, then it gets merged?
16:55:02 <angdraug> mihgen: no
16:55:07 <dshulyak> mandatory reviewers should understand where it goes and start or prevent merge of code
16:55:10 <angdraug> you're pulling a strawman argument
16:55:17 <dshulyak> otherwise we will end up with +2k changes
16:55:25 <dshulyak> which nobody will ever review properly
16:55:27 <evgeniyl___> dshulyak: +1
16:56:01 <mihgen> angdraug: so explain the approach then
16:56:09 <mihgen> I don't get what we've agreed upon
16:56:11 <angdraug> mandatory reviewers should all agree with the design, it's ok to have cosmetic comments and explanations that are not required for general understanding of the design to be left for a follow-up patchset
16:56:27 <mihgen> angdraug: ok, and then merge?
16:56:33 <angdraug> yes
16:56:38 <xarses> mihgen: the spec details, the meat of it should be 100% conensus from the mandatory reviewers, and therefore merged prior to the code
16:56:40 <mihgen> and address minor things as separate patch?
16:56:46 <angdraug> yes
16:56:50 <xarses> yes
16:56:50 <evgeniyl___> dshulyak: had a lot of small patches, I'm sure that he wouldn't be able to deliver the feature if we were waiting for spec to be merged
16:56:53 <evgeniyl___> xarses: -1
16:57:23 <mihgen> we are going circles I think
16:57:29 <dshulyak> angdraug: what the goal of merging spec?
16:57:29 <evgeniyl___> mihgen: agree
16:57:32 <mihgen> let's move to openstack-dev then guys
16:57:40 <xarses> 3 min guys
16:57:43 <mihgen> let's think about it thotoughly
16:57:51 <evgeniyl___> probably we should start a new thread there
16:57:57 <xarses> evgeniyl___: +1
16:57:57 <mihgen> evgeniyl___: +1
16:58:08 <mihgen> cool let's do it.
16:58:16 <mihgen> finishing meeting
16:58:22 <mihgen> if no more questions - closing
16:58:42 <mihgen> thanks guys
16:58:44 <agordeev> thanks!
16:58:47 <mihgen> see you next week!
16:58:48 <dshulyak> buy
16:58:49 <evgeniyl___> thanks
16:58:53 <mihgen> #endmeeting #fuel