18:01:04 <elmiko> #startmeeting sahara
18:01:04 <openstack> Meeting started Thu Aug 20 18:01:04 2015 UTC and is due to finish in 60 minutes.  The chair is elmiko. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:01:05 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
18:01:08 <openstack> The meeting name has been set to 'sahara'
18:01:18 <vgridnev> o/
18:01:19 <huichun> hello
18:01:24 <elmiko> hi
18:01:35 <egafford> Hi all!
18:01:42 <elmiko> let's give folks a minute or two to get here
18:02:06 <crobertsrh> hello/
18:02:14 <elmiko> #chair SergeyLukjanov
18:02:21 <openstack> Current chairs: SergeyLukjanov elmiko
18:02:24 <tosky> hi
18:02:51 <elmiko> ok then
18:02:57 <apavlov> hi
18:03:00 <elmiko> #topic sahara@horizon status (crobertsrh, NikitaKonovalov)
18:03:04 <crobertsrh> #link https://etherpad.openstack.org/p/sahara-reviews-in-horizon
18:03:08 <crobertsrh> Review early and often
18:03:10 <elmiko> thank you sir
18:03:17 <elmiko> any news to report?
18:03:24 <crobertsrh> Once we get 3 or 4 +1s, we can bug a horiz core.
18:03:29 <elmiko> or just review, review, review ;)
18:03:31 <SergeyLukjanov> sorry folks, a bit late
18:03:33 <SergeyLukjanov> hi all
18:03:36 <elmiko> ok, cool
18:03:44 <NikitaKonovalov> no news here from my side
18:03:48 <elmiko> hey SergeyLukjanov, just started. all yours =)
18:03:55 <crobertsrh> Even if your review is mostly a functional review of how great the code ran for you, we'll take it.
18:04:34 <SergeyLukjanov> crobertsrh, is it already helping us to make changes merging?
18:04:38 <crobertsrh> Does anyone know of more UI changes needed for L functionality?
18:04:42 <SergeyLukjanov> I mean move to the contrib
18:04:50 <sreshetnyak> o/
18:05:06 <crobertsrh> Not sure if the move to contrib really helps anything, but the agreement that we can do reviews on our stuff and get a single +2 has helped a bit
18:05:36 <crobertsrh> It is probably in our best interest to get our own repo going again at some point, maybe for M.
18:05:58 <elmiko> lol, the move back to sahara-dashboard?
18:06:01 <crobertsrh> It will be a bit sad to lose the eyes of the cores on our stuff though.
18:06:11 <elmiko> +1
18:06:15 <SergeyLukjanov> +1
18:06:19 <tosky> first let's see how it works, I would say
18:06:46 <SergeyLukjanov> but anyway we should evaluate pros and cons of staying in the horizon repo vs moving back to the sahara-dashboard
18:07:06 <SergeyLukjanov> tosky, yeah, I think we could wait till the summit
18:07:18 <elmiko> that's a wise move, imp
18:07:22 <elmiko> imo*
18:07:49 <SergeyLukjanov> and if there will be no significant improvements on the review time then we'll need to move back to the sahara-dashboard IMO
18:08:16 <egafford> +1.
18:08:18 <crobertsrh> +1
18:08:20 <NikitaKonovalov> +1
18:08:26 <vgridnev> +1
18:08:34 <elmiko> +1
18:08:37 <david-lyle> I honestly think that's a wise move
18:08:54 <david-lyle> I keep pushing for more reviews, but they're not happening
18:09:07 <crobertsrh> Yeah, it's just the nature of the beast
18:09:40 <david-lyle> I will make sure more of those patches merge before liberty
18:09:59 <egafford> Thanks david-lyle.
18:10:19 <david-lyle> but the split will be much easier now, if desired
18:10:22 <SergeyLukjanov> david-lyle, thx!
18:11:00 <SergeyLukjanov> so, I think that it'll be a great topic for the summit discussion
18:11:10 <elmiko> awesome
18:11:19 <SergeyLukjanov> will see how the contrib model will work in the next two month
18:11:50 <SergeyLukjanov> IMO it could work good enough for us, just core reviewers needs some time to start using it
18:12:08 <SergeyLukjanov> anything else to chat about in the ui topic?
18:13:00 <SergeyLukjanov> okay, let's move on
18:14:00 <SergeyLukjanov> #agreed monitor how contrib and +1 from sahara team == +2 helps merging patches to horizon and have a session about it on summit
18:14:03 <SergeyLukjanov> #topic News / updates
18:14:24 <apavlov> Working on acl pathces and grenade job. Finally make it work: https://review.openstack.org/#/c/193101/
18:15:23 <egafford> Potentially mergeable version of the interface UI is up. Beyond that, bug triage and prep for bugfix @ https://etherpad.openstack.org/p/sahara-liberty-bug-fix-day. We have a bit of pruning of old bugs to do, especially in the medium prio range.
18:15:28 <elmiko> i have a few keystone session patches up, and i'm working to get the improved secret store patches ready for push.
18:15:31 <NikitaKonovalov> not much of an update form me. mostly busy with internal activities
18:15:42 <vgridnev> no so much from me, filed important bug: https://bugs.launchpad.net/sahara/+bug/1486544
18:15:42 <openstack> Launchpad bug 1486544 in Sahara "[Spark1.3] Spark jobs doesn't work with Swift" [Critical,Triaged] - Assigned to Trevor McKay (tmckay)
18:15:43 <sreshetnyak> I'm working on wait condition support for heat engine. #link https://review.openstack.org/#/c/214568/
18:15:50 <elmiko> SergeyLukjanov: the spec has been approved, but will i need ffe for the improved sec stuff if i propose the patches after the freeze?
18:16:11 <SergeyLukjanov> yay! we have scenario tests in the gate based on the fake plugin now, that are still non-voting, but I think we could make them voting next week. this jobs are testing (nova-net or neutron) X (direct or heat), so, we could say that proviosning engines are very good covered in the upstream gate (sample https://review.openstack.org/#/c/214205/ )
18:16:26 <huichun> still working on edp job features
18:17:00 <SergeyLukjanov> elmiko, yes
18:17:13 <elmiko> SergeyLukjanov: ack, i'll post something to the ML
18:17:24 <SergeyLukjanov> elmiko, but you have 2 weeks till the FF
18:17:44 <elmiko> SergeyLukjanov: cool, i'm mainly waiting on the castellan update to pypi. so i'm not sure when that will happen
18:17:50 <tosky> no much from me either, I will start more testing after the freeze
18:17:56 <SergeyLukjanov> egafford, awesome etherpad for the bug fix day!
18:18:00 * redrobot pokes head in
18:18:09 * elmiko waves at redrobot
18:18:23 <tmckay> hey there
18:18:32 <redrobot> elmiko we're waiting on a python-barbicanclient release first... it's in progress now.  As soon as that's done we can release a new Castellan
18:18:56 <egafford> SergeyLukjanov: Thanks; there are some more mediums to grab in once we've pruned down, but there's certainly more than enough to start. :)
18:19:08 <elmiko> redrobot: awesome, thanks!
18:19:15 <SergeyLukjanov> redrobot, if you need help from the release mgmt side, please, ping me, I can help with it
18:19:42 <redrobot> thanks SergeyLukjanov.  We're currently waiting for a change to gerrit acls to allow relmgt team to push tags
18:20:09 <SergeyLukjanov> redrobot, send me a link, I'll +2 it
18:20:17 <redrobot> SergeyLukjanov https://review.openstack.org/#/c/215269/
18:20:21 <elmiko> SergeyLukjanov: on the triage effort, we made it most of the way through the 33 untriaged bugs, a few new ones were added, and many were closed out
18:20:39 <SergeyLukjanov> elmiko, yeah, that's great!
18:21:14 <egafford> (Thanks for picking all that up and running with it, elmiko!)
18:21:23 <elmiko> np
18:21:45 <tmckay> so, I have a solution to the swift/spark bug found by vgridnev, just working out the best way to incorporate in current code
18:22:02 <tmckay> summary, bug in spark was fixed, so sahara broke :)
18:22:02 <SergeyLukjanov> tosky, have you seen new gate jobs based on scenario tests/
18:22:19 <SergeyLukjanov> tmckay :)
18:22:27 <tmckay> vgridnev, ++ to you for adding spark/swift integration job
18:22:35 <tmckay> I always meant to do that ....
18:22:40 <tosky> SergeyLukjanov: I have seen the proposals and the reviews, not yet the jobs themselves
18:23:44 <SergeyLukjanov> tosky, thay are now non-voting check pipeline
18:23:56 <SergeyLukjanov> tosky sample: https://review.openstack.org/#/c/214205/
18:24:16 <elmiko> kudos to all on the scenario tests. i've been running them locally and i'm impressed with how well they integrate
18:24:18 <SergeyLukjanov> tosky, gate-sahara-dsvm-scenario-<network>-<engine>
18:24:54 <SergeyLukjanov> hopefully, we'll keep improving them :)
18:25:17 <SergeyLukjanov> #topic Bug triage / fix days / doc days progress
18:25:27 <huichun> hi Trevor, need help on scheduler edp. patch review and recurrence spec review ^_^
18:25:34 <SergeyLukjanov> elmiko, was it added by you? https://bugs.launchpad.net/sahara/+bug/1419643
18:25:34 <openstack> Launchpad bug 1419643 in Sahara "saharaclient should check the input param when cluster-create" [Undecided,Confirmed] - Assigned to warewang (wangguangcai)
18:25:48 <elmiko> yes
18:26:01 <elmiko> this bug looked like we needed to discuss what should happen
18:26:10 <elmiko> or even if we need to change anything
18:26:12 <huichun> tmckay: ^_^
18:26:31 <tmckay> huichun, yes, sorry.  I let myself get distracted.
18:26:40 <elmiko> so, if a user runs the sahara cli tool and provides no stdin with --json, the tool will wait indefinitely
18:26:47 <elmiko> obviously, waiting for input
18:27:04 <elmiko> the question is, is this even a bug, or just bad behavior ;)
18:27:21 <SergeyLukjanov> hm
18:27:26 <SergeyLukjanov> elmiko good question :)
18:27:30 <elmiko> hehe ;)
18:27:53 <elmiko> we have no way of detecting whether the user is getting ready to pipe something or not. so i'm not sure
18:28:00 * egafford votes for exiting with an error if stdin is not immediately extant; better to not block scripts without explanation than to support freestyle json at the bash prompt (which I can't really see many people doing.)
18:28:09 <SergeyLukjanov> I would say that it's a normal behavoir
18:28:09 <egafford> (Just one opinion.)
18:28:30 <SergeyLukjanov> because I see uage of it to pipe cat blabla.json
18:28:40 <elmiko> yea, i can see this either way. i mean there are plenty of cli tools that will do the same
18:29:10 <egafford> Can we time out at least?
18:29:11 <tellesnobrega> i agree that its normal behaviour but i think we should exit and error
18:29:49 <tellesnobrega> an*
18:29:56 <elmiko> egafford: i thought about that, but then people will just get p.o.'d waiting for the timeout
18:30:13 <egafford> elmiko: Valid.
18:30:33 <elmiko> that being said, i'm also ok with just erroring out if nothing is immediately provided
18:30:57 <elmiko> also, it might be worth looking at the openstackclient to see what it does when it expects some json
18:31:00 <tmckay> my opinion, works the same as cat
18:31:18 <tmckay> as long as it's documented in the help
18:31:23 <elmiko> tmckay: so, just leave it as is then?
18:31:23 <SergeyLukjanov> IMO if user exec "sahara cluster-create --json" than it's absolutely okay to wait for the input
18:31:40 <tmckay> elmiko, I think so
18:31:43 <SergeyLukjanov> it's like user creating cluster but not specifying anything
18:31:48 <elmiko> right
18:31:53 <tosky> can't we use the '-' to say that it's stdin?
18:31:57 <SergeyLukjanov> it's okay - if you're not specify any options :)
18:32:15 <elmiko> tosky: that's a nice idea too
18:32:27 <tmckay> hmm, we could force an argument and use "-" for stdin as tosky says
18:32:35 <elmiko> right
18:32:50 <elmiko> would someone mind making this comment on the bug?
18:32:52 <sreshetnyak> tosky: +1
18:33:00 <egafford> I prefer that. Leave the option, but make people reach for that, instead of get confused by the possibly unintended default behavior.
18:33:08 <tmckay> any precedent for this on other openstack clis?
18:33:13 <elmiko> #link https://bugs.launchpad.net/sahara/+bug/1419643
18:33:13 <openstack> Launchpad bug 1419643 in Python client library for Sahara "saharaclient should check the input param when cluster-create" [Undecided,Confirmed] - Assigned to warewang (wangguangcai)
18:33:23 <tosky> (thanks all, but credit to pinotree for the idea :)
18:33:29 <SergeyLukjanov> I'm ok with - for stdin
18:33:29 <elmiko> tmckay: that was my question about the openstackclient
18:33:42 <egafford> +1 to all Toscanos.
18:33:48 <egafford> (Blanket rule.)
18:33:49 <elmiko> tosky: no, if he doesn't speak up then you get the credit ;)
18:33:54 <elmiko> lol
18:34:10 <tmckay> so do we break backward compatibilyt if someone has scripted around the CLI? :)
18:34:32 <elmiko> #agreed change saharaclient to exit with error if not provided any input, and take "-" argument to specify that stdin input is expected
18:34:38 <elmiko> tmckay: i think so
18:34:43 <tmckay> like, a --json << would work, wouldn't it?
18:35:19 <elmiko> good question
18:35:37 <tmckay> I mean, what if we just put a "***** Waiting for Input! ******" line on the screen?
18:35:41 <SergeyLukjanov> I don't think that it will work
18:35:45 <tmckay> how big a deal is this?
18:36:01 <elmiko> i don't think it's a huge deal, but just came up on the bug
18:36:05 <tosky> it's already kind of not working even now
18:36:15 <elmiko> and honestly, we should move on and just add comments to the bug report imo
18:36:30 <egafford> tmckay: I think we only break backward compatibility if someone has scripted around the idea that they'll run a script that will prompt them to write a JSON file halfway through. Are there other breaking cases?
18:36:48 <tmckay> k. I'll +2 anything, completely don't care ;-)
18:36:53 <elmiko> lol
18:37:11 <elmiko> let's move on, enough bike-shedding, please add comments on the bug report =)
18:37:45 <elmiko> thanks for talking it through though
18:37:48 <SergeyLukjanov> anything else re bug/doc days?
18:38:03 <elmiko> doc pad will be up next week
18:38:12 <SergeyLukjanov> cool
18:38:22 <SergeyLukjanov> #topic Client release plans
18:38:48 <SergeyLukjanov> so, we have 0.10.0 released and then 0.10.1 with a hot fix for backward compat issue
18:41:26 <elmiko> sounds good
18:42:13 <SergeyLukjanov> I'm expecting to have 0.11.0 release in the end of next weeK (?) with rest of the features merged
18:42:43 <elmiko> quick turn around ;)
18:43:37 <SergeyLukjanov> :)
18:43:49 <SergeyLukjanov> #topic API v2
18:43:52 <SergeyLukjanov> elmiko, please
18:44:12 <elmiko> ok, then
18:44:34 <elmiko> SergeyLukjanov, you brought up the idea the cluster actions in the v2 spec review
18:44:53 <elmiko> i'd like to talk about that in the larger scope of creating a /tasks and /tasks/{task_id} endpoints
18:45:13 <elmiko> instead of just creating one for clusters, we could make a more general purpose endpoint
18:45:27 <elmiko> and this would give us a nice way to handle asynchronous operations
18:45:47 <elmiko> it would entail creating a new database table for the tasks, but i think we could make good use of it
18:46:08 <SergeyLukjanov> sounds interesting
18:46:21 <SergeyLukjanov> do we have something similar to it in other openstack projects?
18:46:39 <elmiko> it has been proposed before by jaypipes
18:46:48 <elmiko> i will need to look around for examples
18:47:29 <elmiko> the idea i took from it was to replace the actions stuff in nova with a more generic interface that treats the individual actions(tasks) as resources
18:47:32 <SergeyLukjanov> okey
18:48:38 <SergeyLukjanov> anything else to chat here?
18:48:41 <elmiko> i think for the first run at the v2 api, we could mention this, whether its cluster actions or tasks, and then create a spec to address it
18:49:10 <elmiko> no, just that i'd like more reviews on the v2 spec as i'd like to start creating the patches for the experimental api
18:50:21 <egafford> elmiko: I love the idea of a tasks resource. I think there's a significant discussion on whether you, say, PUT to the tasks endpoint to create a cluster, or PUT to the cluster endpoint and get back a ref to a cluster and a task as a side effect, which you can then poll, but the idea itself is really solid, and will allow us to do some really clarifying things to our flows.
18:51:05 <elmiko> egafford: agreed, i'm not sure about that either. ideally we wouldn't want to change too much of the mechanics around clusters and jobs, but it's a thought
18:51:09 <SergeyLukjanov> ++
18:51:29 <SergeyLukjanov> so, let's continue discussion on the spec review
18:51:34 <elmiko> thanks
18:51:51 <SergeyLukjanov> elmiko, thx for keep pushing the v2 API :)
18:51:54 <SergeyLukjanov> #topic Open discussion
18:52:01 <elmiko> SergeyLukjanov: i'm trying ;)
18:52:14 <elmiko> did you want to talk about zaqar and guest agent?
18:52:38 <crobertsrh> ooh, another good topic elmiko
18:52:54 <elmiko> (in fairness i think SergeyLukjanov added it to the agenda)
18:53:20 <SergeyLukjanov> oh, I've missed it
18:53:22 <tmckay> lets do it before the freeze :)
18:53:25 <SergeyLukjanov> #undo
18:53:25 <openstack> Removing item from minutes: <ircmeeting.items.Topic object at 0x9f0a750>
18:53:37 <SergeyLukjanov> #topic Sahara / Zaqar integration for guest agents implementation
18:54:03 <SergeyLukjanov> so, flaper87 said that mostly all need functionally is ready from the Zaqar side
18:54:03 <elmiko> i think zaqar is a good path to investigate for a guest agent
18:54:15 <SergeyLukjanov> so, we need to start working on the specs from sahara side
18:54:26 <SergeyLukjanov> does somebody wants to work on it?
18:54:42 <tosky> guest agent, wasn't there an idea for an unified guest agent (like one year ago)?
18:54:48 <tmckay> so, what is the goal?  Are we trying to get rid of all the ssh usage?
18:54:53 <elmiko> i'm doing some zaqar investigation, i'd be happy to help
18:55:06 <tmckay> Should we start with a subset, and switch incrementall?
18:55:07 <SergeyLukjanov> tmckay, yeah
18:55:07 <tellesnobrega> i want to help out as well
18:55:09 <elmiko> but i don't want to take lead on this one
18:55:19 <SergeyLukjanov> the goal is to migrate to the better communication with VMs
18:55:19 <tmckay> do we use it during provisioning, too?
18:55:31 <elmiko> good question
18:55:43 <tmckay> I think we could start with a small test case
18:55:48 <elmiko> if we put the agent on during image creation then we could use it during provisioning
18:55:49 <SergeyLukjanov> in fact to replace ssh with unified agents with configurable transport
18:55:56 <egafford> elmiko: I'd like to help as well; I work with two of the Zaqar folks pretty closely, and have meant to get involved for a while. Also unclear whether I can take lead on that, though.
18:56:17 <elmiko> cool, sounds like we have a good group that could take on the work
18:56:32 <egafford> So maybe we can just gaggle our way to success. :)
18:56:36 <egafford> COMMUNITY.
18:56:42 <tellesnobrega> :)
18:56:44 <elmiko> well, someone is gonna need to take point on creating the spec
18:56:48 <egafford> True.
18:57:11 <elmiko> and there are a few sources i can point towards in terms of the larger openstack effort to solve some of the issues surrounding this
18:57:12 <SergeyLukjanov> elmiko, egafford, please involve me into discussions too, I have a bunch of ideas :)
18:57:19 <elmiko> SergeyLukjanov: definitely
18:57:27 <SergeyLukjanov> flaper87 said that he will send some materials on it later
18:57:50 <tmckay> I'll be involved too.
18:57:59 <SergeyLukjanov> cool
18:58:07 <tmckay> If nobody else wants to lead the spec, I can
18:58:10 <elmiko> SergeyLukjanov: also, i've added a topic about this to the security mid-cycle meetup, see "Secure communications between control plane and tenant plane" on https://etherpad.openstack.org/p/security-liberty-midcycle
18:58:18 <SergeyLukjanov> #topic Last minute open discussion
18:58:24 <egafford> Once we have a solid idea of what we're going for and a minimal POC, I'm good taking on the spec if we like. Oh, cool tmckay. Sounds good. :)
18:58:31 <SergeyLukjanov> elmiko, yeah, I'm dreaming about it
18:58:38 <elmiko> hehe
18:58:53 <tmckay> egafford, really, you can do it :)
18:59:00 <egafford> tmckay: We can talk about it.
18:59:13 <SergeyLukjanov> in fact it'll be great to figure out how to build unified agent on top of zaqar and probably start it as a separated project and then use in sahara
18:59:22 <elmiko> +1
18:59:23 <egafford> SergeyLukjanov: YES.
18:59:33 <SergeyLukjanov> 30 sec left
18:59:46 <tosky> +1 for unified agent
19:00:10 <SergeyLukjanov> #endmeeting