18:02:19 <SergeyLukjanov> #startmeeting sahara
18:02:20 <openstack> Meeting started Thu Mar  5 18:02:19 2015 UTC and is due to finish in 60 minutes.  The chair is SergeyLukjanov. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:02:21 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
18:02:22 <weiting> hi
18:02:24 <openstack> The meeting name has been set to 'sahara'
18:02:26 <SergeyLukjanov> #link https://wiki.openstack.org/wiki/Meetings/SaharaAgenda
18:02:31 <huichun_> :)
18:02:32 <tosky> hi
18:02:33 <SergeyLukjanov> #topic sahara@horizon status (crobertsrh, NikitaKonovalov)
18:02:42 <SergeyLukjanov> #link https://etherpad.openstack.org/p/sahara-reviews-in-horizon
18:02:43 <crobertsrh> Looks like we're up to about 10 changes pending review at the moment, but we have seen a bit of an uptick in reviews over the past couple weeks.
18:03:08 <crobertsrh> Some are small ones that should [hopefully] be picked-up quickly
18:03:09 <NikitaKonovalov> crobertsrh: yes looks like we are getting reviews
18:03:31 <NikitaKonovalov> I'm still working on event log support
18:03:54 <tosky> I added two test related reviews to the list; hopefully others will come soon
18:04:20 <crobertsrh> tosky:  I'll keep an eye out for test reviews.
18:04:27 <NikitaKonovalov> it looks more or less working, but there are still a lot of things to be done
18:04:57 <SergeyLukjanov> okay
18:05:23 <SergeyLukjanov> #topic News / updates
18:05:26 <SergeyLukjanov> folks, please
18:05:38 <SergeyLukjanov> #info FPF is today
18:05:54 <elmiko> working on the barbican integration patch, hope to land it today or tomorrow. also putting together a bug for the swift passwords to get them migrated into barbican.
18:05:55 <sreshetn1ak> working on patch with renaming sahara service
18:06:05 <tosky> still working on tests (horizon, sahara) - and I would like to talk about tempest them especially if sreshetn1ak is around
18:06:10 <tosky> (he is around)
18:06:16 <SergeyLukjanov> it means that all CRs for new features should be proposed till tomorrow, FPF exception are ok, please, ping me to discuss if you need an exception
18:06:17 <sreshetn1ak> patches in tempest devstack and horizon on review
18:06:18 <tmckay> I am working on job-types-endpoint changes, and default templates (is_default flag review is up, but not the default template CLI yet)
18:06:30 <Nikolay_St> working on the new logging style. hopefully it'll be done until Monday.
18:06:39 <crobertsrh> I have a few more tweaks to the job execution guide.  Those tweaks have also caused me to go back and fix-up a couple other forms in the UI so they'll play nicely with the guides.
18:06:41 <huichun_> working on hbase common lib on HDFS when running edp job https://blueprints.launchpad.net/sahara/+spec/edp-add-hbase-lib
18:06:47 <elmiko> SergeyLukjanov: just to be clear you are talking about specs and not patch reviews?
18:07:00 <alazarev> I was busy with internal customer-oriented stuff, so only updating my patches, making review
18:07:26 <tmckay> seems like we have some gate errors that are common across CRs.  Tempest service catalog issue, and heat_hdp_2 issue
18:07:45 <weiting> One bad news from Cloudera, finally they decided not to host cloudera cdh image on their site.
18:07:47 <tmckay> elmiko, +1 on question
18:08:02 <SergeyLukjanov> elmiko, I'm talking about code change requests - https://wiki.openstack.org/wiki/Kilo_Release_Schedule
18:08:10 <SergeyLukjanov> https://wiki.openstack.org/wiki/FeatureProposalFreeze is FPF description
18:08:30 <elmiko> SergeyLukjanov: that language on the wiki is vague
18:08:36 <SergeyLukjanov> elmiko, yeah
18:08:43 <SergeyLukjanov> let me try to explain
18:08:50 <elmiko> SergeyLukjanov: specifically, if a spec has been approved do we need an exception to land changes against that spec?
18:09:00 <SergeyLukjanov> elmiko, yup
18:09:12 <SergeyLukjanov> but we're doing FPF for the first time
18:09:18 * elmiko scratches head
18:09:41 <elmiko> so, technically, if i don't put the barbican integration code up (even though spec is approved), i need an exception?
18:09:48 <tmckay> so, I will need an exception against default-templates.  And, my job-type-endpoint spec is not merged yet (more during open discussion)
18:09:54 <SergeyLukjanov> elmiko, yeah, that's correct
18:09:59 <elmiko> ooph, ok
18:10:05 <tmckay> elmiko, I think so.  but, exceptions come from the core team, not the os release wranglers
18:10:06 <SergeyLukjanov> I've send an email with reminder a few days ago
18:10:13 <tmckay> so we control it ourselves
18:10:19 <SergeyLukjanov> tmckay, yup
18:10:28 <elmiko> SergeyLukjanov: i saw the email, but i thought it applied to specs not code reviews
18:10:38 <SergeyLukjanov> The project PTL, with the assistance of the core developers of the associated product, will evaluate the request and grant or deny the exception.
18:10:54 <elmiko> ack, and we should send to ML for exceptions?
18:11:04 <SergeyLukjanov> nope, let's just do it in IRC
18:11:37 <elmiko> ok, i need to request an exception for the code part of bp/improved-secret-storage =)
18:11:48 <tmckay> SergeyLukjanov, can we set up an etherpad so we know which CRs are granted exceptions, so we don't workflow+1 non-approved changes by accident?
18:11:49 <SergeyLukjanov> it's enough for us to have a brief discussion on IRC
18:11:56 <elmiko> SergeyLukjanov: sounds good
18:11:57 <SergeyLukjanov> and i will maintain wiki page with a list of exceptions
18:12:07 <tmckay> ah :)
18:12:09 <tmckay> thanks
18:12:15 <SergeyLukjanov> :)
18:12:21 <elmiko> what about bug fixes ?
18:12:35 <SergeyLukjanov> elmiko, it's only about new features
18:12:39 <elmiko> k, awesome
18:12:48 <tmckay> elmiko, if you can make it a bug, you can implement a new feature :)
18:12:58 <SergeyLukjanov> :)
18:13:01 <tmckay> bug: security needs more work
18:13:02 <elmiko> my intention is to make a bug to move the swift passwords into the external key manager
18:13:06 <elmiko> tmckay: lol
18:13:26 <SergeyLukjanov> elmiko, I will definitely approve the fpf exception for experimental integration with barbican
18:13:35 <elmiko> \o/
18:13:47 <SergeyLukjanov> because risks are low and benefits are huge enough
18:13:58 <tmckay> +1
18:14:05 <SergeyLukjanov> I really like that our security dreams are becoming true :)
18:14:11 <elmiko> me too!
18:14:17 <tmckay> I feel safer already
18:15:16 <elmiko> egafford: we talked about the fpf exceptions, you will need to make requests for the code portions of the specs that are up now. fyi
18:15:25 <SergeyLukjanov> tmckay, +1 ;)
18:15:42 <egafford> Working on shell job patches mostly. elmiko: Ah, good to know.
18:16:51 <SergeyLukjanov> oh, I've checked ACLs in gerrit for our stable branches in proejcts
18:17:11 <SergeyLukjanov> for all projects except sahara itself the core team has all permissions for stable branches
18:17:15 <SergeyLukjanov> so, I think it's ok
18:17:52 <SergeyLukjanov> any other updates?
18:18:08 <Nikolay_St> nope from my side
18:18:12 <Nikolay_St> oh!
18:18:29 <SergeyLukjanov> you want to advertise your logs CR? :)
18:18:39 <egafford> Question: I've been poking the job interface map change; because UI support definitely won't make Kilo, though, is it safe to assume this should slip to Liberty?
18:18:48 <Nikolay_St> I remember. now hacking check don't verify if import groups a correct.
18:19:14 <SergeyLukjanov> Nikolay_St, is it a bug or as designed?
18:19:30 <SergeyLukjanov> egafford, honestly I think it's safier to postpone to L
18:19:34 <Nikolay_St> SergeyLukjanov: really don't know
18:19:46 <egafford> (R/T updates as work on the job interface map is status, though I'll be happy to reintroduce the question in discussion. :) ) SergeyLukjanov: I agree; it's a non-trivial change.
18:19:47 <Nikolay_St> SergeyLukjanov: we just faced it today during CR
18:19:57 <Nikolay_St> SergeyLukjanov: I'll check it tomorrow
18:20:32 <SergeyLukjanov> egafford, ok
18:20:49 <SergeyLukjanov> Nikolay_St, could you please investigate it? IMO groups checking is very important...
18:21:01 <Nikolay_St> SergeyLukjanov: yeap, sure
18:21:06 <elmiko> speaking of logging, did anyone notice this http://lists.openstack.org/pipermail/openstack-dev/2015-March/058226.html ?
18:21:12 <tmckay> egafford, it's an awesome improvement, but okay for L.  It was not in scope until long after Paris.
18:21:17 <elmiko> a new logging working group
18:21:27 <Nikolay_St> #action investigate hacking checks for import groups
18:21:59 <elmiko> given all the logging work we've done recently, i think we should have someone as liaison to that gorup
18:22:02 <elmiko> *group
18:22:02 <Nikolay_St> elmiko: not me
18:22:03 <egafford> tmckay: Ack.
18:22:14 <Nikolay_St> elmiko: I'll attend next meeting
18:22:28 <elmiko> Nikolay_St: yay!
18:22:39 <Nikolay_St> SergeyLukjanov: what do you think about it?
18:22:39 <elmiko> thanks, i think it's worthwhile
18:22:55 <SergeyLukjanov> #action Nikolay_St to investigate hacking checks for import groups
18:23:19 <SergeyLukjanov> Nikolay_St, elmiko +1
18:23:55 <Nikolay_St> #action Nikolay_St attend next Log Working Group meeting
18:24:05 <tmckay> #link https://review.openstack.org/161448
18:24:05 <tmckay> #link https://review.openstack.org/161250
18:24:05 <tmckay> #link https://review.openstack.org/157563
18:24:05 <tmckay> I need reviews on these.  Specifically, there is a question about the output format from the python client, sample output is in the spec.  And SergeyLukjanov had a question about relabeling fields in the REST response that was never resolved.  I would like to get the spec and the client/engine CRs merged soon.
18:24:40 <SergeyLukjanov> Nikolay_St, I'll add you to the cross project liasons list as the sahara liasons for the log wg
18:25:07 <SergeyLukjanov> tmckay, relabeling?
18:25:16 <Nikolay_St> SergeyLukjanov: ok
18:25:37 <vgridnev_> SergeyLukjanov, Nikolay, this hacking checks was removed from 0.10 version of hacking. They decided this changes too hard, and removed this checks
18:25:41 <tmckay> SergeyLukjanov, yeah, let me find the comment.  old patch set on the spec.  You suggested different field names, but the field names are from the template records
18:26:10 <Nikolay_St> vgridnev_: ok. but that sucks
18:26:24 <Nikolay_St> can we enable this check locally in Sahara?
18:26:25 <SergeyLukjanov> tmckay, oh, shame on me so :)
18:26:26 <huichun_> tmckay:  does it mean i have the question with egafford? that i should merge horizon patch before their deadline? not our time on 3/19 ?
18:26:33 <tmckay> SergeyLukjanov, https://review.openstack.org/#/c/157563/5/specs/kilo/edp-job-types-endpoint.rst
18:26:49 <SergeyLukjanov> Nikolay_St, IMO we should add them to Sahara (as a 3rd party checks)
18:26:55 <huichun_> tmckay :same question like egafford
18:27:23 <SergeyLukjanov> Nikolay_St, can you do it? (probably copy paste check from the git history of hacking :) )
18:27:26 <tmckay> huichun_, hmm, I missed the earlier question.  Yes, though, I think for a horizon change we need to land on their schedule
18:27:32 <Nikolay_St> SergeyLukjanov: yeap
18:27:36 <tmckay> egafford, ^^ is this your understanding?
18:27:43 <NikitaKonovalov> tmckay: horizon follows the common schedule
18:28:16 <NikitaKonovalov> the changes(if they are proposed in time) should be merged before the code freeze
18:28:25 <SergeyLukjanov> tmckay, oh, I understand, probably we can do it as nesting this section under the single plugin field?
18:28:28 <Nikolay_St> SergeyLukjanov: should I do it ASAP? before FPF?
18:28:30 <tmckay> NikitaKonovalov, are they using FPF?
18:28:49 <NikitaKonovalov> tmckay: not sure about that
18:28:59 <egafford> tmckay: I believe so, yes; we need to land Horizon changes per the Horizon team's guidelines.
18:29:02 <SergeyLukjanov> Nikolay_St, it's not a new feature, so, it could be just a bug, so FPF will not apply for it
18:29:13 <Nikolay_St> SergeyLukjanov: ok, get it
18:29:20 <SergeyLukjanov> #topic Open discussion
18:29:27 <SergeyLukjanov> I forgot to change topic ;)
18:29:38 <egafford> (Speaking of which, there is a patch in Horizon re: Shell job forms which should be reviewed. :) )
18:30:36 <tmckay> SergeyLukjanov, nesting this section under the plugin field, not sure what you're referring to.  Please add comments on the CRs and I will understand :)
18:30:44 <NikitaKonovalov> egafford: is it in this list https://etherpad.openstack.org/p/sahara-reviews-in-horizon ?
18:30:57 <NikitaKonovalov> if not, please add it there
18:31:23 <SergeyLukjanov> tmckay, ack
18:31:45 <NikitaKonovalov> I've seen horizon cores use that etherpad to find patches for review
18:31:56 <SergeyLukjanov> tmckay, I'll review it again today
18:32:05 <tmckay> huichun_, so it sounds like you should follow the Horizon schedule, which sounds like a freeze on 3/19.  Sooner is better :)
18:32:14 <tmckay> SergeyLukjanov, thank  you
18:32:25 <egafford> NikitaKonovalov: Done; thanks.
18:32:52 <Nikolay_St> guys, need your attention here
18:32:53 <tmckay> huichun_, ^^ note this etherpad.  Add something here when you have a CR up.
18:32:54 <huichun_> NikitaKonovalov: hi NikitaKonovalov， so does it mean that i should have add my horizon patch on the etherpad?
18:33:05 <Nikolay_St> #link https://review.openstack.org/#/c/154037/
18:33:30 <NikitaKonovalov> huichun_: yes, please add your horizon patches to that etherpad
18:33:37 <huichun_> tmckay: thanks you Trevor ^^
18:33:39 <tmckay> Nikolay_St, ack
18:34:19 <Nikolay_St> tmckay: thx
18:34:43 <alazarev> I have several patches in horizon waiting for new sahara python client, I should wait for global requirements update, right?
18:34:51 <alazarev> please review https://review.openstack.org/#/c/155428/
18:35:00 <tmckay> so, do we know what is causing gate failures?  Seems like common errors across CRs, spells trouble
18:35:35 <egafford> tmckay: +1; we're seeing almost universal Tempest failures and very frequent HDP-heat failure.
18:35:46 <tmckay> #link https://review.openstack.org/#/c/160814/
18:35:48 <tmckay> for example
18:36:10 <huichun_> me too +1
18:36:11 <tmckay> service catalog problem (recent service name change?)
18:36:55 <tmckay> and heat-hdp_2, not sure about that one.  maybe a message queue timeout?
18:37:00 <elmiko> "APIException: Error occurred during validation" so informative lol
18:37:21 <elmiko> tmckay: that's what i suggested before!
18:37:36 <tmckay> elmiko, yeah, depends what log you look in
18:37:39 <elmiko> it only seems to happen on that hdp2 test though
18:38:02 <tmckay> I wonder if we should make that hdp 2 test non-voting?  Is that too aggressive?
18:38:10 <elmiko> i wonder if there is a slow network issue that is exagerated by nova+heat+hdp2?
18:38:15 <tmckay> it's holding things up.  tempest too
18:38:21 <tmckay> could be
18:38:29 <egafford> Are we registering as data_processing or data-processing in our tempest tests?
18:38:47 <tmckay> egafford, ? don't know
18:38:51 <egafford> Looks like data_processing.
18:39:17 <sreshetn1ak> as data_processing in mirantis ci
18:40:45 <egafford> Okay. elmiko and I were talking about this earlier; I believe he theorized that we might need to switch to data-processing (though if that is the case, we have a fairly troublesome backward compatibility issue.)
18:40:47 <tmckay> I wonder if we can register as both?
18:41:03 <elmiko> that's my speculation
18:41:12 <tmckay> sreshetn1ak, ^^
18:41:18 <elmiko> tmckay: i think we could, but we need 2 sets of endpoints then
18:41:27 <elmiko> for the 2 different service ids
18:41:34 <tmckay> we can't register the same endpoint under 2 names?
18:41:45 <elmiko> no, they resolve to service ids
18:42:01 <elmiko> i don't think it's an issue though since you are still going through keystone for the endpoints
18:42:05 <egafford> Service type is a field of the service object, yeah. Endpoint just pks to service.
18:42:13 <egafford> /s/pks/fks/
18:42:32 <sreshetn1ak> tempest-sahara-tests don't use endpoints from service catalog
18:42:47 <elmiko> ahh, well there goes that theory ;)
18:43:14 <elmiko> sreshetn1ak: why are those errors complaining about the service catalog then?
18:43:14 <tmckay> APIException: Failed to parse response from Sahara. Check if service catalog configured properly.
18:43:25 <tmckay> so that error is not meaningful?
18:43:29 <elmiko> maybe it's just a bad error msg
18:43:36 <egafford> Presumably the client is suggesting that as the most likely problem in the most likely case.
18:43:43 <elmiko> yea
18:43:47 <egafford> (Where the most likely case is not our Tempest tests. :)
18:44:03 <egafford> Still, good to know that's not the issue.
18:44:03 <sreshetn1ak> i think need to see api logs
18:44:28 <elmiko> sreshetn1ak: https://sahara.mirantis.com/logs/14/160814/3/check/tempest-sahara-tests/9dea7b8/console.html
18:44:36 <elmiko> (one example)
18:44:42 <tmckay> yeah, it's failing on everything.
18:45:26 <elmiko> SergeyLukjanov: did you ever get in touch with the person who was looking for Kilo release information from us?
18:45:29 <tmckay> I wonder if we can log the response, where it says "Failed to parse response from Sahara"
18:45:48 <elmiko> SergeyLukjanov: malini1 was the name
18:46:11 <SergeyLukjanov> elmiko, hm, probably yes
18:46:30 <elmiko> k, just following up =)
18:47:20 <sreshetn1ak> ekmiko: example don't contain sahara-api logs :(
18:48:28 <elmiko> doh!
18:49:02 <tosky> before closing: tests. Question about the approach here, if you generally agree: https://review.openstack.org/#/c/161370/ and about few points raised by sreshetn1ak/sreshetnyak
18:49:05 <tosky> and related question to alazarev and https://review.openstack.org/#/c/142632/
18:49:12 <tosky> (when you are done with the log issue)
18:49:34 <egafford> sreshetn1ak: Yeah, logs for that specific test don't seem to contain sahara-api logs even in the positive case (so we have null information rather than negative information.)
18:50:48 <sreshetn1ak> degorenko working on problem with logs
18:50:54 <alazarev> https://review.openstack.org/#/c/142632/ need probably moved to heat repo
18:51:26 <alazarev> *be moved
18:52:20 <alazarev> and question "what to do if there is no fake plugin" is definitely make sense, probably need to implement custom check in tempest
18:53:24 <tosky> alazarev: about this: if you check the other review, I had to face the same issue for API tests; maybe you can recycle the templates
18:53:36 <tosky> s/recycle/reuse/
18:54:53 <tosky> the question I have is: I defined only one template for each plugin, choosing a plugin version available in all supported openstack versions (as for tempest rules)
18:55:35 <tosky> should I change it to have more versions for each plugin, so that at least the latest one is always used for each plugin, or doesn't it matter?
18:56:23 <elmiko> good question
18:57:17 <elmiko> i'm not sure on that one. i guess it depends how deep we want the testing to go
18:57:29 <elmiko> also, will we need to then worry about deprecating older plugins?
18:57:58 <tosky> so, the templates are used for testing the basic API, so you don't need many combinations there
18:58:21 <elmiko> ah ok, in that case why not?
18:58:24 <tosky> but, as I was suggesting, they could be used elsewhere (see alazarev patch), so maybe having more templates for more versions could be more useful
18:58:32 <elmiko> yea
18:58:37 <SergeyLukjanov> 1 min left
18:58:39 <SergeyLukjanov> :)
18:58:47 <tosky> in the future they could be merged with the input templates used for the new set of tests too
18:59:05 <elmiko> makes sense to me
18:59:34 <tosky> any other opinion? You can comment also on the review(s)
18:59:38 <tosky> pleeease do :)
18:59:49 <SergeyLukjanov> #endmeeting