18:02:19 <SergeyLukjanov> #startmeeting sahara 18:02:20 <openstack> Meeting started Thu Mar 5 18:02:19 2015 UTC and is due to finish in 60 minutes. The chair is SergeyLukjanov. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:02:21 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 18:02:22 <weiting> hi 18:02:24 <openstack> The meeting name has been set to 'sahara' 18:02:26 <SergeyLukjanov> #link https://wiki.openstack.org/wiki/Meetings/SaharaAgenda 18:02:31 <huichun_> :) 18:02:32 <tosky> hi 18:02:33 <SergeyLukjanov> #topic sahara@horizon status (crobertsrh, NikitaKonovalov) 18:02:42 <SergeyLukjanov> #link https://etherpad.openstack.org/p/sahara-reviews-in-horizon 18:02:43 <crobertsrh> Looks like we're up to about 10 changes pending review at the moment, but we have seen a bit of an uptick in reviews over the past couple weeks. 18:03:08 <crobertsrh> Some are small ones that should [hopefully] be picked-up quickly 18:03:09 <NikitaKonovalov> crobertsrh: yes looks like we are getting reviews 18:03:31 <NikitaKonovalov> I'm still working on event log support 18:03:54 <tosky> I added two test related reviews to the list; hopefully others will come soon 18:04:20 <crobertsrh> tosky: I'll keep an eye out for test reviews. 18:04:27 <NikitaKonovalov> it looks more or less working, but there are still a lot of things to be done 18:04:57 <SergeyLukjanov> okay 18:05:23 <SergeyLukjanov> #topic News / updates 18:05:26 <SergeyLukjanov> folks, please 18:05:38 <SergeyLukjanov> #info FPF is today 18:05:54 <elmiko> working on the barbican integration patch, hope to land it today or tomorrow. also putting together a bug for the swift passwords to get them migrated into barbican. 18:05:55 <sreshetn1ak> working on patch with renaming sahara service 18:06:05 <tosky> still working on tests (horizon, sahara) - and I would like to talk about tempest them especially if sreshetn1ak is around 18:06:10 <tosky> (he is around) 18:06:16 <SergeyLukjanov> it means that all CRs for new features should be proposed till tomorrow, FPF exception are ok, please, ping me to discuss if you need an exception 18:06:17 <sreshetn1ak> patches in tempest devstack and horizon on review 18:06:18 <tmckay> I am working on job-types-endpoint changes, and default templates (is_default flag review is up, but not the default template CLI yet) 18:06:30 <Nikolay_St> working on the new logging style. hopefully it'll be done until Monday. 18:06:39 <crobertsrh> I have a few more tweaks to the job execution guide. Those tweaks have also caused me to go back and fix-up a couple other forms in the UI so they'll play nicely with the guides. 18:06:41 <huichun_> working on hbase common lib on HDFS when running edp job https://blueprints.launchpad.net/sahara/+spec/edp-add-hbase-lib 18:06:47 <elmiko> SergeyLukjanov: just to be clear you are talking about specs and not patch reviews? 18:07:00 <alazarev> I was busy with internal customer-oriented stuff, so only updating my patches, making review 18:07:26 <tmckay> seems like we have some gate errors that are common across CRs. Tempest service catalog issue, and heat_hdp_2 issue 18:07:45 <weiting> One bad news from Cloudera, finally they decided not to host cloudera cdh image on their site. 18:07:47 <tmckay> elmiko, +1 on question 18:08:02 <SergeyLukjanov> elmiko, I'm talking about code change requests - https://wiki.openstack.org/wiki/Kilo_Release_Schedule 18:08:10 <SergeyLukjanov> https://wiki.openstack.org/wiki/FeatureProposalFreeze is FPF description 18:08:30 <elmiko> SergeyLukjanov: that language on the wiki is vague 18:08:36 <SergeyLukjanov> elmiko, yeah 18:08:43 <SergeyLukjanov> let me try to explain 18:08:50 <elmiko> SergeyLukjanov: specifically, if a spec has been approved do we need an exception to land changes against that spec? 18:09:00 <SergeyLukjanov> elmiko, yup 18:09:12 <SergeyLukjanov> but we're doing FPF for the first time 18:09:18 * elmiko scratches head 18:09:41 <elmiko> so, technically, if i don't put the barbican integration code up (even though spec is approved), i need an exception? 18:09:48 <tmckay> so, I will need an exception against default-templates. And, my job-type-endpoint spec is not merged yet (more during open discussion) 18:09:54 <SergeyLukjanov> elmiko, yeah, that's correct 18:09:59 <elmiko> ooph, ok 18:10:05 <tmckay> elmiko, I think so. but, exceptions come from the core team, not the os release wranglers 18:10:06 <SergeyLukjanov> I've send an email with reminder a few days ago 18:10:13 <tmckay> so we control it ourselves 18:10:19 <SergeyLukjanov> tmckay, yup 18:10:28 <elmiko> SergeyLukjanov: i saw the email, but i thought it applied to specs not code reviews 18:10:38 <SergeyLukjanov> The project PTL, with the assistance of the core developers of the associated product, will evaluate the request and grant or deny the exception. 18:10:54 <elmiko> ack, and we should send to ML for exceptions? 18:11:04 <SergeyLukjanov> nope, let's just do it in IRC 18:11:37 <elmiko> ok, i need to request an exception for the code part of bp/improved-secret-storage =) 18:11:48 <tmckay> SergeyLukjanov, can we set up an etherpad so we know which CRs are granted exceptions, so we don't workflow+1 non-approved changes by accident? 18:11:49 <SergeyLukjanov> it's enough for us to have a brief discussion on IRC 18:11:56 <elmiko> SergeyLukjanov: sounds good 18:11:57 <SergeyLukjanov> and i will maintain wiki page with a list of exceptions 18:12:07 <tmckay> ah :) 18:12:09 <tmckay> thanks 18:12:15 <SergeyLukjanov> :) 18:12:21 <elmiko> what about bug fixes ? 18:12:35 <SergeyLukjanov> elmiko, it's only about new features 18:12:39 <elmiko> k, awesome 18:12:48 <tmckay> elmiko, if you can make it a bug, you can implement a new feature :) 18:12:58 <SergeyLukjanov> :) 18:13:01 <tmckay> bug: security needs more work 18:13:02 <elmiko> my intention is to make a bug to move the swift passwords into the external key manager 18:13:06 <elmiko> tmckay: lol 18:13:26 <SergeyLukjanov> elmiko, I will definitely approve the fpf exception for experimental integration with barbican 18:13:35 <elmiko> \o/ 18:13:47 <SergeyLukjanov> because risks are low and benefits are huge enough 18:13:58 <tmckay> +1 18:14:05 <SergeyLukjanov> I really like that our security dreams are becoming true :) 18:14:11 <elmiko> me too! 18:14:17 <tmckay> I feel safer already 18:15:16 <elmiko> egafford: we talked about the fpf exceptions, you will need to make requests for the code portions of the specs that are up now. fyi 18:15:25 <SergeyLukjanov> tmckay, +1 ;) 18:15:42 <egafford> Working on shell job patches mostly. elmiko: Ah, good to know. 18:16:51 <SergeyLukjanov> oh, I've checked ACLs in gerrit for our stable branches in proejcts 18:17:11 <SergeyLukjanov> for all projects except sahara itself the core team has all permissions for stable branches 18:17:15 <SergeyLukjanov> so, I think it's ok 18:17:52 <SergeyLukjanov> any other updates? 18:18:08 <Nikolay_St> nope from my side 18:18:12 <Nikolay_St> oh! 18:18:29 <SergeyLukjanov> you want to advertise your logs CR? :) 18:18:39 <egafford> Question: I've been poking the job interface map change; because UI support definitely won't make Kilo, though, is it safe to assume this should slip to Liberty? 18:18:48 <Nikolay_St> I remember. now hacking check don't verify if import groups a correct. 18:19:14 <SergeyLukjanov> Nikolay_St, is it a bug or as designed? 18:19:30 <SergeyLukjanov> egafford, honestly I think it's safier to postpone to L 18:19:34 <Nikolay_St> SergeyLukjanov: really don't know 18:19:46 <egafford> (R/T updates as work on the job interface map is status, though I'll be happy to reintroduce the question in discussion. :) ) SergeyLukjanov: I agree; it's a non-trivial change. 18:19:47 <Nikolay_St> SergeyLukjanov: we just faced it today during CR 18:19:57 <Nikolay_St> SergeyLukjanov: I'll check it tomorrow 18:20:32 <SergeyLukjanov> egafford, ok 18:20:49 <SergeyLukjanov> Nikolay_St, could you please investigate it? IMO groups checking is very important... 18:21:01 <Nikolay_St> SergeyLukjanov: yeap, sure 18:21:06 <elmiko> speaking of logging, did anyone notice this http://lists.openstack.org/pipermail/openstack-dev/2015-March/058226.html ? 18:21:12 <tmckay> egafford, it's an awesome improvement, but okay for L. It was not in scope until long after Paris. 18:21:17 <elmiko> a new logging working group 18:21:27 <Nikolay_St> #action investigate hacking checks for import groups 18:21:59 <elmiko> given all the logging work we've done recently, i think we should have someone as liaison to that gorup 18:22:02 <elmiko> *group 18:22:02 <Nikolay_St> elmiko: not me 18:22:03 <egafford> tmckay: Ack. 18:22:14 <Nikolay_St> elmiko: I'll attend next meeting 18:22:28 <elmiko> Nikolay_St: yay! 18:22:39 <Nikolay_St> SergeyLukjanov: what do you think about it? 18:22:39 <elmiko> thanks, i think it's worthwhile 18:22:55 <SergeyLukjanov> #action Nikolay_St to investigate hacking checks for import groups 18:23:19 <SergeyLukjanov> Nikolay_St, elmiko +1 18:23:55 <Nikolay_St> #action Nikolay_St attend next Log Working Group meeting 18:24:05 <tmckay> #link https://review.openstack.org/161448 18:24:05 <tmckay> #link https://review.openstack.org/161250 18:24:05 <tmckay> #link https://review.openstack.org/157563 18:24:05 <tmckay> I need reviews on these. Specifically, there is a question about the output format from the python client, sample output is in the spec. And SergeyLukjanov had a question about relabeling fields in the REST response that was never resolved. I would like to get the spec and the client/engine CRs merged soon. 18:24:40 <SergeyLukjanov> Nikolay_St, I'll add you to the cross project liasons list as the sahara liasons for the log wg 18:25:07 <SergeyLukjanov> tmckay, relabeling? 18:25:16 <Nikolay_St> SergeyLukjanov: ok 18:25:37 <vgridnev_> SergeyLukjanov, Nikolay, this hacking checks was removed from 0.10 version of hacking. They decided this changes too hard, and removed this checks 18:25:41 <tmckay> SergeyLukjanov, yeah, let me find the comment. old patch set on the spec. You suggested different field names, but the field names are from the template records 18:26:10 <Nikolay_St> vgridnev_: ok. but that sucks 18:26:24 <Nikolay_St> can we enable this check locally in Sahara? 18:26:25 <SergeyLukjanov> tmckay, oh, shame on me so :) 18:26:26 <huichun_> tmckay: does it mean i have the question with egafford? that i should merge horizon patch before their deadline? not our time on 3/19 ? 18:26:33 <tmckay> SergeyLukjanov, https://review.openstack.org/#/c/157563/5/specs/kilo/edp-job-types-endpoint.rst 18:26:49 <SergeyLukjanov> Nikolay_St, IMO we should add them to Sahara (as a 3rd party checks) 18:26:55 <huichun_> tmckay :same question like egafford 18:27:23 <SergeyLukjanov> Nikolay_St, can you do it? (probably copy paste check from the git history of hacking :) ) 18:27:26 <tmckay> huichun_, hmm, I missed the earlier question. Yes, though, I think for a horizon change we need to land on their schedule 18:27:32 <Nikolay_St> SergeyLukjanov: yeap 18:27:36 <tmckay> egafford, ^^ is this your understanding? 18:27:43 <NikitaKonovalov> tmckay: horizon follows the common schedule 18:28:16 <NikitaKonovalov> the changes(if they are proposed in time) should be merged before the code freeze 18:28:25 <SergeyLukjanov> tmckay, oh, I understand, probably we can do it as nesting this section under the single plugin field? 18:28:28 <Nikolay_St> SergeyLukjanov: should I do it ASAP? before FPF? 18:28:30 <tmckay> NikitaKonovalov, are they using FPF? 18:28:49 <NikitaKonovalov> tmckay: not sure about that 18:28:59 <egafford> tmckay: I believe so, yes; we need to land Horizon changes per the Horizon team's guidelines. 18:29:02 <SergeyLukjanov> Nikolay_St, it's not a new feature, so, it could be just a bug, so FPF will not apply for it 18:29:13 <Nikolay_St> SergeyLukjanov: ok, get it 18:29:20 <SergeyLukjanov> #topic Open discussion 18:29:27 <SergeyLukjanov> I forgot to change topic ;) 18:29:38 <egafford> (Speaking of which, there is a patch in Horizon re: Shell job forms which should be reviewed. :) ) 18:30:36 <tmckay> SergeyLukjanov, nesting this section under the plugin field, not sure what you're referring to. Please add comments on the CRs and I will understand :) 18:30:44 <NikitaKonovalov> egafford: is it in this list https://etherpad.openstack.org/p/sahara-reviews-in-horizon ? 18:30:57 <NikitaKonovalov> if not, please add it there 18:31:23 <SergeyLukjanov> tmckay, ack 18:31:45 <NikitaKonovalov> I've seen horizon cores use that etherpad to find patches for review 18:31:56 <SergeyLukjanov> tmckay, I'll review it again today 18:32:05 <tmckay> huichun_, so it sounds like you should follow the Horizon schedule, which sounds like a freeze on 3/19. Sooner is better :) 18:32:14 <tmckay> SergeyLukjanov, thank you 18:32:25 <egafford> NikitaKonovalov: Done; thanks. 18:32:52 <Nikolay_St> guys, need your attention here 18:32:53 <tmckay> huichun_, ^^ note this etherpad. Add something here when you have a CR up. 18:32:54 <huichun_> NikitaKonovalov: hi NikitaKonovalov， so does it mean that i should have add my horizon patch on the etherpad? 18:33:05 <Nikolay_St> #link https://review.openstack.org/#/c/154037/ 18:33:30 <NikitaKonovalov> huichun_: yes, please add your horizon patches to that etherpad 18:33:37 <huichun_> tmckay: thanks you Trevor ^^ 18:33:39 <tmckay> Nikolay_St, ack 18:34:19 <Nikolay_St> tmckay: thx 18:34:43 <alazarev> I have several patches in horizon waiting for new sahara python client, I should wait for global requirements update, right? 18:34:51 <alazarev> please review https://review.openstack.org/#/c/155428/ 18:35:00 <tmckay> so, do we know what is causing gate failures? Seems like common errors across CRs, spells trouble 18:35:35 <egafford> tmckay: +1; we're seeing almost universal Tempest failures and very frequent HDP-heat failure. 18:35:46 <tmckay> #link https://review.openstack.org/#/c/160814/ 18:35:48 <tmckay> for example 18:36:10 <huichun_> me too +1 18:36:11 <tmckay> service catalog problem (recent service name change?) 18:36:55 <tmckay> and heat-hdp_2, not sure about that one. maybe a message queue timeout? 18:37:00 <elmiko> "APIException: Error occurred during validation" so informative lol 18:37:21 <elmiko> tmckay: that's what i suggested before! 18:37:36 <tmckay> elmiko, yeah, depends what log you look in 18:37:39 <elmiko> it only seems to happen on that hdp2 test though 18:38:02 <tmckay> I wonder if we should make that hdp 2 test non-voting? Is that too aggressive? 18:38:10 <elmiko> i wonder if there is a slow network issue that is exagerated by nova+heat+hdp2? 18:38:15 <tmckay> it's holding things up. tempest too 18:38:21 <tmckay> could be 18:38:29 <egafford> Are we registering as data_processing or data-processing in our tempest tests? 18:38:47 <tmckay> egafford, ? don't know 18:38:51 <egafford> Looks like data_processing. 18:39:17 <sreshetn1ak> as data_processing in mirantis ci 18:40:45 <egafford> Okay. elmiko and I were talking about this earlier; I believe he theorized that we might need to switch to data-processing (though if that is the case, we have a fairly troublesome backward compatibility issue.) 18:40:47 <tmckay> I wonder if we can register as both? 18:41:03 <elmiko> that's my speculation 18:41:12 <tmckay> sreshetn1ak, ^^ 18:41:18 <elmiko> tmckay: i think we could, but we need 2 sets of endpoints then 18:41:27 <elmiko> for the 2 different service ids 18:41:34 <tmckay> we can't register the same endpoint under 2 names? 18:41:45 <elmiko> no, they resolve to service ids 18:42:01 <elmiko> i don't think it's an issue though since you are still going through keystone for the endpoints 18:42:05 <egafford> Service type is a field of the service object, yeah. Endpoint just pks to service. 18:42:13 <egafford> /s/pks/fks/ 18:42:32 <sreshetn1ak> tempest-sahara-tests don't use endpoints from service catalog 18:42:47 <elmiko> ahh, well there goes that theory ;) 18:43:14 <elmiko> sreshetn1ak: why are those errors complaining about the service catalog then? 18:43:14 <tmckay> APIException: Failed to parse response from Sahara. Check if service catalog configured properly. 18:43:25 <tmckay> so that error is not meaningful? 18:43:29 <elmiko> maybe it's just a bad error msg 18:43:36 <egafford> Presumably the client is suggesting that as the most likely problem in the most likely case. 18:43:43 <elmiko> yea 18:43:47 <egafford> (Where the most likely case is not our Tempest tests. :) 18:44:03 <egafford> Still, good to know that's not the issue. 18:44:03 <sreshetn1ak> i think need to see api logs 18:44:28 <elmiko> sreshetn1ak: https://sahara.mirantis.com/logs/14/160814/3/check/tempest-sahara-tests/9dea7b8/console.html 18:44:36 <elmiko> (one example) 18:44:42 <tmckay> yeah, it's failing on everything. 18:45:26 <elmiko> SergeyLukjanov: did you ever get in touch with the person who was looking for Kilo release information from us? 18:45:29 <tmckay> I wonder if we can log the response, where it says "Failed to parse response from Sahara" 18:45:48 <elmiko> SergeyLukjanov: malini1 was the name 18:46:11 <SergeyLukjanov> elmiko, hm, probably yes 18:46:30 <elmiko> k, just following up =) 18:47:20 <sreshetn1ak> ekmiko: example don't contain sahara-api logs :( 18:48:28 <elmiko> doh! 18:49:02 <tosky> before closing: tests. Question about the approach here, if you generally agree: https://review.openstack.org/#/c/161370/ and about few points raised by sreshetn1ak/sreshetnyak 18:49:05 <tosky> and related question to alazarev and https://review.openstack.org/#/c/142632/ 18:49:12 <tosky> (when you are done with the log issue) 18:49:34 <egafford> sreshetn1ak: Yeah, logs for that specific test don't seem to contain sahara-api logs even in the positive case (so we have null information rather than negative information.) 18:50:48 <sreshetn1ak> degorenko working on problem with logs 18:50:54 <alazarev> https://review.openstack.org/#/c/142632/ need probably moved to heat repo 18:51:26 <alazarev> *be moved 18:52:20 <alazarev> and question "what to do if there is no fake plugin" is definitely make sense, probably need to implement custom check in tempest 18:53:24 <tosky> alazarev: about this: if you check the other review, I had to face the same issue for API tests; maybe you can recycle the templates 18:53:36 <tosky> s/recycle/reuse/ 18:54:53 <tosky> the question I have is: I defined only one template for each plugin, choosing a plugin version available in all supported openstack versions (as for tempest rules) 18:55:35 <tosky> should I change it to have more versions for each plugin, so that at least the latest one is always used for each plugin, or doesn't it matter? 18:56:23 <elmiko> good question 18:57:17 <elmiko> i'm not sure on that one. i guess it depends how deep we want the testing to go 18:57:29 <elmiko> also, will we need to then worry about deprecating older plugins? 18:57:58 <tosky> so, the templates are used for testing the basic API, so you don't need many combinations there 18:58:21 <elmiko> ah ok, in that case why not? 18:58:24 <tosky> but, as I was suggesting, they could be used elsewhere (see alazarev patch), so maybe having more templates for more versions could be more useful 18:58:32 <elmiko> yea 18:58:37 <SergeyLukjanov> 1 min left 18:58:39 <SergeyLukjanov> :) 18:58:47 <tosky> in the future they could be merged with the input templates used for the new set of tests too 18:59:05 <elmiko> makes sense to me 18:59:34 <tosky> any other opinion? You can comment also on the review(s) 18:59:38 <tosky> pleeease do :) 18:59:49 <SergeyLukjanov> #endmeeting