18:00:23 <SergeyLukjanov> #startmeeting sahara
18:00:24 <openstack> Meeting started Thu Mar 17 18:00:23 2016 UTC and is due to finish in 60 minutes.  The chair is SergeyLukjanov. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:00:25 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
18:00:28 <openstack> The meeting name has been set to 'sahara'
18:00:32 <vgridnev> hi
18:00:33 <esikachev> o/
18:00:33 <tosky> hi
18:00:34 <SergeyLukjanov> #link http://wiki.openstack.org/wiki/Meetings/SaharaAgenda
18:00:50 <SergeyLukjanov> #topic News / updates
18:01:29 <vgridnev> working almost on bug fixing and testing sahara, some internal stuff
18:01:40 <mionkin> hi, I've just finished support of usage hadoop swiftfs with Keystone API v3 by adding hadoop-openstack.jar into plugins images
18:01:42 <esikachev> worked on nightly jobs. Merged! :)
18:01:43 <elmiko> i'm mainly working on api v2, and a little on talk for summit. also, we have completed the bandit work and it is all green now =)
18:01:43 <SergeyLukjanov> #info I'm stepping down as PTL, staying around w/ a project, always ready to help
18:01:56 <huichun> Working on patch review and EDP
18:02:05 <SergeyLukjanov> the RC1 will be today
18:02:15 <SergeyLukjanov> updating my repos right now
18:02:23 * elmiko raises a glass to SergeyLukjanov ;)
18:02:54 <vgridnev> thanks SergeyLukjanov !
18:02:57 * SergeyLukjanov planning to have a beers in Austin ;)
18:03:03 <elmiko> +1
18:03:08 <huichun> Thanks SergeyLukjanov
18:03:08 <tosky> yes, right, thanks SergeyLukjanov
18:03:49 <tosky> trying to migrate sahara tempest API tests to sahara-tests _with_ history
18:04:38 <vgridnev> and also, I brought my candidacy for PTL role in Newton
18:04:42 <vgridnev> #link https://review.openstack.org/#/c/293987/
18:04:54 <crobertsrh> I've mostly been testing out the UI.  I added myself to a couple bugs that I hope to fix soon.
18:05:02 <elmiko> vgridnev: +1, thanks!
18:05:08 <crobertsrh> great news vgridnev :)
18:05:29 <huichun> Cong vgridnev !
18:05:33 <SergeyLukjanov> tosky no way to migrate with history actually ;)
18:05:45 <tosky> SergeyLukjanov: I think it is
18:06:00 <tosky> SergeyLukjanov: but let's talk later, if there are other topic
18:06:03 <tosky> topics*
18:06:04 <SergeyLukjanov> sure
18:06:11 <SergeyLukjanov> let's move on?
18:07:20 <carolbarrett> I'd like to have a few mins to talk about Sahara input for the Community roadmap, if possible
18:07:42 <SergeyLukjanov> #topic OpenStack Community Roadmap - Sahara info review (Carol Barrett) (I can only attend for the 1st 30 mins)
18:07:46 <SergeyLukjanov> carolbarrett sure
18:07:49 <elmiko> thanks carolbarrett =)
18:07:59 <carolbarrett> Thanks Sergey
18:08:02 <SergeyLukjanov> Link to the draft: https://docs.google.com/presentation/d/1CwHEYE1YDRoHIaFQjN2wwEnj5ZQEUkIsCJwl9NG-f0A/edit?usp=sharing
18:08:02 <SergeyLukjanov> Link to roadmap published for Tokyo Summit: https://www.openstack.org/assets/tokyo-summit/OpenStack-Roadmap-Mitaka-Update.pptx
18:08:15 <carolbarrett> I'll start with a little background
18:08:23 <SergeyLukjanov> carolbarrett sorry for the no replies from my sight...
18:08:29 <elmiko> #link https://docs.google.com/presentation/d/1CwHEYE1YDRoHIaFQjN2wwEnj5ZQEUkIsCJwl9NG-f0A/edit?usp=sharing
18:08:35 <elmiko> #link https://www.openstack.org/assets/tokyo-summit/OpenStack-Roadmap-Mitaka-Update.pptx
18:08:41 <carolbarrett> elmiko: Thanks!
18:09:23 <carolbarrett> I'm part of the Product Work Group. This team produces a couple of deliverables like User Stories to bring requirements from markets (like enterprise) into the community
18:09:55 <carolbarrett> As well as producing a multi-release roadmap that gets released ahead of Summits and updated after the Design Summit dust settles
18:10:08 <carolbarrett> we do this in collaboration with the OpenStack foundation
18:10:23 <carolbarrett> I was asked to reach out to you all and create a page on Sahara
18:10:29 <elmiko> \o/
18:10:32 <carolbarrett> That's in the 1st link posted
18:10:41 <SergeyLukjanov> carolbarrett thank you!
18:10:45 <SergeyLukjanov> I like the slide for Sahara and the items seems to be correct (#1 link)
18:11:07 <carolbarrett> SergeyLukjanov: Thanks! Glad to hear that!
18:11:11 <crobertsrh> Yeah, the slide looks good
18:11:38 <carolbarrett> I could use a bit more info on a couple of these to be able to speak to them.
18:11:56 <huichun> Nice slide carolbarrett
18:11:56 <carolbarrett> I looked in your specs, launchpad, etc to put this together.
18:12:21 <carolbarrett> Is there any other place that I could look to find the details on SPI method for example?
18:12:38 <carolbarrett> And pls - go ahead and edit the slide to improve the content!
18:13:07 <carolbarrett> Is there more we can say, directionally, about Ocata?
18:13:27 <elmiko> carolbarrett: in general, this is the documentation for our spi methods docs.openstack.org/developer/sahara/devref/plugin.spi.html
18:13:38 <elmiko> it's not very descriptive, but gives a developer eye view
18:13:38 <carolbarrett> elmiko: Thanks!
18:14:09 <elmiko> the method in question, i believe was the validate(cluster)
18:14:37 <elmiko> i think we have more info on the way images are validated, let me look
18:14:41 * elmiko rifles through links
18:15:13 <elmiko> http://specs.openstack.org/openstack/sahara-specs/specs/mitaka/validate-image-spi.html
18:15:22 <vgridnev> elmiko, yeah, you are right, but I was not landed into Mitaka release, as far I know
18:15:22 <elmiko> that's the dirty details ;)
18:15:37 <elmiko> vgridnev: i might have the wrong method there
18:16:07 <carolbarrett> vgridnev: Can we verify whether it has landed in Mitaka?
18:16:12 <sreshetnyak> o/
18:16:22 <carolbarrett> And if it didn't land, should we push to Newton?
18:16:23 <vgridnev> 1min
18:17:16 <vgridnev> seems it's on review: https://review.openstack.org/#/c/263424/
18:17:17 <carolbarrett> This is the type of feedback I'm looking for, want to make sure that the info is accurate and covers the most siginificant new capabilities that were added in Mitaka
18:17:36 <vgridnev> Yeah, I think that we should push that to the Newton release
18:17:57 <carolbarrett> vgridnev: will make the change
18:18:06 <elmiko> vgridnev: +1
18:18:20 <carolbarrett> what else do we need to change?
18:18:51 <SergeyLukjanov> (no ideas from my side)
18:19:09 <elmiko> the mitaka portion looks ok to me
18:19:28 <elmiko> do we need to add any other deprecations?
18:19:31 <vgridnev> I think no, but probably we should mention with MapR versions were removed
18:19:57 <carolbarrett> SergeyLukjanov: You think the slide calls out the top priorities for the project as you go into the Newton Design Sessions?
18:20:06 <vgridnev> or just mention which are still there, MapR 500 and MapR 510 are still there
18:20:33 <SergeyLukjanov> carolbarrett I think so
18:20:35 <carolbarrett> vgridnev: What do we need to change on the "Remove" entry under mitaka?
18:21:13 <carolbarrett> SergeyLukjanov: Are there any other ongoing or long term areas for enhancement or expansion that we would want to call out?
18:21:23 <elmiko> apiv2?
18:21:26 <vgridnev> I would say all MapR plugin versions were removed except for 500 and 510
18:21:49 <elmiko> carolbarrett: we are currently working on a new version of our api, it definitely qualifies as long term ;)
18:22:07 <carolbarrett> This could be based upon changes in other OpenStack Proects, Big Tent direction, etc?
18:22:14 <carolbarrett> Modularity, scalability, etc?
18:22:19 <SergeyLukjanov> elmiko yeah, I think v2 API could be added as on-going work for the next releases
18:22:32 <SergeyLukjanov> carolbarrett ^^
18:22:49 <SergeyLukjanov> carolbarrett there is an often requested to support new plugins in old sahara releases
18:22:56 <huichun> SergeyLukjanov:  may I request EDP engine parts enhancement to be added in?
18:23:28 <SergeyLukjanov> carolbarrett so we want to do something with it, but no clear plans yet. there is too high coupling between sahara and plugins to support moving them to previous releases
18:24:09 <carolbarrett> What's the right wording for the v2 API? Is it about enhancing existing APIs or developing new ones or compatibility, etc?
18:24:40 <carolbarrett> SergeyLukjanov: Understand the compatibility challenge....
18:25:02 <carolbarrett> we can leave that out if you want
18:25:07 <huichun> EDP log enhancement and Multiple EDP workflow support
18:25:16 <esikachev> sahara-tests separation appropriate?
18:25:38 <carolbarrett> what about the EDP engine ehancements - should we carry that over to Newton? If so, any specific focus of that work that we can highlight?
18:25:41 <elmiko> carolbarrett: v2 is about enhancing/rewriting portions of existing api, and improving developer experience
18:25:54 <elmiko> it is a major version bump
18:25:56 <SergeyLukjanov> carolbarrett working on a new api that'll provide more clear and flexible endpoint for end users based on the existing one IMO
18:26:39 <carolbarrett> got it.
18:26:53 <carolbarrett> what about EDP engine suggestion?
18:26:59 <huichun> carolbarrett:  EDP log enhancement, we should add user friendly info to end user, and support multiple EDP workflow
18:28:20 <carolbarrett> huichun: Thanks
18:29:01 <carolbarrett> Good stuff!
18:29:01 <SergeyLukjanov> carolbarrett anything else to discuss on this topic?
18:29:28 <SergeyLukjanov> carolbarrett thx a lot, great job in putting this items together!
18:29:38 <carolbarrett> SergeyLukjanov: I think I got what I need. I'll update the slide and send a not on ML for any addl comments
18:29:58 <carolbarrett> SergeyLukjanov: Happy to help out, this is good stuff for our users and market!
18:29:59 <elmiko> +1, thanks carolbarrett !
18:30:16 <carolbarrett> thanks for your help elmiko!
18:30:16 <SergeyLukjanov> #topic API v2 progress
18:30:20 <SergeyLukjanov> #link https://review.openstack.org/#/c/273316/
18:30:20 <elmiko> hey
18:30:23 <SergeyLukjanov> #link https://wiki.openstack.org/wiki/Sahara/api-v2
18:30:33 <elmiko> all the work items have been added to the wiki
18:30:46 <elmiko> i am currently working on writing the specs for some of the bigger foundational features
18:31:05 <elmiko> a couple of red hatters have signed on to work on portions of this as well
18:31:18 <elmiko> so, hopefully over the next few weeks we'll start to put some reviews up
18:31:34 <elmiko> but, these can all wait until after mitaka release is finalized and freeze is over
18:31:57 <elmiko> i think that's about it for current status, any questions?
18:32:33 <SergeyLukjanov> elmiko cool, thx for the update!
18:32:42 <SergeyLukjanov> #topic Scenario tests releases
18:32:42 <elmiko> =)
18:32:49 <SergeyLukjanov> tosky, esikachev your turn
18:33:11 <tosky> so
18:33:16 <esikachev> tosky: what about moving tempest?
18:33:26 <esikachev> tests)
18:33:28 <tosky> I spoke with -infra, and they told me that a way is possible
18:34:16 <tosky> the idea is: push the filtered changes to a clean (orphan) branch, merge it, and request a special ACL to be able to push a merge commit to gerrit for review
18:34:52 <tosky> you can see how it would look like from the master branch here: https://github.com/ltoscano-rh/sahara-tests
18:35:10 <tosky> after that the code is merged, we can move it around/add tempest plugin interface with no issues
18:35:24 <SergeyLukjanov> tosky yeah, I think we can do it, I could help from infra root side
18:35:46 <SergeyLukjanov> about the move itself, I'm pretty neutral
18:35:55 <SergeyLukjanov> sreshetnyak was having some concerns AFAIK
18:35:58 <tosky> ok, so I will recheck that it is working, and if you are fine, I will proceed with this
18:36:04 <tosky> sure, which kind of concerns?
18:36:41 <SergeyLukjanov> while waiting for concerns
18:36:47 <tosky> tempest will (in the long run) keep only core parts and tests (mainly scenario) for core services
18:36:50 * tosky waits
18:36:53 <sreshetnyak> tosky: what about release scenario framework and tempest tests?
18:37:01 <SergeyLukjanov> esikachev tosky what are you guys thinking about release sahara-tests?
18:37:12 <tosky> SergeyLukjanov: as I said, I'm fine with that
18:37:25 <esikachev> SergeyLukjanov: all ready from my side
18:37:32 <sreshetnyak> we want release tempest tests with framework?
18:37:33 <tosky> esikachev: did you try to pip install it? Does it work?
18:37:51 <SergeyLukjanov> esikachev what's the release model, versioning, release criteria?
18:38:18 <tosky> sreshetnyak: if they are part of sahara-tests, yes, tempest-like tests for API would be released as well
18:38:35 <esikachev> SergeyLukjanov: criteria: all tests for testing sahara-release merged
18:39:02 <tosky> but if we keep the sahara-tests like tempest, so branchless and able to support all supported openstack releases, then it's not different from tempest "releases"
18:39:24 <esikachev> tosky: no, i am worked on nightly-jobs. but i tried it today
18:39:28 <tosky> (users could always use the master, like for tempest)
18:39:29 <esikachev> *try
18:39:51 <tosky> esikachev: oh, perfect; I think there could be some issues with some relative paths
18:40:13 <tosky> at least that's the effect I've seen in a draft RPM package for sahara-tests
18:40:21 <tosky> like this morning, and I couldn't investigate
18:40:52 <sreshetnyak> so, can we have tempest tests for testing releases?
18:41:39 <tosky> sreshetnyak: having tempest tests in sahara-tests instead of tempest is not different from the current situation, they are just in a different (but more proper) location
18:41:40 <tosky> imho
18:41:43 <huichun> tosky: tempest test for independent release?
18:42:07 <tosky> huichun: what do you mean by independent release?
18:42:50 <huichun> Current tempest is out of Sahara-tests, right?
18:43:40 <sreshetnyak> tosky: what about packaging? we should pack only scenario framework?
18:43:58 <esikachev> tosky: AFAIK, we need a spec
18:44:11 <tosky> huichun: current tempest tests are for API and are in tempest repo
18:44:34 <tosky> sreshetnyak: why should we package only scenario frameworks? I think everything from that repo could be package
18:45:18 <tosky> I mean, it's all like before, because sahara-tests (for previous discussions) is brancheless like tempest, but it is a better fit because
18:45:36 <tosky> - it's a non-core project
18:45:54 <tosky> - sahara-tests reviews do not require tempest core
18:46:07 <tosky> esikachev: a spec for the move?
18:46:12 <esikachev> yes
18:46:45 <esikachev> SergeyLukjanov: what did you think?^^
18:47:01 <sreshetnyak> tosky: I don't see reasons for pack tempest tests. we don't have ability for installing all dependencies, like tempest
18:47:32 <tosky> sreshetnyak: iirc there are no additional dependencies apart from tempest(.lib) itself
18:47:40 <SergeyLukjanov> esikachev honestly don't have specific thoughts about move
18:48:22 <tosky> sreshetnyak: and if you want to run the tests, you require tempest, so the dependencies come from tempest
18:48:32 <sreshetnyak> tosky: ah, ok. I don't know it
18:48:58 <tosky> sreshetnyak: what were talking about is this: http://docs.openstack.org/developer/tempest/plugin.html
18:49:27 <tosky> sreshetnyak: it's the same mechanism we applied to python-saharaclient tests currently in sahara (sahara/tests/tempest/scenario/data_processing)
18:50:39 <esikachev> any questions?
18:51:07 <SergeyLukjanov> I think it's now cleared out
18:51:16 <SergeyLukjanov> tosky esikachev thx folks for driving the sahara-tests
18:51:20 <SergeyLukjanov> #topic Newton summit topics
18:51:31 <SergeyLukjanov> so, I've created the etherpad for it
18:51:33 <SergeyLukjanov> #link https://etherpad.openstack.org/p/sahara-newton-summit
18:51:33 <elmiko> i added a bunch of stuff to the etherpad
18:51:41 <elmiko> just trying to suggest some things
18:51:45 <SergeyLukjanov> please, start adding discussion topics to it
18:53:52 <SergeyLukjanov> #topic Open discssion
18:53:55 <tmckay> I have a few discussion topics on deprecations, maybe
18:54:08 <SergeyLukjanov> tmckay dump any ideas
18:54:09 <tmckay> 1) depreate internal db for job binaries
18:54:19 <SergeyLukjanov> it's better to have more topics than less ;)
18:54:24 <SergeyLukjanov> tmckay ++
18:54:45 <elmiko> we actually had someone asking about that last night
18:54:48 <tmckay> 2) deprecate use_namespaces -- I've found 2 bugs with it already, maybe a 3rd (currently looking) and it seems very very fragile. problem is ....
18:55:05 <tmckay> with multi-tenant and many private networks that could overlap, is there another good solution?
18:55:14 <tmckay> like configure neutron to just handle it
18:55:53 <tmckay> so bug #1, frozen dict, it didn't work with spark/storm, #2 couldn't get routers, #3 on ubuntu right now I am dying on a pickle error
18:55:55 <openstack> bug 1 in Ubuntu Malaysia LoCo Team "Microsoft has a majority market share" [Critical,In progress] https://launchpad.net/bugs/1 - Assigned to MFauzilkamil Zainuddin (apogee)
18:56:05 <tmckay> and that's after going through the pain of setting up root-wrap
18:56:11 <tmckay> heh
18:56:16 <tmckay> thanks openstack
18:56:39 <tmckay> so, anyway, I'll add to the pad, but there's a summary
18:56:40 <elmiko> whoa, that was cool
18:56:53 <elmiko> bug #2
18:56:56 <tmckay> also, the old topic we always have -- make bug reporting better, and harvest job logs
18:57:08 <elmiko> sorry, just had to try
18:57:15 <tmckay> :) lol
18:57:53 <tmckay> elmiko, talking it over with someone today I realized that multiple private nets say on 192.168.x.y from a script to set up tenants would be ambiguous from the controller
18:57:58 <tmckay> hence the namespaces
18:58:07 <elmiko> that makes sense
18:58:22 <elmiko> i agree in general, though, we should have a session on deprecations
18:58:24 <tmckay> but there may be a way to shepherd around that, or at least make the stuff under the covers less complex / more robust
18:58:38 <elmiko> and everyone can bring their favorite thing to deprecate to that session ;)
18:58:53 <tmckay> hadoop .... just kidding :) spark ftw
18:58:57 <SergeyLukjanov> :)
18:59:02 <elmiko> lol!
18:59:07 <SergeyLukjanov> 1 min left
18:59:39 <tmckay> elmiko, PTL? deprecate SergeyLukjanov ;-)
18:59:51 <SergeyLukjanov> ;(
18:59:56 <huichun> PTL HA?
19:00:02 <tmckay> ++
19:00:09 <tmckay> SergeyLukjanov, you will be missed
19:00:10 <SergeyLukjanov> #endmeeting