12:00:38 <david-lyle> #startmeeting horizondrivers
12:00:38 <openstack> Meeting started Wed Sep 23 12:00:38 2015 UTC and is due to finish in 60 minutes.  The chair is david-lyle. Information about MeetBot at http://wiki.debian.org/MeetBot.
12:00:39 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
12:00:41 <openstack> The meeting name has been set to 'horizondrivers'
12:00:55 <david-lyle> anyone around ?
12:01:08 <tsufiev> hi
12:02:17 <robcresswell> o/
12:03:20 <mrunge> o/
12:03:56 <david-lyle> that feels like the usual suspects
12:04:55 <mrunge> for reference, the agenda is: https://wiki.openstack.org/wiki/Meetings/HorizonDrivers#Agenda_for_September_23_1200_UTC
12:05:01 <david-lyle> Just want to start by thanking everyone for their attention to the FFEs and bugs for Liberty RC-1
12:05:27 <robcresswell> mrunge: You forgot the link! tut tut
12:05:39 <david-lyle> all the FFEs are merged or bumped and just a handful of bugs left
12:05:39 <mrunge> I did? where robcresswell
12:05:51 <robcresswell> with the agenda, the meetingbot command :)
12:06:00 <david-lyle> #link https://wiki.openstack.org/wiki/Meetings/HorizonDrivers#Agenda_for_September_23_1200_UTC
12:06:02 <mrunge> robcresswell, gotyou
12:06:28 <mrunge> #action mrunge to remember linking correctly
12:06:35 <mrunge> ;-)
12:06:36 <david-lyle> #topic https://blueprints.launchpad.net/horizon/+spec/horizon-glance-large-image-upload
12:07:38 <david-lyle> tsufiev: how does this related to the tasks API?
12:07:49 <tsufiev> so it was first hold off due to Travis' words, but then it became clear that he meant a different BP, authored by Janet Yu
12:08:24 <tsufiev> david-lyle, AFAIK, it is not related
12:08:43 <mrunge> tsufiev, I like that, it addresses a long standing issue
12:09:01 <mrunge> but: you need to take care of keystone tokens
12:09:19 <tsufiev> mrunge, yes, that's where I currently stopped
12:09:27 <mrunge> i.e. take care for the fact, that keystone tokens might time out during upload
12:09:34 <tsufiev> ah...
12:09:36 <david-lyle> https://wiki.openstack.org/wiki/Glance-tasks-import
12:09:50 <david-lyle> #link https://wiki.openstack.org/wiki/Glance-tasks-import
12:10:04 <doug-fish> not to get to far into the details, but how would the keystone timeout be addressed?
12:10:18 <tsufiev> david-lyle, thanks for the link, looks like some new API. I targeted the old one
12:10:22 <tsufiev> will read through
12:10:32 <mrunge> doug-fish, I have no idea. this is an issue for cli, too
12:10:37 <doug-fish> oh
12:10:43 <doug-fish> invention required
12:10:52 <david-lyle> tsufiev: the bps are from 2013
12:10:56 <tsufiev> doug-fish, keystone trusts?
12:11:13 <tsufiev> david-lyle, hm... Anyways, will read through :)
12:11:17 <doug-fish> tsufiev: not sure - I don't know enough about that to say
12:11:20 <mrunge> uploading a 20 gigs windows image over a dsl line takes maybe longer than keystone tokens live
12:11:33 <david-lyle> I believe this allows you to tell glance the source and not have to relay the data
12:11:46 <tsufiev> david-lyle, exactly
12:11:49 <david-lyle> new to me too
12:12:05 <amotoki> it seems glance craetes a task resource under a project, so we don't need to take care of token expiration.
12:12:06 <david-lyle> but might save you some work
12:12:17 <tsufiev> I think that taking care of token timeout should be done at Glance side
12:12:19 <robcresswell> "cleverly written JS code" - sounds fantastic
12:12:42 <tsufiev> robcresswell, don't say that to Angular team ;)
12:13:17 <robcresswell> tsufiev: Agreed on timeout, it is a client issue too, so should not be a concern of ours (but needs to be noted, as it may be a blocker)
12:13:52 <david-lyle> if the tasks API won't work you'll have to work on reissuing tokens
12:14:42 <tsufiev> david-lyle, I'll elaborate this issue
12:15:45 <david-lyle> I'd like to hold on this bp and let tsufiev look into glance tasks, the proposal to be CORS and clientside based seems fairly restrictive at this point
12:16:07 <tsufiev> david-lyle, sure, makes sense
12:16:45 <david-lyle> #info moved https://blueprints.launchpad.net/horizon/+spec/horizon-glance-large-image-upload to discussion
12:17:08 <david-lyle> #topic https://blueprints.launchpad.net/horizon/+spec/horizon-large-ldap-users-browsing
12:17:15 <tsufiev> so the next one is also mine
12:18:17 <tsufiev> there are some things blocking us at Keystone side, but first I'd like to evaluate how sane does the proposed workflow look to you at Horizon side
12:18:51 <doug-fish> it looks very reasonable to me
12:19:13 <tsufiev> Keystone limitations for LDAP force us to restrict the amount of table contents by filtering query (no query = no content)
12:19:42 <doug-fish> IMO having an unfiltered list with thousands of users seems very close to useless
12:20:00 <tsufiev> yeah, that's an exact thing we've been talking internally
12:20:14 <mrunge> doug-fish, esp. since AD gives you all kinds of errors with more than 500(?) users in resultset
12:20:16 <david-lyle> in horizon it is
12:20:23 <tsufiev> perhaps that pattern should be generalized for other Horizon tables?
12:20:34 <david-lyle> I wanted to index it for Searchlight
12:20:46 <mrunge> oh, pop corn!
12:20:49 <tsufiev> also I wanted to understand how well this workflow aligns with tables angularization
12:20:58 <robcresswell> tsufiev: Out of curiosity, what is the default display, an empty table?
12:20:59 <david-lyle> and get much faster queries
12:21:21 <tsufiev> david-lyle, there was a concern that nobody is going to allow SearchLight to index LDAP users due to security implications, corporate policies etc
12:21:22 <mrunge> tsufiev, I think it's a bit unexpected, if you don't get results, if no filter selected
12:21:33 <mrunge> yes, tsufiev exactly
12:21:43 <robcresswell> Yeah, you won't get much angular input at this meeting time unfortunately
12:21:47 <tsufiev> robcresswell, perhaps a table with a message 'Provide a query'?
12:21:50 <david-lyle> tsufiev: think is I may be able to indirectly
12:21:57 <doug-fish> david-lyle: would searchlight indexing (if you could do it) mean you'd want to show all of the users in an initial table?
12:22:00 <robcresswell> tsufiev: Seems sane to me.
12:22:24 <mrunge> pumping users from one database to another just to enable searching is braindead
12:22:47 <david-lyle> mrunge: not if you care about performance
12:22:51 <robcresswell> tsufiev: I suppose you'd need some call to see what number you're dealing with, then alter behaviour; if you had 100 users, you may as well just show them all
12:23:13 <mrunge> david-lyle, I agree, it's faster. but why don't you upload users then directly to memcached?
12:23:13 <david-lyle> doug-fish: I don't see a reason to list them all no
12:23:20 <tsufiev> robcresswell, yes, makes sense. I'll add this notion to the bp
12:23:48 <doug-fish> maybe the same idea applies of there are more than 100 objects in any table
12:23:56 <tsufiev> mrunge, 'One of the hardest programming problems is cache invalidation' :)
12:24:10 <mrunge> tsufiev, sure. applies to searchlight, too
12:24:42 <david-lyle> mrunge: definitely applies to searchlight, and ldap doesn't post events to the bus on update
12:25:04 <doug-fish> OTOH the list of users is probably reasonably static
12:25:28 <david-lyle> well let's ignore searchlight
12:25:34 <mrunge> so, the main issue is, ldap doesn't provide the required api we'd need
12:25:47 <david-lyle> I'm just stating why I wanted the exhaustive list
12:25:53 <tsufiev> do I understand correctly that a) the proposed workflow seems reasonable to most of us here b) need to ask Angular folks how well is it aligned with table angularization
12:26:14 <david-lyle> tsufiev: yes
12:26:41 <tsufiev> okay, then I'm going to start the next round of convincing Keystone community which seems to be hardest battle ever
12:26:43 <tsufiev> :/
12:26:43 <david-lyle> and that should work with the magicsearch implementation going into the angularized user's table
12:27:06 <doug-fish> tsufiev: what are the needs from keystone?
12:27:09 <tsufiev> david-lyle,  filtering + limit number of pages?
12:27:10 <david-lyle> tsufiev: with you proposal, what do you have to convince them of?
12:27:12 <doug-fish> it doesn't support filters?
12:28:18 <tsufiev> david-lyle, doug-fish: I need to convince them that pagination for LDAP could be supported solely at Keystone side, by taking the whole chunk of users from LDAP and throwing away the ones not needed according to API request parameters
12:28:42 <mrunge> tsufiev, how does that work with AD backend?
12:28:50 <mrunge> I mean, it doesn't
12:28:59 <tsufiev> given that we limit the size of this chunk thanks to filters, the overhead should be smaller than it was seen before
12:29:03 <mrunge> and I'm a bit worried with performance
12:29:50 <david-lyle> tsufiev: I thought your proposal was: don't show data unless filtered, if too much data, show initial results and instruct to filter more
12:29:51 <mrunge> tsufiev, it looks like you're trying to solve an unsolvable problem
12:30:09 <tsufiev> david-lyle, yes, exactly
12:30:18 <doug-fish> mrunge: isn't AD pretty much the same as LDAP? your question confuses me
12:30:21 <tsufiev> mrunge, it's the only kind of problems that's worth solving
12:30:21 <david-lyle> that should work as is
12:30:44 <mrunge> doug-fish, AD returns smaller result sets, like 100 users at max
12:30:56 <mrunge> where ldap gives you 1000 (?)
12:31:18 <tsufiev> david-lyle, if too much data, either show initial results, or show no results at all - still have to decide
12:31:18 <doug-fish> oh I see
12:31:32 <mrunge> I mean, 100, 1000, that's just numbers
12:31:36 <tsufiev> the final choice is affected both by UX considerations and Keystone/LDAP limitations
12:32:19 <doug-fish> Maybe it's wise to get UX team input on this? (which would obviously take time)
12:32:25 <tsufiev> mrunge, I'm starting to understand Keystone folks' reluctance to give it (cause LDAP implementations are quite different)
12:32:25 <robcresswell> tsufiev: What does keystone currently do if your query is too general? Return initial 500, or just return 0?
12:33:15 <tsufiev> robcresswell, I'd vote for first option, but we may make this configurable
12:33:32 <tsufiev> doug-fish, yes, I'll ask Piet about that
12:33:50 <robcresswell> tsufiev: No I mean, what is the *current* behaviour
12:34:05 <robcresswell> I'm getting confused over how keystone currently handles requests of this nmature
12:34:08 <robcresswell> nature*
12:34:47 <david-lyle> I think we target current keystone ldap behavior, whatever it is. question is if that differs from a DB backed keystone
12:34:54 <david-lyle> and if we change behavior then
12:34:55 <tsufiev> robcresswell, it just handles all the results to Horizon, and if there are too many Horizon is choked
12:35:08 <robcresswell> tsufiev: I see
12:35:57 <tsufiev> david-lyle, there is no pagination capabilities in Keystone v3 and that's a problem needed to be solved first
12:35:58 <amotoki> no upper limit on the number of results returned from keystone?
12:36:12 <tsufiev> (since most likely we're going to use v3)
12:36:55 <david-lyle> tsufiev: I know no pagination in v3, my question is if ldap returns first 500 and error like the DB request would
12:37:19 <david-lyle> if your API result limit is 500
12:38:58 <tsufiev> david-lyle, we're currently investigating its behavior on scale
12:39:42 <amotoki> as an operator perspective, we cannot handle more than 100 users without filtering (e.g. project, role, last updated date, e-mail address or and so on). user name is less helpful unless we no exact user name.
12:40:15 <tsufiev> amotoki, so the point here is that we need faceted search at Identity->Users?
12:40:15 <amotoki> i think this is similar to what keystone folks replied in the dev ML.
12:40:45 <pkarikh> david-lyle: hi! I'm working on it. I've found few Keystone issues and haven't got feedback about it from keystone team. Looks like sometimes keystone just becomes too slow and Horizon breaks with request time out
12:40:49 <amotoki> tsufiev: one possible option I think
12:41:46 <david-lyle> parameterized search
12:41:50 <tsufiev> amotoki, IIUC, that's how filtering is implemented on Instances page, but in a generalized form
12:42:17 <david-lyle> selecting the filters in the search bar
12:42:27 <amotoki> understood now.
12:42:53 <amotoki> tsufiev: agree. this is what I think.
12:43:08 <david-lyle> tsufiev: I think everyone agrees in principle, just need a few more details ironed out
12:43:43 <tsufiev> david-lyle, okay, we're continuing our study then :)
12:44:03 <mrunge> thank you tsufiev
12:44:44 <david-lyle> #info https://blueprints.launchpad.net/horizon/+spec/horizon-large-ldap-users-browsing moved to drafting and targeting Mitaka
12:44:53 <tsufiev> so the summary is a). Contact UX people b). Compare DB storage response and LDAP storage response on large chunks of return data c). Continue working out Keystone API changes
12:44:59 <david-lyle> btw the Mitaka target shows up now :)
12:45:35 <tsufiev> did I miss anything?
12:45:55 <tsufiev> ah, and d) Prefer faceted search for limiting table contents
12:46:16 <mrunge> would be great, if we don't need to care about keystone backend
12:46:28 <mrunge> but maybe that's too unrealistic
12:46:29 <tsufiev> I'm afraid we'll have to :)
12:46:57 <mrunge> why do we have keystone in this case?
12:47:07 * tsufiev shrugs
12:47:13 <david-lyle> projects and roles
12:47:46 <doug-fish> and we don't want AD/LDAP differences to be OUR problem!!
12:48:01 <mrunge> yes doug-fish !
12:49:20 <david-lyle> any other bps to discuss today?
12:50:23 <robcresswell> I unfortunately didn't have time to prepare a list this week
12:50:27 <robcresswell> sorry :(
12:51:18 <david-lyle> no worries
12:52:26 <mrunge> a quick one? https://blueprints.launchpad.net/horizon/+spec/node-management-ui
12:52:39 <mrunge> I believe, that's outdated and nobody is working on that
12:52:54 <david-lyle> #topic https://blueprints.launchpad.net/horizon/+spec/node-management-ui
12:53:31 <mrunge> proposed patches are abandoned
12:54:53 <david-lyle> I think the baremetal driver provides what horizon needs in this case to just use the instances panel
12:55:01 <robcresswell> Yeah, agreed I think mrunge
12:55:04 <david-lyle> *nova driver
12:55:35 <david-lyle> #info https://blueprints.launchpad.net/horizon/+spec/node-management-ui marked obsolete
12:57:04 <david-lyle> https://blueprints.launchpad.net/horizon/+spec/login-workflow-restructure is one I think we should just drop
12:57:29 <david-lyle> amotoki: hence our cleanup effort
12:57:36 <robcresswell> david-lyle: Agreed
12:57:36 <david-lyle> long ways to go
12:57:39 <amotoki> agree
12:57:40 <mrunge> agreed david-lyle
12:58:03 <david-lyle> #info https://blueprints.launchpad.net/horizon/+spec/login-workflow-restructure marked obsolete
12:58:58 <mrunge> a last one? https://blueprints.launchpad.net/horizon/+spec/add-container-only-management-panel-for-object-storage
12:59:22 <robcresswell> Lol, wow. Yep
12:59:29 <mrunge> still drafting, no movement since 2013
12:59:43 <tsufiev> that's a call for UX prototyping
12:59:50 <doug-fish> :-) drop it!
13:00:19 <david-lyle> The need is real, but no one is looking at container related work, no matter how bad the existing UI is
13:00:25 <david-lyle> I'm ok with dropping
13:00:26 <mrunge> if somehow overlaps with the bug zigo filed yesterday: horizon doesn't work without nova installed
13:00:53 <mrunge> dansmith, yes, I agree. there are folks installing only swift, nothing else
13:00:53 <tsufiev> mrunge, you mean the people who just install Horizon for Swift?
13:00:55 <zigo> mrunge: It does, just in *one* screen it doesn't.
13:00:59 <doug-fish> mrunge: do you happen to have the link for that bug?
13:01:08 <mrunge> sorry dansmith , I meand david-lyle
13:01:10 <doug-fish> I might have an out-of-date patch to address it
13:01:14 <zigo> Everywhere else, there's just an error message.
13:01:27 <mrunge> zigo, do you have that bug handy?
13:01:40 <mrunge> tsufiev, yes, exactly.
13:01:55 <tsufiev> I think the only hard requirement for Horizon is Keystone
13:02:03 <david-lyle> #info https://blueprints.launchpad.net/horizon/+spec/add-container-only-management-panel-for-object-storage marked obsolete, still very much needed, but more design work necessary
13:02:12 <mrunge> tsufiev, for swift, you don't need keystone.
13:02:35 <tsufiev> mrunge, you've just ruined my picture of the world :o
13:02:47 <david-lyle> time's up. thanks everyone
13:02:49 <mrunge> and if you use swift with horizon, your users need to have a special hardcoded role
13:02:57 <mrunge> tsufiev, ^^
13:02:57 <david-lyle> #endmeeting