14:01:17 <tosky> #startmeeting sahara
14:01:18 <openstack> Meeting started Thu Feb  1 14:01:17 2018 UTC and is due to finish in 60 minutes.  The chair is tosky. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:01:19 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:01:21 <tosky> #chair tellesnobrega
14:01:22 <openstack> The meeting name has been set to 'sahara'
14:01:28 <openstack> Current chairs: tellesnobrega tosky
14:01:37 <tellesnobrega> o/ thanks tosky
14:01:47 <jeremyfreudberg> o/
14:01:52 <shuyingya> o/
14:02:02 <tosky> #topic News/Updated
14:02:59 <tosky> we are in the process of releasing queens-3
14:03:12 <tellesnobrega> I'm trying to fulfill my goal to clean up bug list
14:03:35 <tosky> (with some delays due to zuul flackiness, but that's a shared problem for all openstack)
14:04:40 <tellesnobrega> also doing some downstream work
14:04:56 <jeremyfreudberg> past few days i've not been doing much - just orbiting two bugs at the moment: s3_hadoop fixes and the dashboard thing
14:04:56 <tosky> I'm personally still fighting with ambari, and babysitting the zuulv3 jobs (the ones for sahara-tests and python-saharaclient can be merged when a openstacksdk regression is fixed)
14:05:57 <shuyingya> some patches to fix oozie sharelib dependency issue on Monday. quite busy with downstream stuff. also help a colleague to finish keypair inject task
14:06:25 <tellesnobrega> shuyingya, that patch looks good, I was reviewing it earlier today, will test it out
14:06:43 <shuyingya> thanks tellesnobrega.
14:06:49 <tosky> talking about oozie, I have few questions and a note - can I briefly switch topic, or do we have other more important topics?
14:06:53 <jeremyfreudberg> yes, i need to review that patch, i do have some comments to give, but it mostly looks good
14:07:10 <jeremyfreudberg> about shuyingya's patch i mean
14:07:33 <jeremyfreudberg> s/shuyingya/shuyingya colleague
14:07:48 <jeremyfreudberg> tosky, you can talk about oozie if you want
14:07:54 <tosky> #topic oozie and artifacts
14:08:00 <shuyingya> tosky, I saw your comment. but didn't get the two patch together. I am ok to put them together
14:08:09 <tosky> shuyingya: yes, that was my question :)
14:08:31 <tosky> if they are so connected (i.e. the first one does not work without the second), it makes sense IMHO to merge them
14:08:42 <tosky> so that we don't commit something that it does not work
14:09:09 <tosky> apart from that, if the job passes, I'm fine with merging it (and backporting it, as the queens branch was created
14:09:30 <jeremyfreudberg> yes, i'd agree that we need the right amount of atomicity/granularity of commits
14:09:55 <shuyingya> ok. I will move them togrther right now
14:09:59 <jeremyfreudberg> i am also slightly confused about how it's hadoop-openstack dependenc, but it needs to be added to the oozie build
14:10:42 <tosky> apart from that, I have some updates on the artifacts; I wrote something on the channel but better also have it in the meeting minutes
14:11:20 <jeremyfreudberg> shuyingya, do you have some insight on my question, before we move on?
14:11:21 <shuyingya> And I can post another new patch to sahara-image-elements to build the 2.8.2 image.
14:12:02 <shuyingya> jeremyfreudberg typing
14:12:12 <jeremyfreudberg> shuyingya, no problem, take your time :)
14:14:08 <shuyingya> let's move on. I will answer jeremyfreudberg later
14:14:23 <tosky> oki, so briefly: before the switch to zuulv3, and the automatic conversion of the zuulv2/jjb jobs, the artifacts were published to tarballs.openstack.org/sahara
14:14:38 <shuyingya> I need test something to clarify  some filename
14:15:09 <jeremyfreudberg> shuyingya: no problem
14:15:11 <jeremyfreudberg> tosky: continue
14:15:34 <tosky> now the jobs (legacy, but it would be probably the same with native jobs) publish to tarballs.openstack.org/sahara-extra
14:16:49 <tosky> I asked infra people (namely the poor Monty who I'm nagging a lot lately) how to proceed: whether have a way to publish to the old location, or to add a symlink, or to change the URL where we use it
14:17:09 <tosky> because, of course, it's a new special case, unmet before :)
14:17:19 <tellesnobrega> we are full of those :)
14:17:28 <tosky> so that's the story - I'm waiting for this to complete the removal of sahara-files
14:17:37 <tosky> end of the story
14:17:52 <jeremyfreudberg> good story
14:18:13 <jeremyfreudberg> i think we all agree that having the old url would be great
14:18:14 <tellesnobrega> I'm not sure monty saw my suggestion, but it makes sense to me (allow to publish if the base name is the same)
14:19:06 <tosky> let's see what will be the result, but it's moving at least
14:19:11 <tellesnobrega> cool
14:19:13 <tosky> anything else on the topic?
14:19:17 <tellesnobrega> not from me
14:19:21 <jeremyfreudberg> no
14:20:42 <tosky> ok, I was going for the PTG, or do you have other suggestions?
14:21:01 <jeremyfreudberg> we can talk about that next, sure
14:21:05 <tellesnobrega> ptg should be fine
14:21:14 <tosky> #topic PTG
14:22:30 <tellesnobrega> about the doc stuff, the last meeting we agreed that we should have a pre-PTG doc day
14:22:48 <tosky> makes sense
14:22:55 <tellesnobrega> iirc monday and wednesday are the best days for jeremyfreudberg, does that work for you tosky?
14:23:09 <tellesnobrega> if so we can schedule for next Wednesday
14:23:24 <jeremyfreudberg> sounds good
14:23:41 <tosky> whole day? Sure
14:24:00 <tellesnobrega> we can start early and see how far we need to go
14:24:38 <shuyingya> I received a message today that foundation didn't approve my tsp. so I have to attend PTG remotely again.
14:24:59 <jeremyfreudberg> shuyingya, sorry to hear that
14:25:01 <tellesnobrega> the idea is to fix simple stuff and have a list of stuff that we need to discuss at PTG
14:25:09 <tellesnobrega> shuyingya, that is too bad :(
14:25:49 <shuyingya> no worry. I will try to talk with you in channel or etherpad :)
14:27:56 <tellesnobrega> anything else on the topic?
14:28:03 <tosky> apart from doc day, thanks to a lot of brave souls that updated the rocky etherpad, it's definitely a list of work items now
14:28:13 <tellesnobrega> it does
14:28:23 <tellesnobrega> I believe that brave soul tracks back to jeremyfreudberg
14:28:30 <tosky> better remind the link
14:28:30 <jeremyfreudberg> correct
14:28:31 <tosky> #info https://etherpad.openstack.org/p/sahara-rocky-ptg
14:28:42 <tellesnobrega> thanks jeremyfreudberg
14:28:54 <jeremyfreudberg> i think the etherpad is full enough now, once again it's a packed schedule (but manageable)
14:29:04 <tellesnobrega> we will do our best
14:29:21 <shuyingya> I read it today, really good. I may fill etherpad next week.
14:29:30 <shuyingya> thanks jeremyfreudberg
14:29:43 <jeremyfreudberg> shuyingya: np. look forward to your suggestions too
14:31:25 <tosky> any other topic, or open discussion?
14:31:54 <tellesnobrega> open discussion and we can talk about the point release of client that jeremyfreudberg talked about
14:32:00 <jeremyfreudberg> right
14:32:13 <jeremyfreudberg> so, tosky can go ahead and change the topic
14:32:23 <tosky> #topic Open Discussion
14:32:50 <tellesnobrega> the floor is yours jeremyfreudberg
14:32:53 <jeremyfreudberg> so, for context, this is about https://review.openstack.org/#/c/536683/ , proposed patch to sahara-dashboard
14:33:37 <jeremyfreudberg> you'll recall that because apiv2 really exists now, and because of how version discovery works, we are recommending new deployments to put an unversioned sahara endpoint in the service catalog
14:33:48 <jeremyfreudberg> that poses a tiny problem for the dashboard
14:34:21 <jeremyfreudberg> namely, that the way horizon works, is that it gives you some token to authenticate your saharaclient with
14:34:46 <jeremyfreudberg> our problem is that saharaclient in its current state can take this token input, but for historical reasons only accepts token input with hardcoded sahara endpoint
14:34:58 <jeremyfreudberg> in other words, it doesn't let token input and version discovery mix
14:35:17 <jeremyfreudberg> so if the user has unversioned endpoint in the catalog, and no changes to sahara-dashboard, all will fail with 404
14:35:31 <jeremyfreudberg> hope that made sense
14:35:36 <jeremyfreudberg> there's two-ish possible solutions:
14:35:38 <jeremyfreudberg> 1)
14:36:11 <jeremyfreudberg> fix on the dashboard side, by providing a session object to saharaclient instead, a known and normal case which allows version discovery to work
14:36:32 <jeremyfreudberg> 2) fix on the saharadashboard side, by allowing direct token input not to require hardcoded endpoint
14:36:38 <jeremyfreudberg> s/saharadashboard/saharaclient
14:37:13 <jeremyfreudberg> both 1 and 2 are equally valid, but with their own set of minor issues
14:37:55 <jeremyfreudberg> fixing on dashboard side, the problem is transient dependency on keystoneauth (in horizon requirements but not in s-d requirements), and also that we probably want to make this fix for all users, not just dashboard things
14:38:27 <jeremyfreudberg> fixing on the saharaclient side, the problem is that we need a new release, and a minimum bump to requirements, and there's potentially red tape there
14:38:57 <jeremyfreudberg> note that when we released saharaclient 1.5.0 last week i specifically said not to bother with the bump
14:39:05 <jeremyfreudberg> and now with 1.5.1 looming i am changing my mind
14:39:08 <tellesnobrega> jeremyfreudberg, I like the client fix better, but we can talk with release people and see if this is a big no no
14:39:35 <jeremyfreudberg> option 2A is fix it on dashboard side, but do it carefully and temporrarily
14:40:08 <tosky> given the experimental status of v2, in order to avoid a respin of the stable client and a dependency bump, in the worst case we could simply go for 2) in queens and then 1) in rocky
14:40:09 <jeremyfreudberg> that is to say, with big TODOs and FIXME comments, and try/except on the import of keystoneauth just in case something happens and someone magically runs sahara-dashboard separately from horizon
14:40:32 <tosky> I don't think that there is even a use case of running any dashboard outside horizon
14:40:38 <tosky> they all relies on core horizon classes
14:41:28 <jeremyfreudberg> tosky, true
14:42:16 <tellesnobrega> I like tosky's idea, but lets make it a priority to fix it asap in rocky
14:42:19 <tellesnobrega> so we don't forget
14:42:30 <jeremyfreudberg> i can prepare patches both for client and dashboard, but not rush to release the client, just put some todo in the dashboard to remove unnecessary code when client bump
14:42:59 <tellesnobrega> that works nicely
14:43:51 <tosky> also we need to be careful: the patches would go to sahara-dashboard master; if we want to backport them, we need to manually test the dashboard properly
14:44:33 <jeremyfreudberg> tosky: the amount of people running pike sahara-dashboard against a new deployment with unversioned endpoint in catalog 'should' be zero
14:44:58 <tosky> backporting to queens
14:44:59 <tosky> :)
14:45:10 <jeremyfreudberg> ugh, right, it's already that time
14:45:20 <tellesnobrega> yes
14:45:44 <jeremyfreudberg> worth noting that this fix should be in before i update the docs and such to recommend unversioned endpoint
14:46:08 <jeremyfreudberg> not sure when i want to land that doc patch, actually
14:46:16 <jeremyfreudberg> we'll talk about that some other time, though
14:46:31 <tellesnobrega> sure
14:47:24 <jeremyfreudberg> well, it seems like we are all board with what i've said, that's good, usually when i have the floor during the meeting it's for transparency's sake and so you know what the heck i'm actually doing, so good
14:47:49 <tellesnobrega> :)
14:48:02 <tellesnobrega> thanks for the work there jeremyfreudberg
14:48:09 <jeremyfreudberg> np
14:48:31 <tellesnobrega> anything else to discuss?
14:49:16 <tosky> apparently not
14:49:26 <jeremyfreudberg> that's it from me
14:49:32 <tellesnobrega> not from me either
14:49:39 <tellesnobrega> we all get 10 minutes back
14:49:40 * tosky looks at shuyingya
14:50:29 <shuyingya> no special from my side
14:50:47 <shuyingya> I just wanna answer jeremyfreudberg's question
14:51:04 <jeremyfreudberg> sure - we can do that now, if you want
14:52:23 <shuyingya> I am not sure you still remember we talked about the httpclient ClassNotFound before, when we execute the oozie command, it need a workflow.xml, and it has some properties to specify the input dir and output dir
14:52:47 <jeremyfreudberg> yes
14:52:50 <shuyingya> If we use swift to store data or j-binary
14:53:39 <shuyingya> I think oozie will use hadoop-openstack to get data from swift
14:55:27 <shuyingya> we should provide the depended package in sharelib for oozie.
14:56:45 <jeremyfreudberg> but my tricky question, wouldn't the dependent package already be inside hadoop-openstack.jar?
14:57:55 <jeremyfreudberg> or i might be wrong
14:58:10 <shuyingya> wow, let me check :) I didn't realize it
14:58:37 <jeremyfreudberg> your fix might be needed, i'm not sure if oozie can find hadoop-opentack.jar in the classpath
14:58:45 <jeremyfreudberg> it is a tricky thing, for sure
14:58:58 <jeremyfreudberg> i just wanted to point out my concern so that we are certain about the fix
14:59:31 <jeremyfreudberg> 30 seconds left in the meeting, i don't think will resolve it instantly
14:59:37 <shuyingya> yes
14:59:45 <tellesnobrega> #action shuyingya to make sure the fix is needed and updates us all
14:59:54 <shuyingya> will do
14:59:58 <tellesnobrega> thanks
14:59:58 <jeremyfreudberg> thanks
15:00:21 <tellesnobrega> see you all next week, on Wednesday for our pre-PTG doc day
15:00:33 <tellesnobrega> I will send out an email
15:00:49 <tellesnobrega> thanks tosky for the help running the meeting
15:01:07 <tosky> o/
15:01:10 <tosky> #endmeeting