13:44:17 <rosmaita> #startmeeting image import refactor
13:44:19 <openstack> Meeting started Fri Oct  7 13:44:17 2016 UTC and is due to finish in 60 minutes.  The chair is rosmaita. Information about MeetBot at http://wiki.debian.org/MeetBot.
13:44:19 * croelandt writes down "be sober"
13:44:20 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
13:44:22 <openstack> The meeting name has been set to 'image_import_refactor'
13:44:35 <croelandt> I was thinking about re-sending the discovery patch next week, not sure exactly what other patches we should take over right now
13:44:43 <Jokke_> so looking the patches proposed ... it's all the API stuff without any of the business logic we actually need
13:44:54 <rosmaita> well, my impression from tuesday was that we'd work inside-out
13:45:02 <croelandt> logic first, API then
13:45:06 <rosmaita> yeah
13:45:06 <Jokke_> yeah I would prefer that
13:45:10 <croelandt> it seems harder to test, but it makes sense
13:45:25 <rosmaita> well, i think it will take the most tiem to implement
13:45:30 <croelandt> indeed
13:45:35 <rosmaita> and we have some design decisions to make
13:45:44 <Jokke_> croelandt: not really as all the unit tests are valid from inside out. You really can't test the api if the logic is not there
13:45:45 <croelandt> I'm just saying that it's harder to make sure we have all the right pieces if we start by writing code that cannot be called
13:45:56 <croelandt> Jokke_: right
13:46:39 <Jokke_> So it has been agreed that we do touch this via the async
13:46:50 <croelandt> hm
13:46:54 <croelandt> not sure I follow
13:47:01 <rosmaita> Jokke_: you mean using tasks?
13:47:05 <rosmaita> (internally only?)
13:47:08 <Jokke_> I mean we do have the task engine that we are gonna utilize
13:47:19 <rosmaita> Jokke_: yes, that was the plan
13:47:33 <rosmaita> we want to wind up with something extensible
13:47:38 <Jokke_> the fact that the tasks api still exists makes the testing of those during development lots easier
13:48:34 <Jokke_> so I think we have majority of what's needed to interact with the glance_store in place but I can take the action to verify that
13:48:52 <Jokke_> cause that's kind of our bottom most part, how those bytes finally land to the store
13:49:09 <rosmaita> yes, if possible, the idea is that tasks will be used internally, but the external api will not expose them
13:49:18 <Jokke_> indeed
13:49:43 <Jokke_> but how the deprecated tasks api works it will expose them if you know how to poke it
13:49:50 <Jokke_> which is fine
13:50:33 <Jokke_> it makes the development easier and we have no need nor intention to document/support any of those tasks being accessed via the tasks api
13:51:17 <croelandt> ok
13:51:26 <rosmaita> one of the items stuart had identified was that the current tasks "know too much" about glance
13:51:41 <rosmaita> we may need to provide a more well-defined interface for operators
13:52:13 <rosmaita> becasue the idea is that they can write custom flows into taskflow to process the imported bits
13:53:02 <Jokke_> Well that's how it was designed ... Only thing we realistically can do is to build filter that permits any access to the tasks outside of the originally released ones, which is moronic as that would render any task someone has written and using via the deprecated api useless
13:53:48 <rosmaita> good point
13:53:51 <Jokke_> so what comes to that, we might need to try to figure out how the flows are compiled form individual tasks and where those tasks and flow descriptions are stored
13:54:01 <rosmaita> maybe i need to take an operator survey of who's using tasks and what for
13:54:14 <rosmaita> was going to do one about image sharing
13:54:24 <rosmaita> and image locations
13:54:29 <Jokke_> well it's deprecated, admin only by default api. I'd say we just don't touch it
13:55:11 <Jokke_> like tasks api should not be any concern anymore. as long as it keeps "working" when we poke the internals
13:55:58 <rosmaita> ok
13:56:18 <Jokke_> but I think extending/refactoring/what-so-ever around our task engine is the secondary goal after we have got the functionality of image import together
13:56:38 <rosmaita> so, i will admit up front that i'm not very familiar with taskflow
13:56:44 <croelandt> +1
13:56:44 <Jokke_> as long as we keep in mind that it's gonna happen and don't rely any crazy thing of it
13:56:47 <rosmaita> how about you, Jokke_ and croelandt ?
13:56:49 <croelandt> nor the task API for that matters
13:57:24 <rosmaita> and, yes +1 for functionality of import first,
13:57:26 <Jokke_> rosmaita: I did review lots of that code when Flavio implemented it but haven't really touched since so I need to take a look and refresh
13:57:39 <openstackgerrit> Darja Shakhray proposed openstack/glance_store: Replace is not correct debug message.  https://review.openstack.org/383745
13:59:56 <rosmaita> ok
14:00:15 <rosmaita> what's our next step, before tuesday?
14:01:14 <croelandt> well
14:01:32 <Jokke_> so very first step I'd like to see us having task that can upload the image data from staging area. Not care about the size constrains of it or anything like that, just taking Image ID, pulling those bits from the stage, uploading them to the store and updating the image
14:01:55 <croelandt> https://etherpad.openstack.org/p/glance-ocata-image-import-sync
14:02:05 <croelandt> let's put that in the etherpad once we agree on something
14:02:29 <rosmaita> croelandt: good point
14:03:12 <rosmaita> Jokke_: do we use the tasks work_dir as the staging area for now?
14:03:13 <Jokke_> then implementing the stage handling. So that our tasks have interface to store their working data to the stage and pick it up from there (this should be in there for current tasks already as well)
14:03:54 <Jokke_> rosmaita: I was thinking of dropping stage_uri to the config and start using that from the beginning
14:05:17 <Jokke_> so I don't want to call it path, because we might want to use glance_store for handling our staging at some point, or some other staging driver so I'd rather have it universal so we don't need to change it again in 2 cycles ;)
14:05:26 <rosmaita> that's an interesing idea ... so the stage could be file:/// or a swift container or anything that could be represented by a uri
14:06:18 <Jokke_> yeah; file, cinder volume, ceph cluster, <pick your piece of favorite cake>
14:06:31 <rosmaita> that sounds good
14:07:02 <Jokke_> as discussed on the initial design, we might not want to have staging tied to the node local filesystem for various reasons
14:07:26 <rosmaita> agreed
14:07:53 <Jokke_> and while for the initial iteration it's definitely is gonna be local FS path, we don't want to it to that for anything admin/user facing
14:08:52 <Jokke_> so lets try to pick any naming of options, etc so that we don't need to deprecate them in Pike :P
14:09:00 <rosmaita> ++
14:11:12 <Jokke_> Very first thing 'though
14:11:24 <Jokke_> What gerrit topic we start using for the patches
14:11:39 <Jokke_> lets make sure we have everything easily discovered
14:11:51 <rosmaita> yes
14:11:52 <croelandt> feature/image-import
14:12:04 <Jokke_> Sounds good to me
14:12:12 <rosmaita> i like it
14:12:35 <Jokke_> rosmaita: mind to throw # agreed for the meeting logs
14:12:37 <openstackgerrit> Merged openstack/glance: Imported Translations from Zanata  https://review.openstack.org/380053
14:12:59 <Jokke_> I'm not sure if the bot takes those only from chair
14:13:06 <rosmaita> #agreed use 'feature/image-import' as gerrit topic
14:14:21 <Jokke_> Should we take until Tue all of us just to get familiar with the current state of the taskflow and agree specific work items then?
14:14:29 <rosmaita> not sure that did anything
14:14:37 <Jokke_> something nice night reading for the weekend :P
14:14:51 <rosmaita> makes sense to me
14:14:51 <Jokke_> #help
14:15:10 <rosmaita> we could vote on it
14:15:29 <rosmaita> #startvote use 'feature/image-import' as the gerrit topic for this work
14:15:30 <openstack> Unable to parse vote topic and options.
14:15:36 <rosmaita> maybe not
14:15:50 <croelandt> oh come on, let's not vote
14:15:59 <rosmaita> #startvote use 'feature/image-import' as the gerrit topic for this work ? yes, no
14:16:00 <openstack> Begin voting on: use 'feature/image-import' as the gerrit topic for this work ? Valid vote options are yes, no.
14:16:01 <openstack> Vote using '#vote OPTION'. Only your last vote counts.
14:16:06 <croelandt> #vote yes
14:16:10 <rosmaita> #vote yes
14:16:16 <Jokke_> well at least that's the command for the bot, dunno if it's currently reacting or if it's just the chair
14:16:28 <Jokke_> #vote yes
14:16:33 <rosmaita> #endvote
14:16:34 <openstack> Voted on "use 'feature/image-import' as the gerrit topic for this work ?" Results are
14:16:35 <openstack> yes (3): croelandt, Jokke_, rosmaita
14:17:03 <rosmaita> well, it's recorded
14:17:08 <Jokke_> yeah
14:17:12 <croelandt> do we really need that? :p
14:17:26 <rosmaita> we want to be completely transparent :)
14:17:41 <Jokke_> croelandt: that's just if people don't obther to read the full meeting log, that will highlight it in the summary
14:17:51 <rosmaita> ok, i like erno's suggestion to read up on taskflow
14:18:02 <rosmaita> croelandt: how is your taskflow-fu ?
14:18:45 <croelandt> rosmaita: looked at taskflow once, wanted to die iirc
14:19:01 <openstackgerrit> Darren White proposed openstack/glance: Glance purge insecure warning  https://review.openstack.org/383757
14:19:33 <rosmaita> croelandt: that is a but unpromising
14:19:38 <rosmaita> *bit
14:19:44 <croelandt> yeah, kinda
14:20:05 <rosmaita> #action read up on taskflow (but get in a good mood first)
14:20:09 * Jokke_ starts preparing http://www.voodoo786.com/wp-content/uploads/2015/11/How-To-Make-A-Voodoo-Doll-For-Revenge.jpg
14:20:09 <croelandt> heheh
14:20:58 <rosmaita> ok, well that gives us something to do to keep up momentum
14:21:12 <rosmaita> anything else?
14:21:46 <Jokke_> yeah, I'll look into the first staging bits as well. I think as we have only 2 working days, it's pretty much all we can expect for now
14:22:36 <rosmaita> ok, sounds good
14:22:48 <rosmaita> hopefully, Jokke_  and croelandt were not netsplit?
14:22:57 <Jokke_> nope
14:23:00 <Jokke_> still here
14:23:17 <rosmaita> not sure about cyril
14:23:26 <rosmaita> well, i'll update the etherpad
14:23:33 <Jokke_> sure
14:24:14 <croelandt> re
14:24:33 <croelandt> yep, like 1.5 days to look into taskflow
14:24:56 <Jokke_> indeed so lets keep it on that
14:25:09 <rosmaita> not a lot of time really, so let's see how it goes
14:25:20 <Jokke_> And obviously we can adhoc in between if needed
14:25:21 <croelandt> next meeting on Tuesday, what time?
14:25:38 <Jokke_> I think that was rolling Tue 13:30 UTC
14:25:53 <rosmaita> just noticed that it doesn't say on the etherpad!
14:26:15 <rosmaita> think it was 14:00
14:26:31 <Jokke_> it's in the mailing list anyways
14:26:54 <rosmaita> that's true
14:27:27 <rosmaita> [openstack-dev] [glance] image import sync - Tuesdays 14:00 UTC
14:27:40 <rosmaita> ok, see you both on tuesday!
14:27:55 <rosmaita> #endmeeting