19:00:21 <briancurtin> #startmeeting python-openstacksdk
19:00:22 <openstack> Meeting started Tue Feb  3 19:00:21 2015 UTC and is due to finish in 60 minutes.  The chair is briancurtin. Information about MeetBot at http://wiki.debian.org/MeetBot.
19:00:23 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
19:00:25 <openstack> The meeting name has been set to 'python_openstacksdk'
19:00:36 <briancurtin> if you're here for the SDK meeting, say hi
19:00:38 <briancurtin> Brian Curtin
19:01:09 <terrylhowe> Terry Howe, HP
19:01:11 <etoews> o/
19:02:11 <briancurtin> #topic Conversations at Cinder meetup, Cinder v2 https://review.openstack.org/#/c/150979/
19:03:05 <briancurtin> so last week i was at the cinder mid-cycle meetup for the day and mike perez gave me the floor for a while. all in all it was an hour-long talk on where we're at, what we do, etc. i think of the ~25 people in the room we had active participation from 12 people
19:03:36 <etoews> what's mike's nick?
19:03:46 <briancurtin> thingee
19:04:18 <stevelle> is there an etherpad of notes from that session?
19:04:18 <briancurtin> it was all really good. a handful had heard about the project already, but after explaining the goals and the status they were really interested. in teh end i had a few people interested in contributing and about an hour after the talk i had a guy submit a simple proxy that could list volumes (a dupe of my own work, but good that he just started hacking
19:04:18 <briancurtin> right away)
19:05:00 <briancurtin> stevelle: i dont think anyone took notes on that particular part as it was less about their roadmap for kilo and just getting them to think about it, but i'll look
19:05:52 <briancurtin> anyway, once i have the proxy part of that cinder.v2 part done, which i just got started, im going to bring in walt boring and john griffith to take a look at it. after that im going to hack out the minimal compute proxy part to integrate the two for a sample app that can show attaching volumes or something
19:06:42 <terrylhowe> sounds like a good sample ap
19:07:23 <stevelle> will need to incorporate a bit of glance for the boot image I presume?
19:08:08 <etoews> a bit future looking but how should we go about getting the project teams to take ownership of development in the sdk?
19:08:10 <briancurtin> stevelle: probably. i dont actually know all of the moving parts in play there, but i also want glance for another thing, so that's also on the list
19:08:42 <etoews> while bootstraping the project we'll need to do a fair amount ourselves but that's not sustainable.
19:09:05 <etoews> i'm fine tabling that topic. just something to think about.
19:09:34 <briancurtin> etoews: not sure, talked about it briefly with cinder and their thought was just having one or two people interested in the project to spend some time looking at it and get to know the cinder parts, then see where it goes
19:10:20 <etoews> ya. and i think at some point soonish we can start being more vocal about it on the openstack-dev ml.
19:10:29 <stevelle> for glance we should have enough if we get the the v2 image proxy merged (https://review.openstack.org/#/c/147314/)
19:10:50 <etoews> "hi all, support for glance has been added, next week is cinder." etc.
19:11:44 <briancurtin> stevelle: sort of tangent but i think we should have built-out proxies like object_store that can do more than be session managers, but that's a start
19:11:49 <briancurtin> #topic Thoughts on doc work in progress? http://briancurtin.com/doc_reorg/
19:12:34 <briancurtin> part of that doc work was driven by etoews starting to work on the project, but also as other projects get involved
19:13:06 <etoews> so far it's been great
19:13:12 <etoews> i +1'd the reorg
19:13:16 <briancurtin> i dont want it to be one huge change so i think im going to add another guide on building out a proxy, and keep expanding user-guide stuff. ideally teh whole thing acts like a cookbook
19:13:25 <etoews> and i've literally been working through the contrib docs
19:13:35 <etoews> i can provide a more complete review soon
19:13:59 <etoews> haven't touched the user stuff yet
19:14:02 <briancurtin> cool, i think you're the most likely to have in depth review since you're actually going through it. anyone else?
19:14:04 <stevelle> +1 for not expanding the current change
19:14:12 <stevelle> easier to review in parts
19:14:40 <stevelle> I have it queued for review tomorrow
19:14:47 <terrylhowe> the contributor docs I went over so far looked good, but I need more time to read it all
19:15:29 <briancurtin> #topic Resource.list consistency (pagination disabled, or not supported but still returning lists) https://review.openstack.org/#/c/147686/
19:16:00 <etoews> wait. one more thing on the docs
19:16:05 <briancurtin> go for it
19:16:31 <etoews> does gerrit spit out a rendered version of each patch set?
19:16:49 <etoews> i know it can do this for docs.openstack.org
19:17:00 <briancurtin> i guess we could make it do that, but it doesn't currently
19:18:14 <briancurtin> i dont know if we get that luxury being a stackforge project or what the deal is behind that. currently the official docs are hosted at http://python-openstacksdk.readthedocs.org/en/latest/
19:18:36 <etoews> maybe annegentle knows...
19:18:36 <briancurtin> i just claimed that name and point them at github master
19:19:28 <stevelle> I'm just running sphinx to review formatting at this point
19:19:48 <terrylhowe> me too
19:19:51 <etoews> python setup.py build_sphinx
19:19:54 <etoews> correct?
19:19:58 <briancurtin> i'll look around and see what the deal is. we do have a doc builder so they are being built per changeset
19:20:05 <briancurtin> etoews: i just do "tox -e docs"
19:20:06 <stevelle> etoews: yes
19:20:20 <etoews> heh. 2 different answers.
19:20:25 <stevelle> both work
19:20:33 <etoews> do they do the same thing?
19:20:42 <briancurtin> yeah, tox runs that
19:20:56 <briancurtin> so if you're in a proper virtualenv you're just doing the manual version
19:21:33 <etoews> alright. i'll do that for now.
19:21:37 <etoews> let's move on
19:22:04 <briancurtin> Britt isn't here, but I'll see if I can get him to run that again. now that the page/list changes are in, that change lets us Resource.list anything that is a list, whether or not it supports pagination, so it's a consistent action across the board
19:22:53 <briancurtin> however, one thing it loses is an optimization i was able to put in where it does an early exit when you get back fewer items than your requested limit. i dont know how often people would do that, but that's the only downside of the change i saw
19:24:15 <briancurtin> we can still make that happen, but it'll require making things a bit more complex. in the interests of actually making Resource.list work i just ripped that stuff out. adding it back in to work would require an extra special case on top of the special casing of this being the first response
19:24:38 <briancurtin> so it's possible but maybe not worth it to add it back in
19:26:16 <terrylhowe> the last_response == response seems like it could be expensive on large pages
19:28:37 <briancurtin> terrylhowe: yeah that's one thing i want to check out. i have a container in rackspace cloud files with ~6k objects that would show how this turns out. i'll bring that auth change over and try it out and see what it does
19:29:18 <briancurtin> terrylhowe: the change is unfortunately quite naive in that it works against devstack, which includes trivially small amounts of data to test against
19:30:00 <etoews> it worked in devstack
19:30:24 <briancurtin> terrylhowe: actually, now that i think about this, we might have to have services override list because this might just be too general
19:30:30 <terrylhowe> worked not works :)
19:31:50 <briancurtin> the whole thing started because network supports pagination but disables it (and you can't know discover that it's disabled), and my optimization thing came from the way object_store works, so they don't work together well and both lose
19:32:24 <briancurtin> i just workflow-1
19:32:33 <terrylhowe> yeh briancurtin too much generalization might just get confusing
19:32:45 <briancurtin> will take it back to the drawing board and maybe only apply that type of thing where really really needed
19:33:47 <terrylhowe> I was thinking at one time about having something like “service preferences” to help with that kind of thing
19:34:32 <etoews> have to ask, do you know the rationale behind making pagination disableable in neutron?
19:34:33 <briancurtin> yeah i remember we talked about that, but i think it would then require that we have pagination turned off unless you say you want it, which might be too heavy handed
19:35:29 <briancurtin> etoews: no clue, i just know in the docs it says it accepts it but is configurable to be off. it seems to be off in devstack since i've requested pages of something with a limit of 1 and just got back all 4 things i had (i think i was trying to list /networks)
19:35:39 <terrylhowe> everyone should support pagination, but we have to deal with some services not like it or not I think
19:36:55 <etoews> i'm fine with services either having it or not having it but the idea of having it but making it disableable is bizarre to me.
19:36:59 <briancurtin> terrylhowe: this might be something we can work out in the proxy level. if object_store supports pagination we can do the iterator containers() call through list. if network /networks doesn't support pagination, we can still call it networks() in the proxy levl but just call get() under the hood, which still gives a list
19:37:39 <etoews> get gives a list?
19:38:24 <etoews> oh. get() pretty much equals GET right?
19:38:27 <briancurtin> yeah
19:39:12 <terrylhowe> well, get() has an id and the GET in list does not
19:39:36 <etoews> this is on Resource right?
19:40:04 <briancurtin> we can still call Resource.get() with no ID and have the full list of /networks returned to us. or at least that's what i think i was doing
19:40:48 <briancurtin> that's why Resource.list() on /networks is an infinite loop, since it calls GET with no ID
19:41:44 <etoews> i think i just need to get more familiar with all of this. i'll ask better questions then.
19:41:59 <terrylhowe> briancurtin: proxy level or resource level.  Seems like resource level would be better.  there are only 2 resources in object store
19:43:46 <briancurtin> terrylhowe: i think proxy. let resources do what they need to do. teh network resource works fine with just a get(), but no list(), but its get() does return a list object, so on top of that we can make it function the same as containers.list() which under the hood can make use of the list() pagination
19:44:25 <briancurtin> terrylhowe: if you run the examples.get against /networks you receive the same as you would hope from doing examples.list against /networks
19:45:40 <briancurtin> in a network proxy we would just need to have our own loop around the body we get back
19:45:58 <briancurtin> maybe i'll toy with that and see if it looks any better or more usable
19:47:37 <terrylhowe> I’d probably need to goof with it a little to form any real opinion on the best approach
19:48:45 <briancurtin> it's -1'ed for now so we can slow down and do it right. i'll talk to britt, who reported it, and get him an example of the thing i just said and see how that works
19:49:26 <briancurtin> #topic async transport backend?
19:50:58 <briancurtin> i've had a few people ask me if there's any thought to supporting something like tornado.http or any of the async versions of requests out there, and im not sure i even really know how to do this. or, i know how to do it, but it would make the entire thing async, i think. i might toy around with this as well, but does anyone know anything about making a
19:50:58 <briancurtin> library that would work both async and sync? seems quite messy, i think
19:51:57 <briancurtin> any time i mention how you can sub out your own transport, async is the first question after that
19:52:13 <terrylhowe> interesting problem, no ideas here
19:52:32 <terrylhowe> I like messages over callbacks though :)
19:53:08 <etoews> o/
19:53:38 <etoews> we had that in jclouds for the longest time and then ripped it out.
19:54:17 <briancurtin> etoews: because people didn't use it or beacuse it was a hassle?
19:54:40 <briancurtin> (or any of the many reasons to remove code :) )
19:54:42 <etoews> both
19:55:04 <etoews> the first thing it did was confuse
19:55:18 <etoews> it forced an early design decision on users
19:55:39 <etoews> oh. i need to choose between sync and async. i have no idea what i need at this point.
19:55:58 <briancurtin> maybe i'll just pin it on the next person who asks me about async to do some of the work themselves. i'm definitely not the best person to do this anyway. i know a little about it, but not enough to do it right
19:56:15 <etoews> then the underlying threading model was never quite right
19:56:19 <etoews> for their use case
19:56:22 <stevelle> I think the best way to do async is with a dedicated proxy model at least
19:57:05 <briancurtin> etoews: it was kyle kelley asking me about async fwiw
19:57:07 <stevelle> mixing the models just makes things harder
19:57:18 * sigmavirus24 is very late
19:57:30 <briancurtin> you have to do 20 pushups
19:57:38 <sigmavirus24> my 2￠: the async versions of requests are all junk
19:58:01 <sigmavirus24> If people want to use then, that's their responsibility. grequests and erequests aren't maintained iirc
19:58:19 <etoews> fwiw, we got very little complaints after we ripped it out of jclouds
19:59:13 <etoews> why not have the users wrap the sdk in async? maybe give them examples of how to do it to get them started.
19:59:34 <etoews> *has opinions, has never done async in python*
19:59:43 <sigmavirus24> etoews: it's not pretty =P
19:59:49 <etoews> take me with a grain of salt
19:59:51 <briancurtin> etoews: that was kyle's first response..."ugh, i'll have to wrap this in something or keep using my own stuff"
19:59:56 <stevelle> only way I would imagine it working well is a reactive approach to async but dunno if that is popular in py
20:00:06 <sigmavirus24> /time
20:00:26 <briancurtin> yeah, i'll follow up with a few people much better at async than me and see what we can/should do
20:00:28 <briancurtin> #endmeeting