19:00:21 #startmeeting python-openstacksdk 19:00:22 Meeting started Tue Feb 3 19:00:21 2015 UTC and is due to finish in 60 minutes. The chair is briancurtin. Information about MeetBot at http://wiki.debian.org/MeetBot. 19:00:23 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 19:00:25 The meeting name has been set to 'python_openstacksdk' 19:00:36 if you're here for the SDK meeting, say hi 19:00:38 Brian Curtin 19:01:09 Terry Howe, HP 19:01:11 o/ 19:02:11 #topic Conversations at Cinder meetup, Cinder v2 https://review.openstack.org/#/c/150979/ 19:03:05 so last week i was at the cinder mid-cycle meetup for the day and mike perez gave me the floor for a while. all in all it was an hour-long talk on where we're at, what we do, etc. i think of the ~25 people in the room we had active participation from 12 people 19:03:36 what's mike's nick? 19:03:46 thingee 19:04:18 is there an etherpad of notes from that session? 19:04:18 it was all really good. a handful had heard about the project already, but after explaining the goals and the status they were really interested. in teh end i had a few people interested in contributing and about an hour after the talk i had a guy submit a simple proxy that could list volumes (a dupe of my own work, but good that he just started hacking 19:04:18 right away) 19:05:00 stevelle: i dont think anyone took notes on that particular part as it was less about their roadmap for kilo and just getting them to think about it, but i'll look 19:05:52 anyway, once i have the proxy part of that cinder.v2 part done, which i just got started, im going to bring in walt boring and john griffith to take a look at it. after that im going to hack out the minimal compute proxy part to integrate the two for a sample app that can show attaching volumes or something 19:06:42 sounds like a good sample ap 19:07:23 will need to incorporate a bit of glance for the boot image I presume? 19:08:08 a bit future looking but how should we go about getting the project teams to take ownership of development in the sdk? 19:08:10 stevelle: probably. i dont actually know all of the moving parts in play there, but i also want glance for another thing, so that's also on the list 19:08:42 while bootstraping the project we'll need to do a fair amount ourselves but that's not sustainable. 19:09:05 i'm fine tabling that topic. just something to think about. 19:09:34 etoews: not sure, talked about it briefly with cinder and their thought was just having one or two people interested in the project to spend some time looking at it and get to know the cinder parts, then see where it goes 19:10:20 ya. and i think at some point soonish we can start being more vocal about it on the openstack-dev ml. 19:10:29 for glance we should have enough if we get the the v2 image proxy merged (https://review.openstack.org/#/c/147314/) 19:10:50 "hi all, support for glance has been added, next week is cinder." etc. 19:11:44 stevelle: sort of tangent but i think we should have built-out proxies like object_store that can do more than be session managers, but that's a start 19:11:49 #topic Thoughts on doc work in progress? http://briancurtin.com/doc_reorg/ 19:12:34 part of that doc work was driven by etoews starting to work on the project, but also as other projects get involved 19:13:06 so far it's been great 19:13:12 i +1'd the reorg 19:13:16 i dont want it to be one huge change so i think im going to add another guide on building out a proxy, and keep expanding user-guide stuff. ideally teh whole thing acts like a cookbook 19:13:25 and i've literally been working through the contrib docs 19:13:35 i can provide a more complete review soon 19:13:59 haven't touched the user stuff yet 19:14:02 cool, i think you're the most likely to have in depth review since you're actually going through it. anyone else? 19:14:04 +1 for not expanding the current change 19:14:12 easier to review in parts 19:14:40 I have it queued for review tomorrow 19:14:47 the contributor docs I went over so far looked good, but I need more time to read it all 19:15:29 #topic Resource.list consistency (pagination disabled, or not supported but still returning lists) https://review.openstack.org/#/c/147686/ 19:16:00 wait. one more thing on the docs 19:16:05 go for it 19:16:31 does gerrit spit out a rendered version of each patch set? 19:16:49 i know it can do this for docs.openstack.org 19:17:00 i guess we could make it do that, but it doesn't currently 19:18:14 i dont know if we get that luxury being a stackforge project or what the deal is behind that. currently the official docs are hosted at http://python-openstacksdk.readthedocs.org/en/latest/ 19:18:36 maybe annegentle knows... 19:18:36 i just claimed that name and point them at github master 19:19:28 I'm just running sphinx to review formatting at this point 19:19:48 me too 19:19:51 python setup.py build_sphinx 19:19:54 correct? 19:19:58 i'll look around and see what the deal is. we do have a doc builder so they are being built per changeset 19:20:05 etoews: i just do "tox -e docs" 19:20:06 etoews: yes 19:20:20 heh. 2 different answers. 19:20:25 both work 19:20:33 do they do the same thing? 19:20:42 yeah, tox runs that 19:20:56 so if you're in a proper virtualenv you're just doing the manual version 19:21:33 alright. i'll do that for now. 19:21:37 let's move on 19:22:04 Britt isn't here, but I'll see if I can get him to run that again. now that the page/list changes are in, that change lets us Resource.list anything that is a list, whether or not it supports pagination, so it's a consistent action across the board 19:22:53 however, one thing it loses is an optimization i was able to put in where it does an early exit when you get back fewer items than your requested limit. i dont know how often people would do that, but that's the only downside of the change i saw 19:24:15 we can still make that happen, but it'll require making things a bit more complex. in the interests of actually making Resource.list work i just ripped that stuff out. adding it back in to work would require an extra special case on top of the special casing of this being the first response 19:24:38 so it's possible but maybe not worth it to add it back in 19:26:16 the last_response == response seems like it could be expensive on large pages 19:28:37 terrylhowe: yeah that's one thing i want to check out. i have a container in rackspace cloud files with ~6k objects that would show how this turns out. i'll bring that auth change over and try it out and see what it does 19:29:18 terrylhowe: the change is unfortunately quite naive in that it works against devstack, which includes trivially small amounts of data to test against 19:30:00 it worked in devstack 19:30:24 terrylhowe: actually, now that i think about this, we might have to have services override list because this might just be too general 19:30:30 worked not works :) 19:31:50 the whole thing started because network supports pagination but disables it (and you can't know discover that it's disabled), and my optimization thing came from the way object_store works, so they don't work together well and both lose 19:32:24 i just workflow-1 19:32:33 yeh briancurtin too much generalization might just get confusing 19:32:45 will take it back to the drawing board and maybe only apply that type of thing where really really needed 19:33:47 I was thinking at one time about having something like “service preferences” to help with that kind of thing 19:34:32 have to ask, do you know the rationale behind making pagination disableable in neutron? 19:34:33 yeah i remember we talked about that, but i think it would then require that we have pagination turned off unless you say you want it, which might be too heavy handed 19:35:29 etoews: no clue, i just know in the docs it says it accepts it but is configurable to be off. it seems to be off in devstack since i've requested pages of something with a limit of 1 and just got back all 4 things i had (i think i was trying to list /networks) 19:35:39 everyone should support pagination, but we have to deal with some services not like it or not I think 19:36:55 i'm fine with services either having it or not having it but the idea of having it but making it disableable is bizarre to me. 19:36:59 terrylhowe: this might be something we can work out in the proxy level. if object_store supports pagination we can do the iterator containers() call through list. if network /networks doesn't support pagination, we can still call it networks() in the proxy levl but just call get() under the hood, which still gives a list 19:37:39 get gives a list? 19:38:24 oh. get() pretty much equals GET right? 19:38:27 yeah 19:39:12 well, get() has an id and the GET in list does not 19:39:36 this is on Resource right? 19:40:04 we can still call Resource.get() with no ID and have the full list of /networks returned to us. or at least that's what i think i was doing 19:40:48 that's why Resource.list() on /networks is an infinite loop, since it calls GET with no ID 19:41:44 i think i just need to get more familiar with all of this. i'll ask better questions then. 19:41:59 briancurtin: proxy level or resource level. Seems like resource level would be better. there are only 2 resources in object store 19:43:46 terrylhowe: i think proxy. let resources do what they need to do. teh network resource works fine with just a get(), but no list(), but its get() does return a list object, so on top of that we can make it function the same as containers.list() which under the hood can make use of the list() pagination 19:44:25 terrylhowe: if you run the examples.get against /networks you receive the same as you would hope from doing examples.list against /networks 19:45:40 in a network proxy we would just need to have our own loop around the body we get back 19:45:58 maybe i'll toy with that and see if it looks any better or more usable 19:47:37 I’d probably need to goof with it a little to form any real opinion on the best approach 19:48:45 it's -1'ed for now so we can slow down and do it right. i'll talk to britt, who reported it, and get him an example of the thing i just said and see how that works 19:49:26 #topic async transport backend? 19:50:58 i've had a few people ask me if there's any thought to supporting something like tornado.http or any of the async versions of requests out there, and im not sure i even really know how to do this. or, i know how to do it, but it would make the entire thing async, i think. i might toy around with this as well, but does anyone know anything about making a 19:50:58 library that would work both async and sync? seems quite messy, i think 19:51:57 any time i mention how you can sub out your own transport, async is the first question after that 19:52:13 interesting problem, no ideas here 19:52:32 I like messages over callbacks though :) 19:53:08 o/ 19:53:38 we had that in jclouds for the longest time and then ripped it out. 19:54:17 etoews: because people didn't use it or beacuse it was a hassle? 19:54:40 (or any of the many reasons to remove code :) ) 19:54:42 both 19:55:04 the first thing it did was confuse 19:55:18 it forced an early design decision on users 19:55:39 oh. i need to choose between sync and async. i have no idea what i need at this point. 19:55:58 maybe i'll just pin it on the next person who asks me about async to do some of the work themselves. i'm definitely not the best person to do this anyway. i know a little about it, but not enough to do it right 19:56:15 then the underlying threading model was never quite right 19:56:19 for their use case 19:56:22 I think the best way to do async is with a dedicated proxy model at least 19:57:05 etoews: it was kyle kelley asking me about async fwiw 19:57:07 mixing the models just makes things harder 19:57:18 * sigmavirus24 is very late 19:57:30 you have to do 20 pushups 19:57:38 my 2¢: the async versions of requests are all junk 19:58:01 If people want to use then, that's their responsibility. grequests and erequests aren't maintained iirc 19:58:19 fwiw, we got very little complaints after we ripped it out of jclouds 19:59:13 why not have the users wrap the sdk in async? maybe give them examples of how to do it to get them started. 19:59:34 *has opinions, has never done async in python* 19:59:43 etoews: it's not pretty =P 19:59:49 take me with a grain of salt 19:59:51 etoews: that was kyle's first response..."ugh, i'll have to wrap this in something or keep using my own stuff" 19:59:56 only way I would imagine it working well is a reactive approach to async but dunno if that is popular in py 20:00:06 /time 20:00:26 yeah, i'll follow up with a few people much better at async than me and see what we can/should do 20:00:28 #endmeeting