19:00:27 <briancurtin> #startmeeting python-openstacksdk
19:00:28 <openstack> Meeting started Tue Mar 17 19:00:27 2015 UTC and is due to finish in 60 minutes.  The chair is briancurtin. Information about MeetBot at http://wiki.debian.org/MeetBot.
19:00:29 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
19:00:32 <openstack> The meeting name has been set to 'python_openstacksdk'
19:00:42 <briancurtin> now it's really time to say hi if you're here for the SDK meeting
19:00:45 <etoews> o/
19:00:54 <terrylhowe> o/
19:01:18 <dtroyer> o/
19:02:15 <dhellmann> o/
19:02:25 <edleafe> o/
19:02:36 <briancurtin> ian and steve are in a training, although steve might make it, but let's go
19:02:43 <briancurtin> #topic API guidelines
19:02:48 <briancurtin> #link https://etherpad.openstack.org/p/proxy_api_guidelines
19:03:11 <briancurtin> so i refined that doc quite a bit, and want to move on with the easier of the things: delete and get
19:03:51 <terrylhowe> I was hoping someone else would take a look at your delete proposal
19:03:53 <briancurtin> there's a review out for applying delete at https://review.openstack.org/#/c/164310/, and get will look basically the same but i do want to keep them separate
19:05:40 <briancurtin> one thing i think i want to change from what's listed is in update. i just saw steve at lunch and he mentioned that enforcing having a resource passed into update ignores the fact htat you could just do it by ID, so i might want to make that one update_resource(resource=None, **attrs)
19:07:06 <briancurtin> but anyway, i think we should probably do delete, get, create, update in that order so we're not piling on everything at once like i had originally written down
19:08:42 <dhellmann> ++
19:10:25 <briancurtin> so i guess i'll just ask: conn.service.delete_resource(resource_or_id) - can it get any better than this?
19:11:23 <edleafe> briancurtin: what is the expected response if the resource doesn't exist?
19:11:33 <terrylhowe> I’d like some eyes on the implementation
19:11:49 <terrylhowe> probably 404 exception edleafe
19:12:05 <edleafe> terrylhowe: I mean from the SDK, not the API
19:12:50 <briancurtin> we can debate the implementation but i want to talk about the API and get that ironed out.
19:13:03 <edleafe> I don't see failures documented in that etherpad
19:13:12 <briancurtin> edleafe: we are talking about the API.
19:13:27 <briancurtin> i want to define what the signatures for everything are going to look like.
19:14:03 <edleafe> briancurtin: So conn.service.delete_resource() is an API call? Something that the SDK would consume?
19:14:19 <briancurtin> it is the sdk
19:14:21 <dhellmann> that's a function in the SDK isn't it?
19:14:36 <edleafe> 404 is not a python response, it is an HTTP response
19:15:32 <edleafe> if I'm using the SDK and calling conn.service.delete_resource(), what should I as a developer expect?
19:15:35 <briancurtin> again. we can talk about how it is implemented on the review for https://review.openstack.org/#/c/164310/. i want to talk about the API. we have to figure out what the APIs provided by the proxy are going to look like.
19:15:55 <etoews> edleafe: we're talking about the api of the sdk here
19:16:01 <etoews> not the http api
19:16:23 <edleafe> etoews: so I'm still confused about returning a 404
19:16:28 <dhellmann> I think edleafe is asking a legit question about the behavior here.
19:16:38 <terrylhowe> https://github.com/stackforge/python-openstacksdk/blob/master/openstack/transport.py#L265
19:16:46 <terrylhowe> ^ that is what happens
19:17:00 <briancurtin> it doesnt return a 404, requests raises a 404. we have not yet determined how exactly to implement the API
19:17:25 <edleafe> briancurtin: ah, ok, that's what I thought was missing from the etherpad
19:17:36 <dhellmann> sure, but if we're talking about inputs and outputs of the methods then it seems reasonable to raise this
19:18:06 <etoews> as in, considering how the sdk handles error responses from the http api as being part of the sdk api
19:18:19 <dhellmann> it sounds like we do whatever the REST API does, transcribing the error to a different type, which means if the APIs are not consistent neither will the SDK be
19:18:25 <dhellmann> etoews: right
19:18:31 <edleafe> etoews: +1
19:18:53 <dhellmann> briancurtin: I like the consistency in the inputs from what I can see in the etherpad
19:19:04 <briancurtin> deletes in general return no body, sometimes headers, so the general case is that None gets returned. i think there are cases where we can make some responses cleaner, but what would you expect to be returned here if there's nothing to delete?
19:19:40 <dhellmann> I wouldn't expect delete to ever *return* anything, but the question is do we throw an exception if the thing we're trying to delete doesn't exist
19:20:05 <briancurtin> well, there is a case somewhere i just saw that delete gives information in headers
19:20:08 <dhellmann> I can see both sides to the answer. Doing nothing is idempotent, but hides potential user errors.
19:20:20 <dhellmann> briancurtin: right, sorry, I would not expect the sdk delete methods to ever return anything
19:20:24 <etoews> the current behaviour is to throw an HttpException with code=404
19:20:29 <dhellmann> this is a hard conversation :-)
19:21:00 <dhellmann> etoews: right, and I can see lots of consumers of the sdk wrapping all delete calls in an exception handler so they can safely ignore that
19:21:24 <dhellmann> how hard would it be to add an ignore_missing_resource argument to the delete methods in the sdk and use that boolean to decide whether to throw or ignore the 404?
19:21:25 <briancurtin> is it useful to know that you tried to delete something that doesn't exist?
19:21:42 <dhellmann> briancurtin: I would think in a lot of cases it's not, but I hesitate to say never
19:22:09 <briancurtin> so then maybe delete_resource(resource_or_id, ignore_missing=False)?
19:22:17 <dhellmann> so if we default ignore_missing_resource=True, we would swallow the 404 exceptions
19:22:32 <dhellmann> I was thinking default of True, but yeah
19:22:50 <etoews> it could be that a user expects something to be there to be deleted, and if it's not then something is very wrong in their system (e.g. some other actor deleted something it shouldn't have)
19:23:17 <dhellmann> etoews: right, that's why we provide an option, but I think in *most* cases people are going to just want the thing deleted and if it's already gone that's ok
19:23:44 <dhellmann> edleafe: is that what you were thinking?
19:23:49 <briancurtin> yeah, defaulting to true is fine
19:23:54 <etoews> ya. i'd lean towards ignore_missing_resource=True
19:23:57 <edleafe> dhellmann: something like that, yeah
19:24:26 <edleafe> dhellmann: I was looking at the etherpad as a consumer of the SDK, and that just caught my eye
19:24:46 <briancurtin> is there any other general enough case that would happen when deleting besides the resource not existing?
19:24:47 <dhellmann> edleafe: yep
19:25:12 <dhellmann> briancurtin: I think auth errors are the only other, and we should throw errors for those
19:25:29 <dhellmann> are the list methods already implemented? I'd sort of like to see all our methods have verbs in them for consistency. Maybe query_resource()?
19:25:36 <dhellmann> or query_resources()
19:25:39 <briancurtin> yeah auth stuff is handled off in auth land
19:26:34 <briancurtin> dhellmann: we currently have some things written as list_resources(), but after toying around with a object store implementation i wrote that as resources() and it worked really well. i wrote about that at the bottom of the doc
19:26:52 <briancurtin> but staying simple, get is along the lines of delete in that it operates on one piece of info, an id
19:27:33 <dhellmann> yeah, I saw. It caught my eye as an inconsistency, but I can live with it if everyone else likes that.
19:27:58 <briancurtin> dhellmann: we'd apply all of them the same way, we just started down different paths originally to see what worked in what way
19:27:58 <dhellmann> right, get makes sense for one, that's why I was suggesting a different verb
19:28:16 <etoews> for get i would expect it to throw an exception if the resource isn't there.
19:28:21 <dhellmann> briancurtin: I'm not sure what you mean "apply all of them the same way"
19:28:31 <dhellmann> etoews: yes, definitely
19:28:47 <briancurtin> dhellmann: compute.servers(), object_store.containers(), image.images()
19:29:02 <briancurtin> whatever signatures we determine here get applied everywhere for that type of action
19:29:03 <dhellmann> briancurtin: oh, right. And I was suggesting to verbify them with query_servers()
19:29:07 <dhellmann> etc.
19:29:25 <dhellmann> but again, if everyone likes the shorter name, I can go along
19:29:46 <briancurtin> hmm, query might actually work with the filtering/searching query params that a lot of them implement
19:30:04 <dhellmann> yeah, that's what I was thinking :-)
19:30:11 <briancurtin> hah, we're sort of jumping back and forth/ahead though
19:30:13 <dhellmann> I do like the argument against using "list"
19:30:47 * dhellmann gets back in line
19:31:01 <briancurtin> is there any extra argument to give to get_resource(x)? like etoews said, if it's not there, i think that's actually exceptional so letting the 404 fly seems reasonable
19:31:27 <briancurtin> can't just soldier on through your code without what you were trying to get. probably
19:31:45 <dhellmann> when we say "resource_or_id" do we mean a uuid for the id part, or for things that have names are the names allowed?
19:32:02 <etoews> do we want to use the generic HttpException in that case or resurrect ResourceNotFound?
19:32:08 <dhellmann> or is it up to the user of the SDK to turn a name into a resource or a uuid somehow?
19:32:29 <terrylhowe> I was thinking find would be used to translate name->id
19:32:30 <dhellmann> etoews: I like the more specific exception
19:32:44 <terrylhowe> I don’t think everyone should take a HTTP hit for that
19:32:45 <dhellmann> terrylhowe: ok, I like that. too many options just complicates our own implementation
19:32:53 <briancurtin> dhellmann: the way i've written things like this in the past are that they can take a resource.Resource and get the id_attribute value off of it (name where that applies), or whatever you have provided manually as an ID
19:33:47 <dhellmann> briancurtin: right, and I was asking more specifically about resources that can be referenced by more than one identifying characteristic. Everything has a UUID, not everything has a name, and not all names are required to be unique. So allowing a search by name adds complexity that can probably better be handled in the application layer.
19:33:48 <edleafe> id = resource_or_id.id if isinstance(resource_or_id, resource.Resource else resource_or_id
19:34:19 <dhellmann> edleafe: ++
19:34:34 <edleafe> oops - forgot to close the parens :)
19:34:35 <briancurtin> dhellmann: oh yeah. i think what terrylhowe said would work in that case
19:34:54 <dhellmann> briancurtin: ++
19:35:23 <etoews> and the user is expected to explicitly call find to do name->id
19:35:45 <etoews> i think that's reasonable.
19:35:46 <dhellmann> etoews: by "user" you mean "application developer"?
19:35:59 <etoews> yes
19:36:08 <dhellmann> ok, just checking :-)
19:36:35 <briancurtin> i agree with etoews. either you pass in a resource with an ID, or you pass in the ID itself, and we make it so you can get the ID by name (we already have something like it, need to beef it up)
19:36:55 <etoews> the app dev should always have a lot of control over how many requests they're making.
19:37:08 <etoews> briancurtin: i might have to rethink the orchestration stuff
19:37:14 <etoews> in light of this discussion
19:37:19 <briancurtin> yep, if you know you have the ID string, just pass it in and you're good, no extra calls
19:37:49 <briancurtin> if you have a bunch of names only, you'll have to do an extra jump but we can get it
19:37:57 <etoews> it does a funky name to id http redirect that none of the other apis do
19:38:03 <dhellmann> so all of the command line apps will always have to call find(), which means for delete they will need to decide whether to ignore the error for a missing object or report it
19:38:35 <etoews> so to make it consistent with the other services might require skipping on the functionality.
19:39:48 <briancurtin> to make who consistent with what?
19:40:40 <dhellmann> etoews: does that meant it's not possible to just query the orchestration service by name and have it give an id back?
19:40:59 <dhellmann> it seems like we could intercept the redirect, if we know that's how the API works
19:41:34 <etoews> briancurtin: to make orchestration consistent with the sdk api of all of the other services
19:44:06 <briancurtin> etoews: let me know if i can help out with that. i havent looked at orchestration much but we've gone through a few sort of quirky things already
19:44:08 <etoews> dhellmann: it's definitely possible. we just have to make a point of making it work like all of the other services.
19:44:16 <dhellmann> etoews: right
19:45:13 <etoews> briancurtin: i wish i could point you at some functional test code to show you how it works compared to the other services. ;)
19:45:32 <briancurtin> so in general, get_resource(resource_or_id) is cool to work with either a resource or an ID, but we also enhance the ability to get the ID based on a name. is that in line what what people are thinking?
19:46:05 <etoews> briancurtin: that's not the impression i got.
19:46:26 <briancurtin> and?
19:46:36 <dhellmann> I thought we were talking about a separate find() method that would search by name. Although we could to that with the query_resource() method, too, and avoid having another method.
19:46:39 <etoews> app dev's would have to use find_resource
19:46:41 <briancurtin> that exists
19:46:56 <briancurtin> yeah they would
19:46:59 <dhellmann> oh, ok, so find_resource is like get but takes something other than an id?
19:47:20 <terrylhowe> id or resource
19:47:21 <briancurtin> find is currently a name-to-id
19:47:39 <etoews> sorry it's resource.find
19:47:39 <dhellmann> ok, I think I've got it
19:47:42 <briancurtin> terrylhowe: maybe find just goes away and get takes it over?
19:48:02 <edleafe> briancurtin: -1 on removing find.
19:48:05 <briancurtin> eh, now i dont like this. find sort of duplicate this.
19:48:05 <terrylhowe> no, find is complicated
19:48:06 <dhellmann> yeah, I wonder if we really need both? if get was get(resource_or_id=None, name=None)
19:48:12 <edleafe> It's a much more natural name
19:48:15 <briancurtin> find isn't necessary
19:48:20 <briancurtin> but ive taken way too much time
19:48:21 <briancurtin> #topic functional testing
19:48:27 <briancurtin> #link https://review.openstack.org/#/c/162210/
19:48:50 <etoews> i really need functional tests
19:49:08 <terrylhowe> ah yeh, I was just trying to write a script to create a devstack we could all use
19:49:09 <etoews> in the little bit of functional testing i've done it's caught stuff the unit test dont'
19:49:10 <briancurtin> same times a million. ive been writing too many personal scripts i just run locally
19:50:01 <etoews> terrylhowe: that script will be useful but it's not 100% necessary to have before we can start writing functional tests.
19:50:16 <dhellmann> is this cloud-init script the way the CI systems expect functional tests to run? I haven't looked into that yet
19:50:17 <terrylhowe> there are gates that create a devstack instance, this is not intented to replace that, it is just for manually functional testing
19:50:39 <dhellmann> ok
19:51:06 <dtroyer> dhellmann: no.  stevemar built one for OSC that works the other way around…point it to an existing devstack and it runs the tests
19:51:32 <terrylhowe> true etoews I just thought we’d want to develop some standard test bed
19:51:43 <dhellmann> dtroyer: ok, I thought sdague was expecting functional test jobs to rely on the devstack plugin stuff, but if terrylhowe had something else in mind for this script that's fine
19:52:04 <etoews> terrylhowe: yep. i just don't want the merging of that script to hold up merging functional tests
19:52:26 <dtroyer> AIUI the plugin bits are to get devstack set up, then d-g runs tox -efunctional.  that's the bit stevemar setup does
19:52:46 <dhellmann> ah, ok, I didn't realize we were using tox for that step
19:52:53 <dtroyer> but I also haven't been too involved in those details
19:53:03 <etoews> where would functional tests go in the tree?
19:53:19 <etoews> i just want to submit some functional test code.
19:53:33 <stevemar> dtroyer, yep you got it. etoews they are under osc/functional
19:53:47 <briancurtin> openstack/tests/functional/ and then follow the same tree layout from there as unit tests?
19:53:52 <dhellmann> etoews: we should look around at some of the other projects that are starting to do functional tests to see what pattern is emerging
19:54:01 <dhellmann> briancurtin: probably
19:54:06 <terrylhowe> I was thinking so briancurtin
19:54:15 <etoews> +1
19:54:50 <dhellmann> do we have a way for that test suite to know about the cloud it is supposed to talk to, yet? a config file or something?
19:54:53 <briancurtin> i dont know if we would want to move the unit tests from being straight under tests/ to being tests/unit
19:55:19 <dhellmann> briancurtin: moving them will make it easier to run testr
19:55:21 <etoews> briancurtin: it's tempting do to that...
19:55:36 <etoews> dhellmann: i'm just using env vars right now for that
19:55:43 <dhellmann> etoews: ok, that works, too
19:55:44 <etoews> the config that is
19:55:48 <dhellmann> right
19:56:10 <dtroyer> dhellmann: it is simple enough to find the devstack root dir and pull credentials from there via openrc or one of the accrc/* files
19:56:28 <dhellmann> dtroyer: that works if the test suite is running against a devstack deployment vs. a real cloud
19:56:38 * dhellmann needs more caffeine before his next meeting, brb
19:56:41 <dtroyer> right
19:57:06 <terrylhowe> os-client-config might be nice for our examples/testing
19:57:29 <etoews> ya. we should have a look at that.
19:58:40 <briancurtin> two minute warning
19:58:51 <dtroyer> it's prety sweet
19:59:03 <etoews> #action etoews to have a look at os-client-config for functional test config
19:59:31 <stevemar> dhellmann, the script that kicks off the functional testing for osc also looks for a localrc file that has an auth_url/uname/pass
19:59:47 <stevemar> but it can definitely be improved
20:00:02 <dhellmann> stevemar: ok, so if I'm testing against something that isn't devstack I can always fake it that way, but it sounds like os-client-config might be a good option, too
20:00:09 <etoews> stevemar: is the auth_url fixed?
20:00:50 <briancurtin> looks like our time's up here. back to -sdks to continue?
20:00:52 <terrylhowe> those reviews are still out for osc
20:00:53 <dhellmann> next meeting is starting, thanks everyone
20:00:58 <briancurtin> #endmeeting