19:02:07 <briancurtin> #startmeeting python-openstacksdk
19:02:07 <openstack> Meeting started Tue Nov 18 19:02:07 2014 UTC and is due to finish in 60 minutes.  The chair is briancurtin. Information about MeetBot at http://wiki.debian.org/MeetBot.
19:02:07 <sigmavirus24> s/et/t/
19:02:09 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
19:02:11 <openstack> The meeting name has been set to 'python_openstacksdk'
19:02:17 <briancurtin> now i did!
19:02:20 <sigmavirus24> :D
19:03:03 <briancurtin> anyways, if jamielennox or dtroyer come in, cool, but let's go
19:03:13 <briancurtin> a couple of these are quick
19:03:17 <jamielennox> o/
19:03:48 <briancurtin> if anyone has a few minutes to check out the switch from manual limit/marker to an iterator, that would be awesome: https://review.openstack.org/134105
19:04:17 * dhellmann arrives late
19:04:29 <briancurtin> we also had sigmavirus24's entry_points review at https://review.openstack.org/131314 - however, our talk at the summit leans toward this not being something we should pursue, right?
19:04:41 <briancurtin> #topic entry_points to load services - https://review.openstack.org/131314
19:04:50 <sigmavirus24> briancurtin: that's what I thought, but there seem to be mixed opinions
19:05:12 <sigmavirus24> I'm on the fence and given the somewhat conflicting opinions I've not abandoned it, yet
19:05:55 <terrylhowe> I think jamielennox ’s points against entry points were good, but we still need to do something
19:06:31 <terrylhowe> I was thinking there would be maybe a dict to pass UserPreference constructor that would override and/or add services
19:06:32 <sigmavirus24> That said, I'm also liable to be very busy for the next couple weeks for personal reasons so if someone wants to commandeer that patchset, by all means, go ahead
19:06:47 <jamielennox> i see the need for them, i just think we need to make it clear that it's not part of our lib or we confuse users when services aren't present
19:07:06 <jamielennox> for example anything in openstack.ext.* is fair game
19:07:30 <jamielennox> s/services/libraries
19:08:00 <dhellmann> jamielennox: I'm not sure I understand, are there notes from that discussion? I hate to make you rehash it here
19:08:33 <terrylhowe> I was thinking maybe create pref = user_preference.UserPreference({‘lbaas’: ‘hp.lbaas’})
19:08:37 <dhellmann> the comment on the patch says something about the initialization order of the preferences object and command line parsing, but I didn't think we were building a command line tool in the sdk
19:08:42 <terrylhowe> something like that if someone wanted to add a service
19:09:06 <dhellmann> so the application developer would have to explicitly add the services?
19:09:37 <terrylhowe> I guess so
19:09:44 <dhellmann> you can still do that with entry points, fwiw, using the NamedExtensionManager or EnabledExtensionManager
19:09:56 <jamielennox> dhellmann: that was a fairly informal discussion with just the few of us so probably not. My concern with allowing arbitratry entry points on the main client object is the UX of a user not having the designate library installed for example, or dealing with versions of those libraries
19:10:29 <dhellmann> ah, ok, sure -- we'd have to define an API for those plugins to adhere to in any case, so we could take those factors into account
19:10:52 <briancurtin> i dont think  you should have to enable services manually - we just have to figure something out between load up everything we have (which may not be in your service catalog), and nothing
19:10:53 <jamielennox> if we don't know if it's going to be installed, or we don't control the functions in it i'd prefer to just namespace it a little so it's obviously an addition to the library
19:11:21 <dhellmann> the alternative is to do some sort of thing where the caller tells us how to import a module, and that is just so ripe for errors that are hard to debug I hate to think of the usability issues
19:11:21 <jamielennox> briancurtin: i agree
19:11:41 <jamielennox> dhellmann: yea, that's not a good approach
19:12:07 <dhellmann> how do we decide when something moves between the special namespace and the default?
19:12:18 <terrylhowe> so are you guys saying we could use entrypoints if the namespace is openstack.ext?
19:12:34 <jamielennox> as i said the best i've though of was a ext obj, which had not much more than a __getattr__ that hands off to a NamedExtensionManager from that value
19:12:36 <terrylhowe> the only thing is we have to do this before we have a service catalog
19:12:49 <briancurtin> ah, yeah
19:12:55 <dhellmann> the actual plugin namespace is invisible to everyone except the developer, so I'm not sure that name makes a lot of difference. I assumed you meant that the sdk objects would treat extensions somehow differently from builtins
19:13:40 <dhellmann> it seems like deciding which plugins to load is independent of the service catalog, isn't it?
19:14:10 <sigmavirus24> jamielennox: so something like openstack.services.builtin vs openstack.services.ext?
19:14:14 <jamielennox> dhellmann: i would think so, you just get an exception if you don't have the entry in SC
19:14:32 <dhellmann> jamielennox: or we could disable the plugin once we do have the service catalog
19:14:59 <dhellmann> but yeah, and exception is probably better since it's an opportunity to explain what's going wrong instead of just having a feature disappear
19:15:04 <terrylhowe> well, the service catalog is used to create the Connection class, it just wouldn’t have the attribute
19:15:11 <jamielennox> sigmavirus24: i was more thinking with an object, c = openstack.Connection()   c.identity vs c.ext.dns
19:15:13 <briancurtin> fail as early as we can, but no earlier
19:15:44 <dhellmann> jamielennox: how do we decide when c.ext.dns is "good enough" to become c.dns?
19:16:15 <jamielennox> dhellmann: i think it's a matter of explaining the failure, if the entry isn't in the catalog and you get a NoCatalogEntryExc as opposed to a AttributeException for it just not being there
19:16:31 <jamielennox> dhellmann: i was thinking when it moved into the tree
19:16:48 <jamielennox> dhellmann: at least initially the point was to have no external deps
19:16:53 <dhellmann> jamielennox: changing the name of the thing when it moves into the tree is just going to break any apps that use it, though
19:17:56 <dhellmann> we don't need external dependencies to "mix" access to the plugins, do we? I think I don't understand the concerns, still.
19:18:26 <jamielennox> there's a transition period anyway where you no longer have to install the external library, if the library was still available it would still work under c.ext
19:18:56 <dhellmann> yeah, but then you have an opportunity for 2 versions of the same thing to be present as well
19:18:58 <jamielennox> my concern is more for when the dns people add a bunch of functions we don't want to support in SDK, the project graduates and we are left supporting all those functions
19:19:00 <briancurtin> i would think of something like c.ext for where rackspace non-standard auth goes, instead of something like a designate lib
19:19:28 <dhellmann> jamielennox: why would we not support the full API?
19:20:00 <jamielennox> it's not the API that's the problem - you should see the amount of random crap that gets thrown in these client libraries
19:20:38 <dhellmann> I guess you're anticipating an incubation period for a library that would be outside of the sdk tree, and then at some point that code moving into the main tree?
19:21:00 <dhellmann> for oslo that graduation step is a trigger for us to re-evaluate the API of the module and clean it up
19:21:05 <briancurtin> all we should be concerned with, at least at the start, is supporting the REST API. conveniences on top of that, if that's what youre talking about, seem like the "even higher level"
19:21:57 <briancurtin> (where 'even higher level' is that shade-like lib)
19:22:32 <dhellmann> yeah, I would expect something like that to be built on top of this, rather than using this extension mechanism
19:22:41 <sigmavirus24> So I think I understand jamielennox's POV but I'm not sure I agree with it. If we don't want to immediately include code of incubated libraries then why not give openstack projects the ability to use the entry points without having to do c.ext
19:22:52 <dhellmann> so maybe we just say "this is how you make a plugin to support your rest API" and then later we have other plugins for higher-level functions
19:22:58 <sigmavirus24> I was thinking of c.ext much the same was as dhellmann so *shrug*
19:23:14 <jamielennox> anyway, it's just my opinion and not one i need to hold to firmly, i'm just concerned that given our goal is to make a nice API if we let everybody join with there own entrypoint we have no control over that and for those that are not intimitely familiar with the library and how entrypoints work we end up in a mess of what is supported and not
19:23:32 <terrylhowe> What if the user had to manually say ‘allow plugins’ and we’d look at entry points
19:24:10 <dhellmann> the plugins we're talking about are one per service, or one per version of a service, right?
19:24:22 <sigmavirus24> terrylhowe: like c = Connection(load_plugins=True)?
19:24:32 <terrylhowe> yes and yes
19:24:44 <terrylhowe> one per service
19:24:55 <jamielennox> i was trying to visualize this from a coming to openstack and possibly to python what is intuitive
19:25:11 <dhellmann> jamielennox: I can see your point, but I'm worried that segregating the plugins into 2 groups will make the SDK harder for an app developer to use
19:26:26 <jamielennox> plugins into 2 groups? was thinking purely what we maintain (which should be almost everything that is graduated) vs what we don't. What is internal isn't (at least currently) a plugin
19:26:47 <dhellmann> jamielennox: ah, ok, I thought those would just go into a different namespace
19:27:25 <terrylhowe> so, if load_plugins=True we don’t override existing services
19:27:28 <jamielennox> dhellmann: that's at least not currently the plan, we were talking entrypoints only for out of tree
19:27:47 <dhellmann> ok, that wasn't my impression from the patch just mentioned, sorry for the confusion
19:28:15 <jamielennox> i may be wong here
19:28:18 <dhellmann> terrylhowe: "override existing services"?
19:28:37 <terrylhowe> well, you couldn’t override out compute services
19:28:55 <dhellmann> yeah, I don't think we ever want to allow that, do we?
19:29:04 <jamielennox> ah, that patch loads everything via entrypoint - sorry
19:29:44 <dhellmann> jamielennox: right, and I like the "pick one way and do it" approach
19:30:50 <jamielennox> as a preference i'd say lets not use entrypoints for things we can call directly in python, i don't see why we'd want to let anyone change those values
19:31:13 <briancurtin> agreed there
19:31:27 <dhellmann> yeah, I guess the point is to say "here's how we load the services we know about" and then your code doesn't have 2 separate paths for setting things up
19:31:50 <dhellmann> if we don't want plugins because we think we'll end up with a lot of crap, let's just not use them at all to begin with
19:32:18 <dhellmann> I'd much rather see a well-thought-out sdk that works with a small number of services and tackle the extension issue in a later rev
19:32:46 <jamielennox> dhellmann: ++
19:33:01 <jamielennox> punt on extensions until someone really wants them
19:33:07 <briancurtin> works for me
19:33:29 <dhellmann> and at that point we'd have some sense of our requirements to be able to do them more cleanly
19:34:03 <dhellmann> I personally have less concern about the crap issue, since anything out of tree is going to have to be installed separately and someone else is going to maintain it, but I'm willing to wait and see
19:34:52 <briancurtin> since we're a little over halfway through, moving on to get to a few other things
19:35:08 <briancurtin> #topic make prop access return None by default instead of raise AttributeError - https://review.openstack.org/134632
19:35:39 <briancurtin> terrylhowe +2'ed that, but it could use looks from others
19:35:54 <dhellmann> do we want to return None or raise a more specific exception for that case?
19:36:02 <dhellmann> I assume some properties can have None as a value?
19:36:20 <stevelle> dhellmann: I was considering that case but haven't found any useful way yet
19:36:22 <briancurtin> that's one concern, but having it raise an attribute is kind of ugly to work with
19:36:45 <terrylhowe> If someone cared, that ould ‘attr’ in resource
19:37:31 <dhellmann> briancurtin: I'm concerned this is going to hide errors caused by typos
19:37:53 <dhellmann> values would always come back None instead of a NoSuchProperty error
19:38:23 <briancurtin> the current one definitely hides typos unless you're very specific about the try/except
19:38:27 <dhellmann> that try:except block in security_group.py should probably be inside the for loop instead of outside, but otherwise that doesn't look bad to me
19:38:39 <dhellmann> yeah, we definitely need a different exception type
19:39:02 <dhellmann> NoSuchPropertyError should subclass AttributeError probably
19:39:23 <jamielennox> so i understand that AttributeError is weird because we defined the attribute with prop, i think None is a bad default and we should make a new exception
19:39:37 <briancurtin> i just think if you want to check if an object has a header value set, you just access and either get back a value or not, like how you do dict.get(key, default) instead of try: dict[key] except KeyError
19:40:20 <jamielennox> briancurtin: so add a default=None to the prop()
19:40:23 <jamielennox> ?
19:40:43 <briancurtin> i had done that a long time ago but either abandoned it or got rejected, but id like to revisit that, i think
19:40:48 <dhellmann> that might work
19:40:52 <terrylhowe> this code would be accessed in the resource.prop case, so it is a little more typo proof
19:41:07 <jamielennox> i'd be happy with default= for that case
19:41:07 <briancurtin> so perhaps two things: betteer exception, an optional default return value
19:41:23 <dhellmann> so if I type "resource.porp" I'll get an AttributeError but if the person who set up "resource.prop" made a type I will get None?
19:41:38 <dhellmann> s/type/typo
19:41:39 <dhellmann> heh
19:42:39 <dhellmann> if I understand correctly, then having a default for the property makes more sense now so I'm OK with returning None
19:43:35 <terrylhowe> resource.porp wouldn’t hit this code, only resource.prop
19:45:02 <briancurtin> are we actually ok setting None as *the* default return value, or just an user-enabled one
19:45:53 <briancurtin> although, i think i would just want to set every single prop ever to be default=None, defeating the purpose of it being user-set (since we're the user in this case)
19:46:27 <stevelle> is the value proposition of a default to really indicate the property by that name exists?
19:46:57 <stevelle> otherwise we might be discussing another requirement for Resource
19:47:20 <dhellmann> briancurtin: I'm ok having None as the default default
19:47:46 <briancurtin> i think so. i'd like to be able to do "obj.read_ACL" and i either get the ACL back or something telling me it exists but isn't actually set, so None
19:47:52 <dhellmann> stevelle: isn't the fact that the developer declared the property an assumption that the value will be there?
19:47:57 <jamielennox> yea, that could work, still expose default= on the prop but default default is None
19:48:08 <dhellmann> jamielennox: ++
19:48:44 <briancurtin> so i just have to shift https://review.openstack.org/#/c/134632/ around a bit then and we're good?
19:48:55 <jamielennox> if you want to get tricky later if isinstance(default, Exception): raise else return
19:49:18 <dhellmann> briancurtin: I think I'm happy with it as is, if you add the default handling in another patch
19:49:30 <briancurtin> cool
19:49:47 <dhellmann> if there are no objections, I'll approve it now?
19:50:09 <stevelle> none here
19:50:41 <dhellmann> jamielennox?
19:50:49 <jamielennox> that's fine
19:50:54 <dhellmann> ok, +2a
19:51:15 <briancurtin> #topic jenkins/swift high level views
19:51:43 <briancurtin> terrylhowe: what's needed to proceed with the jenkins example and related code? anything?
19:51:46 <terrylhowe> I have two features in my jenkins thing that need to be pulled out into their own patches
19:52:14 <terrylhowe> swift could be rebased on master now
19:52:53 <briancurtin> yeah i need to do that, but i also think im going to start breaking out into other reviews for later things so i dont smash too much in this supposed "first cut"
19:55:24 <briancurtin> once we have these couple of high-level interfaces in, how do we want to go about building them out? do we just go a few different ways, reconvene soon, and see what works? bulk up the examples, do some blog posts, and try to get people to bang on the interfaces to see what they like?
19:56:16 <terrylhowe> I’d like to try a high level and thing POC with OSC
19:56:29 <terrylhowe> s/thing/thin
19:57:13 <terrylhowe> it’d be nice to have the thin code in master even if we scrap it later on just for the POC
19:57:30 <briancurtin> i'll check out that review you have up for it
19:58:39 <terrylhowe> 2 minutes.  Unicode?
19:58:54 <briancurtin> one last question with 2 minutes on the clock: in some of my toying with the high-level swift interface, i started wondering if we need to make a decision on unicode/bytes, or if we just pass through what we got and return what we received?
19:58:56 <briancurtin> yep
19:59:50 <dhellmann> it seems like the answer to that depends on the API we're calling, doesn't it?
19:59:53 <briancurtin> it came up because i was trying to determine if you gave me a Container object or a container name, so then i was checking against six.text_type, then i wondered if that's right, or if we should just act as a "you gave me some Python object that is a name, we send that name"
20:00:33 <dhellmann> we can pass the container instance and its name to the same method and that's supposed to work?
20:00:36 <briancurtin> dhellmann: it does, but i guess i havent seen any details around saying "this property must be utf8" or somethig, but i havent dug super far
20:01:02 <briancurtin> dhellmann: i'm toying with something high-level, not set in stone yet
20:01:06 <dhellmann> ok
20:01:35 * dhellmann has to run to the tc meeting and doesn't have a good answer for that one :-/
20:01:43 <briancurtin> dhellmann: it's kind of wonky for now, but i wanted to easily be able to create an object in a container, whether that container is a container.Container or "mycontainer"
20:01:58 <briancurtin> i guess we'll toy with it and see where it goes, don't need to be rigid up front, i think
20:02:20 <briancurtin> anyways, that's time, so...
20:02:22 <briancurtin> #endmeeting