| *** rods has quit IRC | 00:16 | |
| *** ioggstream has quit IRC | 01:10 | |
| *** gkadam has joined #openstack-shade | 03:41 | |
| *** gouthamr has joined #openstack-shade | 04:31 | |
| *** squid has joined #openstack-shade | 04:41 | |
| *** calebb has quit IRC | 04:43 | |
| *** squid is now known as calebb | 04:43 | |
| *** calebb has quit IRC | 04:52 | |
| *** squid has joined #openstack-shade | 04:53 | |
| *** squid is now known as calebb | 04:53 | |
| *** gouthamr has quit IRC | 05:16 | |
| *** yfried has joined #openstack-shade | 05:59 | |
| *** slaweq has quit IRC | 06:05 | |
| *** slaweq has joined #openstack-shade | 06:29 | |
| *** rods has joined #openstack-shade | 06:48 | |
| openstackgerrit | Rui Chen proposed openstack-infra/shade master: Support to get resource by id https://review.openstack.org/492080 | 10:57 |
|---|---|---|
| *** noshankus has joined #openstack-shade | 11:04 | |
| *** gkadam has quit IRC | 11:52 | |
| *** rods has quit IRC | 12:24 | |
| samueldmq | morning shade | 12:27 |
| samueldmq | mordred: what lib do you use as a grpc client so that it's possible to test oaktree | 12:27 |
| samueldmq | ? | 12:28 |
| samueldmq | it might be great if I start adding some tests in that project | 12:29 |
| mordred | samueldmq: if you look in devstack/test.py in the oaktree repo you'll see a small client script | 12:30 |
| mordred | samueldmq: grpc libs produce their own client libs - so it's easy ;) | 12:30 |
| *** slaweq has quit IRC | 12:31 | |
| samueldmq | mordred: oh that's easy! | 12:31 |
| samueldmq | creating functional tests for that seem easy then | 12:31 |
| *** ioggstream has joined #openstack-shade | 12:31 | |
| *** slaweq has joined #openstack-shade | 12:32 | |
| mordred | yes indeed, I agree! | 12:32 |
| samueldmq | mordred: why do we need a new repo to define the oaktree model? I have a couple of questions if you're up for a quick convo now | 12:35 |
| *** slaweq has quit IRC | 12:36 | |
| samueldmq | ok. that adds protobuf into the mix, I need to learn that | 12:38 |
| mordred | samueldmq: I stuck the model is in a separate repo because it is just the protofbuf definitions and the auto-generated code for python, c++ and go - it requires sepcial tooling to build | 12:38 |
| mordred | yah | 12:38 |
| mordred | the library produced by oaktreemodel can be used by clients to connect as well as the server | 12:38 |
| samueldmq | mordred: so that generates the server and client versions for other languages too ? | 12:39 |
| mordred | oaktree itself is just the server - so it uses the oaktreemodel library for the grpc/protobuf definitions but actually connects those stubs with shade calls on the backend | 12:39 |
| mordred | samueldmq: it does | 12:39 |
| mordred | samueldmq: in oaktreemodel we actually commit the generated go sources to the repo, so that go get will work with it | 12:39 |
| samueldmq | mordred: why do we need that then? other languages would be able to use the oaktree grpc APIs, so we enable other languages that way | 12:39 |
| mordred | but the build jobs can produce the python lib and c++ lib | 12:39 |
| samueldmq | we wouldnt need ot run the servers in those languages | 12:40 |
| mordred | that's right | 12:40 |
| mordred | those are just for client bindings | 12:40 |
| mordred | someone can always just grab the grpc protobuf files and compile their own client - but it's easy enough for us to produce client libs for multiple languages - and that way we can test them in the gate too | 12:40 |
| samueldmq | mordred: so to use grpc you need to provide protobuf stuff that is used to build the clients | 12:41 |
| samueldmq | and then only that way the clients will be able to talk to your grpc server | 12:41 |
| samueldmq | is that correct? | 12:41 |
| mordred | that's right in general - but when we produce client libs for people, those people dont' have to do any protobuf/grpc build steps themselves | 12:42 |
| samueldmq | I thought grpc was like, let's say soap, where there would be clients out in the wild there for different languages | 12:42 |
| samueldmq | one could just grab a generic client and use it | 12:42 |
| samueldmq | mordred: aha so you're going one step ahead. and making native clients for other languages | 12:43 |
| mordred | nope - grpc is strongly pre-typed - it's done for network efficiency | 12:43 |
| mordred | yah. exactly | 12:43 |
| samueldmq | as opposed to just enabling shade/oaktree to other languages | 12:43 |
| mordred | so someon eCAN just pip install oaktreemodel | 12:43 |
| mordred | and they don't have to do any grpc/protobuf things | 12:43 |
| mordred | or they can do a go depend on git.openstack.org/openstack/oaktreemodel | 12:43 |
| samueldmq | oh that's awesome, and they would have a native client for whatever language we provide already/they want to build a client for | 12:44 |
| mordred | but - since it's grpc, if someone wants to do a language that we haven't made a client for, the grpc files exist and are public, so they can also build one themselves | 12:44 |
| *** rods has joined #openstack-shade | 12:44 | |
| mordred | samueldmq: ++ | 12:44 |
| samueldmq | mordred: I wonder if the model itself (objects oaktree server returns) should be with the server code (or in third repo?) | 12:46 |
| samueldmq | and the magic that does things for the various clients with protobuf etc would be a separate thing,as it looks to be | 12:46 |
| samueldmq | it sounds like let's say objects keystone return (abstract it is http api) but those objects are defined in ksc | 12:47 |
| mordred | samueldmq: so - the thing with grpc is that there is no difference between the objects the server returns and the client uses | 12:48 |
| mordred | it's the exact same objects (which is one of the cool things) | 12:48 |
| mordred | so we COULD have oaktreemodel, oaktreeclient and oaktree - but oaktreeclient would not have much code in it | 12:48 |
| mordred | we MIGHT still want to do that - and for go and C++ - because we might find that the gRPC api is too low-level and a 'nice' api wants to wrap it... | 12:49 |
| mordred | we do this a LITTLE in model.py in oaktreemodel | 12:49 |
| samueldmq | mordred: yeah, but the distiction would be clear. (not much code isn't an issue, we show grpc is easy!) | 12:49 |
| * samueldmq nods | 12:49 | |
| mordred | yah - might be worth splitting those for that reason for sure | 12:49 |
| samueldmq | cool, and we're in a good timing to make those changes | 12:50 |
| samueldmq | I can probably get a mentee from Outreachy to work with us too. I mean, I will try | 12:50 |
| *** slaweq has joined #openstack-shade | 12:51 | |
| samueldmq | mordred: I will experiment it and we can probably have more of these interesting discussions at the pt | 12:51 |
| samueldmq | ptg | 12:51 |
| rods | mordred Hi, I'm back :) | 12:52 |
| rods | sorry I've been a little busy at work lately | 12:52 |
| rods | what's the next step? what can I help with? | 12:52 |
| slaweq | mordred: hello, can You check my patch https://review.openstack.org/#/c/491033/ ? | 12:53 |
| openstackgerrit | Monty Taylor proposed openstack/oaktreemodel master: Add python make_client helper factory function https://review.openstack.org/492524 | 12:54 |
| slaweq | and take a look on https://review.openstack.org/#/c/491248 also - I don't know why QoS tests are still not running on functional tests | 12:54 |
| slaweq | maybe You will have some idea what I missed here or in project-config repo | 12:54 |
| mordred | slaweq: ok. I'll look ... | 12:54 |
| slaweq | mordred: thx a lot | 12:54 |
| slaweq | :) | 12:54 |
| mordred | rods: welcome back! I _may_ have just signed us up for a new set of fun work ... but I need to write up thouhts on that | 12:55 |
| mordred | in the mean time - I have a maybe fun challenge if you're interested | 12:55 |
| rods | mordred sure | 12:56 |
| mordred | rods: you know how we do the list then client-side filtering for get operations? well - we got a complaint over in the ansible modules that this is expensive/slow for more simple cases | 12:56 |
| mordred | so Rui Chen wrote this: https://review.openstack.org/492080 ... I'm not super crazy about it being the only interface to do that | 12:56 |
| mordred | rods: I was thinking we could make a flag for the constructor - like "use_direct_get" or something - that defaults to false | 12:57 |
| mordred | rods: but if it's true then the get_entity function could, if it is passed a dict with an id in it, use the get call instead of the list/filter | 12:57 |
| rods | mordred ya, makes sense | 12:58 |
| mordred | rods: that way for consumers like ansible, we can always set that flag and people won't pay the cost of the list - but the code/interface in shade can remain consistent | 12:58 |
| mordred | rods: I looked at it briefly yesterday and was thinking that instead of passing the search method to get_entity, we could just pass the cloud object and resoure name - then have get_entity look for the flag, and do a getattr to find either get_{resource_name} or search_{resource_name} depending on the flag | 12:59 |
| rods | mordred ok, I'll work on it | 13:00 |
| mordred | maybe we go ahead and also accept the Rui Chen patch, so get entity can look for get_{resource_name}_by_id or search_{resource_name} - it's not terrible idea ot have an explicit method people can use | 13:00 |
| mordred | rods: awesome - thanks! | 13:00 |
| mordred | rods: also - if you get a chance, you might want to read the mailing list thread "[openstack-dev] [python-openstacksdk] Status of python-openstacksdk project" | 13:01 |
| mordred | rods: because I have just proposed adopting the openstacksdk project, making some changes to it and then potentially merging shade and openstacksdk into one library - you may have thoughts/feedback on such an idea | 13:01 |
| rods | mordred will do , thx | 13:02 |
| mordred | slaweq: so - we're getting a neutron did not start: http://logs.openstack.org/48/491248/1/check/gate-shade-functional/9ab482d/logs/devstacklog.txt.gz#_2017-08-10_05_31_38_037 | 13:16 |
| mordred | slaweq: I see we got qos into the service plugins: http://logs.openstack.org/48/491248/1/check/gate-shade-functional/9ab482d/logs/etc/neutron/neutron.conf.txt.gz | 13:17 |
| mordred | so that's good | 13:17 |
| mordred | slaweq: ah: http://logs.openstack.org/48/491248/1/check/gate-shade-functional/9ab482d/logs/screen-q-svc.txt.gz#_Aug_10_05_30_40_975341 | 13:17 |
| mordred | slaweq: so it seems the qos driver wasnt found - does an additinoal library need to be installed or something? | 13:18 |
| openstackgerrit | Monty Taylor proposed openstack/oaktreemodel master: Add glide dependency tracking files https://review.openstack.org/492530 | 13:18 |
| openstackgerrit | Monty Taylor proposed openstack/oaktreemodel master: Symlink tools/test-setup.sh to bootstrap.sh https://review.openstack.org/492531 | 13:18 |
| slaweq | mordred: thx for help | 13:18 |
| slaweq | I forgot about it | 13:18 |
| slaweq | I will do another patch to project-config to fix it | 13:18 |
| slaweq | it would be better if there would be possibility to run shade on top of such patch from project-config to test it before merge | 13:19 |
| slaweq | do You know if there is such possibility? | 13:19 |
| slaweq | I think that "Depends-On" is not enough for that | 13:19 |
| mordred | slaweq: there is not today - but there is in zuulv3 which we will be rolling out at the PTG | 13:21 |
| mordred | slaweq: lucky for us, we're also using shade as one of the projects to develop zuulv3 jobs, so I'll be making zuul v3 devstack-gate jobs very soon (start working on it today or tomorrow) | 13:22 |
| mordred | and that will make doing these things MUCH easier | 13:22 |
| mordred | in fact, maybe I'll get a simple version started and we can use this as a test case of whether it works :) | 13:22 |
| slaweq | ok, we can use it as a test :) | 13:23 |
| slaweq | but for sure we should fix it quickly because for now I think that I broke all functional tests for shade :( | 13:24 |
| mordred | oh - yah - we should fix that for sure | 13:30 |
| slaweq | mordred: ok, I will handle it today for sure | 13:34 |
| slaweq | I will ping You when patch will be in review | 13:35 |
| mordred | sweet | 13:59 |
| *** gouthamr has joined #openstack-shade | 14:00 | |
| *** slaweq has quit IRC | 14:19 | |
| *** rods has quit IRC | 14:26 | |
| *** rods has joined #openstack-shade | 14:50 | |
| *** rods has quit IRC | 14:54 | |
| kmalloc | mordred: was looking at cache-control... I think we should just bake most of that stuff into ksa | 15:00 |
| kmalloc | Requests-mock should do nothing of the sort, imo | 15:01 |
| *** slaweq has joined #openstack-shade | 15:10 | |
| slaweq | mordred: hello again | 15:11 |
| slaweq | I just pushed patch https://review.openstack.org/#/c/492567/ to add qos extension driver in functional tests | 15:11 |
| slaweq | it should be enough now :) | 15:11 |
| slaweq | (I hope) | 15:11 |
| slaweq | and funny thing is that IIRC I was author of code which validate required extensions and raise this exception which we have now in shade functional tests | 15:12 |
| slaweq | and I forgot about it :) | 15:12 |
| *** yfried has quit IRC | 15:40 | |
| mordred | slaweq: that happens to me all th etime :) | 15:43 |
| mordred | kmalloc: so - cache-control / requests-mock - the main issue there is that you can't use requests-mock to test something that uses cache-control because they both mount into the requests.session | 15:44 |
| mordred | kmalloc: the update I think we need to requests-mock is to figure out how to allow such mounted adapters to be stacked - so that what you'd wind up with (or could if you chose) was requests_mock mountde underneath cache-contrl | 15:46 |
| mordred | that way in unittests you can still have requests_mock intercept the remote connection as it works currently for the times when cachecontrol decides that it does need to make a remote connection and not serve content from its local cache | 15:47 |
| mordred | the main issue at hand is that since requests-mock would override cache-control you have no way to verify that you've plumbed cache-control into things correctly, or that a patch hasn't broken the cache-control integration. obviously in most cases you can assume cache-control is going to do its job properly and that shouldn't really be the subject of your unittests | 15:48 |
| *** rods has joined #openstack-shade | 16:09 | |
| kmalloc | mordred: like i said, i think we should hook that into KSA at a low level | 16:50 |
| kmalloc | or.. need requests to support layered adapters | 16:50 |
| kmalloc | i don't think cache-control should be baked into requests-mock (at all) | 16:50 |
| mordred | kmalloc: I'm not following what you mean by "hook that into ksa" | 16:50 |
| mordred | oh - golly no | 16:50 |
| mordred | I totally agree - it should definitely not be baked into requests-mock | 16:51 |
| kmalloc | cache-control is not great looking at it | 16:51 |
| kmalloc | it's passible | 16:51 |
| kmalloc | bake the caching constructs into ksa or make ksa able to leverage cache-control instead of needing to hook into requests as well. move the cache-stuff up a layer | 16:51 |
| kmalloc | cache-control is beta-ish (by not great, it needs some more maturity) | 16:52 |
| kmalloc | not bad code. | 16:52 |
| kmalloc | it *seems* like requests-mock should be at a lower level than cache-control | 16:52 |
| mordred | yah - so - there's other things that need both cache stuff and requests-mock that don't necessarily use ksa - so that would be an improvement for the ksa case, but wouldn't fix the larger issue | 16:52 |
| mordred | yes. I agree | 16:52 |
| kmalloc | if we're battleing to be at the same level... we should work with requests | 16:52 |
| kmalloc | it's likely an issue in how requests is limited in these ytypes of adapters | 16:53 |
| *** slaweq has quit IRC | 16:53 | |
| mordred | in a perfect world I believe we'd want requests_mock to be at the lower level than cache-control - so yeah, we might need to work with requests to solve this | 16:53 |
| kmalloc | that was what i was getting at | 16:54 |
| mordred | kmalloc: I mean, for now just adding cache-control header awareness to ksa would, I believe, be fine for all the cases we've actually got | 16:54 |
| kmalloc | looking into it, it looked like a limitation in requests. | 16:54 |
| kmalloc | the interface is simple. | 16:54 |
| kmalloc | which is good... and bad. | 16:54 |
| mordred | ++ | 16:54 |
| *** slaweq has joined #openstack-shade | 17:16 | |
| *** ioggstream has quit IRC | 17:42 | |
| *** yfried has joined #openstack-shade | 17:44 | |
| *** slaweq_ has joined #openstack-shade | 17:46 | |
| *** slaweq has quit IRC | 17:50 | |
| *** gouthamr_ has joined #openstack-shade | 17:51 | |
| *** gouthamr has quit IRC | 17:51 | |
| *** yfried has quit IRC | 18:17 | |
| *** rods has quit IRC | 18:42 | |
| *** rods has joined #openstack-shade | 18:45 | |
| *** openstackgerrit has quit IRC | 19:03 | |
| *** slaweq has joined #openstack-shade | 19:33 | |
| *** slaweq_ has quit IRC | 19:36 | |
| slaweq | mordred: plop | 20:44 |
| slaweq | mordred: http://logs.openstack.org/48/491248/1/check/gate-shade-functional/d3fe2ef/logs/testr_results.html.gz | 20:44 |
| slaweq | it's better now | 20:44 |
| slaweq | at least functional tests are running and QoS tests are passing as expected | 20:44 |
| slaweq | mordred: sorry for this wrong patch which broke functional tests for shade | 20:45 |
| mordred | no worries - I'm glad we've got qos testing now ... now to figure out why the console log test is unhappy | 20:46 |
| slaweq | I think it's now related to Qos tests :) | 20:47 |
| slaweq | there was some timeout reached | 20:47 |
| *** gouthamr_ has quit IRC | 20:50 | |
| *** openstackgerrit has joined #openstack-shade | 20:58 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade master: Make get_server_console tests more resilient https://review.openstack.org/492683 | 20:58 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade master: Remove keystoneclient and ironicclient as direct depends https://review.openstack.org/492684 | 20:58 |
| mordred | slaweq: https://review.openstack.org/492683 should fix it - best I can tell it's a race-condition bug - we can test that thing in a more resilient manner | 20:59 |
| slaweq | mordred: I commented Your patch | 21:04 |
| mordred | slaweq: yah - so - chatted with mriedem a little bit in #openstack-infra ... in this case there actually was no console output for nova to report - my thinking is that making sure we're getting back the {'console': ''} payload is all we can safely test for - otherwise we're testing nova and devstack not shade ... but I could also see just letting unittest take care of it | 21:08 |
| mordred | Shrews: ^^ you have any thoughts? | 21:11 |
| slaweq | mordred: I can agree that checking only if {'console': ''} was returned should be fine | 21:11 |
| * Shrews reads sb | 21:12 | |
| slaweq | but I'm not sure then why You need this private method also | 21:12 |
| mordred | slaweq: the private method is becaue the normal method returns '' on http errors - so we wouldn't know in the test if we got a '' because nova did the right thing, or if the method just returned '' | 21:13 |
| mordred | slaweq: by splitting it and then using the private one - if we don't get {'console': str} back or have an http error, the method will fail and so will the test | 21:13 |
| slaweq | ok, I understand | 21:13 |
| slaweq | so it can be like that for me | 21:14 |
| Shrews | mordred: yeah, i don't think we should be testing content there | 21:14 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade master: Make get_server_console tests more resilient https://review.openstack.org/492683 | 21:14 |
| mordred | of course, if I could write python correctly it would be better :) | 21:14 |
| Shrews | pffft | 21:14 |
| slaweq | mordred: some unit tests are also failing in this patch: http://logs.openstack.org/83/492683/1/check/tox-py35-constraints/c852b4b/testr_results.html.gz | 21:15 |
| mordred | yah - I changed the name of a variable and didn't change it everywhere :( | 21:15 |
| slaweq | :) | 21:16 |
| slaweq | mordred: sorry for my questions but IMO get_server_console will not return {'console': ""} in case of other HTTP errors | 21:18 |
| slaweq | it will then return None or raise exception, no? | 21:18 |
| mordred | yes - that's right - it will only return "" if it gets BadRequest | 21:19 |
| slaweq | ok, and thanks this private method You will not get "" in such case bad BadRequest exception will be raised and test will fail for known reason | 21:20 |
| slaweq | sorry for many questions but now I understand it (I hope) :) | 21:21 |
| *** gouthamr has joined #openstack-shade | 21:23 | |
| *** ioggstream has joined #openstack-shade | 22:06 | |
| *** openstackgerrit has quit IRC | 22:18 | |
| *** slaweq has quit IRC | 22:23 | |
| mordred | questions are good! | 22:28 |
| *** openstackgerrit has joined #openstack-shade | 22:37 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade master: Make get_server_console tests more resilient https://review.openstack.org/492683 | 22:37 |
| openstackgerrit | Merged openstack-infra/shade master: Make QoS rules required parameters to be not optional https://review.openstack.org/491033 | 23:06 |
| *** gouthamr has quit IRC | 23:12 | |
| *** slaweq has joined #openstack-shade | 23:23 | |
| *** slaweq has quit IRC | 23:28 | |
| *** ioggstream has quit IRC | 23:31 | |
| *** pabelanger has quit IRC | 23:47 | |
| *** pabelanger has joined #openstack-shade | 23:47 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!