Monday, 2017-03-27

*** wirehead_ has quit IRC02:01
*** wirehead_ has joined #craton02:03
*** VW has quit IRC02:16
*** VW has joined #craton02:16
*** Syed__ has quit IRC03:25
*** VW has quit IRC03:32
*** VW has joined #craton03:33
openstackgerritMichael Porras proposed openstack/craton master: Adding wrapper functions to tools  https://review.openstack.org/44923004:40
anonymikefsaad: I think I'm gonna take your advice and swap my laptop this week. This is happening every couple of hours http://imgur.com/a/9edEI crazy frustrating04:44
*** VW has quit IRC05:15
*** acabot has quit IRC07:17
*** klindgren_ has quit IRC07:48
*** klindgren_ has joined #craton07:49
*** acabot has joined #craton07:52
*** openstackgerrit has quit IRC08:33
*** VW has joined #craton11:00
sulooh yikes i didnt realize this was still pending https://review.openstack.org/#/c/447580/11:03
sulogit-harry: are you clear on what sigmavirus is referring to there ?11:05
sulowould really love to get that patch in if possible11:05
sulothomasem: jimbaker: do take a look at ^11:05
git-harrysulo: yes but I was waiting for the opinions of others. I'll add another comment.11:12
sulook11:15
*** VW has quit IRC12:39
*** VW has joined #craton13:09
*** VW has quit IRC13:13
*** VW has joined #craton13:14
thomasemsulo: git-harry: done13:21
thomasem:)13:21
thomasemanonymike: that's no fun. I'm sorry you're having to deal with that. :(13:22
thomasemAnd good morning / afternoon!13:22
*** VW has quit IRC13:29
*** openstackgerrit has joined #craton13:29
openstackgerritThomas Maddox proposed openstack/craton master: JSON Path-like querying for variables  https://review.openstack.org/44394113:29
*** VW has joined #craton13:32
fsaadmorning guys13:35
fsaadanonymike: yeah that's pretty bad13:36
thomasemgit-harry: https://bugs.launchpad.net/craton/+bug/167643713:39
openstackLaunchpad bug 1676437 in craton "error responses do not comply with OpenStack API WG guidelines" [Undecided,New]13:39
git-harrythomasem: ta13:41
thomasemda13:41
* thomasem waves hands13:41
*** VW has quit IRC13:41
*** VW has joined #craton13:43
anonymikemorning all and yeah, kind of frustrating but it'll be fixed soon13:55
*** Syed__ has joined #craton14:01
openstackgerritThomas Maddox proposed openstack/craton master: JSON Path-like querying for variables  https://review.openstack.org/44394114:01
thomasemAlright, writing up final tests for that patch. Please feel free to review what's done anyway^^ :)14:28
*** VW has quit IRC14:32
*** VW has joined #craton14:32
jimbakersulo, git-harry - i did take a look and also discussed with sigmavirus re https://review.openstack.org/#/c/447580/14:59
jimbakermeeting...14:59
fsaadmmm, I'm there by myself15:01
fsaadam I in the wrong channel? :S15:02
git-harryfsaad: #openstack-meeting-415:02
fsaadthanks git-harry15:02
*** VW has quit IRC15:12
*** VW has joined #craton15:12
*** VW has quit IRC15:32
*** VW has joined #craton15:33
*** klindgren__ has joined #craton15:35
*** klindgren_ has quit IRC15:38
jimbakergit-harry, so is the plan to fix up your patch such that it complies with https://specs.openstack.org/openstack/api-wg/guidelines/errors.html ? just seems like this could be a second step?15:55
thomasemjimbaker: I am in favor of us making that a second step, myself.15:56
thomasemgit-harry: ^^15:56
jimbakerfirst step: one exit point; second step: follow guidelines. i think the only tricky part is that iirc the patch currently removes some of the flask error reporting that these guidelines effectively expand upon15:57
thomasemI only intended to point out the thing we probably want to move towards with  https://bugs.launchpad.net/craton/+bug/167643715:57
openstackLaunchpad bug 1676437 in craton "error responses do not comply with OpenStack API WG guidelines" [Undecided,New]15:57
jimbakerthomasem, got it, and i think that's certainly the right place to go15:57
thomasemWhich is why I made a separate bug for it. :)15:57
thomasemCool15:58
jimbakernot certain how it impacts graphql mind you...15:58
jimbakerbut separate discussion. long term discussion15:58
jimbakermaybe different endpoint too15:58
jimbakerthe other thing that comes to mind is better internal logging of problems. i want my stack trace back...15:59
jimbakerideally running on 3.6 while we are at it :) (because it does a far better job of logging stack overflows)16:00
jimbakeranyway, not immediate. just putting it out there16:00
thomasemLol16:00
thomasemYeah... I have no idea what happened to the tracebacks.16:01
thomasemBut it's been really annoying.16:01
thomasemMaybe I'll dig into that a bit and see if it's just something silly (probably is).16:01
jimbakeralmost certainly is16:01
jimbakerwe did a fair amount of refactoring to be systematic about codes being returned, which this recent patch by git-harry is an extension of16:02
thomasemRight16:02
*** klindgren__ is now known as klindgren16:04
jimbakerbtw i highly encourage we use chained exceptions (http://stackoverflow.com/a/16414892/423006)16:05
jimbakerthomasem, this could be a simple bug (i would think) for someone to take care of - most of this needs to be done in craton/db/sqlalchemy/api.py16:06
jimbakerthat way we could avoid losing context16:06
sulosorry folks i missed the meeting today, i had a doctors apt16:07
jimbakersulo, no worries, we did give you some work :)16:08
sulojimbaker: nice16:09
suloalthough i alrady got some16:09
jimbakeri bet you do16:09
suloso back to that patch from git-harry16:09
sulolet me see if ya'all got a chance to talk about it or make comments16:09
jimbakerno, it was just the outstanding client stuff for network resources16:09
suloi need that patch(or revisions of it) for some stuff i am doing16:10
sulohttps://review.openstack.org/#/c/447580/16:10
thomasemjimbaker: good call, re: chained exceptions. I'm fixing that in my patch.16:12
jimbakerso i think the consensus here is that we want to bring back the error array reporting; but do it with respect to wg guidelines16:12
jimbakerso two steps16:12
sulojimbaker: ok sounds good to me, as long as there is some consensus ..16:14
jimbakerthomasem, yeah, something we can think about in our reviewing. all exceptions should be chained. i cannot think of a reason why not (potentially one could want avoid in certain scenarios where exceptions are being used at a high rate. doesn't apply here)16:14
suloi just need the output of how the error handled ... etc16:14
git-harryI'm not sure the example in the guidelines applies directly to this situation but some variation on it would probably make sense.16:15
jimbakeryeah, i'm not sure how many projects adhere to these guidelines...16:16
jimbakerstill we have enjoyed helping set direction here, as we saw with pagination16:17
jimbakerhttps://developer.openstack.org/api-guide/compute/paginated_collections.html is merged in...16:19
jimbakergit-harry, also it looks there's a chance to reconcile https://developer.openstack.org/api-guide/compute/faults.html with https://specs.openstack.org/openstack/api-wg/guidelines/errors.html16:21
jimbakerthe upshot: definitely more than just one patch required here :)16:21
jimbakerthomasem, i see an opportunity to report back to OSIC on fixing the openstack ecosystem inconsistencies, one small sisyphean boulder at a time16:23
jimbaker:)16:23
jimbakerthomasem, re https://review.openstack.org/#/c/447580/3/craton/tests/functional/test_cell_calls.py, are we good with postponing that in a second phase of work?16:32
thomasemTesting for multiple validation errors?16:33
jimbakeryep16:33
thomasemI don't think so. That's something that's supposed to be supported now.16:33
thomasemSince this code changes that piece, I think it makes sense to assert that it works.16:33
jimbakerok. i just see it as being superseded by api wg guidelines, that's all16:33
thomasemI see it as making sure the existing implementation meets the expectations we have of it. I don't know how fast a follow the new "standard" may be, and I don't think it's that much more work to add a few tests to assert >1 validation errors.16:34
thomasemIf I'm misunderstanding or thinking that's a bunch of extra work to do, please tell me. It seems like the same test as before, just another thing wrong with the request and asserting that the message provides both errors.16:35
jimbakerthomasem, ack16:36
thomasemor we're thinking*16:36
thomasemcool :)16:37
thomasemAlso, it lets us add the extra docstring stuff sulo was suggesting, which will help anyone else reading this code.16:38
thomasemSo, we had a little bit of reason for a change to it anyway.16:38
jimbakerSyed__, any update on https://review.openstack.org/#/c/425463/ ?16:38
thomasemSyed__: Also, did that fix work that we discussed a couple weeks ago?16:39
thomasemregarding tests for the patch jimbaker mentioned16:39
openstackgerritThomas Maddox proposed openstack/craton master: JSON Path-like querying for variables  https://review.openstack.org/44394116:39
anonymikeQuick clarification for the fields stuff. Filtering the desired fields from the returned object is all being done through the client, correct? We aren't crafting queries based on requested fields yet16:40
suloanonymike: querys are filter based16:40
sulonot sure thats what you are saying though .. --fields is different16:41
suloits just display iirc16:41
thomasemyeah, there's no crafting queries to the API based on --fields, it's simply taking what the API already gives us and modifying how it's displayed16:42
thomasematm16:42
anonymikeperfect16:42
anonymikethomasem16:42
anonymikethanks16:42
*** chrisspencer has quit IRC16:42
jimbakeranonymike, yeah at some point we may want to pushdown this "select" info into craton16:43
thomasemYep16:43
jimbakerbut for now, everything is purely on client side for this work16:44
thomasemWould be nice to have the same capabilities on the API in the future.16:44
anonymikecool, I got a little confused when you brought that up jimbaker, i thought thats what i was doing now16:44
thomasemEspecially when it comes to scaling this thing16:44
jimbakeranonymike, nope, that's why we mentioned graphql16:44
anonymikewhew okay16:44
jimbakerbecause we are doing it terribly inefficiently16:44
jimbakerbut correctness first16:44
jimbakerthen optimize16:44
jimbakerone of many mantras :)16:45
jimbakermy many16:45
jimbakeranonymike, also applies to aliasing. let's do this client side16:46
anonymikei agree, and that was going to be my initial approach. I was just a little confused at the mentions of how it SHOULD be done and started freaking out lol16:47
jimbakeranonymike, as in, what i did just volunteer for?16:49
anonymikeexactly16:49
anonymikeconfident, but gonna take much longer16:49
jimbakeranonymike, yeah, i think we just wanted to impress upon how much this touches. but fortunately all highly useful stuff16:49
anonymikeI'm still running into some weird issues with the craton client. I think i really messed up my environment while I was juggling the compose/elastic search stuff and installing the client manually. I'm going to rebuild my instance while I lunch.16:53
anonymikebtw jimbaker, I made some good progress last night on the elastic search stuff. I would really like to play around with displaying some data through kibana. It sounds like that should go on the backburner, but Its something I'm still interested in completing in the next few days16:54
jimbakeranonymike, it is a "back burner" project, especially kibana16:55
jimbakerhowever, i think we can readily do something like tie in to craton as follows16:55
jimbakerv1/resources?q=... proxies the query to ES, getting back a set of json docs. then applies rbac to set and returns16:56
jimbakeranonymike, so this probably can do what we want with search. kibana is trickier, because we need craton to provide rbac as an api it can use. but that's why i suggested sidestepping these integration points for now and just see if the functionality was at all useful16:59
jimbakerbefore even exploring further17:00
anonymikeyeah, i was just wondering how we'd secure those attributes with kibana17:00
jimbakeranonymike, yeah, and do it cheaply as well17:01
jimbakerif if it needs to call craton for every attribute...17:01
jimbakerso best to defer for now17:02
anonymikejimbaker: are there any thoughts or plans around data visualization? Something like the Galaxy UI (observatory)  I think that'd go a long way in a demo and help buy in from support. However I completely understand that a lot more has to be completed before we can justify setting up a UI17:05
anonymikejust curious...17:06
antonymi'm guessing craton-dashboard plugin for horizon was an attempt at that17:07
thomasemYeah17:08
antonymbut yeah, i'd guess that'd be useful for support17:08
jimbakerright. i think the horizon idea had some merit, but the reality is that we want a better UI17:08
jimbakeranonymike, at one point we were getting into discussion of visualization. mostly with respect to workflows17:09
jimbakerso how we build out a "cloud progress bar"17:09
anonymikegotcha17:09
jimbakerin any case, the key thing here is how to highlight problems17:09
jimbakernote that the vars stuff doesn't directly give us that17:10
jimbakerit doesn't say - what is the expected values, and what if not in compliance17:10
anonymikepeople like quick tables of inventory as well. drilling down on a webpage is a lot easier than writing a script or entering a long command to see the hosts in a cell17:10
jimbakerit is possible we could say, maybe we could add such things to a namespace17:10
anonymikeright17:10
jimbakeranonymike, agreed17:10
jimbakerso i think kibana actually could be a good way for quickly trying out ideas here17:11
jimbakereven if it's not the real dashboard17:11
anonymikeJust some stuff I was wondering about :) didn't mean to derail any work. I'm gonna relocate to a coffee shop and continue this work17:13
jimbakeranonymike, no derailing, just great questions!17:15
jimbakergit-harry, sulo , we just need a quick fix on https://review.openstack.org/#/c/447580/3, and we are good17:16
sulojimbaker: git-harry: yeah, cool. my change is based on that change so i am just waiting for whatever17:16
jimbakerspecifically the requested test for https://review.openstack.org/#/c/447580/3/craton/tests/functional/test_cell_calls.py17:16
jimbakersulo, should be a minor to get in17:17
sulojimbaker: rgr17:17
*** harlowja has quit IRC17:18
*** harlowja has joined #craton17:24
openstackgerritThomas Maddox proposed openstack/craton master: JSON Path-like querying for variables  https://review.openstack.org/44394117:42
openstackgerritMerged openstack/python-cratonclient master: Remove link to modindex  https://review.openstack.org/42858017:48
jimbakerjust cleaned up our review queue with some minor (including taking care of "robot" patches)17:49
thomasemthanks jimbaker17:49
jimbakerwe need to get some progress on jovon's doc patches17:50
jimbakereverything else looks like we are moving forward on at this time17:50
jimbakeror at least pings made17:51
thomasemyeah17:51
openstackgerritMerged openstack/python-cratonclient master: Update link to documentation and bug tracker  https://review.openstack.org/44972517:51
openstackgerritMerged openstack/craton master: Remove link to modindex  https://review.openstack.org/42857917:52
jimbakeri'm sure we will want to re-add modindex but no need to have a broken link17:52
thomasemAlright, going to write some docs up on how to use this JSON Path thing and submit those next.17:53
jimbakerthomasem, so i take it that the json path func testing is done, and we just need to wrap up reviews on it17:59
jimbaker(which i assume will be quick, given how much time we have collectively looked at it)17:59
thomasemjimbaker: yep17:59
thomasemreviews + any changes that come out of reviews. I, too, hope it's fairly quick.17:59
jimbakercool. brb, need to walk dog, but will do that next17:59
thomasemGreat, thanks!17:59
thomasemI need to head to an appointment over a late lunch. bbiab18:00
openstackgerritMerged openstack/python-cratonclient master: Drop py34 target in tox.ini  https://review.openstack.org/44989418:02
openstackgerritMichael Porras proposed openstack/craton master: Adding wrapper functions to tools  https://review.openstack.org/44923018:36
thomasemHow was everyones' weekend?20:52
antonymcould have been longer :) good outside weather though20:56
thomasemHaha, that's fair.20:56
thomasemYeah, it was pretty nice out.20:56
anonymikeSeriously. I played kickball for a friends birthday party at a local park. It was beautiful outside20:59
*** VW_ has joined #craton21:00
thomasemNice21:00
*** VW_ has quit IRC21:02
*** VW_ has joined #craton21:02
*** VW has quit IRC21:04
openstackgerritgit-harry proposed openstack/craton master: Ensure JSON responses result from failure  https://review.openstack.org/44758021:19
antonymso with the rbac stuff, we should be able to have read only users right?21:22
thomasemI think that's the idea, but I'd ping jimbaker, as he's been ideating on it much longer and has more context.21:56
thomasemAlright, I'm outta here. Have a lovely evening/day!22:06
anonymikelater thomasem!22:19
anonymikeI'm gonna go for a run but I'll be on later22:19
*** VW has joined #craton22:22
*** VW_ has quit IRC22:22
*** Syed__ has quit IRC22:25
*** VW has quit IRC22:54
*** VW has joined #craton22:55
*** VW has quit IRC22:58

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!