| *** Syed__ has quit IRC | 00:05 | |
| *** wirehead_ has quit IRC | 02:12 | |
| *** wirehead_ has joined #craton | 02:13 | |
| *** anonymike has quit IRC | 02:17 | |
| *** anonymike has joined #craton | 02:17 | |
| *** anonymike has quit IRC | 02:33 | |
| *** openstackstatus has quit IRC | 02:58 | |
| *** openstackstatus has joined #craton | 03:00 | |
| *** ChanServ sets mode: +v openstackstatus | 03:00 | |
| *** tojuvone has joined #craton | 04:48 | |
| *** VW has joined #craton | 04:51 | |
| *** VW has quit IRC | 05:51 | |
| *** ediardo_ has joined #craton | 06:30 | |
| *** ediardo has quit IRC | 06:33 | |
| *** ediardo_ is now known as ediardo | 06:33 | |
| *** klindgren_ has joined #craton | 10:06 | |
| *** klindgren has quit IRC | 10:08 | |
| sulo | ok, so search by label is actually broken because of pagination stuff | 14:12 |
|---|---|---|
| sulo | it seems _pagination causes some weirdness with that query | 14:12 |
| sulo | i am not 100% why | 14:12 |
| thomasem | It did not occur to me that we had a generic variables API piped up in the codebase. I'm thinking I ought to switch Project over to using that? | 14:15 |
| thomasem | Ah, it's being added, nvm. | 14:18 |
| thomasem | I'll switch that over once that lands. | 14:18 |
| *** VW has joined #craton | 14:29 | |
| *** acabot has quit IRC | 14:42 | |
| sulo | sigmavirus: fyi: https://bugs.launchpad.net/craton/+bug/1662842 | 14:55 |
| openstack | Launchpad bug 1662842 in craton "Allow search by labels" [Critical,New] | 14:55 |
| sulo | i am not sure why it does this with labels join but not vars join | 14:56 |
| sulo | but its definitely the pagination function | 14:56 |
| sigmavirus | That's very weird | 14:57 |
| * sigmavirus looks at oslo_db's docs again | 14:57 | |
| sigmavirus | I dont' think there was anything about using it only on certain tables | 14:57 |
| sulo | i do see the "Unique keys not in sort_keys" get logged | 14:57 |
| sigmavirus | That said, I'm also chairing a meeting so give me time | 14:57 |
| sulo | sigmavirus: no worries | 14:57 |
| *** VW_ has joined #craton | 14:57 | |
| sigmavirus | sulo: Oh, so the default sort_keys are created_at and id | 14:57 |
| sigmavirus | because we need some keys to sort on | 14:58 |
| *** VW_ has quit IRC | 14:58 | |
| *** VW_ has joined #craton | 14:58 | |
| *** VW__ has joined #craton | 15:01 | |
| *** VW has quit IRC | 15:01 | |
| sigmavirus | sulo: yeah, this is why I wanted to add tests but when you started depending on it I thought I'd push those off for another patch | 15:01 |
| *** VW__ has quit IRC | 15:02 | |
| thomasem | May I get another core review of https://review.openstack.org/#/c/427777? | 15:02 |
| *** VW has joined #craton | 15:02 | |
| thomasem | and https://review.openstack.org/#/c/428996/ | 15:03 |
| thomasem | Both already have a +2 | 15:03 |
| *** VW has quit IRC | 15:03 | |
| *** VW_ has quit IRC | 15:03 | |
| *** VW has joined #craton | 15:04 | |
| *** VW has quit IRC | 15:05 | |
| *** VW has joined #craton | 15:05 | |
| sulo | sigmavirus: so it seems to have mixed up the query somehow https://gist.githubusercontent.com/sulochan/8faff00f20a13351d0e3f35f95f419b6/raw/50c6a5cec56cd1c22d3e76975a055e2b2638b744/gistfile1.txt | 15:05 |
| sigmavirus | wow that's huge | 15:06 |
| sulo | SELECT anon_1.devices_created_at AS anon_1_devices_created_at, anon_1.devices_updated_at AS | 15:07 |
| sulo | that cant be right | 15:07 |
| sigmavirus | yeah that was confusing to me | 15:08 |
| sigmavirus | I have no clue what SQLA is generating there | 15:08 |
| sigmavirus | I suspect that oslo_db's pagination helper wasn't designed to work on queries with variatic association | 15:09 |
| sigmavirus | perhaps we can't actually paginate that | 15:12 |
| sigmavirus | sulo: that line number in the _paginate function doesn't match up for what the latest patch has | 15:17 |
| sigmavirus | I'm sure this will still happen, but you could you rebase so I could figure out what exactly is generating that exception? | 15:17 |
| thomasem | haha git-harry, looks like cell-list doesn't work now, too. :P | 15:27 |
| git-harry | thomasem: damn it, I guess we need some more tests there. | 15:31 |
| thomasem | Yeah, including integration tests | 15:32 |
| * sigmavirus wishes our unit tests weren't so fragile | 15:39 | |
| *** anonymike has joined #craton | 15:39 | |
| thomasem | Could always mock out the function you're testing. They'll never break. | 15:39 |
| thomasem | :P | 15:39 |
| * thomasem shields face | 15:40 | |
| thomasem | So, why were those options available previously for those calls? | 15:46 |
| thomasem | in the CLI | 15:46 |
| *** acabot has joined #craton | 15:48 | |
| *** VW has quit IRC | 16:01 | |
| *** VW has joined #craton | 16:02 | |
| jimbaker | thomasem, agreed, we need to support tox -e integration for python-cratonclient | 16:02 |
| *** VW has quit IRC | 16:03 | |
| *** VW has joined #craton | 16:03 | |
| jimbaker | interesting about that query | 16:04 |
| *** VW_ has joined #craton | 16:05 | |
| *** VW_ has quit IRC | 16:05 | |
| jimbaker | it can be useful to control the specifics of this query by doing such things as suppressing eagerness in the query | 16:05 |
| *** VW has quit IRC | 16:05 | |
| *** VW has joined #craton | 16:05 | |
| jimbaker | http://docs.sqlalchemy.org/en/latest/orm/loading_relationships.html | 16:07 |
| jimbaker | it's worth pointing out that if we don't want the variable details, we shouldn't be doing any joins against variables and the variables association table | 16:09 |
| jimbaker | sigmavirus, sulo, ^^^ | 16:09 |
| sigmavirus | I'm reading those docs | 16:09 |
| sigmavirus | jimbaker: am I remembering wrong, or are labels not variable associations? | 16:10 |
| jimbaker | sigmavirus, labels are treated differently | 16:10 |
| *** jovon has joined #craton | 16:11 | |
| jimbaker | they are just an associated table for devices; we don't actually implement labels for other resources | 16:11 |
| jimbaker | that could make sense... labeling a region could be reasonable... but it's possible we might even stop using labels and just use variables. that's a more interesting question i think | 16:13 |
| thomasem | There was a subquery rendering bug, iirc, in SQLAlchemy related to eagerness | 16:19 |
| thomasem | I thought it was fixed, though | 16:19 |
| thomasem | :In Quark we attempted to use lazy='joined' to eagerly load, specifically, the 'associations' relationship on the IPAddress model and this was causing some functional tests to break due to a rendering bug where it would render 'ORDER BY address' in the outer query where it should have been 'ORDER BY <outer_query_alias>_address'.: | 16:21 |
| thomasem | Blurb from my bug about it way back when ^^ | 16:21 |
| jimbaker | thomasem, we have this pinning: SQLAlchemy<1.1.0,>=1.0.10 # MIT | 16:26 |
| thomasem | Yeah, I just saw that in my venv | 16:26 |
| thomasem | So, it _should_ be fixed. Anyway, I don't want to throw red herrings out here. Sorry. | 16:26 |
| jimbaker | latest sqlalchemy is at 1.1.5. maybe this requirement can be updated? | 16:27 |
| jimbaker | thomasem, so in this case, i think we don't need this fix because we should not be joining in these tables | 16:28 |
| sigmavirus | jimbaker: it should get auto-updated | 16:28 |
| jimbaker | even if we implement ?...details=all, that can be through a separate query | 16:28 |
| jimbaker | sigmavirus, but it is pinned to be before <1.1.0 | 16:29 |
| thomasem | Agreed. | 16:29 |
| thomasem | And we can definitely cross that bridge when we get there. | 16:29 |
| tojuvone | Evening | 16:29 |
| sigmavirus | jimbaker: we synchronize from upstream requirements. When that requirement is updated there, we'll receive it auto-magically | 16:30 |
| thomasem | git-harry: I'm going to remove those options from the project-* calls in my CLI patch. Are you working on any of the other clean up around the issue we were discussing earlier? | 16:31 |
| thomasem | project-list will only support --name, --detail, and --limit | 16:32 |
| thomasem | until further functionality is introduced into the API for it | 16:32 |
| thomasem | Oh, and --fields :) | 16:33 |
| git-harry | thomasem: sounds like a plan. I'm not currently, I'm still going through reviews | 16:33 |
| thomasem | Okay, cool | 16:33 |
| thomasem | I'll push this up and then do the same. | 16:34 |
| tojuvone | going to update nova spec for single and same url for admin and owner of VMs in host. Just need to take care of having policy working on Craton side | 16:34 |
| tojuvone | https://review.openstack.org/428070/ | 16:35 |
| tojuvone | Now spec is not mentioning Craton, I do not know if it should. | 16:35 |
| tojuvone | Anyhow planning to have there a link for the main idea of "planned host maintenance" and that will mention plan to use Craton | 16:37 |
| tojuvone | Need to get that spec work going if to have still in Pike | 16:38 |
| tojuvone | Thoughts? | 16:41 |
| *** Syed__ has joined #craton | 16:42 | |
| sulo | tojuvone: where do you see craton coming into play for that spec ? | 16:56 |
| tojuvone | sulo, url that will be set there will point to craton | 16:57 |
| tojuvone | but it can be just generic URL | 16:57 |
| sulo | tojuvone: ah, but that could be any url right ? | 16:57 |
| tojuvone | sure | 16:57 |
| tojuvone | the point being just if pointing to Craton | 16:58 |
| tojuvone | I would keep single url instead like it is currently proposed there | 16:59 |
| tojuvone | if set there url like: https://gerrit.opnfv.org/gerrit/28379 | 16:59 |
| tojuvone | that is same that will be shown to owner of server/vm | 16:59 |
| tojuvone | ups | 16:59 |
| tojuvone | lol, wrong link' | 17:00 |
| tojuvone | http://${CRATON_IP}:8080/v1/hosts/{id}/host_details | 17:00 |
| sulo | tojuvone: so you want nova to know how to set the url ? | 17:00 |
| tojuvone | just need policy to work when implementing to craton | 17:00 |
| tojuvone | that server owner and admin can query that | 17:01 |
| tojuvone | Now admin will set the url that points to Craton | 17:01 |
| jimbaker | hi, we should probably get started on our craton meeting | 17:01 |
| tojuvone | sorry | 17:01 |
| sulo | tojuvone: ok, not sure i am following this right now | 17:01 |
| sulo | lets chat more after the meeting | 17:02 |
| jimbaker | sulo, or during the meeting | 17:02 |
| jimbaker | this is a good topic | 17:02 |
| jimbaker | #startmeeting craton | 17:02 |
| openstack | Meeting started Thu Feb 9 17:02:25 2017 UTC and is due to finish in 60 minutes. The chair is jimbaker. Information about MeetBot at http://wiki.debian.org/MeetBot. | 17:02 |
| sulo | sure, i dont mind | 17:02 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 17:02 |
| openstack | The meeting name has been set to 'craton' | 17:02 |
| jimbaker | #chair sigmavirus sulo jimbaker | 17:02 |
| openstack | Current chairs: jimbaker sigmavirus sulo | 17:02 |
| jimbaker | #link https://etherpad.openstack.org/p/craton-meetings | 17:02 |
| jimbaker | #topic Roll Call | 17:03 |
| thomasem | o/ | 17:03 |
| jovon | o/ | 17:03 |
| Syed__ | o/ | 17:03 |
| * sigmavirus can't chair this today | 17:03 | |
| jimbaker | sigmavirus, np, i will chair | 17:03 |
| jimbaker | and hopefully keep short | 17:03 |
| git-harry | hi | 17:03 |
| tojuvone | hi | 17:03 |
| sulo | hello | 17:04 |
| jimbaker | so at some point we agreed that we are not supposed to have this meeting if we didn't have an agenda in advance... | 17:05 |
| jimbaker | but given lots of stuff happening... i believe we should disregard and have a short meeting regardless :) | 17:05 |
| jimbaker | everyone ok with that? | 17:06 |
| Syed__ | yep | 17:06 |
| jovon | indeed | 17:06 |
| jimbaker | awesome. let's just move to our standing agenda, and fit in what we need to communicate | 17:06 |
| thomasem | yep | 17:06 |
| jimbaker | #topic Action Items from Last Meeting | 17:07 |
| jimbaker | http://eavesdrop.openstack.org/meetings/craton/2017/craton.2017-02-06-15.00.log.html | 17:08 |
| jimbaker | bunch of action items. so i haven't taken care of doing the dusty's req doc as launchpad | 17:09 |
| jimbaker | bug | 17:09 |
| thomasem | I have not gotten around to the BP yet this week. I took the bulk resources endpoint bug, but that's lowered in priority right now. | 17:09 |
| thomasem | Had some good discussion around the latter, though, and will continue once we get some more critical things done. | 17:09 |
| jimbaker | thomasem, agreed. we will want to figure out deployment specifics, and if you want to continue to drive point on that, that would be great | 17:10 |
| thomasem | You bet | 17:10 |
| jimbaker | re dusty's doc, we got some very useful triaging in our discussion with toan and antonym | 17:11 |
| jimbaker | which to recap for others here (such as tojuvone) | 17:11 |
| jimbaker | we are trying to figure out for rackspace if we can immediately use craton for some rackspace support needs | 17:12 |
| jimbaker | separately i discussed with farid getting these into launchpad so we can track | 17:12 |
| jimbaker | i believe as your project lead :) that the most important thing we can do is see craton get adopted for real usage. it will get us in front of real problems, which this project is all about solving | 17:13 |
| jimbaker | *so earlier is better here* | 17:13 |
| jimbaker | so that's my action item and thomasem's | 17:14 |
| jimbaker | sigmavirus is out this point, but we can discuss that later | 17:14 |
| jimbaker | sulo, it was mentioned as info, but it could be considered an implicit action item as well given priority | 17:15 |
| * sigmavirus is here | 17:15 | |
| sigmavirus | just also in another meeting | 17:15 |
| thomasem | Yeah. We have some discussions that need be had with another contender for these use-cases. | 17:15 |
| sigmavirus | and can't focus on chairing this one :) | 17:15 |
| jimbaker | sigmavirus, no worries. do you want to report on your action item at this time? | 17:15 |
| jimbaker | sulo, also please give us feedback on Parent/Child Relationship work is the highest priority work for this week (was marked as #info) | 17:16 |
| sigmavirus | Just constantly spinning between projects. Everytime I come back to craton I'm working on testing things | 17:16 |
| sigmavirus | (Things I have in flight) | 17:16 |
| jimbaker | sigmavirus, and that's your action item right, testing the CLI - hopefully will reduce some of that spin as well | 17:17 |
| Syed__ | cool | 17:18 |
| jimbaker | ok, let me report on the last high priority item, that might have been marked as #action; more work on the getters/setters in the client/CLI, mostly by clean up; and addition of delete support | 17:19 |
| jimbaker | i will push that up momentarily. did find that our DELETE on variables doesn't actually do anything, so i have a patch for that as well | 17:20 |
| Syed__ | Sounds good | 17:20 |
| Syed__ | for me, i have been looking into https://bugs.launchpad.net/craton/+bug/1662842 | 17:20 |
| openstack | Launchpad bug 1662842 in craton "Allow search by labels" [Critical,New] | 17:20 |
| jimbaker | i like the curious symmetry of "addition of delete" :) | 17:20 |
| Syed__ | and have some patches in the review, need to write functional test for update calls of projects and users and that patch will be good to go | 17:21 |
| jimbaker | Syed__, that's cool, but we are currently focused on action items | 17:21 |
| Syed__ | Sounds good, i guess once bugs are raised and referenced properly, helps a lot in picking those up | 17:21 |
| jimbaker | Syed__, right, that was part of my conversation with farid is everyone doing a better job there. which seems to have improved. and more importantly, making it easier to use the bug info to get at useful reporting | 17:22 |
| jimbaker | so we will work on that, and that's very much related to the "requirements as a bug/feature" (or better, bugs) action item i started off with | 17:23 |
| Syed__ | yeap and that would be great once we have that in place | 17:24 |
| Syed__ | +1 | 17:24 |
| jimbaker | awesome. i think that's it for action items, which all seem to be in flight at this point | 17:24 |
| jimbaker | #topic Stand-up | 17:25 |
| jimbaker | #topic Stand-up - Syed | 17:25 |
| jimbaker | (let it be noted that the meetbot seems currently unresponsive. but it's in the log) | 17:26 |
| Syed__ | Sounds good | 17:26 |
| jimbaker | Syed__, anything you want to add to your earlier comments? | 17:26 |
| Syed__ | No i guess thats all i am doing. Will be looking into picking some CLI bug apart from other things i am doing | 17:27 |
| jimbaker | Syed__, that would be great | 17:27 |
| jimbaker | #topic Stand-up - git-harry | 17:27 |
| jimbaker | fwiw, git-harry, i'm still thinking about my response to your spec on variables in https://review.openstack.org/#/c/428190/ | 17:28 |
| jimbaker | and i see you have some changes now for review | 17:28 |
| git-harry | I've got some patches in review for tidying up the variables a bit. Currently I'm going through the review queue. If there's high priority stuff that needs picking up, point me in the direction of it and I'll take a look. | 17:29 |
| git-harry | jimbaker: that stuff is just clean up to make any future changes easier | 17:29 |
| git-harry | done | 17:29 |
| jimbaker | git-harry, got it. dupe removal is good. i do have concerns about some of the API changes you proposed, namely around PUT | 17:30 |
| jimbaker | for variables | 17:30 |
| jimbaker | we can discuss further | 17:30 |
| git-harry | okay | 17:30 |
| jimbaker | #topic Stand-up - jovon | 17:30 |
| jimbaker | we will circle back to jovon | 17:31 |
| jimbaker | #topic Stand-up - thomasem | 17:31 |
| jovon | currently working on documenting users, projects, and devices. | 17:31 |
| jimbaker | jovon, +1 | 17:32 |
| thomasem | Need reviews: | 17:32 |
| thomasem | https://review.openstack.org/427777 | 17:32 |
| thomasem | https://review.openstack.org/428996 | 17:32 |
| thomasem | Been keeping those tidy | 17:32 |
| thomasem | Currently working on wiring up Cloud resource in between Project and Region | 17:32 |
| thomasem | Going through the review queue as well | 17:32 |
| jimbaker | thomasem, awesome, sounds like you're keeping busy! | 17:32 |
| thomasem | I try! | 17:33 |
| jimbaker | i expect any number of conflicts here, but we will get the changes in | 17:33 |
| thomasem | Other than that, had a lot of meetings distracting me this week, but that won't continue much. | 17:33 |
| thomasem | Need to write up a bug for the issue git-harry found this morning with an API change that went in, so putting that on my list right now. | 17:34 |
| thomasem | done | 17:34 |
| jovon | question. are we planning to create a separate definition for devices? as it is currently nested within net devices? | 17:34 |
| jimbaker | thomasem, thanks for reminding us of that part of standup protocol | 17:35 |
| thomasem | You bet! | 17:35 |
| jimbaker | jovon, consider it like an abstract class | 17:35 |
| jovon | understood | 17:35 |
| jimbaker | it should not be directly instantiated | 17:35 |
| jovon | just curious | 17:36 |
| jimbaker | not certain if sqlalchemy would prevent, but our apis certainly do not support | 17:36 |
| jimbaker | jovon, no worries. but let's move to open discussion if more. everyone, please remember your open discussion topics! | 17:36 |
| jimbaker | #topic Stand-up tojuvone | 17:37 |
| tojuvone | ok.. | 17:37 |
| tojuvone | I want to make "planned host maintenance" using Craton | 17:37 |
| tojuvone | Adding namespace, something like: http://${CRATON_IP}:8080/v1/hosts/{id}/host_details | 17:37 |
| tojuvone | I want to add this url link to Nova | 17:37 |
| tojuvone | Now in Craton namespace host_details is added by "admin" | 17:38 |
| tojuvone | but in Nova side also owner of server/VM will see the same link | 17:38 |
| tojuvone | I guess we can handle the policy somehow in Craton to make this possible? | 17:38 |
| tojuvone | I want to update my Nova spec with different url to link to Craton than it currently state there: https://review.openstack.org/428070/ | 17:38 |
| jimbaker | tojuvone, so we should have the level of support necessary for this in rbac | 17:38 |
| jimbaker | but rbac + namespaces is not something we have considered just yet | 17:38 |
| jimbaker | having said that: i think namespaces are exactly the way one would want to organize some set of variables for rbac | 17:39 |
| jimbaker | we have considered separately that the prefix of a variable is something we would directly control with rbac, and is mentioned briefly in the rbac bp that is posted | 17:40 |
| jimbaker | (this follows similar practice in tooling like zookeeper or consul) | 17:40 |
| sulo | tojuvone: your nova service permissions are different from craton .. unless you are using all within the same keystone domain/context | 17:41 |
| jimbaker | so that is the extended answer to yes | 17:41 |
| jimbaker | sulo, and that's a great question, what that cross service permission model means | 17:41 |
| sulo | tojuvone: i am not sure why you are worried about policy right now .. i think what should matter here is how does nova know to create the link | 17:41 |
| sulo | tojuvone: the answer is possibly, it doesnt | 17:41 |
| tojuvone | We need to set it there | 17:42 |
| tojuvone | when making http://${CRATON_IP}:8080/v1/hosts/{id}/host_details | 17:42 |
| sulo | so unless this url is something you are manually looking to set in nova .. i dont see how this will work | 17:42 |
| sulo | tojuvone: what is id ? | 17:42 |
| sulo | id here is craton id .. nova doesnot know that | 17:42 |
| jimbaker | sulo, so we see some similar sort of bidirectionality required in webhooks | 17:42 |
| tojuvone | "id" I guess we have in Craton host the "id" | 17:43 |
| sulo | jimbaker: how do you mean ? | 17:43 |
| jimbaker | but can i ask we move this to the open discussion? | 17:43 |
| sulo | tojuvone: yes, but how does nova know that id ? | 17:43 |
| sulo | tojuvone: are you going to manually push the url for each host ? | 17:43 |
| jimbaker | please remember this topic, we can get back to it | 17:43 |
| jimbaker | sulo, since you are here | 17:44 |
| tojuvone | yes, need to push it to Nova | 17:44 |
| jimbaker | #topic Stand-up sulo | 17:44 |
| sulo | jimbaker: iirc nova does not have any webhooks, neither do we | 17:44 |
| jimbaker | sulo, no, but it's in the same idea. what can we learn? | 17:44 |
| jimbaker | : | 17:44 |
| jimbaker | :) | 17:44 |
| sulo | I am just chasing a sqlalchemy bug thats causign search by labels to fail | 17:44 |
| sulo | its on pagination function | 17:45 |
| sulo | but looks like something weird on sqla | 17:45 |
| jimbaker | sulo, ok, is this something we can ignore for now | 17:45 |
| sulo | not unelss we want search by labels to work | 17:45 |
| jimbaker | by avoiding that join? | 17:45 |
| sulo | join ? | 17:45 |
| jimbaker | to the labels table | 17:46 |
| jimbaker | but back up | 17:46 |
| jimbaker | the relevant question is search by labels | 17:46 |
| sulo | not sure i follow, why woud that matter ? | 17:46 |
| jimbaker | so sure, that needs to work :) | 17:46 |
| sulo | although, it is freaking out on labels join .. we do join on variables too .. which seems to work | 17:47 |
| sulo | well, its not exactly join its freakign out on | 17:47 |
| jimbaker | sulo, earlier we were discussing unnecessary joins and how that was impacting pagination. specifically variable details when such details were not requested (not even implemented!) | 17:47 |
| sulo | its the part where pagination is adding limit | 17:47 |
| sulo | jimbaker: thats different ? .. we are eager loading | 17:48 |
| sulo | we can chagne that to select load | 17:48 |
| sulo | but that wont fix this problem | 17:48 |
| jimbaker | sulo, and eager loading can be great. but not here | 17:48 |
| jimbaker | let's back up. i wanted to find out earlier about parent/child and supporting cabinet to container views | 17:49 |
| jimbaker | since this is the highest priority task we have right now, other than variable support down to the CLI | 17:49 |
| sulo | parent_id is a easy fix, ill push a review soon | 17:49 |
| sulo | i need this pagination work to get done for that | 17:50 |
| jimbaker | sulo, ok, just want to make sure we are addressing | 17:50 |
| sulo | since it piggybacks on the links support added by ian | 17:50 |
| jimbaker | ah, ok | 17:50 |
| jimbaker | and not so separable | 17:50 |
| jimbaker | as coded now | 17:50 |
| sulo | yeah its tied to this code | 17:50 |
| sulo | my change is based on this work | 17:50 |
| sulo | everything works .. except the seach by labels test is failing | 17:51 |
| jimbaker | got it. i was hearing pagination, and thinking, that's not in any of our immediate reqs | 17:51 |
| jimbaker | even though we know it's super important for the future | 17:51 |
| sulo | anyway need to figure out why this is happening | 17:51 |
| sulo | as it seems there might be other conditions this can cause pain with | 17:51 |
| jimbaker | yes. let's spend some more time on this | 17:52 |
| jimbaker | but we may need to separate out the link code and defer pagination if too much | 17:52 |
| jimbaker | just want to give us that out | 17:52 |
| sulo | yeah so the search function i had added while back | 17:52 |
| sulo | and without the pagination bit it works | 17:52 |
| jimbaker | got it | 17:52 |
| sulo | its a much have feature righ tnow | 17:53 |
| sulo | example: seach for all compute nodes | 17:53 |
| sulo | or api nodes etc | 17:53 |
| sulo | by labels | 17:53 |
| jimbaker | in terms of vars | 17:53 |
| sulo | more than one service on a host | 17:53 |
| sulo | identified by host labels | 17:54 |
| jimbaker | right, which is more efficient with labels than going into json | 17:54 |
| jimbaker | #info clearly we have gone to open discussion... | 17:54 |
| jimbaker | sulo, let me state something provocative/time saving: i do wonder if we will end up not caring about labels after all | 17:55 |
| sulo | how so ? | 17:56 |
| jimbaker | what would happen if we just removed labels? | 17:56 |
| sulo | we wont have a nice way to tag hosts ? vars are not well suited for this ? | 17:56 |
| thomasem | Yes... what WOULD happen if we just removed labels? | 17:56 |
| jimbaker | could we do the same sort of queries, by using variables? | 17:56 |
| sulo | maybe | 17:56 |
| sulo | it might be ugly | 17:57 |
| jimbaker | sulo, especially given namespaces, which we will be implementing | 17:57 |
| sulo | example: host1 needs 3 labels "compute", "api", "scheduler" | 17:57 |
| sulo | even with namespace | 17:57 |
| jimbaker | so let's say we have the labels namespace | 17:57 |
| sulo | labels (namespace) will have which k:v pair | 17:57 |
| jimbaker | labels/compute=true | 17:58 |
| jimbaker | basically we get everything we need | 17:58 |
| sulo | yeah, i dunno how that seach will look like | 17:59 |
| jimbaker | yes, right now our variables search is expensive, because it's not indexed. but it should be cheap with the right json indexing | 17:59 |
| jimbaker | in this case, it might be still be fine, since we only care about the presence of keys, not even the values | 17:59 |
| thomasem | It would be weird to have labels/compute=false | 18:00 |
| thomasem | and treat it as a compute. | 18:00 |
| sulo | yeah | 18:00 |
| sulo | i mean we can make it work if desired | 18:00 |
| jimbaker | thomasem, but that's why we have the labels namespace | 18:01 |
| jimbaker | i should point out that the usual representation of a set is a dict... | 18:01 |
| sulo | yeah i dunno | 18:01 |
| sulo | what happens when someone does compute=false | 18:01 |
| sulo | it wil mean we have to query on k and filter on value | 18:01 |
| jimbaker | cpython may have an alternative implementation; but that's how it's done in java for example: key=true | 18:02 |
| jimbaker | implicitly | 18:02 |
| jimbaker | sulo, so we constrain with the namespace | 18:02 |
| jimbaker | we can use convention for now | 18:02 |
| sulo | like ? | 18:02 |
| jimbaker | but one thing tojuvone and i discussed is that if we had a "maintenance/" namespace, we could constrain the allowed transitions | 18:03 |
| sulo | so what do we gain from doing k:v labels, vs doing labels the way it is now ? | 18:03 |
| sulo | i think if we gain more .. then its a no brainer | 18:03 |
| jimbaker | so one can just set keys to arbitrary values | 18:03 |
| jimbaker | sulo, so we get k:v labels for free | 18:03 |
| jimbaker | and we implement less db code, and complex queries | 18:04 |
| sulo | i get that .. but whats the use case for that | 18:04 |
| jimbaker | which are currently breaking | 18:04 |
| jimbaker | also, labels don't work on anything but devices | 18:04 |
| sulo | right | 18:04 |
| sulo | then its the equivalent of getting rid of labels | 18:04 |
| tojuvone | Actually might extend /maintenance to more generic /host_details | 18:04 |
| sulo | k:v is just vars | 18:05 |
| jimbaker | yes! | 18:05 |
| sulo | we can still keep the endpoint etc .. | 18:05 |
| sulo | not saying i am convinced this is good yet | 18:05 |
| jimbaker | sulo, because it knows about the labels/ namespace | 18:05 |
| sulo | yeah | 18:05 |
| sulo | i am mostly worried about perf hit when there is large data seach on vars | 18:05 |
| sulo | but if we dont think that will be a probl | 18:06 |
| sulo | then its fine | 18:06 |
| jimbaker | sulo, i think it's less of a problem than a multitable join | 18:06 |
| tojuvone | I need to go, but be back in as hour or so | 18:06 |
| sulo | jimbaker: you will have that join regardless .. because varxmixin | 18:06 |
| jimbaker | tojuvone, thanks for your participation | 18:06 |
| sulo | jimbaker: currently we can get rid of vars in labels seach | 18:06 |
| sulo | thats a differnt issue | 18:06 |
| jimbaker | sulo, there are joins regardless :) | 18:07 |
| jimbaker | just fewer joins | 18:07 |
| jimbaker | in a complex query | 18:07 |
| sulo | heh sure | 18:07 |
| jimbaker | so consider labels + variables in the search query. with pagination. uh-oh | 18:07 |
| * thomasem needs to go investigate the labels implementation | 18:08 | |
| sulo | jimbaker: yeah, i think we should experiement for sure | 18:08 |
| jimbaker | thomasem, it is simple, and a delight to read | 18:08 |
| jimbaker | but it's too simple imho | 18:08 |
| sulo | this is somethign we've talked about for a while anyway | 18:08 |
| sulo | maybe its time to just try it out | 18:08 |
| jimbaker | sulo, yeah, we kept on simplifying variables | 18:08 |
| jimbaker | sorry, simplifying *labels* | 18:09 |
| jimbaker | ultimate simplification: remove them | 18:09 |
| sulo | well anyway, in terms of priority work to get done | 18:09 |
| sulo | we needs labels right now | 18:09 |
| jimbaker | but again if we can just use convention now | 18:09 |
| jimbaker | labels/compute=true | 18:10 |
| jimbaker | then figure out how to make that cleaner later... | 18:10 |
| jimbaker | then we can get this done now | 18:10 |
| sulo | thats a much bigger change right now ? | 18:11 |
| sulo | that needs change on all layers including api maybe | 18:11 |
| jimbaker | but vars work now | 18:11 |
| sulo | will have to look | 18:11 |
| sulo | labels works too | 18:11 |
| jimbaker | just not enough | 18:11 |
| jimbaker | anyway... good discussion | 18:11 |
| sulo | well its pretty bad if we are getting rid of label because somethig else is breaking it | 18:11 |
| sulo | lets figure out why paginaiton is breaking it | 18:12 |
| sulo | rather than getting rid of it | 18:12 |
| jimbaker | sulo, so i would first relax the requirement on sqlalchemy | 18:12 |
| jimbaker | and use 1.1.5 | 18:12 |
| jimbaker | as we earlier discussed | 18:12 |
| jimbaker | on this channel | 18:12 |
| jimbaker | thomasem mentioned that he saw a similar bug | 18:12 |
| jimbaker | i suggest we wrap up our meeting. we are well over | 18:13 |
| jimbaker | discussion can continue of course | 18:13 |
| jimbaker | seeing no objections.... | 18:13 |
| thomasem | That bug might have been fixed, btw. I wanted to throw out an idea of where to look, but I haven't gone and dug in very deep on it. | 18:13 |
| jimbaker | #endmeeting | 18:13 |
| openstack | Meeting ended Thu Feb 9 18:13:58 2017 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 18:14 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/craton/2017/craton.2017-02-09-17.02.html | 18:14 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/craton/2017/craton.2017-02-09-17.02.txt | 18:14 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/craton/2017/craton.2017-02-09-17.02.log.html | 18:14 |
| thomasem | That bug was in SQLAlchemy < 1.0, iirc. I don't know if it's still there. | 18:14 |
| jimbaker | thomasem, so very early SA | 18:14 |
| thomasem | I seem to remember installing 1.0 caused it to disappear, but that was years ago. | 18:14 |
| jimbaker | but still worthwhile trying in the 1.1 series | 18:14 |
| jimbaker | just to see if it goes away | 18:14 |
| thomasem | Gotcha | 18:15 |
| sulo | so check this https://gist.github.com/sulochan/28d2827c80d580091c9c5bb4e706b4a7 | 18:16 |
| jimbaker | that it happens to come because of a certain variable modeling is not so relevant. it's the joins | 18:16 |
| sulo | that function is ..https://github.com/openstack/oslo.db/blob/master/oslo_db/sqlalchemy/utils.py#L138 | 18:16 |
| sulo | so it looks like i get some werid behavior when doing limit | 18:16 |
| sulo | if you commit out line 261 where its adding the limit, then everything is happy | 18:17 |
| sulo | so this must be some sqlalchemy weirdness | 18:17 |
| jimbaker | stuff like that makes me want to have a simple path for the usual query | 18:18 |
| jimbaker | and just do the limit in python | 18:18 |
| sulo | yeah | 18:18 |
| jimbaker | (ordering still has to be done of course) | 18:18 |
| jimbaker | the usual: top N results are the most common query | 18:18 |
| jimbaker | optimize for that first before anything special for the next set of pages | 18:18 |
| sulo | anyway afk for now | 18:19 |
| *** VW has quit IRC | 18:20 | |
| jimbaker | sulo, np, and thanks! | 18:20 |
| jimbaker | great meeting everyone, much appreciated! | 18:20 |
| *** VW has joined #craton | 18:20 | |
| *** VW has quit IRC | 18:20 | |
| *** VW has joined #craton | 18:21 | |
| jovon | great job team! (just a little moral support) :) | 18:21 |
| jimbaker | :) | 18:24 |
| *** VW has quit IRC | 18:42 | |
| *** VW has joined #craton | 18:42 | |
| *** VW has quit IRC | 18:47 | |
| *** VW has joined #craton | 18:50 | |
| tojuvone | back for a while | 19:06 |
| thomasem | wb! | 19:07 |
| tojuvone | thx | 19:07 |
| *** VW_ has joined #craton | 19:08 | |
| *** VW_ has quit IRC | 19:10 | |
| *** VW_ has joined #craton | 19:10 | |
| *** VW_ has quit IRC | 19:11 | |
| tojuvone | I think I know now how I will change my spec | 19:11 |
| *** VW_ has joined #craton | 19:11 | |
| tojuvone | Can deal with the details how things will be done in Craton side later | 19:12 |
| *** VW has quit IRC | 19:12 | |
| tojuvone | Btw, going to be in London from Saturday till Tuesday and also off holiday on week 8. | 19:15 |
| jimbaker | tojuvone, next week? you can potentially meet up with sulo and git-harry - sulo i know is near heathrow, and i believe git-harry is too | 19:18 |
| tojuvone | jimbaker, Otherwise nice, but I go with the family :) | 19:24 |
| tojuvone | Harry Potter studios and stuff :D | 19:25 |
| tojuvone | yes, would be so great to meet otherwise | 19:27 |
| jimbaker | tojuvone, awesome, i enjoyed visiting the harry potter attraction in orlando a few years back. should be fun! | 19:49 |
| Syed__ | git-harry: you around ? | 19:53 |
| *** VW_ has quit IRC | 20:00 | |
| *** VW has joined #craton | 20:01 | |
| thomasem | ruh roh, network troubles | 20:13 |
| -openstackstatus- NOTICE: Restarting gerrit due to performance problems | 20:18 | |
| thomasem | aha | 20:19 |
| thomasem | That's what it is | 20:19 |
| *** VW has quit IRC | 20:29 | |
| *** VW has joined #craton | 20:29 | |
| *** harlowja_ has joined #craton | 20:35 | |
| *** harlowja has quit IRC | 20:37 | |
| thomasem | I must run to an appointment. bbiab | 20:41 |
| jimbaker | np | 20:43 |
| *** VW has quit IRC | 20:48 | |
| *** VW has joined #craton | 21:24 | |
| *** VW_ has joined #craton | 21:26 | |
| *** VW_ has quit IRC | 21:26 | |
| *** VW has quit IRC | 21:26 | |
| *** VW has joined #craton | 21:27 | |
| *** harlowja_ has quit IRC | 21:58 | |
| *** harlowja has joined #craton | 22:05 | |
| jimbaker | thomasem, please refactor your change for https://review.openstack.org/#/c/427777/ based on the work i just +2'ed for the var refactoring | 22:20 |
| jimbaker | just love it when we can remove code, and make it easier to implement changes like what is needed here for project vars support | 22:20 |
| jimbaker | (i have my own minor refactoring to now do based on this change, with respect to properly support deletion) | 22:21 |
| *** jovon has quit IRC | 22:22 | |
| thomasem | jimbaker: Sounds good. I was going to come in behind and do that anyway, so might as well do it now since that got in. :) | 22:43 |
| jimbaker | thomasem, i was only helping you :) | 22:52 |
| jimbaker | and this minimizes a fix i need to make as well. plus better testing i can use | 22:52 |
| thomasem | jimbaker: oh, yeah! No worries. | 22:56 |
| jimbaker | i'm just glad that the vars work uses a mixin approach in the model; now the api; and soon the client. this is going to make var support easier to manage, especially as we start adding support like namespaces | 22:58 |
| thomasem | +1-- | 22:58 |
| thomasem | +100* | 22:58 |
| thomasem | lol | 22:58 |
| jimbaker | another thing we will soon want to decide is if we want to remove labels. i'm increasingly in favor of this, given that vars work just as well | 23:01 |
| jimbaker | one fewer abstraction to implement, and vars are more pervasive. right now labels only support devices, and i think that's a legacy of an older design idea | 23:01 |
| jimbaker | another consideration would be searchlight support - if we just need this with vars, that just makes it easier | 23:02 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!