Wednesday, 2017-02-22

*** VW has joined #craton01:15
*** VW has quit IRC01:20
*** openstackgerrit has quit IRC01:23
*** VW has joined #craton02:08
*** VW has quit IRC02:13
*** VW has joined #craton02:14
*** valw has joined #craton02:59
*** VW has quit IRC03:11
*** VW has joined #craton03:17
*** valw has quit IRC03:26
*** openstackgerrit has joined #craton05:06
openstackgerritJim Baker proposed openstack/craton master: Only generate prev and next links if necessary.  https://review.openstack.org/43676005:06
*** pwnall1337 is now known as zz_pwnall133705:12
*** VW has quit IRC06:39
suloo/ hola09:39
* sulo feels like i missed a busy meeting week this week ;)10:26
*** izaakk has quit IRC13:02
*** izaakk has joined #craton13:04
openstackgerritsulochan-acharya proposed openstack/craton master: Change config default to use pymysqldb  https://review.openstack.org/43691513:18
suloFYI: https://bugs.launchpad.net/craton/+bug/166689013:25
openstackLaunchpad bug 1666890 in craton "Query by filter should raise 404 when no entry found" [High,New] - Assigned to Sulochan Acharya (sulochan-acharya)13:25
*** tojuvone has quit IRC13:29
openstackgerritMerged openstack/craton master: Change config default to use pymysqldb  https://review.openstack.org/43691513:36
*** VW has joined #craton13:54
thomasemHey there13:58
*** valw has joined #craton14:00
*** VW has quit IRC14:14
*** VW_ has joined #craton14:14
openstackgerritThomas Maddox proposed openstack/craton master: WIP: Add Cloud resource  https://review.openstack.org/43170514:14
*** valw has quit IRC14:16
*** valw has joined #craton14:20
sulohey thomasem14:27
*** david-lyle has joined #craton14:27
faridhi guys14:32
sulohi farid14:33
faridhi sulo how's it going14:34
*** VW has joined #craton14:46
openstackgerritThomas Maddox proposed openstack/craton master: WIP: Add Cloud resource  https://review.openstack.org/43170514:46
*** VW_ has quit IRC14:49
*** VW_ has joined #craton14:49
*** VW has quit IRC14:50
openstackgerritThomas Maddox proposed openstack/craton master: Add PyMySQL to project requirements  https://review.openstack.org/43696314:58
*** VW_ has quit IRC14:59
thomasemsulo sigmavirus ^^14:59
thomasemMaster is broken right now because of that14:59
thomasemAt least the test env we've been using14:59
*** VW has joined #craton14:59
*** david-lyle_ has joined #craton15:00
thomasemThanks!!15:00
*** david-lyle has quit IRC15:01
sigmavirusI broke it, I better fast-track the fix =P15:01
sigmavirusProbably should have run the functional tests on sulo's patch before I approved it15:01
thomasemThey're getting moved to gating, though, right?15:02
thomasemso that'll catch it in the future. :)15:02
sigmavirusRight15:02
thomasemExcellent15:02
sigmavirusThat said, I think most of #openstack-infra is at the PTG15:02
thomasem:(15:03
sigmavirusSo if someone wants to nag fungi or someone else in there to approve that patch, that would greatly accelerate things =)15:03
openstackgerritMerged openstack/craton master: Add PyMySQL to project requirements  https://review.openstack.org/43696315:05
*** Syed__ has joined #craton15:11
thomasemSyed__: out of curiosity, how do you feel about IRC bouncers?15:14
Syed__I used to have weeChat, but not now15:15
openstackgerritThomas Maddox proposed openstack/craton master: WIP: Add Cloud resource  https://review.openstack.org/43170515:15
Syed__thinking to either get limeChat installed, or maybe pay for irc cloud, since its 5$ monthly i guess thomasem15:16
thomasemSyed__: I just popped a ZNC bouncer on a very tiny AWS node.15:16
thomasemhttps://github.com/thomasem/docker-znc15:16
thomasemAnd connect via LimeChat.15:17
thomasemJust an idea if you don't want to spend $. :)15:17
Syed__that sounds good15:17
Syed__maybe later i might need your help, will set it up today15:17
thomasemYou bet. Feel free. I've set up tons of those things.15:18
thomasemBy the way, that repo maps to https://hub.docker.com/r/tmaddox/znc/15:19
thomasemSo, you don't need to pull it down or anything. :P15:19
thomasemJust docker pull the image and you should be set15:19
sulothomasem: sigmavirus: oh my bad. I though i had added that to requirements. :(15:19
thomasemsulo: no worries! Fixed promptly, little issue.15:19
thomasemsulo: after chatting with jimbaker yesterday, I got the impression you'd have some stuff to talk about regarding this bug: https://bugs.launchpad.net/craton/+bug/166669915:32
openstackLaunchpad bug 1666699 in craton "able to create unexpected relationships via API" [Undecided,New]15:32
jimbakersulo, re https://review.openstack.org/#/c/436760/ - agreed on those tests. i will look at some modifications of what is already there15:32
jimbakerzz_pwnall1337, https://review.openstack.org/#/c/436760/ should hopefully work around the problem you saw, which is the client currently fails due to try to page regions and projects15:33
thomasemI also imagine sigmavirus would like to chime in on that bug15:33
jimbakerthomasem, something we will need to resolve of course for real for projects (i guess this is related to https://review.openstack.org/#/c/435599/)15:34
jimbakerneed to look into regions as well15:34
thomasemjimbaker: regions could be due to https://review.openstack.org/#/c/435595/15:35
thomasemIf there's a default of None in the schema, it breaks the generated links15:35
thomasemfor params15:35
jimbakerthomasem, re https://bugs.launchpad.net/craton/+bug/1666699, yeah we need to add the checks15:35
openstackLaunchpad bug 1666699 in craton "able to create unexpected relationships via API" [Undecided,New]15:35
* thomasem sighs15:36
jimbakerthomasem, ahh, somehow missed that change. i will look into combining with mine15:36
sulo thomasem: oh ?15:38
sulothomasem: do we need extra checks ?15:38
jimbakersulo, that's the net of it15:38
sulook15:39
jimbakerit's similar to the bug that git-harry found with respect to being able to create cycles in the parent relationship15:39
suloyeah15:39
suloyou can create parent_id from anywhere too15:39
jimbakerwhich if it happens, corrupts the db in the sense that the api calls fail15:39
jimbakereasy enough to fix with a simple check. the denormalization we have done for the relationships for cloud, region, and cell have somewhat similar checks we need to do15:41
thomasemsulo: I disagree with our data model, is the gist of that, really. I realize short-term it may have to be checks, but I don't think we're asking for pain with this data model.15:44
thomasemI think we're asking for pain with this data model*15:44
thomasemHaha, wires crossed.15:44
*** david-lyle_ has quit IRC15:44
sulothomasem: oh ok :) how so ?15:44
thomasemTo be clear, I think we need to normalize and leverage the integrity constraints that the DB is good at.15:44
sulook, can we do that with cells ?15:45
thomasemrelational table15:45
thomasemYes15:45
thomasemtable(s)15:45
*** valw has quit IRC15:46
suloyeah thats fair enough i guess15:46
thomasemThe data model as it is has caused us to have to reference all parents of a resource in both the API and cLI15:46
thomasemwhen creating, and doesn't enforce any constraints that are supposed to be enforced by the data model itself15:46
thomasemSo, instead we'll have to do it in the code itself, which introduces a lot of potential for bugs, and still doesn't quite improve the UX of having to know all parents to create a resource.15:47
thomasemIt makes more sense to infer based on the immediate parent who the next ancestor is15:47
sulothomasem: all parents as in having to know both region and cell ?15:48
suloor parent_id should know which parents and we allow to overwrite it ?15:49
jimbakersulo, normalization would certainly fix. i believe the only arg for denormalization is that it can improve performance by removing intermediate joins15:49
thomasemBut we don't even know we have that performance problem, jimbaker15:49
sulojimbaker: yeah i think thats why we did this15:49
thomasemI'd prefer we not start pre-optimizing and over complicating when we don't know.15:49
jimbakerthomasem, oh for sure15:49
thomasemAnyway, I do understand that short-term it's probably not viable to up-end the data model15:50
thomasembut I do want to get the wheels turning on migrating to something easier to work with.15:50
thomasemDoes that seem fair?15:50
jimbakerthomasem, and btw, there's this bug: https://bugs.launchpad.net/craton/+bug/1664707 - which is much more impactful in terms of the unnecessary joins that are being performed15:51
openstackLaunchpad bug 1664707 in craton "SQLAlchemy queries are not lazy enough" [High,New] - Assigned to Jim Baker (jimbaker)15:51
jimbakerselective laziness on queries would help a lot15:51
sulothomasem: sounds fine to me .. but we did see this problem with large public cloud data back in the day15:51
sulopre mongo days15:51
sigmavirusthomasem: sorry, was focusing. which bug might I have input on?15:51
*** valw has joined #craton15:51
thomasemsigmavirus: https://bugs.launchpad.net/craton/+bug/166669915:52
openstackLaunchpad bug 1666699 in craton "able to create unexpected relationships via API" [Undecided,New]15:52
sulojimbaker: yeah +1 on that .. we are loading vars and labels on everything :)15:52
thomasemsulo: yeah... there's some other ways around that using features of the DB, like views.15:52
thomasemI will probably sit down with my housemate (another Racker with a lot of data modeling experience) and see if we can come up with a data model that fits the bill.15:53
sulothomasem: sounds good .. if you are doing that then i have one more thing to add to that15:53
thomasemhmm?15:53
sulosomethig i've been meaning to dig into15:53
sigmavirusthomasem: that's a good find15:53
sulocurrently our netowks model is bad15:53
thomasemLol15:53
sulowell bad in the sense that it coudl be better15:53
thomasemI love how matter-of-fact that was.15:54
openstackgerritIan Cordasco proposed openstack/craton master: Ignore None param values when generating links  https://review.openstack.org/43559515:54
suloso we have network / devices / network-interface15:54
sulobut the relations is not well defined15:54
sulothat needs a bit of modeling too tbh15:54
suloif we are going to poke models then worth looking at that too15:54
thomasemOkay, neat. Let's make note of that on that bug.15:55
thomasemSo, cells is the crux of the issue? You can either have a cell parent or a region parent?15:55
sulono cell is optional15:56
thomasemrather, either an immediate cell parent or immediate region parent15:56
thomasemThat's why I say 'immediate'15:56
thomasemIf you don't have an 'immediate' cell parent, then there's no cell (therefore optional) and the region is your parent instead.15:56
thomasemif you do have a cell, that's your parent, and the cell has a region parent.15:56
*** valw has quit IRC15:56
suloyeah15:56
thomasemCool, got it.15:56
sulobut there were talks of cross cells devices too15:57
sulonot sure where that fits here15:57
suloi thinks thats one of the asks to us etc15:57
thomasemYeah, it was.15:57
thomasemI'll consider that, too.15:57
thomasemI'd like to collaborate on it a bit, too, so should probably be in a spec.15:58
sulo+115:58
thomasemBut, for the initial proposal... yeah, you get the picture.15:58
thomasemOkay, excellent. Thank you for hearing me out on this. :)15:59
thomasemsigmavirus: thanks, yeah. I was going through wiring up Clouds and thought "wow... i'm having to put cloud_id EVERYWHERE... why?"15:59
sigmavirusthomasem: ?16:00
* sigmavirus shakes fist at launchpad timing out while trying to post a freaking comment16:00
thomasemsigmavirus: https://review.openstack.org/43170516:00
thomasemAdding Clouds meant I had to add cloud_id to all children of cloud16:01
thomasemaccording to the existing pattern16:01
thomasemthis lead me to think about what I could abuse in the data model via the API16:01
thomasemFor context16:01
sigmavirusah right16:02
sigmavirussorry16:02
thomasemSpeaking of, I added the CLI bug for clouds, too.16:02
thomasemsigmavirus: no problem16:02
* sigmavirus changes contexts too quickly16:02
thomasemit's all good. :)16:02
* sigmavirus is going to work on the client some more today16:02
sigmavirusadding tests16:02
thomasemSo, we're going to have to merge those in rapid succession.16:02
sigmavirusreworking existing tests that are not the best16:02
thomasemSince adding clouds breaks the existing CLI16:03
thomasemGotcha16:03
thomasemSounds good!16:03
suloalso i picked https://bugs.launchpad.net/craton/+bug/1666890 up as my back from break work16:04
openstackLaunchpad bug 1666890 in craton "Query by filter should raise 404 when no entry found" [High,New] - Assigned to Sulochan Acharya (sulochan-acharya)16:04
sulonot sure if anyone noticed that yet16:04
sigmavirusjimbaker: had assigned me to work on the cilent last week anyway16:05
sigmavirussulo: I saw it cross my email but didn't look closely at it16:06
sigmavirussulo: I'm not sure I'd call an empty list of hosts garbage data16:06
thomasemsigmavirus: feel free to assign yourself to https://bugs.launchpad.net/craton/+bug/1665779, you're definitely doing the bulk of that work.16:06
openstackLaunchpad bug 1665779 in craton "error using generated link for collection with parameter defaulted to None" [High,In progress] - Assigned to Thomas Maddox (thomas-maddox)16:06
sulosigmavirus: heh16:07
sigmavirusthomasem: I just wrote tests16:07
sigmavirussulo: that said, I'm not sure we can generate a 404 from that16:07
thomasemTests were the bulk of that work. :P16:07
sulosigmavirus: can we not ?16:07
sigmavirusBecause we don't have a test for that label existing in advance of the joined query16:07
sigmavirusOr rather, not easily16:07
sigmavirusthomasem: I wrote two tests16:07
thomasemLOL okay, okay16:07
sigmavirusI don't care who's assigned to what16:07
sigmavirusJust that it gets done =P16:07
thomasemFair enough.16:08
sigmavirusThat said, farid is free to assign me to it =P16:08
sulosigmavirus: i mean in general any query fileter does this16:08
suloreturns empty list with links data16:08
sigmavirussulo: sure, but do we want to add checks for the filter conditions being sensible?16:08
farid:)16:08
sigmavirussulo: yeah, so it seems I need to update our pagination spec because the way I understood our design was to always return those links16:08
sigmavirusAnd it seems that's not what I wrote16:08
sigmavirus(Or not what got merged)16:09
jimbakerthomasem, so any revisiting of the data modeling should take in account shared devices; https://bugs.launchpad.net/craton/+bug/166257416:09
openstackLaunchpad bug 1662574 in craton "Support device sharing between clouds, regions, and cells" [High,New] - Assigned to Jim Baker (jimbaker)16:09
sigmavirusfarid: not sure what kind of task tracking you have in mind for craton (Kanban, etc.) so if assignees matter to you, that's a different story IMO16:10
jimbakerbut definitely great if we can get your housemate's input on this16:10
sigmavirusIOW, I defer all task tracking decisions to farid =P16:10
sulosigmavirus: ok, because right now, it doesnt make sense .. also i thought the db already raises16:10
jimbakersulo, i assume you mean that prev/next links should not be generated for empty lists?16:10
sulojimbaker: yes16:10
sigmavirussulo: so the pagination work won't mask those at all16:11
jimbakeryeah, hopefully my patch takes care of that. but will test16:11
sigmavirus(if I remember correctly)16:11
sulosigmavirus: rgr .. it looks like it is not16:11
suloso it looks like its returning an empty list :/16:11
sigmavirusright16:11
*** klindgren has quit IRC16:11
sigmaviruswhich is why we return a 200 OK with an empty list16:11
sigmavirusI would expect an empty list from a filtered query if the label exists but nothing has that label if that makes any sense16:12
jimbakeryes, we want a 200 OK in this circumstance16:12
jimbakeran empty collection is a valid response16:12
sulosigmavirus: yes, that was the though behind that anyway16:12
faridsigmavirus: that's something we can discuss in our meeting here coming up16:12
sigmavirusI disagree that there shouldn't be prev/next links for empty lists16:12
faridwe ended up using just milestones in launchpad to get started with something16:13
sulosigmavirus: there should be ?16:13
jimbakerand what happens when these are chased?16:13
sigmavirussomeone might receive an empty list at the end of pagination series. There should definitely be a prev link for that16:13
sigmavirusjimbaker: ostensibly users are going to stop chasing ilnks when nothing comes back in the list16:13
sigmavirusif they keep chasing them, they'll get empty lists16:13
sigmavirusempty lists for days16:13
thomasemjimbaker: yes, I'm planning to account for that.16:14
suloheh16:14
jimbakersure...16:14
sulofarid: i just added a few bugs to that16:14
sulomilestone16:14
farido/16:14
sigmavirushttps://review.openstack.org/43559516:14
thomasemjimbaker: So, there's the bug already made for the issue... I'll write a BP for actually addressing the data model. We do have an immediate problem with the bug. Does that sound good?16:14
sigmavirusjimbaker: sulo ^16:14
jimbakerthomasem, so let's also look at this in the context of graphene16:15
jimbakeri believe any reasonable adj model should be compatible with graphene support for sqlalchemy16:15
jimbakerbut might as well have that part of our consideration16:16
openstackgerritThomas Maddox proposed openstack/craton master: Add Cloud resource before Region  https://review.openstack.org/43170516:16
*** jovon has joined #craton16:16
thomasemjimbaker: I'll look into that, too. Not terribly familiar, I'm afraid.16:17
thomasemDo you have any immediate concerns around that with a more normalized data model?16:18
sigmavirusokay, so here's a probably very stupid question... why do we want to do `GET /v1/projects?id=...`16:18
*** valw has joined #craton16:19
sigmavirusIsn't that always going to be a single project return and less valuable than forcing `GET /v1/projects/:id`16:19
thomasemsigmavirus: I agree wholeheartedly. It makes no sense.16:20
sulosigmavirus: +116:20
sigmavirusthomasem: so part of this review of yours is just to remove that ;)16:20
thomasemsigmavirus: which review? lol16:20
sigmavirusIad8733594ca83c3bc061017fd9eae6861f31ee5d16:21
sigmaviruser16:21
sigmavirusthat's it's change-id16:21
sulosigmavirus: however, there might be a case (now) that we have root project that controls other projects ?16:21
sigmaviruschange-id Iad8733594ca83c3bc061017fd9eae6861f31ee5d16:21
sigmavirussulo: wouldn't that be filtered by parent_id?16:21
thomasemhttps://review.openstack.org/#/c/435599/16:21
sigmavirussulo: unless I'm misunderstanding yo16:21
thomasemmisunderstanding, yo*16:21
thomasemFTFY16:21
sulosigmavirus: i mean tenant 112 controls all other tenants ?16:22
suloyeah i hear me .. that makes no sense16:22
sigmavirussulo: right, I'm still not sure how that affects filtering by id.16:22
sigmavirusId's are constrained to be unique16:22
sigmavirusUnless we want fuzzy id matching16:22
sigmavirusgimme all projects starting with a16:22
sulosigmavirus: hehe cool cool .. i agree16:22
sigmavirusBut I don't think we want fuzzy matching at the moment16:22
thomasemYeah, but does that really make sense with UUIDs?16:23
sigmavirusthomasem: My UUID has a first name... I think it starts with a16:23
sigmavirusMy UUID has a second name, I think it starts with b16:23
sigmavirus♬16:23
faridsigmavirus / sulo / thomasem / jimbaker can y'all meet a bit early today?16:24
*** valw has quit IRC16:24
thomasemfarid: for you, I'll meet now, even.16:24
suloi can, would appreciate some extra time16:24
faridgit-harry in case he's around too, so that uk folks have more day time16:24
faridgood deal16:24
faridyeah now works if everyone is around16:24
faridmy room is available, let's give this a shot16:24
* sigmavirus is free up until 10:4516:24
jimbakerfarid, i cannot, sorry16:24
sigmavirusI have a meeting at 10:4516:24
sigmavirus;)16:24
faridouch!16:25
sulojimbaker: noooooooooooooooooooo16:25
faridjimbaker: if you can't let's wait it out then, it's just 20 mins16:25
jimbakerneed to take my son to school. brb16:25
faridnp! see you then in a bit.16:25
thomasemsigmavirus: I like your song. How much for it?16:26
sulothomasem: what song ?16:26
sulosigmavirus: what song ?16:26
sigmavirusthomasem: well I charge $3.50 but if you want extra toppings, those come at $0.25 a pop16:26
sigmavirussulo: I sang it to you above16:26
sulonice16:26
thomasemsigmavirus: might want to go a bit cheaper if you ever want to get platinum status. :P16:28
Syed__haha16:28
sigmavirusthomasem: but my songs come with free hot dogs16:28
thomasemOh, you didn't say that!16:28
thomasemWait... that's not "free".16:28
sigmaviruslmao16:29
sigmavirusokay, I'll level with you, what you're paying for is the hot dog and my lawyer who protects me from this pesky thing called "copyright infringement"16:29
thomasemBahahahahaa16:29
thomasemReminds me of this call I had with an ISP who also provided cable. I was trying to tell them all I wanted was internet service, and they kept repeating "but, when you bundle, you save...". I felt like I was actually inside of the movie Idiocracy.16:31
thomasemThey were going to sell me a cable package with my internet, period.16:31
*** valw has joined #craton16:31
sulothomasem: thats how it works in the UK mostly16:32
thomasemOh man... yikes16:32
suloall the big providers that have area coverage force you to16:32
thomasem:(16:32
sigmavirusthomasem: so the way I tend to debate that is to ask them to get a pen and paper16:33
sigmavirusthen I explain to them how I will use the cable package exactly never16:33
sigmavirusso I'm paying more for something I'll never use16:34
sigmavirusSo when you compare the price of teh package and the internet alone, I'm actually saving money by not bundling16:34
thomasemYou have to be careful with that. They have a very fragile mental constitution... it might shatter.16:34
thomasemWhen they realize everything's been a lie from the start.16:35
sigmavirusthen I explain to them how I've never in my life had cable16:36
sigmavirusThat usually results in me being switched to someone whose mind hasn't been blown16:36
thomasem(yet)16:36
thomasemsulo: I guess they just know you can't go anywhere else that won't do the same thing?16:37
sigmavirussulo: don't you also have a national license fee/tax that pays for BBC, etc.?16:37
sulothomasem: yes, sigmavirus: yes16:38
sulosigmavirus: and its quite expensive too16:38
* sigmavirus nods16:39
suloteam sync still on ?16:44
jimbakeryes16:44
*** david-lyle has joined #craton16:46
*** david-lyle has quit IRC16:47
*** zz_pwnall1337 is now known as pwnall133716:58
thomasempwnall1337: https://review.openstack.org/43559516:58
thomasemthat one first16:59
thomasempwnall1337: then this one https://review.openstack.org/43559916:59
pwnall1337thanks16:59
thomasempwnall1337: you bet, sorry for the issues there.17:00
pwnall1337all good :)17:00
thomasemWe'll have them in master by EoD, I hope.17:00
* sigmavirus won't mention graphql17:00
*** valw has quit IRC17:14
*** valw has joined #craton17:18
jimbakersigmavirus, i have warmed up to graphql17:22
*** VW has quit IRC17:22
thomasemLooks pretty slick17:23
jimbakerbut not for this milestone of course17:23
sigmavirusI feel like graphql could provide that search functionality pretty well17:23
*** VW has joined #craton17:23
*** VW has quit IRC17:23
sigmavirusYeah clearly17:23
thomasemLunch17:23
sigmavirusBut that cross-project search functionality would probably be well suited to graphql17:23
*** VW has joined #craton17:23
jimbakerplus in general stuff like being able to say, i just want these variables17:23
jimbakerso yeah, +1 to the potential, and hopefully graphene makes it easy to implement17:24
jimbakerhttps://developer.github.com/early-access/graphql/17:25
suloafk for a few17:31
openstackgerritIan Cordasco proposed openstack/craton master: Use String for Project ID in schema  https://review.openstack.org/43559917:33
sigmavirusjimbaker: so between ^ and https://review.openstack.org/435595 the pagination bugs we talked about should be fixed for projects17:33
sigmavirusyou mentioned there were problems for regions too?17:33
openstackgerritJovon McCloud proposed openstack/craton master: Adds Networks to docs  https://review.openstack.org/42911017:34
jimbakersigmavirus, yeah. let me revisit17:34
jimbakersigmavirus, so regions = list(craton.regions.list()) in the client (master) against the server (master) produces a 400 (bad request)17:39
jimbakeriirc this is because of the marker for region_id is not picked up as an int; and thomasem has a patch for that17:40
jimbakermight as well try it out...17:40
jimbakerhttps://review.openstack.org/#/c/435599/17:41
jimbakernope, still fails with that patch17:43
jimbakerwill dig into it17:43
sigmavirushuh17:43
sigmavirusjimbaker: that's for projects17:43
sigmavirusI'll work on the region id fix if you want17:43
jimbakerahh, yes, sorry17:44
jimbakertoo many patches :) good problem to have i suppose17:44
jimbakersigmavirus, +1 for you working on that fix17:45
jimbakersigmavirus, so it looks like something you already worked on; this url "/v1/regions?sort_keys=created_at%2Cid&sort_dir=asc&marker=2&limit=30" produces this response {"errors": ["Additional properties are not allowed ('sort_keys', 'sort_dir' were unexpected)"], "message": "Bad Request"}17:47
jimbakerso just a basic validation problem17:48
jimbakersigmavirus, btw, i was thinking that it might make sense to templatize in some easy fashion those definitions, so we can share the schema for pagination across paginated resources17:49
jimbakeroh never mind, it looks this has already been merged in17:50
jimbakerhttps://github.com/openstack/craton/blob/master/craton/api/v1/schemas.py#L161617:50
jimbakeretc17:50
jimbakerjust missing on https://github.com/openstack/craton/blob/master/craton/api/v1/schemas.py#L85117:51
jimbakerfor regions GET, so again an easy fix17:51
jimbakeractually i'm correct, we have duplication here: https://github.com/openstack/craton/blob/master/craton/api/v1/schemas.py#L930, but this specific fix is not applied with respect to https://github.com/openstack/craton/blob/master/craton/api/v1/schemas.py#L86717:55
jimbakerso in particular, sort_keys and sort_dir are not defined17:55
jimbakersigmavirus, let's make the definition for the req then be more like the response body, and have a similar function like paginated_resource to inject the necessary schema elements into the request definition17:56
openstackgerritIan Cordasco proposed openstack/craton master: Use String for Project ID in schema  https://review.openstack.org/43559917:58
sigmavirusjimbaker: sounds like a plan18:00
jimbakercool, thanks!18:00
sigmavirusI just want to get ^ tests passing18:05
sigmavirusThen I'll move onto the region fixes18:06
sigmavirusjimbaker: do you have a bug report?18:12
*** valw has quit IRC18:19
*** valw has joined #craton18:21
*** david-lyle has joined #craton18:22
jimbakersigmavirus, i did not file one18:25
jimbakerbut i can do so now18:25
sigmavirusThat's fine18:25
sigmavirusI'll do it18:25
jimbakercool, thanks!18:25
* sigmavirus wanted to make sure he wasn't doubling up bugs18:28
thomasemjimbaker: my patch was for project_id, not region_id?18:29
jimbakerthomasem, https://review.openstack.org/#/c/435599/, correct18:30
thomasemOkay. That's what I thought. Saw you mention region_id above....18:33
jimbakerjust my mix up18:33
thomasemOhhh, gotcha. Okey doke! Carry on.18:33
jimbakerso sigmavirus will fix the region stuff, which is actually a problem in defining pagination requests. i think there may have been a marker issue at some point, but certainly not now18:34
jimbakerjust sort_dir and sort_asc params, and making sure this validates18:34
jimbakereasy stuff, just want to abstract this out so we don't repeat18:34
thomasemOh wild, gotcha18:35
thomasemGood find18:35
*** david-lyle has quit IRC18:39
jimbakerthomasem, interesting, the api is generating this url for the next link on projects with that patch: "http://127.0.0.1:8080/v1/projects?sort_dir=asc&limit=30&sort_keys=created_at%2Cid&marker=b9f10eca-66ac-4c27-9c13-9d01e65f96b4&name=None"18:39
jimbaker&name=None is problematic18:40
thomasemjimbaker: that's the first patch in flight18:40
thomasemhttps://review.openstack.org/#/c/435595/18:40
thomasemjimbaker: ^^18:40
jimbakercan we link?18:40
jimbakershould be a dependency then on that change18:40
thomasemThey weren't necessarily dependent, though. You can fix one problem without the other.18:41
jimbakerwell, it would be fine if it were not broken :)18:41
thomasemboth are needed to fix generated links, but they're not related problems.18:41
jimbakerbut sure, i suppose they are separable18:42
jimbakeranyway, i agree with git-harry - being able to specify None (or null) in the query should be fine18:42
thomasemI was about to post a question - is that standard practice for null param values?18:43
thomasem"None"?18:43
thomasemin OpenStack APIs18:43
jimbakerthomasem, i don't know that answer18:43
jimbakeri do know it makes sense with respect to vars18:43
jimbakerand in particular, we might want to ask ?cell_id=null18:44
thomasemSure, but that's different from allowing18:44
thomasemNone18:44
jimbakeryeah, None is just some string18:44
jimbaker"None"18:44
jimbaker:)18:44
thomasemRight. So, depends on how we wish to handle that.18:44
thomasemAnyway, I'm cool w/ removing the defaults from the schema instead. I know sigmavirus had some opinions on that.18:45
thomasemThe point I was making regarding "None" vs. "null" is that "null" would be independent of this particular issue. What happens here is the None value gets literally translated to the "None" string, and it was never a part of the query to begin with.18:46
jimbakerthere's no question that the management of missing/default/nonexistence/and similar special values is problematic18:46
jimbakerthomasem, yeah, that's clear18:47
thomasemWhich is actually the case for removing it from the schema anyway18:47
thomasemSooo yar18:47
thomasemI think sigmavirus's main argument was this fixed it in one place for all occurrences.18:48
thomasemBut, if we intend to actually use ?key=None, then, yeah18:49
thomasemAgh VPN cut my SSH pipe.18:49
* thomasem frowns18:49
thomasemgit fetch ssh://thomasm@review.openstack.org:29418/openstack/craton refs/changes/95/435595/2 && git checkout FETCH_HEAD18:50
thomasemBah, sorry18:50
jimbakerthomasem, hmm, i think we will take on this tech debt now and figure out a better solution18:50
jimbakersince your patches actually do fix a real and immediate problem18:51
jimbakersounds good?18:51
thomasemTo me, it does. I would _very_ much like for us to solve it in a more future-proof way fast-follow.18:52
jimbakercool. let's do it that way then18:52
jimbakerwe will file a bug to make sure we don't lose track of course18:52
thomasem+118:53
jimbakerbut basically if we use None for a default value, we need to ensure we can use null if it makes sense (i can only think of cell_id); and regardless no loose typing such that None is stringified to "None"18:53
thomasemAgreed. Thanks, jimbaker!18:54
thomasemI'll throw that bug together.18:55
thomasemunless you already are18:55
jimbakerthomasem, if you are on it, please do submit that bug18:57
jimbakerand do cite the change i just approved18:57
jimbakerthanks!18:57
*** valw has quit IRC18:57
openstackgerritMerged openstack/craton master: Ignore None param values when generating links  https://review.openstack.org/43559518:59
openstackgerritMerged openstack/craton master: Use String for Project ID in schema  https://review.openstack.org/43559919:00
*** valw has joined #craton19:01
*** VW has quit IRC19:04
*** VW has joined #craton19:04
jimbakerpwnall1337, so the above fixes projects listing; sigmavirus is working on a fix for regions listing19:04
jimbakerfarid, this now works: $ craton project-list19:05
farido/19:08
jimbakeralthough interestingly this does work $ craton region-list, but not list(craton.regions.list()) in the client. i would have assumed the command used the client as i described19:08
faridawesome ;)19:08
jimbakerbut apparently not19:08
jimbakersigmavirus, maybe something you want to look at19:08
faridyeah that's odd19:08
sigmavirusjimbaker: that is odd19:09
jimbakersigmavirus, really want to add this minor bit to the client: https://bugs.launchpad.net/python-cratonclient/+bug/166673419:09
openstackLaunchpad bug 1666734 in Craton's Python Client "Support debug logging for client/CLI" [High,New]19:09
jimbakerjust to get some transparency on what is going on19:10
jimbakerwhether it's using --debug=http or --debug=all or something else, i don't care about those details. but i do want to be able to see the full request/response19:11
sigmavirusjimbaker: I can work on that next19:12
jimbakersigmavirus, thanks, i think that will be super helpful19:12
*** VW_ has joined #craton19:16
*** VW__ has joined #craton19:18
*** VW_ has quit IRC19:18
thomasemjimbaker: https://bugs.launchpad.net/craton/+bug/166707519:19
openstackLaunchpad bug 1667075 in craton "Support filtering on null values" [Undecided,New]19:19
*** VW has quit IRC19:19
*** VW__ has quit IRC19:23
thomasemhttps://review.openstack.org/#/c/431705 is ready for review, btw.19:28
thomasemworking on CLI changes right now19:28
thomasemfor https://bugs.launchpad.net/python-cratonclient/+bug/166694719:29
openstackLaunchpad bug 1666947 in Craton's Python Client "Support Clouds resource" [Critical,New] - Assigned to Thomas Maddox (thomas-maddox)19:29
*** VW has joined #craton19:31
*** david-lyle has joined #craton19:33
*** VW has quit IRC19:34
*** VW has joined #craton19:35
openstackgerritThomas Maddox proposed openstack/python-cratonclient master: Adds basic Cloud resource CRUD to CLI  https://review.openstack.org/43709819:44
jimbakera little gist that i have found useful for wrapping curl; uses the same env vars to setup as the craton CLI - https://gist.github.com/jimbaker/9089e381af38d9db6bc4af59baed6d4719:46
thomasemAdded to that a little helper I use for setting up client creds.19:52
thomasemModify however you wish.19:52
openstackgerritIan Cordasco proposed openstack/craton master: Add support to regions for sort_dir and sort_keys  https://review.openstack.org/43710419:53
*** valw has quit IRC19:53
sigmaviruspwnall1337: jimbaker ^19:53
jimbakerthomasem, yeah, i already have those env vars set in my .bash_profile19:54
thomasemohhoho, fancy19:54
jimbakerbut adding the admin switch could be nice19:54
jimbakerat some point, one doesn't want to type s* in again19:54
jimbaker;)19:54
thomasemof course19:54
thomasemI made a helper just because I use all of those uers19:54
thomasemusers*19:54
jimbakerfortunately we have a solution: we code19:54
thomasemwhen vetting code paths19:54
jimbaker;)19:54
thomasemDoes your eye ever start twitching from all of your winking?19:55
thomasem:P19:55
thomasemMust have some seriously strong... what is that muscle even called?19:55
jimbakermy daughter is taking intro to programming. she complained about their first assignment: they had to repeatedly do X to some collection of objs in a graphical programming env. i told her, you have the mind of a developer19:55
jimbakerthomasem, no eye strain today, but it has happened in the past :)19:56
thomasemLOL19:56
jimbakerspeaking of removing tedium, sigmavirus just removed a whole bunch of duplicated code in that patch. nice!19:58
*** VW_ has joined #craton20:00
thomasemVery nice!20:00
*** VW has quit IRC20:00
jimbakerthomasem, you will to use for the cloud support20:02
jimbakeranyway, fast track approval since obvious fix, and make pwnall1337's life much nicer20:02
*** valw has joined #craton20:02
thomasemjimbaker: sure, ezpz.20:02
thomasemWill rebase once that goes in20:03
sigmavirusjimbaker: did you run check experimental on that patch? Just wanted to get independent verification of my own results20:06
sigmavirus'specially given we've seen the functional tests run differently in completely clean environments20:06
*** valw has quit IRC20:06
*** david-lyle_ has joined #craton20:07
openstackgerritMerged openstack/craton master: Add support to regions for sort_dir and sort_keys  https://review.openstack.org/43710420:07
*** david-lyle has quit IRC20:07
jimbakersigmavirus, i did not, and i forgot we are still waiting on that gate change20:08
sigmavirusNo worries. jimbaker maybe you should go bug infra about that patch ;)20:09
sigmavirusJust need another +2 https://review.openstack.org/43655620:10
jimbakersigmavirus, right. let me do this a little bit later today20:10
sigmavirus:+1:20:11
jimbakerbiab20:12
thomasemWerid... my Mac just randomly restarted, lol.20:30
*** david-lyle_ has quit IRC20:30
faridw00t first bug https://bugs.launchpad.net/craton/+bug/166709920:32
openstackLaunchpad bug 1667099 in craton "Allow environment export from client" [Undecided,New]20:32
faridthanks thomasem for pointing out we didn't have this use case in a bug/ticket20:32
thomasemjimbaker: After our discussion this morning, sounds like we were planning to address export short-term in the same way we were going to address import (an out-of-band script)?20:33
thomasemWanted to bring that up since export is the thing we actually committed to short-term.20:33
thomasemI also remember mention of handling that in the CLI. Are we thinking something like craton export project <id> or something?20:34
thomasemSure thing!20:35
thomasemfarid: ^^20:35
thomasemfarid: might open one here, too: https://bugs.launchpad.net/python-cratonclient20:37
thomasem:D20:37
thomasemthe craton-api one might actually turn into something related to the bulk endpoint or get absorbed into that body of work (https://bugs.launchpad.net/craton/+bug/1661714)20:38
openstackLaunchpad bug 1661714 in craton "Bulk endpoint for working with resources" [High,In progress] - Assigned to Thomas Maddox (thomas-maddox)20:38
thomasemWeird.. ZNC isn't playing nice with channel buffers.20:40
thomasemOh well, problem for another time.20:40
openstackgerritThomas Maddox proposed openstack/craton master: Add Cloud resource before Region  https://review.openstack.org/43170520:47
thomasemIntegrates with sigmavirus's add_pagination_args ^^20:48
*** valw has joined #craton20:49
*** valw has quit IRC20:52
*** valw has joined #craton20:55
faridthomasem: ah thanks for pointing out20:55
faridsounds like I should just move it to the python-cratonclient altogether, not in addition, right ?20:56
thomasemfarid: yeah, probably. Until we know we wish to do something with the API other than the bulk endpoint we already plan to do.20:56
faridallright!20:57
*** VW_ has quit IRC20:57
*** VW has joined #craton20:58
faridallright, created and included in client milestone20:59
faridhttps://bugs.launchpad.net/python-cratonclient/+bug/166711020:59
openstackLaunchpad bug 1667110 in Craton's Python Client "Allow environment export from client" [Undecided,New]20:59
*** VW has quit IRC21:17
*** VW has joined #craton21:20
*** valw has quit IRC21:23
*** valw has joined #craton21:24
pwnall1337hola21:24
pwnall1337best way to create admin user in craton? just insert sql? does admin user require a project id or just null?21:24
*** valw has quit IRC21:25
*** valw has joined #craton21:31
*** VW_ has joined #craton21:34
*** VW has quit IRC21:34
pwnall1337thomasem:  do you know?21:34
thomasempwnall1337: if you don't have a bootstrap user, best way is inserting into the MySQL, iiuc.21:35
thomaseminserting into the DB* lol21:35
thomasemthe MySQL21:35
pwnall1337gotcha21:35
thomasempwnall1337: admin or root? There's a distinction... root can create and manage projects, admin cannot.21:36
thomasemadmin is project admin21:36
thomasemroot is root over all the things21:36
thomasemProjects being the top-level resource: Project -> Region -> Cell (optional) -> ...21:37
pwnall1337im wanting to project-list21:37
thomasemThen root21:37
thomasemis what you want21:37
thomasemso, when inserting set is_root=True21:37
thomasemfor your user in the Users table21:37
pwnall1337is root true and keep is admin 0?21:38
thomasemyep21:38
thomasemThat should work21:38
pwnall1337what's the different between admin/root?21:38
thomasemif it doesn't, call me out!21:38
pwnall1337admin can add/remove hosts on specific prject?21:38
thomasemAdmin is project admin, root trumps admin.21:38
thomasemcorrect21:38
pwnall1337okie dokie21:38
thomasemadmin can add regions, cells, networks, devices, etc. to a project they're admin over.21:38
jimbakerthomasem, yes, let's address both import/export through the client21:54
jimbakerwhether that's a builtin with the CLI itself, hmm, interesting idea21:55
thomasemjimbaker: yeah, I guess that's what I'm asking... how will that really look?21:58
jimbakerthomasem, maybe we prototype a plugin structure?21:59
jimbakersubcommands can lend themselves to doing this21:59
jimbakeri would first prefer actually write the client code before we put this plugin stuff together however...22:00
*** valw has quit IRC22:11
*** valw has joined #craton22:14
*** valw has quit IRC22:17
openstackgerritOpenStack Proposal Bot proposed openstack/craton master: Updated from global requirements  https://review.openstack.org/43717822:25
*** VW_ has quit IRC22:30
thomasemWondering - why do we use uuids without the dashes?22:35
thomasemjimbaker: yeah, we probably need something sooner. :P22:35
*** VW has joined #craton22:41
*** VW has quit IRC22:45
*** jovon has quit IRC22:52
*** VW has joined #craton22:58
*** VW has quit IRC23:00
*** VW_ has joined #craton23:00
*** VW has joined #craton23:45
*** VW_ has quit IRC23:48
*** VW has quit IRC23:50
*** klindgren has joined #craton23:53

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!