Tuesday, 2017-02-21

*** VW_ has joined #craton01:35
*** VW has quit IRC01:38
*** VW_ has quit IRC01:39
*** VW has joined #craton03:20
*** VW has quit IRC03:24
*** VW has joined #craton03:25
*** VW has quit IRC03:46
*** VW has joined #craton03:46
*** VW has quit IRC04:16
*** VW has joined #craton04:17
*** tojuvone has joined #craton05:51
*** VW has quit IRC06:28
*** openstackgerrit has quit IRC07:32
*** logan- has quit IRC09:16
*** VW has joined #craton12:16
*** logan- has joined #craton12:35
*** VW has quit IRC12:54
*** VW has joined #craton13:09
*** VW has quit IRC13:11
*** VW has joined #craton13:48
sigmavirusHey all13:49
sigmavirusFor people who find our test setup script confusing, I wrote up a tiny explanation here: http://www.coglib.com/~icordasc/blog/2017/02/configuring-docker-on-openstack-ci.html13:50
*** david-lyle has joined #craton14:12
*** Syed__ has joined #craton14:26
thomasemHihi14:32
sigmavirusMorning thomasem14:33
* sigmavirus Syed__: since you missed it, I wrote up http://www.coglib.com/~icordasc/blog/2017/02/configuring-docker-on-openstack-ci.html to give a little background on our test-setup script.14:33
thomasemsigmavirus: thanks for sharing!14:33
sigmavirusNo problem14:34
Syed__sigmavirus: thanks for the link, will look it up in a while14:35
Syed__Morning everyone :)14:35
thomasemGuten Morgen14:35
*** david-lyle has quit IRC14:45
*** david-lyle has joined #craton14:46
*** david-lyle has quit IRC14:46
*** david-lyle has joined #craton14:53
*** harlowja has quit IRC14:56
*** david-lyle has quit IRC14:57
thomasemI noticed DefinitionNetwork doesn't have "region_id" as a required attribute but it's nullable=False in the DB schema. Is this intentional or a bug?14:58
thomasemhttps://github.com/openstack/craton/blob/master/craton/api/v1/schemas.py#L395-L39814:58
git-harrythomasem: https://bugs.launchpad.net/craton/+bug/166176115:07
openstackLaunchpad bug 1661761 in craton "Region ID should be required in DefinitionNetworks" [Undecided,New]15:07
*** openstackgerrit has joined #craton15:13
openstackgerritThomas Maddox proposed openstack/craton master: WIP: Add Cloud resource  https://review.openstack.org/43170515:13
thomasemgit-harry: ahh, thanks, whoops.15:18
thomasemShould have looked there first. Apologies.15:18
thomasemAdded example to the bug. Moooooving on.15:21
openstackgerritIan Cordasco proposed openstack/craton master: Fixes device search by label  https://review.openstack.org/43228315:28
*** david-lyle_ has joined #craton15:29
sigmavirusthomasem: jimbaker https://review.openstack.org/#/c/432283/ has tests asserting what you wanted to see out of that15:42
thomasemsigmavirus: looking15:42
*** VW has quit IRC15:46
*** VW has joined #craton15:47
thomasemsigmavirus: I'm submitting another patch to that review with a minor fix from my comment.15:50
thomasemCool?15:50
thomasemGetting the hang of this gerrit thing. :P15:51
*** david-lyle has joined #craton16:02
*** david-lyle_ has quit IRC16:02
openstackgerritThomas Maddox proposed openstack/craton master: Fixes device search by label  https://review.openstack.org/43228316:04
jimbakersigmavirus, nice explanation of the setup you did for CI, thanks for getting it out there on your blog16:08
sigmavirusthomasem: I was going to update that16:09
sigmaviruslol16:09
thomasemSorry. :\ I was trying to not nit pick and make you do more when I already had the fix in mind... I added an extra test to further assert what we expect16:09
thomasemI will refrain next time16:10
*** valw has joined #craton16:15
*** david-lyle has quit IRC16:27
*** david-lyle has joined #craton16:34
*** jovon has joined #craton16:34
openstackgerritThomas Maddox proposed openstack/craton master: WIP: Add Cloud resource  https://review.openstack.org/43170516:35
sigmavirusthomasem: yeah, just because we can take over the work, doesn't mean we always should :)16:41
thomasemI totally agree. I don't know what my deal is this morning. The wrong thing to do occurs to me way faster than the right.16:42
sigmavirusNo worries :)16:42
sigmavirusJust glad I checked before pushing16:43
sigmavirusIn other news, ordered a pair of gunnar's16:43
thomasemAwww yiss!16:44
*** VW has quit IRC16:48
*** valw has quit IRC16:51
*** VW has joined #craton16:51
*** valw has joined #craton16:54
*** pwnall1337 has joined #craton16:56
pwnall1337.16:57
sigmaviruso/ pwnall133716:58
pwnall1337:)16:58
*** david-lyle has quit IRC16:58
thomasemfarid: It's Craton-recordable on Vidyo, fyi.17:04
sigmavirus https://review.openstack.org/436556 btw17:04
openstackgerritMerged openstack/craton master: Fixes device search by label  https://review.openstack.org/43228317:05
faridthanks thomasem17:09
thomasemSure thing!17:09
*** david-lyle has joined #craton17:16
*** david-lyle has quit IRC17:16
sigmavirusthomasem: I'm constantly thinking that we need GraphQL for Craton, thoughts?17:27
sigmaviruss/constantly/increasingly/17:27
sigmavirushttp://graphql.org/ < Reference17:27
*** valw has quit IRC17:37
*** valw has joined #craton17:46
sigmavirushttps://github.com/graphql-python/graphene-sqlalchemy/ http://graphene-python.org/17:47
sigmavirusAlso, I'm lost in this meeting17:47
*** valw has quit IRC17:51
jovonhttp://raml.org/developers/raml-200-tutorial17:54
jovonhttps://pypi.python.org/pypi/ramlfications/17:54
thomasemsigmavirus: Yeah, I've seen GraphQL before. I'd be interested to see what we get when we slap it in front of our API.18:04
thomasemNever set it up before :)18:04
sigmavirusI've not set it up before, either, but GitHub has hooked it up to their MySQL database and it seems to be cruising along for them18:04
sigmavirusObviously they did more work than that but still18:04
thomasemLol, yeah.18:04
thomasemSounds like Saturday morning entertainment.18:05
thomasemWhilst eating fruit loops or something.18:05
jovonside note: i linked the tutorial for raml because it had nice visual of exporting schema as well as parameterizing resources18:05
jimbakersigmavirus, very nice18:10
jimbakerre graphene - i had looked for something similar earlier re graph support with sqlalchemy, but didn't see such support18:11
*** openstack has joined #craton18:13
jimbakerit will be interesting to see what the ramifications are for implementing graphene, but in principle it looks compatible with what we are doing18:15
*** valw has quit IRC18:18
openstackgerritgit-harry proposed openstack/craton master: Add devices endpoint  https://review.openstack.org/43500518:24
*** valw has joined #craton18:24
*** Syed__ has quit IRC18:35
*** david-lyle has joined #craton18:46
*** david-lyle has quit IRC18:57
jimbakerany other comments for git-harry on https://review.openstack.org/#/c/435005/ ?18:58
jimbakerfrom what i can tell, this follows what we discussed in today's meeting, and addresses the concerns of sulo, sigmavirus, and thomasem18:58
*** VW has quit IRC19:00
jimbakergit-harry, re descendants query, it should be straightforward, and it should work fine with pagination, from what i can tell in my researching the relevant mysql/sqlalchemy docs19:02
jimbakernext step: see how it really does work out!19:02
*** VW has joined #craton19:03
*** valw has quit IRC19:08
*** valw has joined #craton19:09
*** valw has quit IRC19:14
*** pwnall1337 is now known as zz_pwnall133719:22
*** VW_ has joined #craton19:28
*** valw has joined #craton19:30
*** VW__ has joined #craton19:31
*** VW has quit IRC19:31
*** VW__ has quit IRC19:32
*** VW_ has quit IRC19:32
*** VW has joined #craton19:32
*** VW has quit IRC19:33
*** zz_pwnall1337 is now known as pwnall133719:34
*** VW has joined #craton19:34
*** valw has quit IRC19:35
*** VW has quit IRC19:38
*** VW has joined #craton19:38
sigmavirusPsst https://review.openstack.org/#/c/436556/19:42
*** valw has joined #craton19:54
*** pwnall1337 is now known as zz_pwnall133720:01
*** david-lyle has joined #craton20:01
*** valw has quit IRC20:10
*** valw has joined #craton20:18
*** zz_pwnall1337 is now known as pwnall133720:21
*** david-lyle has quit IRC20:23
thomasemjimbaker: looking20:28
jimbakerthomasem, thanks20:29
jimbakerwould like to workflow this so git-harry can get this implemented :)20:29
jimbakerbut no pressure20:29
*** david-lyle has joined #craton20:38
thomasemLol20:40
thomasemjimbaker: lgtm, thanks for calling attention20:49
thomasemgit-harry: ^^20:49
thomasemsigmavirus: looking20:49
*** david-lyle has quit IRC20:49
*** VW has quit IRC20:53
sigmavirusthomasem: would it be helpful if I wrote some tests for  https://review.openstack.org/435595 ?20:53
thomasemsigmavirus: if you like, since I'm working on finishing up clouds atm. That was going to be my next thing. :)20:54
thomasemsigmavirus: I was going to write a test that exhausts pages for /projects to test that out.20:56
*** david-lyle has joined #craton20:56
thomasemvia the generated links20:56
sigmavirusthat's easy :)20:58
*** VW has joined #craton20:58
*** david-lyle has quit IRC20:59
thomasemYep!21:00
thomasemsigmavirus: actually, you'll need that and this: https://review.openstack.org/#/c/435599/ for that test to work.21:01
thomasemsigmavirus: otherwise, it'll just be a reference to 'self' that you could test.21:01
thomasemBombs on 'marker' not being an int otherwise.21:02
sigmavirushah21:03
thomasemPretty jazzy, huh?21:03
sigmavirusThat's 100% my fault21:10
jimbakersigmavirus, thomasem, thanks for the reviews on the devices endpoint spec that git-harry is working in21:14
jimbakeron21:14
jimbakeri workflowed it for now, but we can revisit up (and is it down?) relationship links, but given prior art, seemed to make sense to continue with rel: up21:15
thomasemsigmavirus: Lol, it's understandable. Projects is the weirdo in the API. :)21:15
jimbakerand we only assigned thomasem to work on it as an easy task to introduce him to the project :)21:16
openstackgerritMerged openstack/craton master: Add devices endpoint  https://review.openstack.org/43500521:17
jimbakerat least the result is, thomasem knows a lot about how the code works. just a little pain, that's all21:17
thomasemOh, yeah. It's no problem. I'm just tickled I get to write code again.21:17
thomasem;)21:17
thomasemTrust me... it's a breath of fresh air.21:17
jimbakerthat's right, you had been getting on that mgmt track. and code is definitely fun21:17
jimbakersorry farid... ;)21:18
thomasemYerp. I was more of a team lead than a developer (which you can probably see coming out now and then).21:18
faridlol21:18
* thomasem takes over meetings21:18
thomasem:P21:18
jimbakerthomasem, no worries, you do a great job. much better than me truly taking over the meetings :)21:18
thomasemBahahaha. Sweet. Thanks for saying so. :)21:19
*** VW has quit IRC21:19
jimbakersigmavirus, one thing: you mentioned in https://review.openstack.org/#/c/436556/ that the doc builds work, with some evidence - but all of these builds are with respect to the functional tests21:21
*** valw has quit IRC21:22
sigmavirusjimbaker: so, they build now, yes. But on future changes they may break21:24
sigmavirusOr generate more warnings21:24
jimbakernow i fully expect tox -e docs to work...  this is much more standard than what we did with tox -e functional21:24
sigmavirusright21:24
sigmaviruswe get several warnings, but I'm unconcerned by those21:25
jimbakeranyway, it's worth the minor risk21:25
sigmavirusAlso have to run21:25
*** VW has joined #craton21:25
sigmavirusSo I'll probably finish tackling these tests tomorrow thomasem21:25
jimbakerok, i will +1 and see how it builds once that change goes to zuul21:25
thomasemsigmavirus: sounds good!21:26
thomasemMight be good to combine the two, or make them dependent patches... I would recommend the fix for None being the parent of the string fix for the schema.21:26
pwnall1337running into an issue with craton client21:38
pwnall1337    raise Exception("Versioning for this project requires either an sdist"21:38
pwnall1337from pbr21:38
pwnall1337is it because there are no git release tags?21:38
pwnall1337full failure https://gist.github.com/pwnall1337/6d721:39
*** valw has joined #craton21:40
openstackgerritThomas Maddox proposed openstack/craton master: WIP: Add Cloud resource  https://review.openstack.org/43170521:40
jimbakerpwnall1337, i'm no distribution expert, but exporting PBR_VERSION=1.10.0 should work21:44
*** valw has quit IRC21:44
jimbakerpwnall1337, the other thing: please use this WIP branch for client work: https://review.openstack.org/#/c/427032/21:44
jimbakersince you will want this for vars support21:44
*** VW has quit IRC21:47
*** VW has joined #craton21:47
*** VW has quit IRC21:49
*** VW has joined #craton21:50
thomasemLooks like project_id in POST requests isn't being honored? https://bugs.launchpad.net/craton/+bug/166669521:53
openstackLaunchpad bug 1666695 in craton "project ID not honored in requests" [Undecided,New]21:53
thomasemUses the ID from X-Auth-Project instead.21:53
*** valw has joined #craton21:54
thomasemShould we remove that attribute as allowed from the request, then? Seems to confuse. I suppose that keeps resources scoped to the actual project the user is a part of.21:55
*** valw has quit IRC21:58
thomasemI can create incorrect relationships with our API, it looks like: https://bugs.launchpad.net/craton/+bug/1666699 I was able to create a network in cell 1 and region 2 (which cell 1 does not belong to)22:09
openstackLaunchpad bug 1666699 in craton "resources can be in a region the cell isn't a part of" [Undecided,New]22:10
jimbakerthomasem, thanks for keeping us honest!22:11
jimbakerthis is very much related to https://bugs.launchpad.net/craton/+bug/1666536 - we just need to do some additional checks22:12
openstackLaunchpad bug 1666536 in craton "Cyclical hierarchies using parent_id" [Critical,New] - Assigned to Jim Baker (jimbaker)22:12
thomasemOhhhh22:12
jimbakerthankfully in a SQL transaction we can do the following:22:12
jimbakerread - think - do an update or not :)22:12
jimbakerwhere "think" means some arbitrary python code execution22:12
jimbakerACID FTW!22:12
jimbaker;)22:12
thomasemSo, what would be the pain if we normalized the schema to disallow these sorts of relationships?22:13
jimbakerthomasem, yeah, i hear you. so this was the original design. sulo convinced me that this would make reads more expensive. and he's right22:15
jimbakerfortunately we are doing this through an API layer22:15
thomasemCould we leverage things like views to get around that?22:15
jimbakerwhich means we can check22:15
jimbakerplease, don't go down that path... ;)22:15
thomasemwhy?22:15
jimbakerit's just that denormalization is best addressed by procedural code22:16
jimbakerthey really go together22:16
jimbakerupdateable views are tricky22:16
thomasemwhy's that? Seems that's what the database is for?22:16
jimbakerthomasem, you sound like what i was saying yesterday, on a related topic.... :)22:17
thomasemOh?22:17
jimbakeryes22:17
thomasemWhich topic are you referring to?22:17
thomasemcommunication breakdown, it's always the same22:18
thomasem:P22:18
thomasemthe problem is now we have to go and enforce the data model we want in the Python code rather than the data model itself.22:20
jimbakerThanks to you I'm much obliged22:20
* thomasem wishes he was more of a DB expert now22:21
jimbakerthomasem, but this is going to be true in general22:21
jimbakerdenormalization implies this22:21
jimbakerrecursive aspects like DAGs or even trees also imply this22:22
*** jovon has quit IRC22:22
jimbakerthe good thing is that in a transaction, we can do this check, so the rest of the world always see this as consistent22:22
thomasemOn what grounds do we claim it'd make the reads to expensive, though?22:23
thomasems/to/too/22:23
jimbakerfortunately sqlalchemy's unit-of-work pattern that is pervasive in its design/usage means we can do this in a reasonable fashion22:23
jimbakerthomasem, more normalization is not a bad thing22:23
jimbakerso i'm not opposed to revisiting that decision22:24
thomasemOkay, cool. Because I think we're setting ourselves up for pain if we don't wrangle in our data model and start normal and denormalize as we need to, with tight controls.22:24
jimbakeri just would be opposed to thinking that we can do all of this checking in the db itself, via a view or trigger22:24
jimbakerthat's all22:25
thomasemNo, no. I'm suggesting we start with a cleaner data model, drop the extra columns and have resources relate to their direct ancestor(s) only.22:25
jimbakeragain, my normalization instincts suggested the design you suggested. my pragmatic instincts agreed with sulo22:25
jimbakerbut i'm ok with making this change22:25
jimbakersince it's not going to be visible to any client, except it might prevent errors :)22:26
thomasemHaha, well, ideally. At the moment it's quite visible. :P22:26
jimbakerindeed!22:26
thomasemOkay. This will be a good topic, I think, for others' thoughts as well.22:27
jimbakerit's just that the check can be done in python code, and easily22:27
thomasemWe might have to deal with it for a time, but I think getting this model wrangled in and cleaner will save us tons of headache later on.22:27
jimbakerand with a little work, we can even do this consistently. presumably22:27
jimbakeragain, good either way22:27
jimbakerjust want to point out it's feasible to do it properly in the current modeling22:27
*** VW_ has joined #craton22:28
thomasemYes, just with a lot of extra code, potentially.22:28
thomasemWhich as amount of code increases, so does the potential for bugs.22:28
jimbakermy only concern is support of cell. it's a rather different setup22:28
thomasemand the time taken to implement new things.22:28
jimbakerbut metaprogramming... what about metaprogramming??? ;)22:28
jimbakeranyway, good code, including modeling changes - that will help us decide22:29
thomasemYeah. I'd be really interested in others' thoughts. I tend to be a bit of a purist when it comes to data models and such, so.22:29
thomasemThat could be my own folly.22:29
thomasemI've found that, very often, software problems start at the data model they're interacting with.22:30
*** VW has quit IRC22:31
thomasemAnd software can become much simpler when leveraging the relational DB for what it's good at: especially maintaining relational integrity.22:31
jimbakerthomasem, as someone who as taught monadic programming (in my univ course), i understand a desire for a certain degree of purity...22:31
jimbakerbrb22:32
*** VW_ has quit IRC22:32
thomasemI've got to run, myself. Will catch up tomorrow.22:33
thomasemCheers!22:33
jimbakerthomasem, later!22:53

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!