Tuesday, 2017-01-31

faridempanada: nice01:00
faridcan have just sombrero fridays and call it done01:00
empanadafarid   \o/01:00
empanadagreat idea01:00
*** Syed__ has quit IRC01:25
*** valw has joined #craton01:48
*** valw has quit IRC01:52
*** VW has joined #craton02:52
*** VW has quit IRC02:56
*** VW has joined #craton03:17
*** VW has quit IRC04:05
*** VW has joined #craton04:05
*** valw has joined #craton04:46
*** valw has quit IRC05:12
*** VW has quit IRC05:13
jimbakersulo, https://review.openstack.org/#/c/427032/ hopefully is all *set* for you to *get* (terrible attempt at humor about this ;)05:59
jimbakerone other thing - i briefly worked on getting deletion working; plus having names work in addition to IDs to improve ergonomics. but not enough time. maybe thurs? :)05:59
jimbakerbut the usual overrides, etc, work. so it's pretty nice in that way06:00
jimbakeralso need to fix up PEP8 and some unit tests. btw, i also fixed `craton host-list` (and cells) so that it doesn't require region to be specified; and the region id is now listed. which broke at least some of the unit tests. pretty minors, but may want to separate in a different change06:04
*** valw has joined #craton07:12
*** jimbaker has quit IRC07:16
*** valw has quit IRC07:17
*** jimbaker has joined #craton07:19
*** jimbaker is now known as Guest4345507:19
*** valw has joined #craton08:14
*** valw has quit IRC08:18
*** valw has joined #craton12:15
*** valw has quit IRC12:20
*** VW has joined #craton12:45
Guest43455sulo, were you able to try out https://review.openstack.org/#/c/427032/  ?13:05
Guest43455(nice, nick problem, one moment)13:05
*** Guest43455 is now known as jimbaker13:06
*** jimbaker has quit IRC13:06
*** jimbaker has joined #craton13:06
*** ChanServ sets mode: +o jimbaker13:06
jimbakersulo, ok, that is better13:06
*** VW has quit IRC13:25
tojuvoneHi Cratoners. Is there a demo today?13:26
jimbakertojuvone, yes. at the usual time13:27
jimbakerin terms of our tues meeting13:28
tojuvonejimbaker, Great :)13:28
sulojimbaker: i have not yet, i am still working on network cli stuff13:28
jimbakersulo, no worrie13:28
sulowill be trying it out shortly though13:29
jimbakeri'm fixing up that WIP change so it passes the gates13:29
sulocool .. lets get it merged in13:30
jimbakeralso i will see about fixing the column reordering problem with pretty printing. not certain if you have noticed, but host-list etc reorders columns on every pass. at least on python3513:30
suloyeah i have13:30
jimbakersulo, i don't think it can be merged in as is13:30
sulok13:31
jimbakerbut soon13:31
jimbakercertainly this week13:31
*** valw has joined #craton13:48
*** valw_ has joined #craton13:57
*** valw has quit IRC13:59
jimbakersulo, updated patch. there was a missing file. and it now works on 2.7, without extraneous prints14:19
jimbakerall gates currently pass, except pep814:19
jimbaker(just missing doc strings, will do another pass to get those)14:20
jimbakeralso the included CLI snippet in the commit message for the review is executable as-is, assuming the usual fake data has been generated14:20
*** valw_ has quit IRC14:23
*** VW has joined #craton14:34
*** valw has joined #craton14:41
sulojimbaker: ok i have uploaded the network-X commands cli15:20
jimbakersulo, cool, i will check out15:20
sulothose 3 patches needs to merge with your patch for the demo15:20
jimbakersulo, should we just do this via gerrit dependencies?15:21
jimbakermaybe have a patch that depends on all 4 in gerrit?15:21
jimbakerjust to simplify mgmt15:21
sulowell my patches have a dependency chain.15:22
sulonetworks -> net device -> net interface15:22
jimbakerok, then half the battle15:22
jimbakerhave you tried out the vars commands yet?15:22
sulonope i cant get all these patches to merge locally15:23
jimbakerah, ok15:23
jimbakerthen let me try15:23
sulooh i think i was making it too complicated ...15:25
sulosimply git pull worked on top of my last commit15:25
jimbakersulo, here are my notes on some additional cli/client stuff we need to fix - fix arbitrary (2.7)/randomized (3.5) sort order of <resource>-list; delete variables; parent/children; labels; --format=json15:25
jimbakercould be worthy things for thomasem to look at as well15:26
jimbakera couple of these have reported bugs, others need to do so15:27
jimbakeror rather, have such bugs reported15:27
*** valw has quit IRC15:27
sulojimbaker: ok the vars stuff works nicely15:28
jimbakerthanks!15:28
sulojimbaker: https://gist.github.com/sulochan/beb26f68a3846458ae33d4bc91fa5f0315:30
sulovars and network commands15:30
jimbakernow we just need to add vars to network resources! but not before the demo, it's good enough15:30
*** valw has joined #craton15:31
suloyes one of them have it .. need to add it to network-interface15:31
suloand net-devices15:31
sulothere is actually a bug right now due to this in craton15:32
jimbakeralthough i suppose a combo patch that combines would be easy enough to do. probably no more than 30 min to actually implement, given the VariableMixin15:32
suloi have local fix that makes it work15:32
suloyeah15:32
jimbakersulo, ahh, for rest api support for those resources?15:32
suloyeah15:32
sulobut they are one line changes15:32
jimbakergot it. they have been that, haven't they? :)15:33
jimbakerthe "almost works" because so much is tested. but not a critical end-to-end aspect15:33
suloyeah15:33
suloi dont have func tests for networks yet15:33
suloso didnt catch faults there15:33
jimbakerright15:34
sulothe cli is not perfect yet15:34
suloneed to circle back maybe later today after the demo and stuff15:34
jimbakerand we will want to finally get integration testing going against cli/client with service15:34
jimbakernow that the functional testing stuff gives us the appropriate framework to build upon15:34
sulobut need to get this sorted today/tomorrow ... there are a few things we need to add for Thursday15:35
jimbakerright15:35
sulojimbaker: we need to put --format=json for all cli commands :)15:37
jimbakersulo, yes, we most certainly do15:54
jimbakerand generalize accordingly15:54
*** VW has quit IRC15:56
jimbakersulo, yes, we most certainly do want --format=json for everything. let's get that done by thurs. should be easy to do in a general fashion15:56
jimbakermay want to do a metadecorator to apply common options15:57
jimbaker"metadecorator" :)15:57
jimbakeralso it would be nice to support csv, env, to avoid having to write complex pipelines with jq15:57
jimbakerbut json is most important15:57
*** VW has joined #craton16:01
sulosearch by vars is broken :(16:03
jimbakerin rest, or client/cli?16:04
suloapi16:04
jimbakerso that's a priority fix. plus functional testing to stay good16:05
*** VW has quit IRC16:05
*** VW has joined #craton16:05
jimbakeralso, hopefully impl this using json indexing on mysql16:05
jimbakeri wonder when mysql will go out of preview16:06
jimbakermysql 8.0, that is16:06
sulojimbaker: so its weird, its half working https://gist.github.com/sulochan/72ae2206cbc58a14d618d9ed4fdd36e716:13
suloso a:x does not exist to its giving nothing back which is good16:13
sulobut a:b should have only returned one region back.. its returning both16:14
*** klindgren_ has quit IRC16:14
suloits almost like when the var key is present it doesnt care which id its for :?16:14
sulowe havnet changed anything in models have we ?16:14
*** Syed__ has joined #craton16:15
Syed__Hey everyone16:15
Syed__Can anyone kindly tell me when is the demo ?16:15
*** klindgren has joined #craton16:24
thomasemo/16:27
thomasemSyed__: in 30 minutes16:28
jimbakerSyed__, we are having it during our usual 11a CT dev call16:28
Syed__Cool16:32
Syed__Thanks16:32
*** VW has quit IRC16:32
thomasemYou bet!16:32
*** VW has joined #craton16:32
thomasemjimbaker: I'd be happy to look at any. Got links handy? I didn't see any that obviously relate in the launchpad bugs list.16:45
jimbakerthomasem, i'm actually going through launchpad bugs because we need some cleanup there16:46
jimbakeri will add accordingly16:46
jimbakerbut here's the first one for you: https://bugs.launchpad.net/craton/+bug/164862616:47
openstackLaunchpad bug 1648626 in craton "Projects should support variables, which other entities use in resolution" [Wishlist,New]16:47
thomasemjimbaker: Thanks! I'll assign myself.16:47
jimbakerthomasem, probably want to coordinate a bit with git-harry on the rest api aspects. i would hope that we can make the rest api addition of working with variables something that can be readily mixed in to any resource where it makes sense16:48
jimbakeri have tried to do with the recent wip vars work on the  cli/client side as well. it's also true of the data model16:49
jimbakeralready16:49
jimbakerso the data model part is trivial. getting it end-to-end, some more work16:49
jimbakerit will also be good guidance for similar work on mixing in scoped role assignments for rbac16:50
thomasemGoootcha. Alright, I'll dig in. Out of curiosity, I was reviewing the functional tests patch. Would I be able to run functional tests after just pulling down the repo, or do I need to have an API/DB running for them to work?16:50
jimbakerhttps://blueprints.launchpad.net/craton/+spec/craton-rbac-support16:50
jimbakerthomasem, tox -e functional, assuming you have docker set up. you do have docker set up, right? ;)16:51
jimbakerthomasem, fyi, tox -e functional only works on ubuntu for some reason16:51
thomasemDepends on what you mean by having Docker set up?16:51
jimbakerthomasem, just the basics, just the basic16:51
jimbakers16:51
jimbaker;)16:51
thomasemI have a Docker daemon running. I set up a dev environment with a separate DB, network, and API service running.16:52
jimbakeryep, that's what you want16:52
thomasemNot the all-in-one thing from the Dockerfile yesterday16:52
jimbakerso i run all three setups16:52
jimbaker1. tox -e functional. let sulo's good work on this, work for us16:52
jimbaker2. use the instructions in the dockerfile. useful for some basic debugging. i will post my slightly more robust gist of the command i run in a moment16:53
jimbakerthen i can use docker logs -f craton-api; docker exec -it craton-api mysql -ucraton -pcraton craton; docker exec -it craton-api bash, etc16:53
jimbaker3. run a complete setup. this is what i run if i'm hacking on the schema for example16:54
jimbakerthomasem, we also need to run 4. an actual productized cluster16:54
jimbakernew guy on the team, i wonder what he can work on? ;)16:54
thomasemFiled this BP yesterdaym btw: https://blueprints.launchpad.net/craton/+spec/docker-compose-dev-environment16:54
thomasemYeah, got it. I didn't realize the tests and the all-in-one container were so coupled.16:55
jimbakerthomasem, yeah, that BP looks awesome. very much related to #416:56
thomasemSo, that's where my confusion was. I wasn't sure if it was just set a couple vars and the tests will just work as long as they're pointed at the correct things.16:56
jimbakerwe certainly want to containerize the actual prod setup16:56
thomasemOr if the tests made assumptions about the environment like that.16:56
jimbakerthomasem, that Dockerfile is purely for testing16:56
thomasemGotcha16:56
jimbakeri hope the use of password=craton etc makes that clear :)16:57
thomasemNot really. Could be a little more explicit, lol. That could just as easily be simply a dev environment.16:57
thomasemOr an example16:57
jimbakerright, or for dev/example. just not for real usage16:58
thomasemYerp16:58
thomasemAnyway. So, it sounds like set up using the checked-in Dockerfile and then run the functional tests (tox -e functional)?16:58
jimbakerok, not enough time to file more bugs16:58
jimbakerthomasem, so tox -e functional iirc simply uses that top-level Dockerfile16:59
jimbakervia docker-py16:59
thomasemInteresting, it was bombing for me. I'll dig in a bit more in a bit.16:59
thomasemAfter demo16:59
jimbakerok, meeting in 30 seconds everyone!16:59
thomasemI was wanting to test the functional test patches you put up to review them. See y'all in Vidyo.16:59
*** VW has quit IRC17:00
thomasemOr that were put up, can't remember if that was specifically you. :)17:00
*** VW has joined #craton17:06
*** valw has quit IRC17:08
*** jovon has joined #craton17:09
thomasemWonder if tox -e functional only works on Ubuntu because Docker for Mac wraps the daemon in a thin VM.18:00
suloyeah it probably wont work on mac18:00
thomasemI'll file a bug to look at that. That's going to be a drag going forward, at least for me.18:01
sulobut not sure ... i've always done it on my linux box18:01
sulook18:01
thomasemYeah, I gotcha. Haha, I used to run a Linux box exclusively for work, but then... well, let's just say security cracked down and I lost the ability to do a lot of things since some tooling was fighting with Docker, like a lot.18:02
jimbakerthomasem, awesome if you can get that to work. as for the other stuff, wfh can be nice :)18:02
suloi'd think it should work on mac as well actually18:02
sulodocker on mac exposes docker api right18:03
suloand docker-py is simply talking to it18:03
thomasemIt does, but the IP is different.18:03
sulothere maybe some catch on networking18:03
suloyeah18:03
thomasemYeah, the daemon runs in a thin VM (I forget the name of it)18:03
thomasemand gets a separate network stack and such18:03
thomasemCaused a ton of issues with Carina when we were getting CI/CD set up.18:03
thomasemEspecially because I developed the stuff on a Linux box, and it bombed when folks went to run it on their Macs.18:04
jimbakerthomasem, please file a bug on mac tox -e functional, i don't think that has been done18:04
thomasemjimbaker: will do!18:04
jimbakerat least the usual dockerfile stuff works just fine on mac18:05
jimbakerso it's probably something minor18:05
tojuvoneThanks guys, nice demo :)18:05
thomasemYep, probably18:05
thomasemIt was just a bit of environment introspection to fix it for Carina, so, might be something similar here.18:06
thomasemNot a biggie, just might look at it if I'm feeling especially productive on a Saturday or something18:06
thomasemIn the meantime, I suppose I'm going to spin up a separate Ubuntu box and forklift my configs over.18:07
*** VW has quit IRC18:07
jimbakerthomasem, https://gist.github.com/jimbaker/e74a7b98bc60519033fd455a22163ad2 is a robust version of what's in the Dockerfile for starting up a craton service, with fake data18:07
jimbakeror reasonably robust18:07
thomasemjimbaker: perfect. Thanks for the assist!18:08
jimbakertojuvone, thanks! hopefully you can see how we can start adding maintenance support18:08
tojuvonejimbaker, yes, indeed.18:09
sulook heading out for now18:09
thomasemCheers, sulo!18:09
jimbakeri'm thinking that maybe we can use namespaces + json schemas to get this sort of functionality18:09
jimbakerit would also help address what toan was asking about in terms of making it more robust in terms of what can be done (or not)18:09
jimbakerso if a namespace could had some additional enforcements... that could be interesting18:10
jimbakereg, only valid transitions between states as recorded in variables in that namespace18:11
jimbakerthen add notifications on any variable change...18:11
tojuvoneyes18:11
jimbakerwhich is the most important part of auditing18:11
jimbakerfor craton, that is18:12
tojuvonebuild "own" logic specific to namespace18:12
jimbakerexactly18:12
tojuvonewould it be to buidl some code with plugin mapping to that or something18:13
jimbakerbecause this would just go through normal variables for the client, everything we showed today would just work with host-set-vars, etc18:13
jimbaker$ craton host-set-vars my.host.example.com foo/baz=true18:14
jimbakerso foo/ is the namespace18:14
tojuvoneok, like that18:15
tojuvonecool18:15
jimbakerthe only part that doesn't work on the client is the lookup for the host; but that's easy enough to fix and should this week18:15
*** VW has joined #craton18:15
jimbakerthen on the service side, we add the namespace support, which can enforce, etc18:15
jimbakerrbac will work on this, because it's just a variable. same with notifications. etc.18:16
jimbakerwith thomasem's work, this could be applied to the project. it would also work with regions, cells, now, and other types of resources as we add them18:17
jimbakerwith standard resolution/overrides applying of course18:17
tojuvoneok. Yeah, availability zone was mentioned already as "grouping hosts". but there are other things coupling hosts to, like chaining services18:18
jimbakertojuvone, so we have labels for that18:18
jimbakerbut need to expose to cli/client. there is some support in the rest api, i need to double check how robust it actually is. in the python layer, it works great18:19
jimbaker"python object model" layer, which is the craton.dbapi stuff18:19
*** VW has quit IRC18:20
jimbakerso far we have just needed minor fixups as we have been going through first functional testing (to doublecheck rest api); and then next, integration testing (to doublecheck pythonclient/cli against craton service)18:20
thomasemReading scrollback18:21
jimbakerthomasem, assigned you more stuff ;)18:21
thomasemlol18:21
jimbakernahh, just kidding18:21
thomasem:P18:21
tojuvonejimbaker, Ok. Thanks, I will see what learned today towards what I have thought at.18:22
jimbakertojuvone, right, very good brainstorming, based on toan's feedback18:26
jimbakeri like simple, well rounded concepts. namespaces are starting to make sense18:26
jimbakerneed to open up a blueprint!18:26
jimbakerwe also need one for distributed consolidation18:26
jimbakernot quite a warehouse18:27
jimbakeroops, there is in fact one for that. but still needs revisiting: https://blueprints.launchpad.net/craton/+spec/craton-warehouse18:28
jimbakertojuvone, https://blueprints.launchpad.net/craton/+spec/craton-notifications18:31
jimbakertojuvone, https://blueprints.launchpad.net/craton/+spec/craton-namespaces18:42
*** VW has joined #craton18:43
*** VW has quit IRC18:47
tojuvonejimbaker, Thanks, day started 5am is coming to an end 9pm.18:51
jimbakertojuvone, got it! please enjoy the rest of your evening :) and do get some rest! :)18:52
tojuvonejimbaker, The CFP deadline also approaching, if we want to propose something together18:52
jimbakeri also made an early start, but glad to do so18:52
jimbakertojuvone, yep, let's discuss further tomorrow. if not, we can bring this into an operator session instead. so at least 2 options18:52
tojuvonejimbaker, yes, seen your last post in 8am my time18:52
jimbakeralso still need to figure out op summit18:52
tojuvoneyes18:53
jimbakeryes, up late, up early18:53
jimbakerwhat it takes18:53
jimbakerok, see you tomorrow! thanks again!18:53
tojuvonejimbaker, Yep, have a good rest of the day! This was a great evening for me with the demo and all18:54
jimbakerindeed, i'm quite happy how it went! lots of good discussion. and it all worked!18:55
*** VW has joined #craton18:55
tojuvonedome with effect, but no demo effect :)18:55
jimbaker;)18:56
tojuvonedome = demo18:56
jimbakeryep, humor still worked. we did have a couple of typos after all during the dome, but still worked just fine18:57
jimbaker;)18:58
jovonas it relates to networks in our schema. what does "nss" stand for?19:45
*** valw has joined #craton20:31
klindgrennss is networks is usually "network segmentation size" or aka mtu.20:46
*** valw has quit IRC20:56
*** rainya has joined #craton21:23
*** valw has joined #craton21:30
*** VW has quit IRC21:51
*** VW has joined #craton21:51
*** valw has quit IRC21:53
*** VW has quit IRC21:55
*** rainya has quit IRC21:57
*** valw has joined #craton22:02
*** VW has joined #craton22:03
*** VW has quit IRC22:08
*** VW has joined #craton22:09
thomasemFiled that bug regarding functional tests on OS X: https://bugs.launchpad.net/craton/+bug/166082722:20
openstackLaunchpad bug 1660827 in craton "functional tests fail on mac os x" [Undecided,New]22:20
thomasemAnd set up a separate Ubuntu box for testing and am getting different errors there. Investigating...22:20
thomasemSeems to be getting a 500 on container start... wonder if this is something to do with it being Docker 1.13.0?22:22
thomasemdocker.errors.APIError: 500 Server Error: Internal Server Error ("b'{"message":"oci runtime error: container_linux.go:247: starting container process caused \\"exec: \\\\\\"tools/docker_run.sh\\\\\\": permission denied\\"\\n"}'")22:22
thomasemHmmm, interesting22:22
thomasemOhhhhhhh, I think I know why.22:23
thomasemmy rsync probably didn't retain file permissions XD22:23
thomasemlame22:23
thomasemsaddyface, okay. I'll fix this22:24
*** valw has quit IRC22:25
jimbakerno worries22:26
*** jovon has quit IRC22:50
*** rainya has joined #craton23:12
*** rainya_ has joined #craton23:14
Syed__thomasem: is this bug for functional testing happening on OS-X ?23:14
Syed__thomasem: because i just checked with ubuntu and works fine with couple of failures because of regions23:14
*** rainya has quit IRC23:16
*** VW_ has joined #craton23:46
*** VW has quit IRC23:50
*** VW_ has quit IRC23:51

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!