Thursday, 2017-01-26

*** VW has joined #craton00:11
*** valw has quit IRC00:49
*** valw has joined #craton00:55
*** VW_ has joined #craton01:16
*** VW has quit IRC01:16
*** VW_ has quit IRC01:30
*** VW has joined #craton01:58
*** VW has quit IRC01:59
*** VW has joined #craton02:00
*** valw has quit IRC02:13
*** valw has joined #craton02:14
*** valw has quit IRC02:18
*** Syed__ has quit IRC02:45
jimbakersulo, i merged in one outstanding python-cratonclient change from you; i didn't see a second one02:56
jimbakeralso filed a few bugs02:57
*** valw has joined #craton03:02
*** valw has quit IRC03:30
*** VW has quit IRC03:41
*** VW has joined #craton03:51
*** VW has quit IRC03:59
*** VW has joined #craton04:58
*** mudpuppy has quit IRC05:05
*** mudpuppy has joined #craton05:07
*** VW has quit IRC06:01
*** tojuvone has joined #craton07:47
*** VW has joined #craton08:01
*** VW has quit IRC08:06
sulojimbaker: https://review.openstack.org/#/c/425221/12:12
sulothats the second one12:12
*** VW has joined #craton14:11
sulosigmavirus: if you are around can you take a look at https://review.openstack.org/#/c/425221/14:13
sigmavirusIn an IRC meeting, but I'll queue it :)14:13
suloi need someone other than me to at least +2 it for me14:13
sulosigmavirus: cool, thanks14:13
sigmavirusoh wait, all negative review? +2+A automagically ;)14:13
suloheh, its a simple one, needed to rebase something on top of that14:18
*** VW has quit IRC14:49
*** VW has joined #craton14:50
sulowhich one is the right way to do:14:51
sulovariables-list --host=1 and varialbles-list --region=114:52
suloto get vars for host and region etc14:52
suloor14:52
sulohost-vars-list -h=114:52
suloregion-vars-list -r=114:52
sulo??14:52
sigmavirusI don't like variables-list unless you enforce something like --type host15:12
sigmavirusbecause people will try to combine --host=1 --region=115:12
sigmavirusI like `<type>-vars-from` better because it's explicit in the command-name and we won't have a required -- param15:12
sigmaviruss/from/for/15:13
sigmavirusfor is also fewer things to type than list15:13
sulook, that sounds good to me15:28
sigmavirusI also worry that list might imply that we won't resolve variables for an object if they're not immediately associated with the object15:35
git-harryIs the client design going to be discussed in a spec?15:40
sigmavirusgit-harry: it was designed at the start save for variables15:41
sigmavirustl;dr, cratonclient is going to be as awful as every python-<service>client15:42
git-harrysigmavirus: so this vars stuff is the last piece?15:42
sigmavirusto design? yeah15:44
sigmavirusI wouldn't be adverse to a spec submitted to python-cratonclient to design the client15:44
sigmavirusbut jovon doc'd all of this inc raton already iirc15:44
*** Syed__ has joined #craton15:53
sulogit-harry: no, i am just fixing a missing resource, not designing this thing15:56
suloi have very little idea about client15:56
suloalso i am not sure how to do the variables part either15:56
git-harrysulo: yeah, I misunderstood - I've not yet used the client.15:57
sulolooks like this thing expects base path and all .. which i am not sure works for /hosts/xxx/variables et15:57
sulostill looking though15:57
sigmavirussulo: if we had accepted the url-structure spec with nested resources, then the client would be far more flexible already =P16:04
sigmavirusas such, I didn't prioritize it16:04
sulosigmavirus: how so ? you have the exact same url for variables ?16:05
*** jovon has joined #craton16:05
sigmavirussulo: variables was created after most of that work was done in the client16:05
sigmavirusbut necessitating nested resources as a 1st class citizen would have meant needing immediate changes in the client16:05
sigmavirusvariables landed after both the spec iirc16:06
suloah gotcha16:06
sigmavirusAnd I didn't approve/review vars16:06
sigmavirusso I didn't notice whta would become a difficulty16:06
jimbakerwhen we were loser about the api, the client's crud meant that variables used to work for set by using host.update (which would then just use PUT). i don't see why the client would not just automagically use v1/hosts/$host/variables instead16:13
jimbakerit still retrieves the variables when doing GET against v1/hosts/$host16:14
jimbakerlooser16:14
jimbaker:)16:14
jimbakerlove that16:14
jimbakersulo, so that's my recommendation. just support being able to do stuff like craton.hosts.update(item_id=8, variables={'foo': 42, 'bar': [1,2,3,4]}16:15
sulojimbaker: i dont think that works16:18
jimbakersulo, also what's the status of the functional testing with respect to python mysql client? i spent a bit of time trying to resolve it myself, but somewhere in the various bits of config to be done, it was not picking up the recommended PyMySQL16:18
jimbaker(pure python reimpl)16:18
sulopymysql works16:18
suloi havent updated my patch, but did test it16:19
jimbakersulo, can you push up that?16:19
suloi want to add a small bit to deal with users/project in db16:19
jimbakerok16:19
suloand i didnt get to that yet today16:19
jimbakersulo, re not working with update, so validation now strips out variables. it is possible that i didn't check in the past against the underlying store, and my memory is mixed up16:20
jimbakerbut with the validation stuff, it of course will not work. i think that's fine. we should use v1/hosts/$host/variables, but i think treating variables specially here makes sense in doing stuff like craton.hosts.update(item_id=8, variables={'foo': 42, 'bar': [1,2,3,4]}16:21
sulojimbaker: so i dont think this is a issue to solve for in craton core16:23
suloi think this needs to be handled on client side16:23
jimbakersulo, right, it's client side. we already have the rest api, and it clearly works16:24
suloyeah16:24
sulo(dont know if its a problem yet) but looks like its mostly on how the urls are constructed16:24
jimbakeri don't see a problem in the url construction in the client. just not taking advantage of it :)16:25
sulohow do you mean ?16:25
jimbakera good example of where the rest api usage works is the generate_fake_data.py16:26
jimbakerbut there's no specific support right now in cratonclient for variables16:27
jimbakerbrb16:27
*** VW has quit IRC16:43
jimbaker#startmeeting craton16:59
openstackMeeting started Thu Jan 26 16:59:59 2017 UTC and is due to finish in 60 minutes.  The chair is jimbaker. Information about MeetBot at http://wiki.debian.org/MeetBot.16:59
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.17:00
openstackThe meeting name has been set to 'craton'17:00
jimbaker#chair sigmavirus sulo jimbaker17:00
openstackCurrent chairs: jimbaker sigmavirus sulo17:00
jimbaker#link https://etherpad.openstack.org/p/craton-meetings17:00
jimbaker#topic Roll Call17:00
jimbakero/17:00
Syed__o/17:00
jovono/17:00
jimbakeri have posted a short agenda in the meeting notes17:01
tojuvonehi17:01
jimbakertojuvone, thanks for joining!17:01
jimbakerwe will wait a couple more minutes for git-harry, sigmavirus, sulo to join us17:02
sigmaviruso/17:02
git-harryhere17:03
jimbakerok, let's get started17:04
jimbaker#topic upcoming demos17:05
Syed__when is the demo ?17:05
jimbakerso we have been asked to prepare two demos for next week, on tues jan 31 and thur feb 217:05
Syed__cool17:05
sigmavirusWho is "we"?17:05
sigmavirus=P17:05
sigmaviruso/ antonym in case you want to participate in our meeting/sync17:06
jimbakersigmavirus, well i have been asked, and by extension, all of us have been asked ;)17:06
sigmavirusjimbaker: even tojuvone ? =P17:06
jimbakersigmavirus, no i mean to universally quantify all intelligences17:06
Syed__by whom ? where will it be presented?17:06
jimbaker;)17:06
tojuvonesigmavirus, :D17:07
jimbakerSyed__, in due time17:07
jimbakerso we (the craton dev team) have been asked by toan and dusty to do a demo next tues during our regular vidyo call17:07
jimbakerthis will then be prep for a larger demo one week from today, with time/place to be set. but likely in craton vidyo room, possibly at this time17:08
jimbakerby larger demo, i meant to say, in front of a larger audience17:09
sulojimbaker: whats the expections from the demo ?17:10
jimbakerso this is sort of crunch time. we need to ensure everything that is necessary for basic inventory is working. we got the basics17:10
antonymo/17:10
*** valw has joined #craton17:10
jimbakersulo, basic crud ops, with variables17:10
sulok17:10
jimbakerantonym, thanks for joining17:11
antonymno prob17:11
jimbakerso for antonym: we should show off the python client, since he's going to need this for his tooling work17:12
Syed__So later on down the road specially for boston summit, do we have a set of goals infront of us which we are looking to be done by that17:12
Syed__like having UI for Craton etc.17:12
jimbakerSyed__, sorry, but that's a different topic. let me suggest we discuss later in today's meeting. can you add to the agenda please?17:13
Syed__sure17:13
jimbakercurrent topic is demos :)17:13
git-harryjimbaker: do you have a specific list of things that will be done in the demo?17:13
antonymwe were kicking around the idea of starting to use the maas agent to populate the database with server info: https://github.com/virgo-agent-toolkit/rackspace-monitoring-agent/pull/95017:13
git-harryIt will make it easier to identify problems17:13
jimbakergit-harry, very good question! thanks17:14
jimbakerso it turns on crud ops, variables, and access via python client and CLI17:14
suloantonym: i had a chat with maas team about using the agent for audit work few months back17:14
suloby extending the agent17:14
git-harryjimbaker: so are you going to go through every single one of those options?17:15
antonymyep, i think that would be a really quick way to funnel data in from environments where we don't have a lot of access to scan things17:15
jimbakergit-harry, right now for crud ops we have host deletion failing. which requires me to get some basic changes in i made to alembic setup17:15
jimbakergit-harry, i'm going to go against dusty's doc17:15
jimbakerreq doc17:15
suloantonym: only problem is navigating around rax agent vs its opensource version17:15
suloat least for us17:16
Syed__jimbaker: please share that doc17:16
antonymi'd imagine the data we'd need would work in either version?  we could probably keep the implementation pretty generic17:16
jimbakerSyed__, you should have access to that doc, but if not i will pm you privately17:16
Syed__cool17:16
jimbakersorry tojuvone, we need to get that req doc that rackspace private cloud cleaned up for public distribu17:17
suloantonym: yeah i think sould17:17
tojuvonejimbaker, np17:17
jimbakerbut there's nothing in there that's not in our bugs/blueprints17:17
git-harryjimbaker: I guess I'm suggesting something more explicit, like if you take that doc and write the list of things you want to do to show the state then that can be broken down to verify.17:18
jimbakerjust helpful for specific data elements that would be useful for a demo17:18
suloi was thinking making this an option so 1) we have the generic ssh way of doing audits and gathering data17:18
suloand 2) using the agent to do the same17:18
antonymyeah, getting into some of these environments can be difficult so having a push from the host would be nice17:18
jimbakergit-harry, they want to be able to see how we could capture hardware and software inventory aspects17:19
sulogiven the agent is already doing much of what we want to gather .. it might make sense to focus on that .. given audit though ssh will be acheieved regardless when we work on remediation stuff17:19
jimbakerpretty much the same conversation that antonym and sulo are having, except framing it in a specific usage17:19
antonymsulo: but yeah, i'm plus one on those 2 options17:20
jimbakerso the big missing piece here on crud ops is simple - we need to have updating variables work with the python client; and general variable support in the CLI17:21
jimbakergit-harry, to return to your point, i see the reqs doc as a script for the desired demo. because this is what rackspace support wants from a CMDB - it was explicitly gathered from them17:22
jimbakersince we are in the business of generalization/abstraction, we haven't thought about the specific desired data elements so much. but that's what we will do in the demo itself17:23
jimbakerhope that makes sense!17:24
sulojimbaker: the cli is also missing networks network-interface and net-devices17:24
git-harryjimbaker: I might just need to take another look at the doc.17:24
jimbakergit-harry, just pm'ed you the link17:24
jimbakeragain we will share this publicly asap. i will ask dusty what we need to do to make it public17:25
jimbakersulo, yeah, we should get that support in. but i don't see that as critical for tues and probably not even thurs demo17:26
jimbakerin my discussion with john wood from FAWS (= fanatical AWS support), this is something you thought would be helpful for their needs17:27
jimbakeri'm not certain if we discussed network modeling support with antonym or not...17:27
jimbakersulo, so really the important stuff here would presumably be variables to the CLI; and json support so we can show off with jq17:28
sulook17:30
Syed__jimbaker: tried logging but unable to connect, will talk with you on pm once the meeting is over17:30
jimbakerSyed__, np17:31
jimbakerlet's briefly switch topics here before we get back in some of these details17:33
jimbaker#topic functional testing17:33
suloi need to update my patch with pymysql requirement17:33
jimbakersulo, that would be awesome, since we are almost there on the functional testing changes17:34
sulobut i want to add db clenup for user/project as well .. which i havent done yet17:34
suloyea, other than that .. my local branch works well17:34
jimbakercan we do that separately?17:34
suloi have hosts tests that needs to be updated as well17:34
suloi am still witing on that sqlalchemy change for that17:34
sulohost deletes to be exact17:35
jimbakerahh, so we are in deadlock in each other :)17:35
jimbakeri've been waiting on you17:35
sulo?17:35
sulofor what ?17:35
jimbakerso i can have something to test against17:35
jimbakeri like fixes to have corresponding tests17:35
jimbakersulo, let me give you the alembic diff17:36
suloi can upload the test if you want to test with it17:36
jimbakerone of these two options works... let's discuss later17:37
jimbaker(immediately after meeting)17:37
jimbakeroverall, sounds like we are in good shape with the functional testing updates17:38
jimbaker#topic variable support17:38
jimbakerso we were discussing this before the meeting on #craton17:42
jimbakerthe net of it is, the python client when accessing /v1/hosts/$host will get the variables and populate in the host object on the client17:43
jimbakereg usage like host = craton.hosts.get(item_id=8) will result in host.variables being populated17:43
jimbakerbecause we do not support PUT with variables against /v1/hosts/$host, this is not symmetric, so the current simple crud model the python client uses will not work. example: craton.hosts.update(item_id=8, variables={'foo': 42, 'bar': [1,2,3,4]}) to replace the variables for this host17:44
jimbakerso some choices here, but the simplest to me would be to keep the symmetry and have the python client use PUT /v1/hosts/$host/variables17:47
jimbakerwhen using the above update method17:47
jimbaker(the client would hide this distinction)17:48
jimbakergit-harry, sigmavirus, sulo, what say you?17:48
sigmavirusjimbaker: that works for me17:49
sulodid we just decide to do host-vars-for --host=xxx and host-vars-update --host=xxx17:49
jimbakerthis doesn't change the CLI aspects we were discussing earlier, which was craton variables-for-RESOURCE or something like that17:49
suloso that makes cli and python api different17:49
sigmavirusbut if we're going to do /v1/hosts/:host_id/variables I'd expect a user to be able to do `host = host.get(id); host.variables.update(...)`17:50
suloright17:50
sigmavirussulo: why do people think the cli and python api need to reflect each other?17:50
jimbakersigmavirus, sulo, i'm good with that17:50
sigmavirusthe CLI should be what's most ergonomic17:50
jimbakersigmavirus, +117:50
sigmavirusThe Python API should be ergnomic as well17:50
sigmavirusWhat's ergonomic in bash may not be true in Python and vice-versa17:50
sulook, if the vote is thats cool, then i am fine with it17:51
jimbakerand nested objects are a good example17:51
jimbakeryes, we can do jq, but not every usage of say craton host-show should require it :)17:52
jimbakerok, enough. we have agreed17:52
jimbakersigmavirus, so yeah, host.variables.update(...) sounds good to me17:52
jimbakerright now if i'm not mistaken host.variables is a simple dict, but making it an object sounds far better17:52
jimbakergiven that there's stuff like blame, history, etc, that we want to attach to it17:53
jimbakerin the python client17:53
sigmavirusright17:54
thomasemo/17:55
jimbakerthomasem, thanks for stopping by17:55
thomasemYou bet!17:55
sulothomasem: o/17:55
jimbakerthomasem will be joining us to help develop craton!17:55
thomasemThat I will. Starting next Tuesday, :)17:56
jimbakerso welcome to the team. and timing sounds excellent!17:56
thomasemThanks!!17:56
sulojimbaker: sigmavirus: so whats the net here ? who is done what to get this going for the demo ?17:56
sigmavirusjust in time to help us burn craton to the ground for our demo ;)17:56
jimbakersigmavirus, yep, construct the pyre higher17:56
sigmavirussulo: I have zero visibility into any of this demo nonsense17:56
thomasemLol17:56
tojuvonegreat :) and good stuff the demo and variables for host17:57
sigmavirusNo one has included me in craton conversations in over a month17:57
sigmavirusSo I just assume I'm succeeding at not being seen17:57
sigmavirusOr in other words "Not it"17:57
Syed__Same with me sigmavirus17:57
Syed__i haven't been upto date at all on whats been happening17:57
suloi dont think other than what we are talking abou there has been much17:58
*** david-lyle has quit IRC17:59
sulothe demo thing is ... i guess something we are expected to do to show progress17:59
jimbakerright, we are just being asked to demo something. the only thing that is not really negotiable are the dates18:00
jimbakerso we have to figure something out, and dusty's req doc seems to be the best starting point18:00
jimbakersulo, +1 on progress18:00
sulomight also be a good time to record it and send to the community18:01
jimbakersulo, also +118:01
suloi've been meanign to post vidyo to the ops and dev lists18:01
sulobut just havent made a scrrencast nice enough to do so18:01
jimbakerand sulo is a true master of the impromptu demo18:01
jimbakerso we are in good shape there. just want to make sure that we show all the good dev progress we have made18:02
sulowait a sec, you expecting me to do this demo ?18:02
sulo:D18:02
jimbakersulo, ;)18:02
jimbakerso far it has worked well for us. i'm good for the impromptu commentary/comedy bits18:04
sigmavirusjimbaker: you shouldn't have said that until Tuesday18:04
jimbakersulo, but we can always reverse roles if you want18:04
sigmavirusjimbaker: wait, you do comedy?18:04
jimbakeryep, of the pratfall kind18:04
* sigmavirus is teasing18:04
jimbakerdemo goes down into a burning heap18:04
sigmavirusokay, well we're over time18:05
jimbakeranyway, we will figure this out. really i think the key piece is get some key code committed that has been written18:05
jimbakerand finalize variables18:06
jimbakerDONE18:06
jimbakerand we are done18:06
jimbaker#endmeeting18:06
openstackMeeting ended Thu Jan 26 18:06:11 2017 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)18:06
openstackMinutes:        http://eavesdrop.openstack.org/meetings/craton/2017/craton.2017-01-26-16.59.html18:06
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/craton/2017/craton.2017-01-26-16.59.txt18:06
openstackLog:            http://eavesdrop.openstack.org/meetings/craton/2017/craton.2017-01-26-16.59.log.html18:06
sulothomasem: since you are joning from Tuseday, here is something for you ;) https://review.openstack.org/#/c/421337/18:06
suloto start looking at18:06
thomasemGreat. Thanks!!18:06
thomasemsulo: ^^18:06
tojuvoneFor me have my other duties, but now waiting acceptance to be in ops mid-cycle 13th-16th March in Milan18:10
*** VW has joined #craton18:10
sulotojuvone: you presenting ?18:11
tojuvonesulo, well talked to jimbaker earlier if some of you going. So I could also; just for Craton18:12
jimbakertojuvone, sounds good. at this time, i don't expect to be going, but maybe sulo can make it. regardless having someone there to discuss craton would be awesome18:13
suloyeah, i am still undecided.18:13
tojuvoneok18:13
tojuvoneFor Craton, I'd definately would need somebody to talk about it other than me18:15
tojuvoneif going there18:15
jimbakersulo, it would be awesome if we could have you be there. maybe git-harry as well18:17
sigmavirushey all, so regarding pagination, do we plan on supporting multiple sort-keys where each key can have its own direction? (per API-WG guidelines)?18:18
tojuvoneI thought the 13th-14th would be time to look things within Craton and then see about the meetup on 15th-16th.18:19
jimbaker$121 round trip on british airways for those dates from LHR to MXP...18:20
jimbakersigmavirus, makes sense. fwiw, i don't see such multiple sort keys as being high priority at this time18:21
tojuvoneI am not still 100% sure I get acceptance, but my thing has priority in our side18:23
sigmavirusjimbaker: okay then18:23
tojuvoneI get back to "Milan" thingy, when get to know can I go there.18:27
jimbakertojuvone, awesome, and we need to revisit notifications18:31
*** valw has quit IRC18:31
tojuvoneyeah, I am actually taking the notification thing up in Vitrage Pike design session18:31
jimbakerwe also briefly discussed the question of maintenance state18:31
jimbakerin tues meeting18:31
jimbakerin terms of further elaboration of the current active property18:32
tojuvonejimbaker, might not be much yet for notification till that (Vitrage), but to discuss they could utilize it when it would come18:32
jimbakerrelated to this is a transition state when moving devices/resources between clouds18:32
jimbakerwhich i was asked about separately by another potential user18:33
jimbakertojuvone, sounds good about the timing with vitrage18:33
tojuvonejimbaker, a bit proactive as nothing yet for the notification here then, but it is good to know plans for that, so can prepare to consume notifications then.18:34
jimbakertojuvone, right, some form of auditing or notification is a requirement we have been given from rackspace support. i would like for us to design something that can accommodate a range of uses18:36
jimbakerso your input here is highly welcome!18:36
*** valw has joined #craton18:36
tojuvonejimbaker, yes, for me, now, I can just give the use case of that planned host maintenance and what kind of notifications would be needed18:38
tojuvonejimbaker, then another thing is for me to understand how everything can/is done in Craton18:38
tojuvonejimbaker, so can help on making things happen. Demo will be again nice18:39
jimbakertojuvone, +118:43
*** valw has quit IRC19:01
*** harlowja has quit IRC19:02
*** VW has quit IRC19:04
*** VW has joined #craton19:07
*** valw has joined #craton19:22
jimbakersulo, https://gist.github.com/jimbaker/9a19d4e45ac73aa7db63edb54526e09b19:23
jimbakerso fairly small change, and seems to work well. just want to verify against your testing19:23
jimbakerplease note i did choose to change how fk constraints are setup so it models more closely how we do this in models.py - and also make it easy to name the constraint19:23
jimbakeri have a change i have been planning to do for a while to refactor the remaining alembic pk/fk constraint setups to do the same19:23
*** david-lyle has joined #craton19:45
sulosigmavirus: i have commented on that comment :) https://review.openstack.org/#/c/42417920:07
sigmavirussulo: that comment is much more clear =P20:08
sulojimbaker: git-harry please review ^ i am just nit picking that code but would love to get that in for the demo20:08
sigmavirussulo: I'm almost at the end of the pagination spec20:08
sigmavirus(adding in links to response bodies)20:08
sigmavirusThat might be something to hold off on though because cratonclient doesn't understand it yet20:08
suloah true20:08
sigmavirusAlthough it'd make auto-pagination in cratonclient easier20:08
sulowould be awesome to show tuesday though20:08
sigmavirusWell what you commented on can absolutely go in now20:09
sigmavirusThere's' no changes to response body in there20:09
sigmavirusThe next patch I submit, on the other hand, not so much =P20:09
sulook20:09
*** harlowja has joined #craton20:09
*** valw has quit IRC20:34
*** valw has joined #craton20:47
sigmavirusmy kingdom for a batch api20:58
mudpuppykindom of hearts20:59
* sigmavirus never played that21:01
*** valw has quit IRC21:09
sigmavirusI just pushed up a wip for the hypermedia linking21:17
sigmavirusI haven't thoroughly tested it21:17
sigmavirusBut I'd love to write functional tests for it21:17
sigmavirus*nudge nudge*21:18
*** valw has joined #craton21:49
jimbakersulo, i'm absolutely fine with that code. my only nit pick was about pagination_params vs pagination_args. your nit pick seems valid too, but i'm ready to +2 and revisit21:54
*** valw has quit IRC21:54
*** VW has quit IRC22:27
*** VW has joined #craton22:27
*** VW has quit IRC22:31
*** VW has joined #craton22:33
*** david-lyle has quit IRC23:51
*** david-lyle has joined #craton23:54

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!