Tuesday, 2015-03-10

*** bandarji has quit IRC00:00
*** sreshetn1 has joined #openstack-sahara00:16
*** sreshetn1 has quit IRC00:21
*** hdd_ has joined #openstack-sahara00:31
*** tnovacik has quit IRC00:33
*** sreshetn1 has joined #openstack-sahara02:05
*** sreshetn1 has quit IRC02:10
*** tellesnobrega_ has joined #openstack-sahara02:14
*** tellesnobrega_ has quit IRC02:14
*** Networkn3rd has joined #openstack-sahara02:21
*** coolsvap has joined #openstack-sahara02:33
*** tmckay is now known as _tmckay02:40
*** coolsvap has quit IRC02:45
*** coolsvap has joined #openstack-sahara02:46
*** coolsvap has quit IRC02:46
*** coolsvap has joined #openstack-sahara02:46
*** hdd_ has quit IRC02:52
*** chandankumar has joined #openstack-sahara02:57
*** hdd_ has joined #openstack-sahara03:46
*** sreshetn1 has joined #openstack-sahara03:54
*** sreshetn1 has quit IRC03:58
*** Longgeek has joined #openstack-sahara03:59
*** syncroswitch has quit IRC04:11
*** Poornima has joined #openstack-sahara05:06
*** hdd_ has quit IRC05:08
*** sreshetn1 has joined #openstack-sahara05:09
*** sreshetn1 has quit IRC05:14
*** hdd_ has joined #openstack-sahara05:38
*** chandankumar has quit IRC05:38
*** Nikolay_St has quit IRC05:40
*** chandankumar has joined #openstack-sahara05:54
*** Longgeek has quit IRC06:04
openstackgerritOpenStack Proposal Bot proposed openstack/sahara: Imported Translations from Transifex  https://review.openstack.org/16288006:14
*** sreshetn1 has joined #openstack-sahara06:21
*** Longgeek has joined #openstack-sahara06:23
*** hdd_ has quit IRC06:38
*** coolsvap is now known as coolsvap|afk06:47
*** sgotliv has quit IRC06:53
*** coolsvap|afk is now known as coolsvap06:58
*** tnovacik has joined #openstack-sahara07:19
*** sreshetn1 has quit IRC07:20
*** chlong has quit IRC07:21
*** openstackgerrit has quit IRC07:35
*** openstackgerrit has joined #openstack-sahara07:35
*** sreshetn1 has joined #openstack-sahara07:36
*** sreshetn1 has quit IRC07:44
*** sreshetn1 has joined #openstack-sahara07:45
*** coolsvap is now known as coolsvap|afk07:53
*** coolsvap|afk is now known as coolsvap08:00
*** sreshetn1 has quit IRC08:13
*** sgotliv has joined #openstack-sahara08:15
*** sgotliv has quit IRC08:20
openstackgerritVitaly Gridnev proposed openstack/sahara: [HDP] Add validation check for dfs.replication  https://review.openstack.org/16211808:20
openstackgerritVitaly Gridnev proposed openstack/sahara: [HDP] Add validation check for dfs.replication  https://review.openstack.org/16211808:22
*** Nikolay_St has joined #openstack-sahara08:27
*** IBerezovskiy has joined #openstack-sahara08:28
*** sgotliv has joined #openstack-sahara08:33
*** _tmckay has quit IRC08:33
openstackgerritlu huichun proposed openstack/sahara: Add a common HBase lib in hdfs on cluster start  https://review.openstack.org/16265708:38
*** tmckay has joined #openstack-sahara08:48
*** akuznetsov has joined #openstack-sahara08:48
*** witlessb has joined #openstack-sahara08:51
*** akuznetsov has quit IRC09:12
openstackgerritVitaly Gridnev proposed openstack/sahara: [HDP] Add validation check for dfs.replication  https://review.openstack.org/16211809:25
*** sreshetn1 has joined #openstack-sahara09:29
*** Poornima has quit IRC09:42
openstackgerritNikolay Starodubtsev proposed openstack/sahara: Take back upstream checks for commit message  https://review.openstack.org/16293209:44
*** Poornima has joined #openstack-sahara09:45
*** sgotliv has quit IRC10:07
openstackgerritVitaly Gridnev proposed openstack/sahara: [HDP] Add validation check for dfs.replication  https://review.openstack.org/16211810:12
*** chlong has joined #openstack-sahara10:17
*** sgotliv has joined #openstack-sahara10:20
*** Poornima has quit IRC10:30
*** coolsvap is now known as coolsvap|afk10:33
*** tosky has joined #openstack-sahara10:34
*** chandankumar has quit IRC10:37
*** Poornima has joined #openstack-sahara10:42
*** sreshetn1 has quit IRC10:50
*** sreshetn1 has joined #openstack-sahara10:50
*** chandankumar has joined #openstack-sahara10:50
*** chandankumar has quit IRC10:50
*** chandankumar has joined #openstack-sahara10:51
*** chlong has quit IRC10:51
openstackgerritSergey Reshetnyak proposed openstack/sahara: Skip job_execution tempest client test  https://review.openstack.org/16296011:00
*** akuznetsov has joined #openstack-sahara11:03
*** Krast has quit IRC11:09
*** sreshetn1 has quit IRC11:11
*** Longgeek has quit IRC11:14
openstackgerritDenis Egorenko proposed stackforge/sahara-ci-config: Global refactoring of CI scripts  https://review.openstack.org/16210211:16
*** sreshetn1 has joined #openstack-sahara11:18
openstackgerritNikolay Starodubtsev proposed openstack/sahara: Take back upstream checks for commit message  https://review.openstack.org/16293211:26
*** sreshetn1 has quit IRC11:28
*** Longgeek has joined #openstack-sahara11:36
openstackgerritMerged openstack/sahara: Imported Translations from Transifex  https://review.openstack.org/16288011:36
*** Poornima has quit IRC11:45
*** coolsvap|afk is now known as coolsvap11:54
openstackgerritAndrey Pavlov proposed openstack/sahara: Adding cluster, instance, job_execution ids to logs  https://review.openstack.org/14396411:56
openstackgerritAndrey Pavlov proposed openstack/sahara: Fixing log messages to avoid information duplication  https://review.openstack.org/14750411:57
*** sgotliv has quit IRC12:16
openstackgerritVitaly Gridnev proposed openstack/sahara: [HDP] Add validation check for dfs.replication  https://review.openstack.org/16211812:20
*** tmckay has quit IRC12:26
*** sgotliv has joined #openstack-sahara12:29
*** akuznetsov has quit IRC12:32
tellesnobregaelmiko, i'm trying to execute a java job from a curl but i always get {"error_message": "The method is not allowed for the requested URL.", "error": 405}12:32
tellesnobregaI used the cli and worked. Ran with debug, got the post dict but it still doesnt work12:33
tellesnobregaelmiko, this is the curl call http://paste.openstack.org/show/191267/12:33
*** chandankumar has quit IRC12:35
openstackgerritVitaly Gridnev proposed openstack/sahara: Move updating provision progress to conductor  https://review.openstack.org/15942112:37
*** sreshetn1 has joined #openstack-sahara12:39
*** _crobertsrh is now known as crobertsrh12:55
openstackgerritEvgeny Sikachev proposed openstack/sahara: Add unit-tests for new integration tests  https://review.openstack.org/15529813:03
*** DenisEgorenko has left #openstack-sahara13:05
*** DenisEgorenko has joined #openstack-sahara13:05
*** coolsvap is now known as coolsvap|afk13:06
DenisEgorenkoHi! Can somebody tell me, why job is failed? https://review.fuel-infra.org/#/c/4455/1   http://osci-jenkins.srt.mirantis.net:8080/job/6.1.mos.install-rpm/1/console13:07
DenisEgorenkosorry13:07
DenisEgorenkowrong chat13:07
openstackgerritEvgeny Sikachev proposed openstack/sahara: Add unit-tests for new integration tests  https://review.openstack.org/15529813:12
*** tnovacik has quit IRC13:17
*** macjack has quit IRC13:17
*** tmckay has joined #openstack-sahara13:21
elmikotellesnobrega: i think you want to POST not PUT13:26
*** akuznetsov has joined #openstack-sahara13:27
elmikoanyone know if the CI issues have been fixed yet?13:30
openstackgerritVitaly Gridnev proposed openstack/sahara: Add usages for step_type field  https://review.openstack.org/16299713:30
vgridnevseems that there are one fix for tempest job and one skip for some another test of tempest tests13:35
elmikovgridnev: ok, cool. hopefully tempest is working now13:36
vgridnevthis changes is going to be merged only : (13:36
vgridnevsee: https://review.openstack.org/#/c/162960/13:37
elmikooh... =(13:37
*** chandankumar has joined #openstack-sahara13:43
*** sreshetn1 has quit IRC13:46
openstackgerritMerged openstack/sahara: Fix tempest tests for Sahara  https://review.openstack.org/16256813:47
openstackgerritVitaly Gridnev proposed openstack/sahara: Implement poll util and plugin poll util  https://review.openstack.org/15739213:50
openstackgerritMerged openstack/sahara: [EDP] Add Oozie Shell Job Type  https://review.openstack.org/15992013:50
openstackgerritMerged openstack/sahara: Implemented multi-worker solution for Sahara API  https://review.openstack.org/14666113:50
openstackgerritArtem Osadchiy proposed openstack/sahara: Add Spark support for MapR plugin  https://review.openstack.org/16300713:53
openstackgerritMerged openstack/sahara: Add validation for node group templates update  https://review.openstack.org/16081413:55
openstackgerritMerged openstack/sahara: Adding additional validation to node group template edit  https://review.openstack.org/16126713:55
openstackgerritMerged openstack/sahara: Add validation for cluster templates update  https://review.openstack.org/16093913:57
openstackgerritAndrey Pavlov proposed openstack/sahara: Added support of instance locality to engines  https://review.openstack.org/16217213:57
*** hdd_ has joined #openstack-sahara14:01
*** egafford has joined #openstack-sahara14:06
openstackgerritMerged openstack/sahara: Collect errors in new integration tests  https://review.openstack.org/15784214:09
openstackgerritMerged openstack/sahara: Add concurrency support in new integration tests  https://review.openstack.org/15128814:09
*** tnovacik has joined #openstack-sahara14:10
*** sreshetn1 has joined #openstack-sahara14:20
elmikocrobertsrh: if i want to test the guided cluster creation stuff, should i install this patch https://review.openstack.org/#/c/163016/ ?14:21
crobertsrhelmiko:  That will give you the most authentic up-to-date everything14:22
elmikok, thanks. hopefully i can get this working =)14:22
crobertsrhActually, I might want to rebase it real quick14:23
crobertsrhI had a couple changes get merged that are "nice to have"14:23
elmikoyea14:23
crobertsrhOk, rebased14:23
elmikoi'll hold off for a bit, i've got plenty more reviews....14:23
elmikooh nice =)14:23
openstackgerritNikolay Starodubtsev proposed openstack/sahara: Take back upstream checks for import order  https://review.openstack.org/16302114:31
openstackgerritNikolay Starodubtsev proposed openstack/sahara: Rewrite log levels and messages  https://review.openstack.org/15403714:34
*** chandankumar has quit IRC14:39
*** hdd_ has quit IRC14:42
SergeyLukjanovdhellmann, I've released the 0.8.0 client and updated CR to global requirements https://review.openstack.org/#/c/155428/14:56
tmckayanybody know how the "updated_at" fields in the database are supposed to work?  I'm calling udpate on a node group template and the updated_at field is not being filled in.15:02
tmckay"created_at" is there, but updated_at is empty (or None).  New template.  Update doesn't seem to fill it in15:03
* tmckay brb15:03
tmckaySergeyLukjanov, alazarev ^^15:04
SergeyLukjanovtmckay, hm, that's strange15:04
alazarevtmckay, yeap, it's strange, it works for job executions15:04
elmikodoes it change updated_at on an update() or only on a save() ?15:07
*** tosky_ has joined #openstack-sahara15:11
*** tosky is now known as Guest1836915:11
*** tosky_ is now known as tosky15:11
tmckayelmiko, not sure.15:12
tmckaysigh.  okay.15:13
*** ylobankov has joined #openstack-sahara15:13
elmikotmckay: if you're curious how it works, https://github.com/openstack/oslo.db/blob/master/oslo_db/sqlalchemy/models.py#L117 is where it comes from15:13
elmikoi wonder if it's possible that we are just calling save() on the model, when updated_at expects us to call update() or something?15:14
tmckaySergeyLukjanov, elmiko, alazarev, while we're here (I'll have to figure out this update problem) substitution is maybe the biggest issue with default-templates.  I'm not sure how to do it.  The simplest thing I think is to allow replacement values to be specified in a JSON file. So, you might have "flavor_id": "%(small)s" in a node group template json file, and you define "small" (and other values) in another JSON file passed as an ar15:16
tmckaymake sense?15:16
tmckayconfig options don't map well, because the list of things to replace has to be known up front15:16
tmckayI thought of allowing key/value pairs on the command line, but then it has to parse different types, etc, which json.load() will already do15:16
*** chandankumar has joined #openstack-sahara15:17
crobertsrhtmckay:  I'm a little fuzzy on why a config can't be used.15:17
tmckayelmiko, I'll poke at it a little more15:17
elmikoso the user would call the cli app to generate a template, and they need provide all these values but you don't like sending them on the command line?15:18
elmikocrobertsrh++15:18
tmckaycrobertsrh, well, configs will mean that we have to explicitly list every possible substitution15:18
tmckayso, what if you want to set flavor differently for several different templates?15:18
tmckayhow are you going to do that?15:18
tmckay"flavor_id:" .... what?15:19
SergeyLukjanovtmckay, no good ideas on your question ;(15:19
crobertsrhI was thinking that your config could be a map of some sort15:19
tmckaycrobertsrh, so essentially the same as a json dict defining values, but specified in a config file?15:19
elmikowhy wouldn't it make sense for the user to create a config file and use it to generate a template?15:19
crobertsrh{"vanilla": {"1.2.1": {<set of specific configs>}}}15:19
crobertsrhwith a {"default": ....} section winning out if nothing specific was given15:20
crobertsrhtmckay:  I suppose it's pretty similar :)15:20
tmckaytrouble with configs is they have to be stored in a known value15:20
tmckayso, we would need something like "template_values = {}" with entries for each plugin/version, maybe15:21
tmckayand then you look in template_values for the right thing15:21
crobertsrhI was thinking that a config would be easier for a user to find/edit should they need to since  it would be in a "standard" place15:21
toskyoh, it's the same problem I'm facing when I need to provide "good" working templates in tempest for node group/cluster template API tests15:22
tmckaycrobertsrh, could be.  Not sure how powerful configs may be.15:22
tmckaytosky, how are you doing it?15:22
crobertsrhOh good, tosky will solve the problem for us :)15:22
elmikolol15:22
crobertsrhand then he will test it for us...thanks tosky!15:22
* crobertsrh goes out for a drink15:22
tmckaythe trouble here is time15:22
tmckaythe clock is ticking15:22
tmckayloudly15:22
toskybut it's a bit different: I have at most one template for each supported version for each supported openstack releases15:22
crobertsrhand we lost an hour!!!15:23
toskywith some specific keys just to test cluster config15:23
toskynot necessarily the cleanest/user friendly templates15:23
elmikotmckay: just so i understand, this is for getting the values in to the cli app that generates templates?15:23
toskybut anyway, it's just a set of nested dicts/lists :)15:24
tmckayelmiko, well, there is a CLI that reads json files representing node group templates and cluster templates.  It reads them and adds them to the database.15:24
*** tnovacik has quit IRC15:25
tmckayelmiko, the sticking point is what if those json files need substitution? for instance, flavor_id is one that can be customized.  Other things with uuid values need to be substituted.15:25
elmikotmckay: i guess i'm trying to get the workflow down in my head. the user would be running the cli app to create them?15:26
tmckayelmiko, image_id for node_group templates (not used very much, I think) and default_image_id for cluster templates are other cases15:27
tmckayno15:27
tmckaywe create the base set of files, user runs the CLI to add a template set to his database so that when he starts sahara the first time he can run, say, a vanilla cluster without adding anything.15:27
tmckayJust launch15:27
tmckaygoal is to pre-populate sane template sets for the out of box experience15:28
tmckayso, depends on verbage.  user is running CLI app to create node group and cluster templates in the sahara db, from json files that we distribute with Sahara15:28
tmckaywith a few values that can be replaced15:28
elmikoand those values should be replaced by the user running the app?15:29
tmckayyes15:29
elmikohmm15:29
tmckayelmiko, they don't *have* to be.  I made the starting directory an option15:29
elmikowhy not allow the user to create a yaml file with the subst values they want and then name it as a parameter to the cli app?15:30
tmckayelmiko, so, you could say "copy this and fill in the appropriate fields" and then run against that.  But that seems less nice.15:30
tmckayelmiko, why yaml instead of plain old json?15:30
elmikoless verbose15:31
tmckayI don't actually know a whole lot about yaml15:31
elmikoyaml is equivalent to json in many ways15:31
elmikoi think it's actually a superset15:31
*** sreshetn1 has quit IRC15:32
elmikoyou could even do it with a plain old ini file15:32
elmikowe just need key,value pairs from the user right?15:32
tmckayelmiko, well, yes.  There is the question of whether or not to scope a particular set of replacement values to a particular template set, but it can work as a flat set.  Especially since you can limit processing to a single directory (or a single plugin/version)15:34
tmckayelmiko, I thought of command line, but it gets more complex if you want to say make a json file where you allow node_processes to be replaced15:34
tmckaynow you have to pass a list in a key/value pair15:34
elmikoyea, but if you want to replace node_processes aren't we going past the convenience of canned templates?15:35
crobertsrh+1, I think node_process tweaking is out of scope for this (at least for now)15:35
elmikothe more you talk about it tmckay, the more i think ini file might be appropriate15:36
tmckaypossibly.  I could limit replacements to specific options.  Anything that takes a uuid, plus flavor15:36
elmikothe user could specify a [DEFAULT] section that does replacement for every template type15:36
elmikoand then specify [TEMPLATE_NAME] sections for specific template replacements15:36
elmikocould do something similar with yaml as well15:36
elmikoand both formats have convenient python parsers15:37
tmckayelmiko, what's a python parser for an ini file?15:37
elmikoconfigparser in the main lib15:37
tmckayoh, so a standard .conf file.  I don't think of them as "ini"15:38
elmikohehe15:38
tmckayelmiko, that will work as long as the replacement set is restricted to a specific set of fields15:38
tmckayelmiko, we can even do [plugin.version] for section headers15:38
elmikoright15:38
tmckayvery easy to match up15:38
elmikoand you can specify global options as well15:39
tmckayelmiko, only question at that point is flavor.  We may want multiple values -- "small", "medium", "large", "xtra-large", etc.15:39
tmckayI've never tried specifiying a dict in a config file15:40
elmikoyou wouldn't want to do that15:40
tmckaywell, then, perhaps you need [plugin.version.name]15:40
elmikowhy not just let the user specify "flavor=small" for some template?15:40
tmckayso that flavor can very across multiple templates15:41
elmikosure, just have something like15:41
tmckayelmiko, right, but we have sets of templates.  A set is a whole.  so for vanilla 1.2.1, you have a bunch of ng templates and one or more cluster templates15:41
elmiko[template name]15:41
elmikoflavor=small15:41
elmiko[other template name]15:41
elmikoflavor=medium15:41
elmikoat some point we need to define the bounds of what a user can define for use with these templates15:42
elmikoif they want to play with all the knobs and buttons, then they should probably just craft their own json15:42
elmikoi think the whole point here is simplicity, once it gets complex we have gone too far15:43
tmckayalright, so last opinion.  Do we map config sections to name alone (hdp, vanilla 1, and vanilla 2 might all have an ng called "master") and have the user pass different config files for different plugin/version, or do we map config sections to [plugin.version.templatename]15:43
tmckayelmiko, ack on complexity, but it's tough to balance15:44
elmikoi like the latter, just make it 1:1 section:template name15:44
elmikotmckay: agreed15:44
elmikoi think it's easy for a user to understand that if they call out a template name as a section, then any variable will be substituted15:45
tmckayelmiko, just to clarify --- you like the idea of a section called [hdp-2.0.6-master] ?15:45
tmckayok, you meant "the former"15:46
elmikoif that syncs up nicely to the template name, then yes15:46
elmikoyea, i guess former15:46
elmikoi think it should be easy for a user to see a template named "hdp-2.0.6-master.json" and know that they should use that name as the section15:48
*** hdd_ has joined #openstack-sahara15:49
tmckayelmiko, okay, I'll think about how best to do that.  We've got plugin/version/name in the json file itself, and then we have the possibility of naming convention15:49
elmikocool15:49
tmckayelmiko, thanks.  I've got to figure out this update issue then come back to this.15:50
elmikono worries, look at the yaml stuff too. just in case you feel like the user will need more than a simple key,value config file15:50
tmckayelmiko, I think first pass should be simple key value.  If someone wants to/needs to they can copy the default template dir and edit to their hearts content as a fallback15:51
elmiko+115:51
tmckaycrobertsrh, thanks to you too15:52
*** bandarji has joined #openstack-sahara15:56
*** Nikolay_St has quit IRC15:57
tmckaySergeyLukjanov, alazarev, oops, updated_at was not changed because the field values were actually all the same ...16:11
tmckayelmiko, ^^16:11
*** plieb has joined #openstack-sahara16:11
elmikolol, nice16:12
*** akuznetsov has quit IRC16:15
SergeyLukjanovtmckay, :)16:22
*** sreshetn1 has joined #openstack-sahara16:24
*** tnovacik has joined #openstack-sahara16:29
SergeyLukjanovfolks, I'm going to move all non-started specs to liberty and I will create a wiki page for FPF exceptions; if we'll have something to return back to Juno - we'll just revert corresponding changes16:31
SergeyLukjanovtmckay, elmiko ^^16:31
tmckaySergeyLukjanov, okay.  default-templates and jobs-endpoint should both be started16:32
elmikoSergeyLukjanov: sounds good16:32
SergeyLukjanovtmckay, are you expecting need for FF exceptions for them?16:36
*** crobertsrh has quit IRC16:36
tmckaySergeyLukjanov, looks like March 19th?  I hope not.  I am a little worried about not enough review time.16:37
SergeyLukjanovtmckay, yeah16:37
tmckaySergeyLukjanov, trying hard to get someting up for default-templates.  A few issues left.16:37
SergeyLukjanovtmckay, but anyway I think it'll be ok to provide FF exceptions for them16:37
tmckaySergeyLukjanov, okay.  job-types endpoint is mostly done (I need to check reviews)16:38
tmckaySergeyLukjanov, core of default-template is done, just need to do substitution and code cleanup.  And then, we will actually need to make defaults for the plugins :)16:38
SergeyLukjanovtmckay, yeah, I'm in progress of reviewing things, not so fast unfortunately :( and there was a vacation yesterday in Russia16:38
tmckaySergeyLukjanov, one question, the template CLI touches the conductor (needs to in order to get template magic between cluster/node group templates).  So, a bunch of options are registered with oslo from imports which show up in the help.  Do you know a way to deal with that?  I am going to look for a way to "unregister"16:40
tmckayoslo config, I mean16:40
SergeyLukjanovtmckay, oh... I'm not sure that it's really possible, it's very tricky16:41
tmckayokay, I'll see if I can find something. maybe I could intercept the help :)16:42
*** tnovacik has quit IRC16:42
openstackgerritMerged openstack/sahara-image-elements: spark: small syntax hiccup in the vars check  https://review.openstack.org/16258316:42
*** Nikolay_St has joined #openstack-sahara16:52
*** DenisEgorenko has left #openstack-sahara17:00
openstackgerritlu huichun proposed openstack/sahara: Add a common HBase lib in hdfs on cluster start  https://review.openstack.org/16265717:01
*** Longgeek has quit IRC17:06
pliebis there an easy way to build the sahara docs in html from source with sphnix on a system that does not have sahara installed?17:24
*** hdd_ has quit IRC17:25
elmikoplieb: you can run `tox -edocs` from the root of the project if you have tox installed17:26
elmikothat will create a virtualenv to make the docs17:26
*** hdd_ has joined #openstack-sahara17:28
*** coolsvap|afk is now known as coolsvap17:32
*** sreshetn1 has quit IRC17:34
*** crobertsrh has joined #openstack-sahara17:34
*** tnovacik has joined #openstack-sahara17:36
*** Nikolay_St has quit IRC17:40
*** Nikolay_St has joined #openstack-sahara17:41
pliebelmiko: cheers17:43
elmiko;)17:44
toskyinteresting, dfs.replication for every plugin BUT CDH, which uses dfs_replication17:45
toskyI wonder: why?17:45
elmikohuh, oversite maybe?17:47
toskymaybe it depends on CDH itself17:48
*** IBerezovskiy has quit IRC17:54
egaffordHi crobertsrh. Is it useful to rebase the shell action patch now, or should I wait until more changes are merged?18:01
egaffordcrobertsrh: You certainly have a better sense of the pulse of these things in Horizon.18:01
crobertsrhMight want to wait, especially if it isn't ready yet (back end isn't ready)18:01
*** victorfeitosa has joined #openstack-sahara18:02
crobertsrhI don't think it will be a particularly tricky rebase18:02
crobertsrhI think the patch looks pretty good though.  I'll have to try it out.18:02
egaffordcrobertsrh: Yeah, I was happy enough with it. The code was perfectly extensible.18:04
crobertsrhThe Horizon stuff is pretty good....it's even better for our "well behaved" pages.18:04
crobertsrhSome day we'll make the launch/config stuff a better horizon citizen, until then it's tribal knowledge ftw.18:05
* elmiko starts beating the sahara drums18:05
crobertsrhOooh...Openstack Summit drum circle!18:06
*** sgotliv has quit IRC18:06
crobertsrhIt's not exactly the party at the carnival place, but it's something18:06
egaffordI am uncertain that I want my job to be the party at the carnival place.18:06
egaffordThat sounds less productive than other options.18:07
crobertsrhOh, egafford...you would have loved it18:07
elmikoagreed, highly unproductive, yet highly fun =)18:07
crobertsrhI almost died on a vintage bicycle/carousel thingy that went about mach 518:07
crobertsrhMy shoelace came untied and wrapped around the pedal18:07
crobertsrhI got quite a workout trying to get it freed and keep it free of the pedal (of which I had no control once all 30 people were making this contraption go)18:08
egaffordWait, there was an actual party at an actual carnival place?18:09
toskycrobertsrh: oh, I forgot/didn't notice that "incident"18:10
elmikoyea18:10
elmikoegafford: it was a vintage 18/19th century carnival museum18:10
crobertsrhtosky:  I tried to play it cool.18:10
toskymore laaate 19th/early 20th, maybe18:10
crobertsrhIt was my "inside" foot...I had to hold it straight out for 90% of the ride18:10
crobertsrhBest workout I've had in ages18:11
elmikotosky: yea, my bad. 19th/20th18:11
crobertsrhPlus my options for my left foot were bad too....awkwardly keep "pedaling" with it or hold it out too18:11
elmikolol, 18th would have been a little too early18:11
crobertsrhI tried both....both were horrible18:11
egaffordelmiko, crobertsrh: Wow.18:12
elmikotosky won all the games too, he had a huge pile or prizes18:12
elmikoegafford: yea, we kept speculating about how much money hp laid out for that party. it was intense18:13
crobertsrhTosky probably had to declare nearly 2 euros worth of prizes18:13
elmikolol!18:13
elmikoi thought no declarations inside EU ?18:13
toskyalmost, I missed one of the (cheap-fake-retro) robots :)18:13
elmikoaww =(18:13
crobertsrhOh man...you could have taken all of mine18:13
toskyyep, not taxes inside the EU \o/18:13
tosky(no _additional_ taxes)18:13
elmikoyea, that's more like it18:13
elmikotosky: are the taxes different from country to country?18:14
crobertsrhI think I stealthily just laid my prizes in a pile and went for another drink.18:14
toskyelmiko: yes, the VAT rates changes, unfortunately18:14
elmikotosky: ahh, that could get pricey in some places i imagine18:14
elmikocrobertsrh: lol18:14
crobertsrhtosky, egafford, Any chance you'll be in Vancouver?18:14
toskymine are probably close to 018:14
elmiko=(18:15
egaffordcrobertsrh: It sounds like I probably will, though my Tokyo chances are close to 0.18:15
crobertsrhOther plans for Tokyo timeframe?  Or just hating on Asia?18:15
egaffordIt'll be nice to meet everyone in the actual world in which we are human.18:15
elmikoegafford++18:15
elmikoand it's known as "meat space"18:15
egaffordNah, Asia's fabulous.18:16
crobertsrhYeah, it was nice to see that tosky (Luigi) does not wear a green plumber uniform (at least not all the time)18:16
egaffordelmiko: I'm not a fan of the whole brevity thing.18:16
elmikoegafford: lol18:16
* tosky facepalms18:16
elmikocrobertsrh: so bad...18:16
crobertsrhI can't help it, I'm a nintendo kid18:16
crobertsrhok, a nintendo adult18:16
elmikono, you are still a nintendo kid ;)18:16
crobertsrhha18:17
*** victorfeitosa has quit IRC18:17
tmckayelmiko, crobertsrh, did you know that you can register config sections and options with oslo at any point and it will fill in the values from files it has already read?18:17
crobertsrhtmckay:  I'd never tried it, but that sounds pretty neat18:18
tmckaywhich means I can define config sections on the fly for reading as I go through the templates ....18:18
elmikotmckay: that was my understanding18:18
crobertsrhIt's either really neat, or a bug waiting to happen18:18
tmckaydidn't know that18:18
tmckayno, the docs actually say it18:18
openstackgerritArtem Osadchiy proposed openstack/sahara: Add Spark support for MapR plugin  https://review.openstack.org/16300718:21
*** tnovacik has quit IRC18:27
openstackgerritNikolay Starodubtsev proposed openstack/sahara: Take back upstream checks for import order  https://review.openstack.org/16302118:31
openstackgerritNikolay Starodubtsev proposed openstack/sahara: Take back upstream checks for commit message  https://review.openstack.org/16293218:31
openstackgerritNikolay Starodubtsev proposed openstack/sahara: Rewrite malformed imports order  https://review.openstack.org/16312318:31
*** Nikolay_St has quit IRC18:31
*** Nikolay_St has joined #openstack-sahara18:32
*** syncroswitch has joined #openstack-sahara18:40
*** hdd_ has quit IRC18:41
*** plieb has quit IRC18:49
*** hdd_ has joined #openstack-sahara18:56
*** sreshetn1 has joined #openstack-sahara18:56
*** Nikolay_St has quit IRC19:00
*** sgotliv has joined #openstack-sahara19:00
*** Nikolay_St has joined #openstack-sahara19:01
*** tosky has quit IRC19:15
*** coolsvap is now known as coolsvap|afk19:20
openstackgerritVitaly Gridnev proposed openstack/sahara: Add usages for step_type field  https://review.openstack.org/16299719:28
openstackgerritVitaly Gridnev proposed openstack/sahara: Add usages for step_type field  https://review.openstack.org/16299719:30
*** Nikolay_St has quit IRC19:31
*** Nikolay_St has joined #openstack-sahara19:31
openstackgerritVitaly Gridnev proposed openstack/sahara: Move updating provision progress to conductor  https://review.openstack.org/15942119:34
*** syncroswitch has quit IRC19:48
*** Nikolay_St has quit IRC19:50
*** Nikolay_St has joined #openstack-sahara19:51
openstackgerritSergey Kolekonov proposed stackforge/sahara-ci-config: Pin Glance client version  https://review.openstack.org/16315819:54
tmckayelmiko, crobertsrh, okay, I have some config section magic going. Any thoughts for how to indicate that a field should be replaced?19:54
crobertsrh$REPLACEME is always a favorite19:55
crobertsrh%REPLACEME% is kinda windowsy19:55
crobertsrh$(REPLACEME)19:55
openstackgerritVitaly Gridnev proposed openstack/sahara: Add usages for step_type field  https://review.openstack.org/16299719:56
elmikotmckay: did you see my comments on the review?19:56
crobertsrhLooking to see if there are other examples in openstack19:57
tmckayelmiko, oh, no, I've had my head down.19:57
elmikono worries19:57
elmikomy vote is for format syntax, i.e. {REPLACEME}19:57
openstackgerritVitaly Gridnev proposed openstack/sahara: Add usages for step_type field  https://review.openstack.org/16299719:57
tmckayelmiko, I've got the section stuff going.  you can have [plugin] or [plugin_version] or [plugin_version_templatename]19:57
elmikotmckay: nice!19:57
tmckayit will prefer more specific19:58
crobertsrh+1 tmckay!19:58
tmckayelmiko, now I just need an indicator that means "replace me with the corresponding field"19:58
tmckayso, I have something in the config say, and it's "flavor_id = 17"19:58
tmckaywhat should the flavor_id in the json look like?19:58
* tmckay goes to read comments19:59
elmikotmckay: as for the way it stands now, my vote is for "flavor_id = {FLAVOR_ID}"19:59
crobertsrh{<replaceme>} works for me19:59
crobertsrhdoes case matter?20:00
elmikonah20:00
*** tnovacik has joined #openstack-sahara20:00
crobertsrhHeh...as long as it's not [default] :)20:00
elmikoit only matters for doing the name of the interpolation variable in the format call20:00
crobertsrhright, tmckay? :)20:00
tmckayblah.  DEFAULT20:01
crobertsrhWe had a little voyage into oslo_config a couple weeks back....case insensitive.....except for default != DEFAULT20:01
elmikooh man, that hurts like stubbing a toe20:01
crobertsrhIt was kinda fun20:01
*** hdd_ has quit IRC20:02
crobertsrhwell, as much fun as watching someone else stub their toe can be20:02
elmikoshaudenfreude!20:02
crobertsrhheh, you spelled it quicker than I could20:02
tmckayelmiko, crobertsrh, okay, I was thinking along the same lines.  repeating the field name as the value gives reader a clue what to look for in the config files, I think20:02
crobertsrhabsolutely, +120:02
elmikoworks for me20:02
tmckaybreathing hard ... hope I'm coming to the end of this.  Until reviews.20:03
crobertsrhor, just name all variables schadenfreude and watch the users squirm20:03
elmikocrobertsrh: LOL, you are so cruel!20:03
crobertsrhreminds me of the "marklar" episode of South Park20:04
elmikohaha, nice20:05
elmikonow if you replaced them all with marklar, that would be funny20:05
crobertsrhI suspect there is a crazy amount of source code available somewhere with heavy use of marklar as a var name20:06
elmikomaybe in the obscure C coding comeptition20:07
crobertsrhor maybe it's just my code :)20:07
crobertsrhand I wonder why nothing works20:07
elmikolol20:07
crobertsrhvar marklar = marklar(marklar);20:08
crobertsrhof course, I only do that in javascript20:08
crobertsrhsorry, I only do that in marklar20:08
elmikowould be funny to make a language named marklar20:10
elmikoall library functions named marklar20:10
elmikoall types named marklar20:10
elmikoeverything inferred20:10
*** sreshetn1 has quit IRC20:11
tmckayelmiko, crobertsrh, if I expect the case to match, then I can just use  "value".format(**dict(CONF[section]))20:17
tmckayhaven't tried this yet20:17
elmikolooks about right though20:18
tmckayand, I can use String.Formatter.parse(value) as a guess to see if Formatter thinks there is something in there it can replace.20:18
elmikois CONF[section] denoting what the user will create?20:18
tmckayelmiko, yeah, at that point it dict(CONF[section]) will return a dict with user specified conf values, with None for those not specified20:19
tmckayif you replace with a None, not my fault20:19
tmckaythe validator will give an error eventually20:19
elmikocool, sounds good to me20:19
tmckayso you think it's okay to expect case match?20:20
elmikoyea i think so20:20
elmikoespecially if we provide the .conf file for the user to fill in20:21
tmckayelmiko, all caps?20:21
elmikofor the keys or the sections, or ?20:21
*** hdd_ has joined #openstack-sahara20:22
tmckayoh, the keys.  I *could* lower() both of them at the time of replacement.  Then you can have whatever sugar you want.20:22
elmikoyea20:22
elmikoi don't have a preference either way for all caps or all lower.20:22
tmckayhmm , maybe CONF lowers the key values for me already when it reads.  I'll check,20:23
elmikoi guess, all lower would be nice. then it's similar to the main config file. more consistency =)20:24
tmckayok20:24
tmckayopts are apparently the case you register them in.  So I can lower the format string, to be nice, if you want to yell in your JSON20:25
elmikolol, NICE!20:26
*** albertoffb has quit IRC20:27
crobertsrhthat sounds great.  +120:30
tmckayI guess if we require that the lower() of the field in the format string matches the name of the config option, I don't even have to play games with the formatter.20:31
tmckayJust try it20:31
*** tnovacik has quit IRC20:38
*** hdd_ has quit IRC20:39
*** hdd_ has joined #openstack-sahara20:41
tmckaycrobertsrh, hmm, I don't think file modification time can work.  You've got the json file time, but the config file that drives the values could be different.20:44
crobertsrhYeah...that is probably not likely to be a good condition20:44
tmckaycrobertsrh, sqlalchemy is smart enough not to set the update time if the values did not change. Or, I can compare values myself (I am fetching it anyway at this point to check the timestamp)20:45
crobertsrhI think I was slightly divergent in my streams of thought in the spec.20:45
tmckaymaybe we just try to update, and let sqlalchemy work it out.20:45
crobertsrhA manual check is probably fine, if anything20:45
crobertsrhyeah, I was just thinking about leaving it to the db gods20:45
crobertsrhI think my time stamp bit was thinking that this might be an automated check of some sort20:45
crobertsrh...rather than a CLI tool20:46
crobertsrhIf you run it, we'll try to update it20:46
tmckaywell, even automated -- it's the sythesis of the config file and the JSON.  Not really a (simple) way to track that.  Didn't think of that til now20:47
tmckayDarn, I worked hard on that timestamp code20:47
crobertsrhYeah....some things sound better in the spec than in the code20:48
*** sreshetn1 has joined #openstack-sahara20:55
tmckaycrobertsrh, hmm, even if you script this with a cron job, I think it's okay.  Default templates can only be updated from the tool.  If you change a file and/or a config section and re-run, and it produces a delta according to sqlalchemy, then you changed it, right?20:59
crobertsrhMakes sense to me, sounds right.20:59
elmikotmckay: do they really need to change on every file change?21:00
tmckayelmiko, no. If I was an admin, personally, I'd run manually as needed.  Like I decided to modify my defaults.21:01
elmikotmckay: i'm +1 for that as the default modus operandi21:01
tmckayelmiko, we were trying to use file timestamp to skip updates if the file was older than the object in the db21:02
crobertsrhI think that the admin manually running this is the most likely scenario anyways21:02
elmikowhy bother though, if the user asks, just let it update. imo21:02
tmckaybut what we send is file + config, so file timestamp is not the whole picture.21:02
crobertsrhI can't imagine needing to tweak the default templates very often (maybe ever)21:02
elmikoyea, totally21:02
crobertsrhMaybe a few tweaks when you're getting started, but then forget about it.21:03
elmikocrobertsrh: and if you did, you'd want to make sure that running the app did the updating21:03
tmckayyou might decided that your flavor needs to be bigger, or you want to change the node count, or something.21:03
crobertsrhYeah21:03
tmckayYou could just as well copy the templates and make your own non-default modifications21:03
elmikoi just don't like the implicit updating, or not, depending on outside factors21:03
crobertsrhYeah, I could see a flavor update, maybe a node count, but mostly static forever and ever.21:03
tmckayI wonder if the cron thing is really really rare.  As in, we don't make any effort in that direction21:04
elmikotmckay: +1 for no effort21:04
elmikomake the update explicit through the tool21:04
crobertsrhI think cron can go  >/dev/null for now21:04
tmckayalright, we'll let sqlalchemy work it out.  I'll just make the calls and report.21:04
elmikoyea, if the user bones it up. then they'll have to fix it. explicit > implicit21:05
tmckaythanks guys, this is thornier than it seemed at first.21:05
crobertsrhThat seems plenty good.  99.8% of all cases21:05
elmikoglad to help =)21:05
tmckayelmiko, yeah, +1 for *copying* the sahara set before you muck with it.21:05
tmckayI have also included a "delete default templates" operation21:05
crobertsrhIf people have huge problems, we can tweak for L21:05
tmckayso you can wipe the slate and start over21:06
crobertsrhShould be plenty21:06
elmikotmckay: it might be a nice convenience to have an option to instruct the app to copy the templates to a local dir, but i wouldn't worry about it now21:06
*** tnovacik has joined #openstack-sahara21:14
*** crobertsrh has quit IRC21:20
*** sreshetn1 has quit IRC21:23
tmckayelmiko, okay, if I'm not crazy I have this essentially done. I need some code cleanup, some comments, maybe tweak the spec, and hopefully upload something later tonight or as early as possible in the morning21:24
tmckayI'm sure there are gaps that the reviewers will find21:24
elmikotmckay: cool! i'll clear my review plate ;)21:27
tmckayelmiko, heh.  I haven't looked too closely but it looks like maybe gate tests are still messed up?21:32
tmckayoh, maybe not. At least one is a merge conflict21:33
openstackgerritTrevor McKay proposed openstack/sahara: Add an is_default field to cluster templates and node group templates  https://review.openstack.org/16092021:33
elmikotmckay: there have also been some patches merged by hand, i think21:35
elmikomine hasn't rerun yet21:36
tmckayk, thanks.21:36
*** chlong has joined #openstack-sahara21:40
*** chandankumar has quit IRC21:48
*** coolsvap|afk has quit IRC22:03
*** chlong has quit IRC22:07
openstackgerritTrevor McKay proposed openstack/sahara: Move node group template schema definition to its own file  https://review.openstack.org/16231422:11
*** tmckay has quit IRC22:14
*** egafford has quit IRC22:16
*** hdd_ has quit IRC22:20
*** hdd_ has joined #openstack-sahara22:20
*** tnovacik has quit IRC23:15
*** hdd_ has quit IRC23:22
*** bandarji has quit IRC23:50
*** chlong has joined #openstack-sahara23:54
*** hdd_ has joined #openstack-sahara23:57

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!