Tuesday, 2016-07-19

*** limao has joined #tripleo00:28
*** yamahata has quit IRC00:53
*** hanchao has joined #tripleo00:55
*** apetrich has quit IRC00:58
*** apetrich has joined #tripleo00:58
*** yamahata has joined #tripleo01:07
*** saneax is now known as saneax_AFK01:12
*** stendulker has joined #tripleo01:15
*** beagles_biab is now known as beagles01:19
*** [1]cdearborn has joined #tripleo01:29
*** dmacpher has joined #tripleo01:35
*** limao has quit IRC01:41
*** limao_ has joined #tripleo01:41
*** cdearborn has quit IRC01:44
*** rajinir has quit IRC01:45
openstackgerritJon Schlueter proposed openstack/python-tripleoclient: Use osc-lib instead of openstackclient  https://review.openstack.org/34388301:45
*** shivrao has quit IRC01:52
openstackgerritMerged openstack/instack-undercloud: Modify LVM filter fix to blacklist iSCSI  https://review.openstack.org/34310001:56
*** yamahata has quit IRC02:09
openstackgerritMerged openstack-infra/tripleo-ci: Add MysqlNoBracketsInternal to enable-tls  https://review.openstack.org/34361102:19
*** yamahata has joined #tripleo02:21
*** stendulker has quit IRC02:25
*** yamahata has quit IRC02:34
*** ayoung has quit IRC03:01
*** limao_ has quit IRC03:04
*** limao has joined #tripleo03:09
*** akshai has joined #tripleo03:10
*** limao has quit IRC03:12
*** [1]cdearborn has quit IRC03:14
*** akshai_ has joined #tripleo03:26
*** akshai has quit IRC03:29
openstackgerritJeffrey Zhang proposed openstack/diskimage-builder: Remove the escape in the centos7 README file  https://review.openstack.org/34399703:32
*** saneax_AFK is now known as saneax03:33
*** yamahata has joined #tripleo03:35
*** saneax is now known as saneax_AFK03:45
*** akshai_ has quit IRC03:50
*** shivrao has joined #tripleo04:03
*** limao has joined #tripleo04:13
*** limao has quit IRC04:18
*** jerrygb has joined #tripleo04:22
*** saneax_AFK is now known as saneax04:29
*** jrist has quit IRC04:38
*** shardy has joined #tripleo04:39
saneaxGM folks04:50
*** limao has joined #tripleo04:50
*** jrist has joined #tripleo04:51
shardymorning all04:52
*** radek2 has joined #tripleo04:56
*** links has joined #tripleo05:01
*** jaosorior has joined #tripleo05:04
*** masco has joined #tripleo05:07
openstackgerritAndreas Florath proposed openstack/diskimage-builder: Fix packaging problems for Debian  https://review.openstack.org/34334205:12
openstackgerritIan Wienand proposed openstack/diskimage-builder: [WIP] Move elements into dib package  https://review.openstack.org/34401705:13
*** numans has joined #tripleo05:22
*** jerrygb has quit IRC05:30
openstackgerritIan Wienand proposed openstack/diskimage-builder: [WIP] Move elements into dib package  https://review.openstack.org/34401705:31
*** links has quit IRC05:36
*** apetrich has quit IRC05:50
*** skramaja has joined #tripleo05:54
*** apetrich has joined #tripleo05:54
*** milan has quit IRC05:59
*** milan has joined #tripleo05:59
*** abehl has joined #tripleo06:01
*** pkovar has joined #tripleo06:03
*** _milan_ has joined #tripleo06:04
*** jprovazn has joined #tripleo06:04
openstackgerritIan Wienand proposed openstack/diskimage-builder: [WIP] Move elements into dib package  https://review.openstack.org/34401706:04
*** milan has quit IRC06:05
*** anshul_ has joined #tripleo06:09
*** rcernin has joined #tripleo06:10
*** pgadiya has joined #tripleo06:10
*** abehl has quit IRC06:10
*** leanderthal|afk is now known as leanderthal06:11
*** oshvartz has joined #tripleo06:15
*** _milan_ has quit IRC06:16
*** milan has joined #tripleo06:16
*** derekjhyang has joined #tripleo06:17
*** florianf has joined #tripleo06:17
*** radek2 has quit IRC06:22
*** rasca_ has joined #tripleo06:26
*** ccamacho has joined #tripleo06:27
ccamachoGood morning guys!06:31
jaosoriorccamacho hey dude06:32
jaosoriordtantsur: Are you around?06:32
ccamachojaosorior, still fighting with the mistral stuff, kinda painful jajaaj...06:32
*** shardy has quit IRC06:33
*** pcaruana has joined #tripleo06:36
openstackgerritCarlos Camacho proposed openstack/puppet-tripleo: Add pacemaker mistral profile  https://review.openstack.org/34334606:41
*** tremble has joined #tripleo06:44
*** ramishra has quit IRC06:45
*** ramishra has joined #tripleo06:47
jaosoriorccamacho how well can you debug mistral?06:49
*** anshul_ has quit IRC06:49
*** cwolferh has quit IRC06:51
*** hparekh has quit IRC06:54
*** shardy has joined #tripleo06:56
*** tesseract- has joined #tripleo06:58
*** cwolferh has joined #tripleo06:59
*** devvesa has joined #tripleo07:00
*** ohamada has joined #tripleo07:02
jaosoriorHey guys, can we merge this? https://review.openstack.org/#/c/343607/07:02
*** anshul_ has joined #tripleo07:02
*** liverpooler has joined #tripleo07:03
openstackgerritMichele Baldessari proposed openstack/tripleo-heat-templates: openstack-core resource does not have interleave=true  https://review.openstack.org/34272507:04
jaosoriorshardy: how acquainted are you with mistral?07:05
shardyjaosorior: somewhat, I've been testing the new mistral actions/workflows and written some local workflows attempting to deploy direct to ironic07:05
bandiniI see folks are putting "recheck dlrn fail" or comments like that when rechecking. Are there any guidelines about this?07:06
jaosoriorshardy: For some reason it seems that when we re-enabled introspection... it's now broken07:06
jaosoriorand it's weird07:06
shardybandini: Yes, if possible it's best to do recheck bug ####07:06
shardywhere the bug is an existing known issue07:06
jaosoriorcause it's breaking in an action that comes with mistral07:06
bandinishardy: ack,  got it07:06
ccamachojaosorior, the issue is with the deployment, I believe is almost done, but after is installed and working I need to run a set of local tests to check that is actually working properly07:06
shardybandini: for some things like spurious failiures e.g of delorean mirrors or something, we might do just a description07:06
bandinishardy: ok, I see. I thought there was like a common set of keywords that would allow us to gather some numbers about failures07:07
jaosoriorshardy, ccamacho http://logs.openstack.org/69/342669/1/experimental-tripleo/gate-tripleo-ci-centos-7-ovb-nonha/78969cb/console.html07:07
bandinishardy: thanks :)07:07
shardybandini: the main thing to avoid doing repeatedly is "blind rechecks", as sometimes it's pretty clear folks don't even look at the failure reason and just do recheck recheck recheck07:07
*** apetrich has quit IRC07:07
*** hparekh has joined #tripleo07:08
*** bootsha has joined #tripleo07:08
shardybandini: there is a tool called elastic recheck, http://status.openstack.org/elastic-recheck/07:08
bandinishardy: yeah that makes sense, I know I have not been completely innocent on that front ;)07:08
shardybandini: the idea of tagging a bug reference is you can see from those graphs what bugs are causing most failures07:08
bandiniah, this is nice07:09
*** apetrich has joined #tripleo07:09
shardyjaosorior: introspection is now driven via mistral action/workflow defined in tripleo-common07:10
shardyjaosorior: did we land anything related to those while the CI was disabled for introspection?07:10
openstackgerritMerged openstack/tripleo-heat-templates: Add MysqlNoBracketsInternal to enable-tls.yaml  https://review.openstack.org/34360707:12
jaosoriorshardy: seems that's the case07:16
jaosoriorso now either we disable introspection again or figure out how to fix it07:16
jaosoriorshardy: Yep, here it is https://review.openstack.org/#/c/327780/07:17
jaosoriorah wait07:17
shardyjaosorior: I tested that one locally, I suspect it's a later commit that broke things07:18
jaosoriorno, actually I don't see anything that would immediately break those07:18
*** ifarkas_afk is now known as ifarkas07:18
*** zoli_gone-proxy is now known as zoliXXL07:19
bandiniso I have two changes A) puppet-tripleo and B) tht that only make sense together. For now on change B) I put a depends-on A), but A) will never pass CI because it needs B) as well. Can I create a circular depends-on loop?07:20
shardybandini: I'd put the Depends-On in the tht patch, then we'll merge them together07:20
shardyif it passes CI we can derive the puppet-tripleo patch is OK07:20
bandinishardy: ack, got it. thanks07:20
shardyI'm not sure if a circular dependency will work (suspect probably not)07:21
bandiniyeah that is what I assumed07:21
shardyit'll definitely make them both impossible to merge tho :)07:21
bandinieheh right07:21
*** fzdarsky has joined #tripleo07:23
*** jubapa has joined #tripleo07:23
*** fzdarsky has quit IRC07:24
*** paramite has joined #tripleo07:24
hewbroccaRDO promotion is still 12 days behind07:25
hewbroccaIf anyone here can fix any of the issues at the bottom of this etherpad:07:25
hewbroccaPlease do so ASAP07:25
hewbroccahttps://dashboards.rdoproject.org/rdo-dev is useful for monitoring this07:26
openstackgerritIan Wienand proposed openstack/diskimage-builder: [WIP] Move elements into dib package  https://review.openstack.org/34401707:27
*** dsariel has joined #tripleo07:29
*** anshul_ has quit IRC07:29
*** ebarrera has joined #tripleo07:30
*** akuznetsov has joined #tripleo07:30
*** jerrygb has joined #tripleo07:32
*** paramite has quit IRC07:36
*** paramite has joined #tripleo07:37
*** jerrygb has quit IRC07:37
*** jpich has joined #tripleo07:41
*** abehl has joined #tripleo07:43
*** bootsha has quit IRC07:48
ccamachojmiu when available can you check this in your environment? https://review.openstack.org/#/c/34390207:52
*** bootsha has joined #tripleo07:53
openstackgerritMichele Baldessari proposed openstack/tripleo-heat-templates: Remove two races during the L->M Keystone under httpd migration  https://review.openstack.org/33887907:53
bandinimatbu: can you throw one of your upgrade jobs at the review above ^? It should fix the problem marios saw as well07:54
mariosbandini: o/ i c there is an update with the "no outstanding transactions" bit07:56
*** akuznetsov has quit IRC07:56
bandinimarios: there is more than that. I implemented andrew's suggestion from the RH BZ07:56
bandiniI create openstack-core and wait for it to be started. that way we avoid a bunch of spurious restarts when we change the constraints07:57
matbubandini: yes the ci jobs used this review07:58
mariosbandini: ack thanks I will include it on next pass07:58
bandinimatbu: can you run another one with today's change?07:58
matbubandini: yes sure07:58
openstackgerritMarios Andreou proposed openstack/tripleo-heat-templates: Enable Manila integration - as a composable controller service  https://review.openstack.org/18813707:59
bandinimatbu: thank you sir! ;)07:59
mariosjaosorior: merge conflict but it wiped your +2 appreciate a look when you have a chance ^^08:00
matbubandini: de nada :) , the keystone migration needs another fix ?08:00
openstackgerritMerged openstack/tripleo-ui: Root components reorganization  https://review.openstack.org/33386908:01
bandinimatbu: potentially yes, marios hit some snags during that upgrade steps08:01
*** athomas has joined #tripleo08:03
openstackgerritSaravanan KR proposed openstack/tripleo-heat-templates: Added a new composable service for enabling DPDK on compute nodes.  https://review.openstack.org/34408408:03
matbubandini: k, i hit weird issue in the controller upgrade step208:05
matbumarios: ^08:05
marioso/ morning shardy.the manila review had a green run of ci at https://review.openstack.org/#/c/342961/ (see jenkins set 1) - i think we should consider landing https://review.openstack.org/188137 (manila) and https://review.openstack.org/#/c/339437/ (manila password tripleoclient)08:05
matbuonly on a specific hardware08:05
mariosmatbu: which step do you mean controller upgrade?08:05
mariosoh sry, i read that too fast08:05
matbumarios: major controller upgrade step 208:05
mariosyou mean controller_pacemaker_208:05
matbumarios: no pb, all the resource are unmanage08:05
*** tremble has quit IRC08:08
mariosmatbu: hmm it is strange they should be unmanaged... i mean we don't do that for pacemaker1/2 i mean it happens on converge when we do puppet/post-puppet08:09
mariosmatbu: so it would be from error/timeout i mean not expected at this step08:09
mariosmatbu: do you have logs at this point.would be good to capture info into a bug, i assume 8..9 related and specific bm setup? so probably bz at this point?08:10
matbumarios: nop i just have an env where i reproduce it08:11
*** akrivoka has joined #tripleo08:11
matbumarios: i'm looking at the issue for the moment08:11
matbumarios: i'm wondering if it not due to the size of the hardware08:12
mariosmatbu: yeah ok man, i mean capture whatever you can find but you're doing that08:12
matbumarios: (it's a virt env btw)08:12
*** tremble has joined #tripleo08:12
mariosmatbu: so are there very many cores/processors? or you mean not enough ram?08:12
matbumarios: yeah, not enough ram or core08:12
matbumarios: cause it looks like the cluster take too long time to bring resources up08:13
matbuand manage08:13
openstackgerritFlorian Fuchs proposed openstack/tripleo-ui: Remove app.conf to use a JS-based config file  https://review.openstack.org/34365808:13
mariosmatbu: you should see some kind of error like  "ERROR: cluster remained unstable for more than $timeout seconds, exiting." that we have in the /pacemaker_common_functions.sh or ./major_upgrade_controller_pacemaker_2.sh etc08:14
mariosmatbu: might be worth grepping on some of those strings to see in var/log/messages08:15
*** milan has quit IRC08:16
mariosmatbu: you can also try to manually increase those timeouts ./yum_update.sh:27:cluster_settle_timeout=1800 to test the theory08:16
*** derekh has joined #tripleo08:17
matbumarios: yes this is the error shown by heat08:18
*** Guest53586 has quit IRC08:22
*** flaper87 has joined #tripleo08:22
*** flaper87 has joined #tripleo08:22
mariosmatbu: ok well that is a good thing for the most part i think... i mean it sounds like h/w not some issue in the upgrade, though still might be...08:23
mariosmatbu: how much ram are we talking?08:23
mariosmatbu: and is this 3/1 net-iso or bigger?08:23
matbumarios: yep this is a 3 controller 1 compute, 1 UC and net iso08:24
*** lucas-afk is now known as lucasagomes08:25
gchamoulmarios: hi, thx for your patch https://review.openstack.org/#/c/342032/1 but could you please revive it?08:25
matbubandini: do you want me to re-kick the L to M job ?08:27
bandinimatbu: yes please it should fetch the very latest version of the https://review.openstack.org/#/c/338879/ review08:28
mariosgchamoul: done thanks :) didn't want to be 'stealing peoples reviews' so I thought you may have wanted to do it yrself08:30
mariosgchamoul: but because of the 9-related bugzillas i filed as you were away end of the week.08:31
gchamoulmarios: I was away! and you were right to do it! no worries ;-D08:32
matbubandini: k08:32
gchamoulmarios: thx!08:32
*** radek2 has joined #tripleo08:32
openstackgerritGael Chamoulaud proposed openstack/puppet-tripleo: Ensure the Heat Domain parameters before starting heat-engine pcmk service  https://review.openstack.org/34203208:33
mariosgchamoul: np & ty08:33
matbubandini: done here: https://ci.centos.org/view/rdo/view/tripleo-periodic/job/tripleo-quickstart-upgrade-major-liberty-to-mitaka/34/console08:36
bandinimatbu: perfect, thanks. will keep an eye on it08:38
bandiniso all my experimental jobs seem to fail with http://logs.openstack.org/87/338387/13/experimental/gate-tripleo-ci-centos-7-nonha-multinode/0386df0/console.html and all my normal jobs seem to fail due to delorean issues http://logs.openstack.org/04/337304/2/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/c42139c/console.html08:41
bandinierm, how can I help to fix this? Start with a bug or ping delorean folks?08:41
*** aufi has joined #tripleo08:41
jaosoriormarios: One small detail about that submissino08:45
*** shivrao has quit IRC08:46
*** saneax is now known as saneax_AFK08:46
mariosjaosorior: ok thanks for that... gfidente|afk can you confirm later https://review.openstack.org/#/c/188137/83/puppet/services/manila-base.yaml - though jaosorior as noted see the review that adds this in ci (and you can see manila logs, afaics there aren't errors there)08:48
dtantsurmorning folks, an easy patch to review please (1x +2, gate passed long ago): https://review.openstack.org/#/c/331584/08:50
openstackgerritMartin Mágr proposed openstack-infra/tripleo-ci: Add opstools candidate repo  https://review.openstack.org/32509508:51
*** milan has joined #tripleo08:53
jaosoriormarios; YEah,w e had a lot of those issues in another submission. So thankfully gfidente reviewed it and pointed out that if we don't use the NoBrackets version of that endpoint it will break in ipv608:55
*** gfidente|afk is now known as gfidente08:55
jaosoriordtantsur: Hey dude, have you been testing introspection in master?08:55
jaosorioryesterday we re-enabled it and it seems to be broken08:55
dtantsurjaosorior, that's why you don't disable things in gate :/ link to the error handy?08:56
dtantsurI surely remember it worked at some moment yesterday08:56
jaosoriordtantsur: http://logs.openstack.org/69/342669/1/experimental-tripleo/gate-tripleo-ci-centos-7-ovb-nonha/78969cb/console.html#_2016-07-19_06_15_59_17610208:56
*** jaosorior has quit IRC08:57
*** jaosorior has joined #tripleo08:58
*** tremble has quit IRC08:58
dtantsurwow, that's amazingly non-readable08:58
dtantsurhowever, it seems like the workflow itself is broken, which again brings us to "please don't disable things in gate" point :/08:58
jaosoriordtantsur: It's not like I went ahead and disabled it.08:59
shardyTo be fair, the nonha job had been broken for over two days when we finally gave in and disabled it IIRC08:59
shardyI agree it should be a last resort08:59
shardybut some coverage is better than everyone ignoring the job because it's red for days on end09:00
dtantsurand this last resort can probably only happen on a release day09:00
dtantsuror not even then09:00
dtantsurI mean, if neutron was broken, would you remove neutron from tripleo "temporary" or try to fix it?09:00
shardya failing job carries the same risk, because changes to all the puppet modules were getting landed with failing CI09:00
shardybecause the jobs stopped at introspection09:00
dtantsurwell, here we are: something breaks introspection and we have no idea what09:01
*** tremble has joined #tripleo09:01
jaosoriordtantsur, shardy: FWIW it seems to me that the mistral action for introspection is broken for some reason09:01
dtantsurInputException: Invalid input [name=baremetal_introspection.wait_for_finish, class=NoneType, unexpected=[u\'uuids\']]09:01
jaosoriorthat's as far as I could debug with my minimal knowledge of mistral09:01
jaosoriorwhich makes no sense cause the wait_for_finish function in python-ironic-inspector-client does take a list of uuids09:02
* shardy looks around for d0ugal09:02
jpichI think he's at EuroPython all week09:03
jaosoriordtantsur: Anyway, I poked you cause you might have an idea of what's up. But it seems that you're pretty much in the same spot as me09:03
dtantsuralso, can we please kindly move nonha from experimental gate?09:03
dtantsureverything in the experimental pipeline is broken by definition09:03
shardydtantsur: It runs on all puppet module changes09:04
shardywe don't have the capacity to run it on all tripleo repos until rh1 comes back online09:04
*** tzumainn has joined #tripleo09:04
*** electrofelix has joined #tripleo09:04
jaosoriorshardy: What about adding it to the puppet-tripleo repo?09:04
dtantsurshardy, well, then lets fix introspection after it's in the check pipeline09:05
dtantsurI don't see a reason bothering, if it can get broken the next patch09:05
dtantsurI know I'm a bit harsh, but we've learned all this the hard way in Ironic :)09:05
shardydtantsur: We've got a temporary lack of capacity, is it really too much to ask for folks to test things locally once per day until we get that capacity back?09:06
shardyWe found out when it broke because it is in the check pipeline for every single puppet module anyway09:06
dtantsuryes, it is too much. people don't do that, trust me09:07
shardyawesome attitude :\09:07
dtantsurI've seen a lot of patches that are apparently and terribly broken09:07
* shardy sighs, makes a coffee and pulls all-the-things so he can fix this himself09:07
dtantsurit was completely apparent that people never dared doing even simplest checks09:07
dtantsuralso the split between puppet-tripleo and THT makes testing a lot of things a black magic09:08
dtantsurI gave up on trying to test locally any changes that are split between these 2 repos09:08
dtantsur(it's not related to introspection, just a side note)09:08
*** tobbe_fiberdata has joined #tripleo09:09
jaosoriorshardy: the undercloud is running mistral by default now, right?09:11
jpichshardy: Maybe I can try to help? I have a tiny teeny bit of experience poking at Mistral now. Are there instructions somewhere on how to reproduce that kind of CI run?09:11
dtantsurjpich, do you have a link where mistral actions are defined?09:11
dtantsurI can't find the baremetal introspection wait_for_finish action09:12
shardydtantsur: it really isn't that hard, I'll write a blog post later and/or push a docs patch09:12
shardytl;dr upload-puppet-modules -d puppet-modules09:12
shardyit was discussed on last week's deep-dive call09:12
dtantsurwow, I've never heard about upload-puppet-modules thing. a blog post would be awesome09:12
* dtantsur has too many meetings already :(09:12
jaosoriordtantsur: This is the closest thing I've found to a definition in the mistral code-base https://github.com/openstack/mistral/blob/master/mistral/actions/openstack/mapping.json#L103409:13
jpichdtantsur: I think that'd be there for the general workflow -> https://github.com/openstack/tripleo-common/blob/master/workbooks/baremetal.yaml#L173 and around here https://github.com/openstack/tripleo-common/tree/master/tripleo_common/actions for actions09:13
jaosoriordprince is more acquainted with that part, but he's some hours from waking up still09:13
* jpich looks around too09:13
shardydtantsur: https://etherpad.openstack.org/p/tripleo-deep-dive-topics lines 23-31 is what I do09:13
jaosoriorjpich: well, the action that seems to be failing is in the mistral stock actions09:13
shardyhack on puppet-modules/tripleo and just re-run the upload script after making any changes09:14
dtantsurshardy, cool. still my biggest problem was ensuring that my local THT checkout is in sync with whatever ends up being on the image09:14
shardydtantsur: Yeah, this largely solves that09:14
dtantsurinteresting, I'll try the next time I debug my ironic thingy09:14
shardythe same method can be used to update individual RPMs on the node, but I'm not sure you can update both puppet modules *and* RPMs, that may be an RFE09:14
*** jaosorior is now known as jaosorior_lunch09:15
shardyjpich: Sure, if you can help reproduce that would be great09:16
shardyjpich: All I'm about to do is yum -y update my undercloud, then re-run openstack undercloud install09:16
dtantsurI really wonder why class is None in this case09:16
shardy(actually I think just running undercloud install is enough, but I like seeing what has changed)09:16
gfidentemarios, morning09:17
gfidentewould be nice to restore the ipv6 job indeed09:17
tobbe_fiberdatadoes anyone have a good tip of font to use?09:18
tobbe_fiberdatabest offtopic ever.09:18
*** akuznetsov has joined #tripleo09:19
shardywingdings ;)09:19
tobbe_fiberdatashardy, hahaha, funny man :p09:19
openstackgerritMarios Andreou proposed openstack/tripleo-heat-templates: Enable Manila integration - as a composable controller service  https://review.openstack.org/18813709:20
mariosgfidente: jaosorior_lunch thanks and done ^^^09:20
dtantsura side note: I really hope we're not planning to show things like http://logs.openstack.org/69/342669/1/experimental-tripleo/gate-tripleo-ci-centos-7-ovb-nonha/78969cb/console.html#_2016-07-19_06_15_59_176102 to users09:21
dtantsurit will scare the hell out of them :)09:21
tobbe_fiberdataI'm gonna have fun setting up a undercloud and overcloud for production. we'll see how that goes. putting up a lab on my local computer first lol. Wish me luck!09:21
jpichshardy: Ok, thank you, I'll go and do that too - don't let me slow you down if that wasn't actually all that time-consuming though!09:21
mariosgfidente: but gfidente grep tells me we have like ./puppet/services/neutron-server.yaml:69:            neutron::db::mysql::host: {get_param: [EndpointMap, MysqlInternal, host]}09:22
mariosgfidente: on master09:22
shardygood luck tobbe_fiberdata09:22
tobbe_fiberdatashardy, thanks09:22
gfidentemarios, yeah that's wrong09:22
shardyjpich: ack, thanks, I'll pulling the updates now, will see if I can reproduce09:22
jpichdtantsur: ...From what I've seen there's some space for improvement in terms of error reporting when using the workflows from the client09:23
*** bootsha has quit IRC09:23
shardyYeah I hit some non-obvious error paths when testing too, we'll have to improve them for sure09:25
*** mgould|afk is now known as mgould09:26
*** fzdarsky has joined #tripleo09:27
gfidentemarios, I have seen a submission fixing it though09:31
*** radek2 has quit IRC09:31
mariosgfidente: ok thx just checking09:31
*** saneax_AFK is now known as saneax09:35
ccamachoderekh marios gfidente morning! Quick question, I'm hit now by timeout_mins = 80 when checking a submission, is it possible to increase this time in a "test" submission and link the THT one with a depends-on? If so, can you point me where is defined this timeout?09:35
*** abehl has quit IRC09:36
shardyccamacho: it's calculated and based on a hard-limit set by openstack infra https://github.com/openstack-infra/tripleo-ci/blob/master/toci_gate_test.sh#L7309:37
shardyccamacho: you may find that increasing it simply means we hit the infra timeout and the job gets killed09:38
shardyso it'd be better to figure out where the time is going and optimize the walltime really09:38
shardymaybe that 90 can be reduced though, for jobs where we use cached images09:39
ccamachoshardy thanks, Im getting this executing a clustercheck, im my local env also takes time, Ill try to see then why is taking that long.. Thanks!09:40
*** sshnaidm|afk has quit IRC09:46
*** abehl has joined #tripleo09:47
*** yamahata has quit IRC09:48
*** akuznetsov has quit IRC09:48
openstackgerritMartin André proposed openstack/instack-undercloud: Introduce 'enable_ui' option  https://review.openstack.org/34414009:49
*** mcornea has joined #tripleo09:50
*** dsariel has quit IRC09:52
*** stendulker has joined #tripleo09:53
tobbe_fiberdatahow is the upgrade-path look like from mitaka to newton when there's a bit of differences between the versions?09:54
tobbe_fiberdatawhat i mean is for example the templates for network and such things?09:54
tobbe_fiberdataor perhaps it's to soon to say?09:54
shardytobbe_fiberdata: it's a bit too soon to say, but our aim is to allow existing templates for network configs etc to continue working09:55
tobbe_fiberdataah okey09:55
shardythat said, in https://review.openstack.org/#/c/271450/ I'm aiming to enable an alternative new format for os-net-config data09:56
shardyit's not landed yet tho, and the old method should continue working even if we deprecate it09:56
*** bootsha has joined #tripleo09:57
tobbe_fiberdatasounds great shardy!09:59
tobbe_fiberdatathat has always been the biggest problem from what i see. you deploy an environment for production then you never touch it as it works, when the time comes to move up a version or two it's hard to do it just like that. requires alot of planning10:00
tobbe_fiberdatai'm just talking for myself atleast :P10:00
tobbe_fiberdataand our environment running RHEL-OSP7. we are though not planning on running RHEL OSP in the future though. i hope that will not bring me problem.10:02
tobbe_fiberdatabut i bet it will10:02
openstackgerritJiri Tomasek proposed openstack/tripleo-ui: Stop Validation  https://review.openstack.org/33656110:05
openstackgerritJiri Tomasek proposed openstack/tripleo-ui: Triggering validations and messaging  https://review.openstack.org/33656010:05
openstackgerritJiri Tomasek proposed openstack/tripleo-ui: List Validations and Validation results  https://review.openstack.org/33477010:05
*** sshnaidm has joined #tripleo10:09
*** shardy has quit IRC10:16
*** tosky has joined #tripleo10:19
*** shardy has joined #tripleo10:20
*** limao has quit IRC10:24
*** abehl has quit IRC10:25
*** dsariel has joined #tripleo10:26
*** abehl has joined #tripleo10:26
akrivokais there any drawback to changing the parameter type of ControllerCount from number to string? then we could use the allowed_pattern constraint to limit it to odd numbers (allowed_pattern only works with strings and not with numbers).10:29
shardyakrivoka: It'll probably work if you ensure valid input via a contraint, but personally I'd rather add a number compatible constraint to heat10:31
*** devvesa has quit IRC10:32
openstackgerritAdriano Petrich proposed openstack/tripleo-quickstart: WIP gate upgrade  https://review.openstack.org/34216110:32
akrivokashardy: I tried adding a custom constrain, but heat people don't seem keen https://review.openstack.org/#/c/342265/10:32
*** pkovar has quit IRC10:34
shardyakrivoka: ack, thanks - I disagree and have +2'd the patch10:35
shardyakrivoka: I'll see if I can solicit some other opinions10:35
akrivokaheh, thanks shardy !10:35
shardyin the meantime the string method is probably OK if you're blocked on this10:35
tobbe_fiberdatai'm trying to get this virtual lab up but it says that there's no network devices and there's no bootable device. it does PXEboot.10:35
tobbe_fiberdatawhen i run introspection10:36
*** zoliXXL is now known as zoli|lunch10:40
*** jubapa has quit IRC10:43
*** rodrigods has quit IRC10:44
*** rhallisey has joined #tripleo10:44
*** rodrigods has joined #tripleo10:44
*** Goneri has joined #tripleo10:44
openstackgerritGiulio Fidente proposed openstack/instack-undercloud: Allow customization of undercloud disk size  https://review.openstack.org/34416110:45
dtantsurmarios, gate passed on https://review.openstack.org/#/c/331584/ again, mind +A please?10:45
*** devvesa has joined #tripleo10:46
*** jubapa has joined #tripleo10:46
*** jerrygb has joined #tripleo10:51
*** ccamacho is now known as ccamacho|lunch10:52
*** jerrygb has quit IRC10:56
*** osp has quit IRC11:00
*** jaosorior_lunch is now known as jaosorior11:03
hewbroccaAck early, Ack often!11:04
openstackgerritMartin Mágr proposed openstack/tripleo-puppet-elements: Install osops-tools-monitoring-oschecks package  https://review.openstack.org/32407511:06
matbubandini: it passed CI11:07
bandinimatbu: w00t thanks! marios ^11:08
*** pkovar has joined #tripleo11:08
openstackgerritMerged openstack/tripleo-heat-templates: gnocchi: configure db_sync command in the profile  https://review.openstack.org/34390211:10
*** stendulker has quit IRC11:12
openstackgerritMartin Mágr proposed openstack/puppet-tripleo: New composable service: Sensu  https://review.openstack.org/32330311:13
openstackgerritMartin Mágr proposed openstack/puppet-tripleo: New composable service: Sensu  https://review.openstack.org/32330311:17
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Enable keystone to use the SSL middleware  https://review.openstack.org/34135411:18
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Enable glance to use the SSL middleware  https://review.openstack.org/34136711:18
*** bootsha has quit IRC11:20
*** devvesa has quit IRC11:21
*** athomas has quit IRC11:23
*** weshay has joined #tripleo11:25
*** athomas has joined #tripleo11:28
*** radek2 has joined #tripleo11:32
*** bootsha has joined #tripleo11:35
mariosbandini: matbu: ack thanks11:39
mariosdtantsur: looking11:39
mariosdtantsur: done11:40
*** devvesa has joined #tripleo11:40
*** devvesa has quit IRC11:40
*** openstackgerrit has quit IRC11:41
*** openstackgerrit has joined #tripleo11:41
*** ccamacho|lunch has quit IRC11:44
openstackgerritGiulio Fidente proposed openstack-infra/tripleo-ci: Install Ceph Jewel from Ceph repos  https://review.openstack.org/34050411:44
openstackgerritMerged openstack/tripleo-common: Allow setting capabilities as a dict in addition to a string  https://review.openstack.org/33158411:46
*** lucasagomes is now known as lucas-hungry11:49
openstackgerritJuan Antonio Osorio Robles proposed openstack/python-tripleoclient: Handle empty list of introspected nodes  https://review.openstack.org/34418811:55
openstackgerritAdriano Petrich proposed openstack/tripleo-quickstart: WIP gate upgrade  https://review.openstack.org/34216111:56
*** jerrygb has joined #tripleo11:57
openstackgerritJuan Antonio Osorio Robles proposed openstack/instack-undercloud: Use certmonger for automatic cert generation  https://review.openstack.org/33277411:57
openstackgerritJuan Antonio Osorio Robles proposed openstack/instack-undercloud: Use certmonger for automatic cert generation  https://review.openstack.org/33277411:58
*** bfournie has joined #tripleo12:01
*** pkovar has quit IRC12:01
*** fzdarsky is now known as fzdarsky|afk12:01
*** jeckersb is now known as jeckersb_gone12:03
*** ccamacho has joined #tripleo12:05
sshnaidmfyi, https://bugs.launchpad.net/tripleo/+bug/160438012:06
openstackLaunchpad bug 1604380 in tripleo "CI: nodes registrtion in periodic jobs fail because of ironic-inspector error" [High,New]12:06
sshnaidmfyi2, https://bugs.launchpad.net/tripleo/+bug/160403912:07
openstackLaunchpad bug 1604039 in tripleo "CI: delorean build of tripleo-heat-templates fails because wrong spec" [Undecided,New]12:07
*** sshnaidm is now known as sshnaidm|afk12:07
openstackgerritwes hayutin proposed openstack/tripleo-quickstart: [WIP] Do not exit prior to the inventory of the deployment.  https://review.openstack.org/34150612:08
openstackgerritwes hayutin proposed openstack/tripleo-quickstart: [WIP] Do not exit prior to the inventory of the deployment.  https://review.openstack.org/34150612:10
openstackgerritCarlos Camacho proposed openstack/tripleo-heat-templates: Composable Mistral services  https://review.openstack.org/32343612:10
openstackgerritMerged openstack/puppet-tripleo: Add principal to certmonger's haproxy helper  https://review.openstack.org/34371012:12
*** zoli|lunch is now known as zoli12:17
openstackgerritwes hayutin proposed openstack/tripleo-quickstart: [WIP] Do not exit prior to the inventory of the deployment.  https://review.openstack.org/34150612:19
*** pkovar has joined #tripleo12:20
*** masco has quit IRC12:20
*** tobbe_fiberdata has quit IRC12:21
*** Goneri has quit IRC12:22
openstackgerritDmitry Tantsur proposed openstack/python-tripleoclient: Support node capabilities as dictionaries  https://review.openstack.org/34420612:22
openstackgerritJuan Antonio Osorio Robles proposed openstack/python-tripleoclient: Handle empty list of introspected nodes  https://review.openstack.org/34418812:24
*** jayg|g0n3 is now known as jayg12:25
EmilienMdtantsur: hey12:29
dtantsurhi EmilienM12:29
*** trown|outtypewww is now known as trown12:29
EmilienMdtantsur: I'm trying to promote Puppet CI to latest trunk and a commit since last 24h in Ironic is breaking tempest tests or making it unstable12:30
EmilienMour ironic logs are here: http://logs.openstack.org/40/344040/1/check/gate-puppet-openstack-integration-3-scenario002-tempest-centos-7/93fc562/logs/ironic/12:30
dtantsuryeah, I'm looking at them, but I don't see the API service doing anything12:32
*** radek2 has quit IRC12:33
EmilienMdtantsur: looks like 500 error in ironic API, see http://logs.openstack.org/40/344040/1/check/gate-puppet-openstack-integration-3-scenario002-tempest-centos-7/93fc562/console.html#_2016-07-19_10_50_21_04336612:34
dtantsuryeah, but again, there is no signs of it in ironic API logs, unless I misinterpret something12:34
dtantsurthe only candidate I can think of is https://github.com/openstack/ironic/commit/15c366580a18c77963d75cbc37e0805c96bc2a5012:35
EmilienMdtantsur: http://logs.openstack.org/40/344040/1/check/gate-puppet-openstack-integration-3-scenario002-tempest-centos-7/93fc562/logs/apache/ironic_wsgi_error_ssl.txt.gz12:35
honzajtomasek: how frequently should we publish tripleo-ui rpms?  on every commit, on release only?12:36
dtantsurEmilienM, yeah, it's probably my suspect. now I wonder why it does not fail in our gate12:37
*** Goneri has joined #tripleo12:37
hewbroccahonza: if you want the UI to eventually be tested in CI, then you probably want them published on every commit?12:37
openstackgerritMiles Gould proposed openstack/diskimage-builder: Allow installing proliant tools from an OS package  https://review.openstack.org/34421412:38
shardyhonza: Yeah as hewbrocca says you neet to get a delorean build every commit e.g so it lands in https://trunk.rdoproject.org/centos7/current/12:38
shardythat is what we'll use in upstream CI12:38
*** jdob has joined #tripleo12:39
honzahewbrocca: shardy: ok, makes sense, thanks // jtomasek12:39
*** rlandy has joined #tripleo12:40
*** sshnaidm|afk is now known as sshnaidm12:40
*** dprince has joined #tripleo12:41
jaosoriordprince: Hey dude, mistral question12:41
dprincejaosorior: hi, sup?12:42
jaosoriordprince: Where are the introspection actions defined in mistral? Seems that the wait_for_finish one is broken. As seen in this logs http://logs.openstack.org/69/342669/1/experimental-tripleo/gate-tripleo-ci-centos-7-ovb-nonha/78969cb/console.html#_2016-07-19_06_15_59_17610212:42
dprincejaosorior: 'mistral-db-manage populate' is the command that loads them12:44
*** jeckersb_gone is now known as jeckersb12:44
*** pradk has quit IRC12:44
dprincejaosorior: it uses the mapping.json file here...12:44
dprincejaosorior: http://git.openstack.org/cgit/openstack/mistral/tree/mistral/actions/openstack/mapping.json12:45
dprincejaosorior: see the baremetal_introspection actions in that file...12:45
jaosoriorwell, that seems straight forward12:45
jaosoriordprince: Any idea why it would be failing the way it does?12:45
dprincejaosorior: Invalid input [name=baremetal_introspection.wait_for_finish, class=NoneType, unexpected=[u\'uuids\']]\n'}12:46
dprincejaosorior: the last bit of the error shows the meaningful error12:46
openstackgerritMiles Gould proposed openstack/diskimage-builder: Allow installing proliant tools from an OS package  https://review.openstack.org/34421412:47
jaosoriordprince: right, but what puzzles me is that wait_for_finish does take an input like [u\'uuids\']]12:47
jaosoriorwhich is why I don't understand why it's failing12:47
dprincejaosorior: looks like we changed the interface or something. I had it working but we did jiggle the interface names a bit during review12:47
jaosoriorand what is that class=NoneType?12:47
*** [1]cdearborn has joined #tripleo12:48
openstackgerritwes hayutin proposed openstack/tripleo-quickstart: [WIP] Do not exit prior to the inventory of the deployment.  https://review.openstack.org/34150612:50
EmilienMjaosorior: hey dude, quick question: https://review.openstack.org/#/c/343661/ fails - do you think it's related again to quotes? ie; https://review.openstack.org/#/c/343661/3/puppet/services/nova-api.yaml12:50
dprincejaosorior: do you see the baremetal actions getting deployed? 'mistral action-list | grep baremetal'12:51
jaosoriorEmilienM: That is ineed the case12:53
*** myoung has quit IRC12:53
jaosoriorEmilienM: The resulting hieradata is12:53
jaosoriornova::cron::archive_deleted_rows::destination: /dev/null12:53
jaosoriornova::cron::archive_deleted_rows::hour: */1212:53
EmilienMjaosorior: :(12:53
EmilienMjaosorior: so I should put "'blabla'" right?12:53
dprincejaosorior: is your patch like passing an empty array or something?12:54
*** egafford has joined #tripleo12:54
jaosoriordprince: Actually the patch sets inspector to a TLS endpoint, and does nothing to the nodes12:54
openstackgerritwes hayutin proposed openstack/tripleo-quickstart: [WIP] Do not exit prior to the inventory of the deployment.  https://review.openstack.org/34150612:54
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: Remove ::nova::cron::archive_deleted_rows  https://review.openstack.org/34366112:55
*** myoung has joined #tripleo12:56
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: Remove ::nova::cron::archive_deleted_rows  https://review.openstack.org/34366112:56
*** weshay is now known as weshay_mtg12:57
flaper87Am I right to assume that there's currently no way to call this without creating a small script? https://github.com/openstack/tripleo-common/blob/master/tripleo_common/image/image_uploader.py#L2612:59
dprinceEmilienM: shouldn't we be filing bugs for all the remaining composable work?12:59
EmilienMdprince: what roles remain?13:00
flaper87It looks like that code is new(ish)13:00
EmilienMdprince: it's all done, except aodh but it's tracked13:00
dprinceEmilienM: all of this hiera code is a remnant of composability13:00
EmilienMyeah I can file a but for that13:00
dprinceEmilienM: i.e. we havent' fully decomposed things until it is all moved out I think13:00
dprinceEmilienM: I was going to file bugs for my incoming stuff13:00
EmilienMdprince: https://review.openstack.org/#/c/341616/ looks ready13:01
dprinceflaper87: yes, we had "old" version of image build and image upload tools a long time ago in tripleo_incubator13:01
EmilienMjaosorior: ^13:02
EmilienMjaosorior: thx for doing recheck, it's now passing CI13:02
flaper87dprince: ok, so this is the "new" version that we'll use. I'm asking so I can create a script for it13:02
flaper87dprince: kinda need it13:02
EmilienMdprince: /me filing bugs now13:02
flaper87Based on the contents of tripleo-common/scripts I'd assume the right thing to do is to add a script there13:02
flaper87Like, `tripleo-upload-images`13:03
flaper87Or should this be a mistral action?13:03
*** rook-rocketship has quit IRC13:03
dprinceflaper87: is this something like what you are looking for? https://review.openstack.org/#/c/255535/13:04
flaper87dprince: yup13:05
flaper87dprince: exactly that, I guess13:05
flaper87will review13:06
flaper87dprince: and thanks13:06
*** lucas-hungry is now known as lucasagomes13:06
flaper87(review == I'll cry like a baby to get that - or something like that - in asap)13:06
flaper87mmh, 9 weeks ago13:07
flaper87jpeeler: dprince could you share why tripleoclient was chosen over a script in tripleo_common? https://review.openstack.org/#/c/255535/13:08
flaper87not saying it's wrong, I'm just curious to know if we're moving things into tripleo client now13:08
flaper87or what the logic is there13:08
* flaper87 is still tying pieces together in his head13:08
dprinceflaper87: I like scripts too FWIW13:09
*** rook-rocketship has joined #tripleo13:09
*** pkovar has quit IRC13:09
*** lblanchard has joined #tripleo13:09
dprinceflaper87: there are some in TripleO who argue that scripts can't easily be unit tested though. SO I think python was prefered by some for this reason13:10
jpeeleri just built off of what was already done13:10
jpeelerthe docker image uploader is heavily based on the glance image uploader13:11
EmilienMdprince: do you want me to create a single bug to track Hiera move?13:11
flaper87dprince: sure but the script could've been a python script too. That said, I get the point13:11
flaper87jpeeler: fair enough, thanks for sharing :)13:11
flaper87jpeeler: are you planning to rebase that patch?13:11
flaper87jpeeler: mind if I take over ? Is there anything blocking it ?13:11
*** akuznetsov has joined #tripleo13:11
jpeelerflaper87: that's a good question, do people still want to have the docker functionality?13:11
flaper87jpeeler: I believe the answer is yes but I'll defer to dprince or folks that have been around longer13:12
jpeelerflaper87: i don't mind at all. note that the dependent tripleo-common review is still pending as well13:12
flaper87jpeeler: yes sir, I'll take those over too13:13
jpeeleroops i didn't mean to say glance image uploader. i meant image builder13:13
jpeeleri was told to create that too a long time ago13:13
jpeeler(the glance image uploader) not being real clear here13:14
flaper87gotcha, thanks!13:14
jaosoriorEmilienM: so, regarding the issue with default values for resources. I could fix that with hiera, right? So here https://review.openstack.org/#/c/332774/24/elements/puppet-stack-config/puppet-stack-config.yaml.template If I want to configure a CA for every certmonger cert. would setting certmonger_certificate::ca: {{CERTIFICATE_GENERATION_CA}}13:15
EmilienMjaosorior: yes I think so13:15
jaosoriorso setting that ^^ would use the same ca parameter for all the certmonger_certificate instances, right?13:15
jaosoriorok, gotta try then13:15
dprinceEmilienM: how about something like this? https://bugs.launchpad.net/tripleo/+bug/160441213:16
openstackLaunchpad bug 1604412 in tripleo "Composable services: move hiera settings out of compute role" [Undecided,New] - Assigned to Dan Prince (dan-prince)13:16
EmilienMdprince: well, you want to create multiple bugs then13:17
EmilienMfor controller, etc13:17
dprinceEmilienM: sure, why not13:17
EmilienMmhh ok13:17
openstackLaunchpad bug 1604414 in tripleo "Composable services: move hiera data out of controller role" [Medium,New]13:18
dprinceEmilienM: ^^13:18
*** d0ugal has joined #tripleo13:19
dprincejistr: hi, I would like to get this in https://review.openstack.org/#/c/313544/13:20
dprincejistr: one question, instead of a manifest. Could we live with just 'service_names'?13:21
jistrdprince: yea i guess. Don't we want to capture some service properties too perhaps?13:21
dprincejistr: specifically we could just call the services 'glance-api' and 'glance-api-pacemaker'13:21
dprincejistr: and I think that would give you everything you need13:21
dprincejistr: the service properties to me are similar to 'config' I guess13:22
jistrok so we could capture things as hiera keys maybe if needed?13:22
dprincejistr: I could live with the manifest blob. but it seems a bit overkill when I think all we need right now is just a list of services13:22
jistrdprince: ok i'm fine with just a list13:23
dprinceshardy: is there a way get_attr a nested stack that will give you the "name" of the file or something?13:23
jistrdprince: the service names will probably still need to be a keys in a map though, right? so that we can do merge on them13:24
jistror maybe YAQL could help?13:24
*** pkovar has joined #tripleo13:24
dprincejistr: okay, I will try it. I may leave your patch there for comparison for now... but I will co-author you for the good idea. Sound good?13:24
dprincejistr: I was going for an array. Is an array not enough?13:25
jistrdprince: sounds good :)13:25
*** akshai has joined #tripleo13:25
dprincejistr: re. the merging. When would an array not be enough? I guess we can debate that in the implementation though13:26
jistrdprince: yea is fine. just wondering now how does heat fetch array items and merge them into an array13:26
dprincejistr: I would just list_join all the service names and pass them wherever we want them (probably into hiera)13:26
jistrdprince: list_join produces a string though, no?13:26
jistrdprince: i guess we could utilize the Heat->Hiera bug to make an array out of a string :D13:27
dprincejistr: yeah, list_join does give us a string. which like you point out may be fine for now13:28
dprincejistr: perhaps just the raw output (array) from the resource_chain for this new attribute set would be preferable then13:28
dprincejistr: we can play with both implementations and see what works...13:29
jistrdprince: yea right, that's what i was missing. If it's a feature of a resource chain that if you define a string on each resource in the chain, and then query the chain as a whole and get array of strings, then we could perhaps do just that instead of feeding it through list_join.13:30
dprincejistr: I think that is how it works. So we might be good13:30
*** dtantsur is now known as dtantsur|mtg13:31
jistrdprince: ack, thanks. Was thinking it would be nice to start converging to using correct data types where possible, so that one day we could hopefully fix the Heat->Hiera bug.13:31
jistrdprince: now that we have YAQL we shouldn't need to pass 'raw' things into Hiera i hope13:31
dprinceEmilienM: ^^^ I will use this mechanism for the firewall rules mechanism13:31
flaper87dprince: if you have 5mins, mind taking a look at https://review.openstack.org/#/c/309623/ ?13:31
dprincejistr: agree13:31
*** abehl has quit IRC13:33
openstackgerritSteven Hardy proposed openstack/tripleo-heat-templates: Move Keystone and Neutron Server firewall rules into the service  https://review.openstack.org/33852713:35
EmilienMdprince: shardy is producing a PoC of it I guess13:36
shardydprince, jistr, EmilienM: that's one way we could handle it, e.g move the map_merge so we pass both a list of config_settings and the merged map?13:36
shardyEmilienM: I just hacked in my idea for discussion, hope you don't mind me pushing it :)13:36
EmilienMshardy: I love it13:36
dprinceEmilienM: is he? shardy do you have a patch for this?13:36
shardyI think we can use the same method to generate the services that are enabled13:37
shardye.g just add service_name to each config_settings block13:37
shardydprince: I just pushed it to https://review.openstack.org/#/c/338527/7/puppet/controller.yaml13:37
*** bfournie has quit IRC13:37
*** fultonj has joined #tripleo13:37
*** fultonj has quit IRC13:37
*** ayoung has joined #tripleo13:38
shardyAh, although the hiera keys will collide won't they13:38
*** fultonj has joined #tripleo13:38
shardyI'll have to use repeat to add a prefix13:38
dprinceshardy: I would like to not change the services interface at this point I think13:39
shardydprince: why?13:39
*** d0ugal has quit IRC13:39
shardydprince: this actually doesn't change the services interface13:39
shardyonly what we pass to the node in hiera13:39
dprinceshardy: well, I guess so long as the end result is the same. Internal refactor is fine13:39
dprinceshardy: if it works I could go for it I guess13:40
shardydprince: open to other ideas, but FWIW I've been trying some stuff with yaql to enable composable bind_ip's and it gets pretty scary13:40
dprinceshardy: I was going to take a slightly different approach and "prefix" the firewall rules with the service name13:40
EmilienMlet's see if it works, I'll let you know13:40
shardyI actually implemented a new map_replace function for heat this morning so I could avoid using it ;)13:40
EmilienMwow :)13:40
shardydprince: Cool, happy to consider either approach, this is just a 5min idea I had :)13:41
dprinceshardy: I wasn't targetting this mechanism as something we would use for bind/local IPs.13:41
dprinceshardy: I don't see a good way of moving those out just yet, but they aren't causing any harm to maintain in a single blob I think either13:41
shardydprince: No, I was thinking we could either do a hiera lookup for the contents of the NetIpMap, or a map replacement on the merged config_settings13:41
dprinceshardy: we could template them I guess13:42
*** zoli is now known as zoli|brb13:42
dprinceshardy: yep, that could work13:42
dprinceshardy: anyways EmilienM wanted help with firewall rules. Which is different13:42
shardyYeah, I think we just need to dump the content of NetIpMap onto each node and somehow make the services/* config_settings reference it13:42
dprinceshardy: and I think so long as we use map_merge the output would be the same13:42
dprinceshardy: so I was going to create a "prefix" mechanism and just go with that.13:43
shardydprince: Cool, go for it, I'll give this a try anyway & we can compare13:43
shardyEmilienM: I'll push any further prototyping under a different ChangeId so I don't stop on your patch again13:44
shardystomp I mean13:44
dprinceshardy: correct me if I'm wrong. But all you are doing is moving the map_merge location right? and regardless of where we do that it is still going to merge up duplicate keys right?13:44
dprinceshardy: I mean it will...13:44
EmilienMI'm a bit confusing by all your ideas right now13:44
EmilienMI just want to use hiera_merge which is an hiera feature we can't use right now13:45
dprinceshardy: I don't think this works with what EmilienM intends... where as the service_names prefix idea will I think13:45
shardydprince: No, I'm passing the unmerged and merged data to the node via hieradata13:45
shardydprince: because EmilienM said he wanted to use hiera to merge it13:45
shardydprince: Ok, no worries, just trying to kick-off the discussion13:46
*** abehl has joined #tripleo13:46
ccamachoguys did anyone installed tripleo-ui in the undercloud?13:46
dprinceshardy: I see. So the service_configs_list is what you were going for13:46
dprinceshardy: it could work, but the duplicate data structures with the same data are annoying I think13:47
shardydprince: Yeah, then you could look up a list for all services in hiera13:47
dprinceshardy: I gotta say. What I really want here is a proper hiera hook13:47
shardydprince: Initially I added config_settings_list to the service templates, but then I thought this was a lower impact way (despite the data duplication)13:47
dprinceshardy: and then we have a separate hiera file for each service I think. ANd those merge in hiera very nicely13:47
shardydprince: Yeah, did you see stevebaker raised a bug about that?13:48
shardyapparently the o-a-c based hiera script is pretty slow13:48
dprinceshardy: I hadn't seen his bug13:48
shardyso we may want to rewrite it for performance reasons anyway13:48
dprinceshardy: but the script way isn't great. So many o-a-c calls13:48
openstackLaunchpad bug 1596373 in tripleo "40-hiera-datafiles takes over 20 seconds to run" [High,Triaged]13:49
dprinceshardy: throwing twice as much hiera data at the hiera element is only going to make it slower FWIW13:49
EmilienMweekly meeting in 10 min, please add off items if needed: https://etherpad.openstack.org/p/tripleo-meeting-items13:50
shardydprince: Yeah I guess so, but we can fix that - do you have any examples of your proposed approach for comparison?13:50
dprinceshardy: I explained to Emilien that I thought the service names approach was a reasonable compremise now. ANd that long term a proper hiera hook that lives here http://git.openstack.org/cgit/openstack/heat-templates/tree/hot/software-config/elements13:51
dprinceshardy: no code yet. I was about to do it today based on my earlier conversation w/ jistr. EmilienM said you were working on it as well so that is why I asked13:51
dprinceshardy: my approach would require a 'service_name' in each nested stack13:52
EmilienMno, just the PoC for composable variables13:52
dprinceEmilienM: these things are related because they are going after similar mechanisms I think13:52
EmilienMoh nevermind, yes indeed :)13:52
dprinceshardy: anyways, we probably want the service names in the nested stacks regardless because it helps with other things like upgrades. I think it would be useful to go on and add that now13:53
shardydprince: +1 that sounds like a good first step, agreed13:53
ayoungshardy, what are you using for a development setup?  Are you working single-system virt, or multiple physical?  HA?13:54
ayoungdprince, same quesion for you.  We need to get people up to speed on OOO dev and I want to make sure we are doing roughly the same thing as the core devs are doing13:56
ayoungI was under the impression that Quickstart had become the developers tool of choice, but I am starting to question that assumption13:58
*** yamahata has joined #tripleo13:58
EmilienMmeeting in 2 min folks13:58
ccamachoayoung /me using tripleo.sh13:59
ayoungEmilienM, in #openstack-meeting?13:59
EmilienMayoung: -alt13:59
dprinceayoung: I still use a lot of tripleo-incubator and tripleo.sh too13:59
ayoungccamacho, thanks13:59
shardyayoung: I'm using instack-virt-setup and tripleo.sh https://paste.fedoraproject.org/392510/4689367513:59
dprinceayoung: I think we have a mix of old and new tooling at this point13:59
shardybecause it's clsoe to what we currently CI13:59
jaosoriorayoung: I'm using quickstart and tripleo.sj14:00
jaosorior* tripleo.sh14:00
ayoungdprince, shardy ccamacho and focused on HA dev, not .... whateve we call without-ha14:00
jaosoriorayoung: http://jaormx.github.io/2016/deploying-master-tripleo-quickstart/14:00
*** pradk has joined #tripleo14:00
ayoungjaosorior, rock on!14:00
shardyayoung: No, I don't test much with HA due to lack of ram for multiple controllers14:01
ayoungshardy, OK...so we are all in the same boat :)14:01
ccamachoayoung in my case both ha and non-ha  usually following http://www.anstack.com/blog/2016/07/04/manually-installing-tripleo-recipe.html or  http://www.anstack.com/blog/2016/07/07/tripleo-deployment-with-inluch.html  and then playing with the openstack overcloud environments options14:02
jaosoriorayoung: I usually only do ha to be honest14:02
ayoungccamacho, added to my feedly!  Thanks14:02
ayoungjaosorior, multiple physical machines?14:03
ccamachoayoung nice!14:03
jaosoriorayoung: virtual14:03
ayoungjaosorior, how many VMs of what size do you need in order to do that?  I think we only have one machine on the team capable of running sufficient14:05
gfidenteayoung, though you can deploy with HA on single node14:05
openstackgerritJuan Antonio Osorio Robles proposed openstack/instack-undercloud: Use certmonger for automatic cert generation  https://review.openstack.org/33277414:05
ayounggfidente, a quorum of 1?14:05
gfidentethat reads as, with pacemaker even though running on a single controller which is closer to what you want I think14:05
gfidenteayoung, yeah it'll go through the pacemaker codepath14:06
gfidenteeven though there is a single controller14:06
ayounggfidente, yeah, I want to make sure that I am coding in the right set of files to make things work in an HA environement, but only have a single controller's worth of VMs available14:06
jaosoriorEmilienM: hey dude, I need some help with https://review.openstack.org/332774 it's still not taking the CA I'm trying to set as default, and I tried to set it both via hiera and via the resource declaration14:06
jaosoriorcan you take a look?14:06
ayounggfidente, I'm headed to the Keystone midcycle later on today, and we are going to be heavily discussing 0 downtime upgrades.  I want to be able to apply that to Tripleo as soon as we have it working in Keystone14:07
ayoungand I think HA is going to have to play nice with that14:07
ayoungI suspect that what I will really need to test it is a 2 controller node set up14:07
*** akuznetsov has quit IRC14:07
ayoungWhich I think I can squeeze out in my current setup....14:08
jaosoriorayoung: this is pretty much my quickstart config http://paste.openstack.org/show/537433/14:08
*** liverpooler has quit IRC14:08
*** jcoufal has joined #tripleo14:08
ayoungjaosorior, how much RAM do you need to make that happen?14:09
jaosoriorayoung: My machine has around 32G in total14:09
ayoungjaosorior, ok, that is what I have...I'll try your setup\14:09
*** rook-rocketship has quit IRC14:10
*** zoli|brb is now known as zoli14:10
*** zoli is now known as zoliXXL14:10
ayoungjaosorior, BTW, do you symlink ~/.quickstart/playbooks etc to some other repo?  I'14:10
ayoungve had to do that for a few directories in order to have my git repo survive a clean wipe14:10
jaosoriorI don't14:11
EmilienMjaosorior: will look after meeting14:11
ayoungjaosorior, I think it was because I was writing my own playbooks, not just roles14:12
*** bfournie has joined #tripleo14:12
*** rook has joined #tripleo14:13
*** rajinir has joined #tripleo14:17
*** d0ugal has joined #tripleo14:18
weshay_mtgmatbu, apetrich ping re: https://review.openstack.org/#/c/336442/14:20
apetrichweshay_mtg, looking14:20
weshay_mtgapetrich, matbu wondering since we're hitting issues w/ mitaka.. if we could fire up the third party ci for upgrade starting w/ liberty -> mitaka14:21
*** abehl has quit IRC14:21
weshay_mtgapetrich, take a look at EmilienM's comment on that review14:22
apetrichyeah I just did.14:22
apetrichweshay_mtg, I was thinking the same lines. I'll run one14:22
weshay_mtgEmilienM, ^  k.. thank you apetrich14:22
*** d0ugal has quit IRC14:23
apetrichweshay_mtg, EmilienM but right now we just upgrade the UC that has little effect on it. I'm changing the patch for a full upgrade in a few minutes14:23
EmilienMooo meeting in progress FYI14:24
weshay_mtgapetrich, for liberty -> mitaka you have the full upgrade..14:24
weshay_mtgya.. sry14:24
weshay_mtgwill wait14:24
*** paramite is now known as paramite|afk14:24
*** d0ugal has joined #tripleo14:24
weshay_mtgapetrich, we need to figure out what version he needs there.. but I suspect liberty -> mitaka14:24
weshay_mtgwhich is green14:24
apetrichthat patch is for master14:24
apetrichI'm affraid it might bring more stuff than we are gating14:25
matbuEmilienM: there is no working upgrade job atm for master14:27
EmilienMmatbu: one sec please, we're in meeting14:27
*** radek2 has joined #tripleo14:30
*** skramaja has quit IRC14:30
*** ohamada_ has joined #tripleo14:32
*** ohamada has quit IRC14:32
*** jdob has quit IRC14:34
*** abehl has joined #tripleo14:34
*** jdob has joined #tripleo14:34
*** pgadiya has quit IRC14:36
*** apetrich has quit IRC14:38
*** skramaja has joined #tripleo14:39
*** apetrich has joined #tripleo14:42
*** jdob has quit IRC14:43
*** jdob has joined #tripleo14:43
openstackgerritPradeep Kilambi proposed openstack/instack-undercloud: Fix ceilometer auth url on undercloud  https://review.openstack.org/34427914:48
*** julim has joined #tripleo14:49
openstackgerritPradeep Kilambi proposed openstack/instack-undercloud: Fix ceilometer auth url on undercloud  https://review.openstack.org/34428414:54
*** devvesa has joined #tripleo14:58
*** jaosorior has quit IRC14:58
*** d0ugal has quit IRC14:59
*** fzdarsky|afk is now known as fzdarsky15:00
*** beagles is now known as beagles_brb15:00
*** paramite|afk is now known as paramite15:01
*** dtantsur|mtg is now known as dtantsur15:02
*** pkovar has quit IRC15:03
*** mgarciam has joined #tripleo15:04
*** tobias_fiberdata has joined #tripleo15:04
jpichflorianf: I took the liberty of subscribing you to https://blueprints.launchpad.net/tripleo-ui/+spec/tripleo-ui-mistral-refactoring - if you want to flesh out a bit or correct my mistakes in the whiteboard re:plan creation, that'd be awesome :-)15:08
*** kairat has joined #tripleo15:11
*** jcoufal_ has joined #tripleo15:12
*** weshay_mtg is now known as weshay15:13
*** bootsha has quit IRC15:13
*** jcoufal has quit IRC15:15
ayoungtrown, as my team has been trying to (and failing to) affect change in the deployment, I wonder where the line is drawn between what is on the image and what is done by tripleo.  Am I correct in understanding that puppet modules are in the image and hiera data is in the Metadata server, and this is the contract between heat and the controller?15:16
*** milan has quit IRC15:16
*** pkovar has joined #tripleo15:19
*** tobias_fiberdata has quit IRC15:22
shardyayoung: Yes the puppet modules are in the image, and the hieradata is provided via heat metadata15:24
shardyayoung: however for development the puppet modules can be uploaded at deployment time15:24
shardye.g to enable a quicker hack/test loop15:25
shardysee lines 23-31 there15:25
ayoungshardy, ah, thanks15:25
shardybasically we have a script that can download a tarball and unpack it on the node before we run puppet15:25
shardythat script automates pushing it into the undercloud swift and generating a tempurl which is read by the script on deployment15:25
ayoungshardy, so, for Federation, I have a similar issue.  We have a tool that is used to register the Service provider (Keystone in this case) with the SAML2 IdP (Keycloak)15:26
ayoungthe tool is designed to run on the host15:26
ayoungbut, in an HA world, that sounds like it is violating the separat ion of concernts15:26
*** athomas has quit IRC15:26
ayoungas the data would have to be synchronized across multiple HA-controlled Keystone servers15:26
shardyayoung: Not necessarily. we deploy a "bootstrap_nodeid" which allows you to run things only on one node15:27
shardye.g we only run DB sync for services once, on the bootstrap node15:27
shardysounds like this is similar15:27
ayoungshardy, yes15:27
ayoungshardy, so the workflow is something like this:15:28
*** marios has quit IRC15:28
*** ebalduf_ has joined #tripleo15:28
shardyayoung: that shows how the data is consumed15:28
*** marios has joined #tripleo15:28
shardyin various places there are conditionals in the puppet profiles15:28
*** ohamada_ has quit IRC15:29
ayoung1:  call three keystone APIs to build Idp, mapping, protocol.  either before or after, run the python based tool to register.  So all that sould be done once, and then the generated files would get synced15:29
*** athomas has joined #tripleo15:31
openstackgerritMerged openstack/tripleo-heat-templates: Repurpose KeystoneWorkers add keystone::wsgi::apache::threads  https://review.openstack.org/33647215:31
ayoungshardy, in the ansible based prototype we had, we made keystone query calls to see if the various objects had been made to avoid re-creating the objects, and used the file objects to avoid re-calling the registration tool.  I'm guessing...all of that should be done by puppet modules?  And then...the hiera data would have a subsection that would reflect that structure?15:31
shardyayoung: Yeah the first step is to wire in the logic needed to the puppet-keystone module15:32
ayoungshardy, the problem is that the federation puppet module was written for a very different use case15:32
shardyayoung: to clarify, you run something on one node, then we have to automate copuing things to other nodes?15:32
ayoungshardy, someone squatted on the good name.  That is really for Keystone2Keystone federation15:33
greghaynesianw: cinerama I have an old patch https://review.openstack.org/#/c/181162/ that might be off your review radars - I'm hoping to get that merged before another regression slips in15:33
ayoungshardy,  yes15:33
shardyayoung: Ok, I guess you can take that up with EmilienM and the other puppet-keystone maintainers15:33
shardyfrom the tripleo perspective, it can be any module, that just seems like a logical place to wire it in15:34
*** beagles_brb is now known as beagles15:34
ayoungshardy, what is the relationship between puppet and hiera data?  What says that puppet should find the set of Idps (or anything else) to manage from this subsection of hiera?15:34
EmilienMshardy: what? can I have tl;dr15:34
ayoungEmilienM, Federation15:34
ayoungEmilienM, Keystone puppet module stuff...I bugged you about it a few months back15:35
shardyayoung: on each node we create a bunch of yaml files under /etc/puppet/hieradata15:35
*** numans has quit IRC15:35
*** d0ugal has joined #tripleo15:35
shardyayoung: basically they are key/value pairs that are passed into the puppet modules that get applied15:35
ayoungshardy, OK, my HA deploy via quickstart seems to have just completed.  So let me look at the hiera data for Keystone and see if I can make sense of it15:36
shardyayoung: https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/services/keystone.yaml#L10515:36
EmilienMI'm not sure how can I help here, but please let me know15:36
ayoungshardy, I should look at the cached data in /var, right?15:36
shardythat's where it is defined in the heat templates, if you look inside /etc/puppet/hieradata/service_configs.yaml you should see the data (assuming you're on trunk not mitaka)15:37
EmilienMtrown: we can wait before landing https://review.openstack.org/#/c/341616/15:38
ayoungshardy, my understanding of hiera is that there is pre-packaced stuff in /etc/puppet/ but that gets overlayed by os-collect-config15:38
EmilienMtrown: when ready, feel free to +a15:38
ayoungin /var/lib/os-collect-config/  I see a bunch of json files15:38
shardyayoung: No, we just write some files to /etc/puppet/hieradata15:38
ayoungshardy, we update /etc after the node boots?15:38
shardyayoung: well, yeah, but that's kinda confusing the issue, we use os-collect-config and some other agents to write the hieradata yaml files, and then later to run puppet15:39
ayoungok...the json files look similar15:39
trownEmilienM: cool, I will test it right now15:39
shardyayoung: Yes, we write those files after the node boots, here:15:39
EmilienMtrown: thx15:39
*** Goneri has quit IRC15:40
ayoungshardy, I'm a little confused at how orchestration happens.  There are a bunch of steps, like heat_config_ControllerOvercloudServicesDeployment_Step4.json15:42
ayoungI assume there is a callback from the controller to heat saying "ok this is done, give me the next"15:42
ayoungand then the next chunk of metadata is posted?  More hiera data?15:43
shardyayoung: Yes, we apply puppet six times, and have conditionals in the puppet code like if $step >= 315:43
shardyayoung: The hieradata is all written out at the start, we just re-run puppet with the same manifest and a different "step" variable15:43
ayoungshardy, does heat drive that, or is it all done on the node, and heat just collects the status?15:44
shardythis allows us to re-assert all previous steps each time, so e.g step 5 doesn't undo something step3 did by mistake15:44
shardyayoung: http://hardysteven.blogspot.co.uk/2015/05/heat-softwareconfig-resources.html15:44
shardytha may help, as will the deep-dive recording from last week that I did15:45
ayoungshardy, OK, so the Federation code needs to be run after everything else is set up for keystone.  It will, as a last step, need to restart the HTTPD instance15:45
ayoungand that kind of restart needs to play nice with the HA/pacemaker, right?15:45
shardyayoung: tl;dr heat just updates the metadata with a new step value, which makes os-collect-config re-run the agents to run puppet15:45
ayoungshardy, Oh, so Heat *does* control the step15:46
* ayoung lightbulb15:46
shardyayoung: yes ^^15:46
shardyayoung: I'd say focus first on getting puppet to run things at the time you want, then we can talk later about the copying between nodes requirement15:48
ayoungshardy, OK...so that is how Heat orchestrates across multipe servers, too.  So in the case of the Federation generated files, Heat would wait until the generation was done on one server, and then sync it to each in turn, triggering the HTTPD restart15:48
ayoungshardy, I want to spec the process out, soup to nuts15:49
shardyayoung: Ok, well FYI there's a missing interface for the sync/restart part that will require some thought15:52
ayoungshardy, for HTTPD?15:53
shardyone way we might do that is just generate a writeable tempurl on the undercloud swift, then pass it in so the master node writes some data, and the non-master nodes poll the bucket waiting for it to be there15:53
shardyayoung: to have one node pass data to the other nodes15:53
*** akuznetsov has joined #tripleo15:54
*** pkovar has quit IRC15:54
ayoungshardy, I'm a little worried about that.  The SAML stuff, I think, uses a private key.  Ugh, this is a mess15:54
ccamachothrash around?!15:55
*** tremble has quit IRC15:55
ayoungwe want the IdP to think of it as a single Service provider, but then we have 3+ instances actually saying "here is the rone and true identity"15:55
ayoungshardy, let me confer with the SAML2 experts on this and see if we can come to some sane approach15:55
*** pkovar has joined #tripleo15:56
shardyayoung: Ok there are other ways but they all involve bouncing the data via the undercloud (either via heat or swift)15:56
shardythere's no peer-to-peer channel available15:56
*** oshvartz has quit IRC15:56
gfidenteshardy, ayoung, I think the wanted behaviour though is similar to what we need to fix the swift rings building15:56
*** devvesa has quit IRC15:56
shardygfidente: Yeah, I was just thinking that15:56
*** tremble has joined #tripleo15:56
*** tremble has joined #tripleo15:56
gfidenteas we'll want one node to run the build and the resulting file to be copied on the others15:56
shardyperhaps we can solve both15:56
hewbroccaI was just gonna say, don't we already do this with swift?15:56
hewbroccaoh, we don't15:57
shadowershardy: sorry, I'm on vacation and didn't attend the meeting15:57
shadowershardy: could you summarise it, please?15:57
openstackgerritDan Prince proposed openstack/os-collect-config: Only re-exec on config change or HUP  https://review.openstack.org/34433815:58
hewbroccaIt's simple, we just need to finish off that reimplementation of puppetdb15:58
shardyshadower: sure, do you want to maintain tripleo-validations in launchpad, or just use a tag in the main tripleo launchpad?15:58
shardyshadower: basically there's some fragmentation wrt tracking releases and we're trying to have a discussion around cleaning things up and consolidating15:59
hewbroccahey dprince that review above ^^^15:59
shadowershardy: I'm easy. We've put a few issues on the -validations one but whatever you prefer would be fine15:59
shadowerusing tripleo is perfectly fine by me15:59
gfidenteshardy, ayoung so one simple way to solve this outside of heat and within puppet maybe could be an rsync, triggered one step after the files are built15:59
hewbroccadprince: oh, never mind, I misunderstood what it'đ afford15:59
shardyshadower: really it depends on whether you want tripleo-validations to be tagged and released as part of the coordinated TripleO release?15:59
shardyshadower: if we want it to be considered part of the milestone deliverables, I'd rather it was in the main TripleO LP15:59
hewbroccaI misunderstood what it's for16:00
shardyas that's the only one I regularly track in the run up to a milestone16:00
dprincehewbrocca: yep, it isn't directly related to this if that is what you are wondering https://review.rdoproject.org/r/#/c/1678/16:00
shadowershardy: I think we want to do that. Ideally, I'd like them to be closely integrated within the main workflow16:00
dprincehewbrocca: it is more related to optimizing for zaqar websockets...16:00
shadowerwhich means they sohuld be released at the same time, too16:00
hewbroccaignore me, sorry16:00
*** penick has joined #tripleo16:00
dprincehewbrocca: its good man. Keep us in check16:00
shardyshadower: ack, thanks - we can look at creating a validations tag then and retiring the validations (and tripleo-common) LP, thanks!16:00
gfidenteshardy, ayoung not sure there if it should be push or pull, probably push can be done by the producer within single step using puppet dependencies16:01
shadowershardy: great, thanks! mandre ^ FYI16:01
shardyjpich: ^^ FYI16:01
ayounggfidente, I just had a side convo with a coworker...the more I think about it, the more complex it becomes.  It might be a Barbican use case.  Sharing private keys...16:02
hewbroccaBarbican is actually a good idea16:03
dprinceshardy: thoughts on this https://review.openstack.org/#/c/344338/16:03
shardyayoung: ack - what credentials are required to access barbican, e.g if we were to run it on the undercloud?16:03
*** leanderthal is now known as leanderthal|afk16:03
hewbroccait's like puppetdb for keys16:03
dprinceshardy: a cleanup and a step towards not re-execing as much16:03
jpichCool! Sounds like One Tracker To Rule Them All is the preferred approach in general :)16:03
shardydprince: +1, lgtm16:04
*** d0ugal has quit IRC16:04
ayoungshardy, hmmmm, I think these would be considered resources managed by heat-admin?  Or maybe the Keystone user?  I'm not sure, I need to think it through.  I'll discuss with the other Keystone core tomorrowish16:05
shardyayoung: That's the reason for my question, we only ever deploy credentials in the special heat_stack domain, so we don't leak tenant credentials onto the nodes16:06
gfidentealso, I think barbican can work as storage but it won't really fix the issue with orchestrating the data exchange16:07
openstackgerritMerged openstack/instack-undercloud: ironic: turn on ipxe timeout  https://review.openstack.org/29478716:07
*** zoliXXL is now known as zoli|gone16:07
gfidentesounds like we'd have to do it with steps in puppet anyway16:07
hewbroccagfidente: couldn't you just let it race?16:07
hewbroccaIOW, have all three nodes generate keys and try to write them to barbican16:08
ayoungshardy, it might not be approprate for barbican.  There is a file format (PKCS11) that is used for sharing secrets betwen machines.  It might be that we can use that to save the files we need to swift, and then unpack on the remote machines, but It still needs a secret of some sort.  heat could manage that secret, but I don't like that....16:08
hewbroccafirst one wins, the other two give up and then retrieve the one that's already there16:08
*** zoli|gone is now known as zoli_gone-proxy16:08
*** ifarkas is now known as ifarkas_afk16:08
shardyayoung: sure, open to suggestions on that.  FWIW we already store all the paswords for the overcloud services in the heat DB16:09
shardy(encrypted obviously, but the key exists on the undercloud node)16:09
ayoungshardy, ok, we have options...this will be a lovely problem to mull over tonight on a cross country flight16:09
shardyIf someone gets root access to the undercloud it's basically game over anyway tho tbh16:10
gfidentehewbrocca, probably, but for swift it's more than a single ring so we'd have to make it atomic on the entire set of files16:10
gfidentethough we have this bootstrap_nodeid concept in place already so I was thinking to use just that for the time being16:10
shardygfidente: Yeah, that's why I was thinking have the bootstrap node push the data to swift, then in a subsequent step have the other nodes read it16:11
gfidentebut I am not good at big steps, I can only take small steps so maybe there is an enirely different approach which I don't even see16:11
*** akuznetsov has quit IRC16:11
ayoungshardy, let me confirm first that we need to have a single private key.  It gets much simpler if that is not the case16:12
gfidenteshardy, push to undercloud swift?16:12
shardygfidente: Yes, we generate a tempurl and pass it to the overcloud nodes as a parameter16:12
gfidenteok I see16:13
*** beagles is now known as beagles_brb16:13
*** fultonj has quit IRC16:15
openstackgerritAdriano Petrich proposed openstack/tripleo-quickstart: WIP gate upgrade  https://review.openstack.org/34216116:19
openstackgerritDan Prince proposed openstack/tripleo-image-elements: Add zaqar.use_websockets option to o-c-c  https://review.openstack.org/34435216:20
*** fultonj has joined #tripleo16:21
apetrichEmilienM, is this https://review.openstack.org/#/c/336442 a liberty to mitaka issue or a mitaka to master?16:25
ayoungshardy, to be clear:  all of the *_Step[1-6].json  are posted to the metadata at the same time, and all are fetched together via os-collect-config, which updates its local cache.  it also gets the step value which it stores  somewhere in /var/lib/os-collect-config/  .  Then os-refresh-config picks that up as does os-apply-config.  Is it os-apply-config that decides what to run, based on the step value, or is it os-refresh-config, and why are16:27
ayoung they two separate things?16:27
*** penick_ has joined #tripleo16:28
*** bvandenh has quit IRC16:29
*** jpich has quit IRC16:29
*** mgarciam has quit IRC16:29
*** Goneri has joined #tripleo16:30
*** penick has quit IRC16:31
*** penick_ is now known as penick16:31
ayoungalso...what is the mechanism that notifies Heat that the current step is completed?16:32
shardyayoung: pls check the blog post I linked earlier, it does explain all of that IIRC16:36
shardyhappy to answer any remaining questions if clarification is needed16:37
ayoungshardy, OK...thanks16:37
*** abehl has quit IRC16:37
*** dtantsur is now known as dtantsur|afk16:38
shardyhttps://etherpad.openstack.org/p/tripleo-deep-dive-topics also has a link to the software config deep dive I did last week, which contains a bit more detail about what happens on the nodes, and how to debug including running the puppet manifest on the node outside of heat/os-collect-config16:38
*** d0ugal has joined #tripleo16:38
shardyayoung: ^^16:38
ayoungshardy, excellent16:39
ayoungshardy, I'm still going through the one from the week before.  I'll try to catch up16:39
*** trown is now known as trown|lunch16:44
*** tesseract- has quit IRC16:47
*** tremble has quit IRC16:48
mandreshardy: got it, I'm also for retiring the tripleo-validations LP16:48
*** tremble has joined #tripleo16:49
openstackgerritDan Prince proposed openstack/tripleo-heat-templates: WIP: Test SoftwareConfigTransport: ZAQAR_MESSAGE  https://review.openstack.org/34436516:49
*** rcernin has quit IRC16:49
*** abehl has joined #tripleo16:50
openstackgerritDan Prince proposed openstack/tripleo-image-elements: Add zaqar.use_websockets option to o-c-c  https://review.openstack.org/34435216:50
openstackgerritJames Slagle proposed openstack-infra/tripleo-ci: Add mulitnode CI job support to tripleo-ci  https://review.openstack.org/32477716:54
*** ebarrera has quit IRC16:55
slaglereview plz :) ^ or i will wear out the "r-e-b-a-s-e" keys on my kb16:56
*** derekh has quit IRC17:00
*** tremble has quit IRC17:09
*** aufi has quit IRC17:10
*** fultonj has quit IRC17:11
*** dsariel has quit IRC17:12
*** fultonj has joined #tripleo17:12
openstackgerritMerged openstack/instack-undercloud: Allow customization of undercloud disk size  https://review.openstack.org/34416117:19
*** fzdarsky is now known as fzdarsky|afk17:21
*** fzdarsky|afk has quit IRC17:23
openstackgerritDan Prince proposed openstack/tripleo-heat-templates: WIP: Test SoftwareConfigTransport: ZAQAR_MESSAGE  https://review.openstack.org/34436517:27
*** egafford has quit IRC17:27
*** pkovar has quit IRC17:30
*** paramite is now known as paramite|afk17:31
sshnaidmshardy, did you have a chance to talk to apevec about delorean failures?17:35
*** beagles_brb is now known as beagles17:35
openstackgerritMerged openstack/os-net-config: Make os-net-config idempotent with Linux bonds and MAC mapping  https://review.openstack.org/33631317:36
EmilienMapetrich: I don't know if it's an issue, but I want this patch tested from Mitaka to Newton17:36
shardysshnaidm: No, IIRC I suggested you jump into #rdo and ask apevec or trown about it17:37
*** shivrao has joined #tripleo17:37
shardyI won't be able to look into it until tomorrow now unfortunately17:37
openstackgerritDan Prince proposed openstack/python-tripleoclient: Use the messaging-websocket endpoint  https://review.openstack.org/34438417:37
shardyso if someone else can work through fixing that it'd be much appreciated17:37
*** paramite|afk is now known as paramite17:37
sshnaidmshardy, hmm, so I misunderstood you17:39
hrybackiweshay: I'm going to break https://trello.com/c/whHpxvkO/269-investigate-ci-automated-documentation down into sub-cards aligned with the plan of action in ^^17:39
sshnaidmshardy, I was trying to find anybody who will handle this the whole day, it didn't work out17:39
*** shivrao_ has joined #tripleo17:40
sshnaidmshardy, weshay, trown|lunch do you know how can I find somebody to solve this issue with delorean? Running over IRC channels with searching who can take it has appeared as not working solution17:40
*** shivrao has quit IRC17:41
*** shivrao_ is now known as shivrao17:41
sshnaidmis there any interface in RDO that I could come with this issue? bz, launchpad, any other rdo specific issues tracker?17:42
*** egafford has joined #tripleo17:42
*** penick is now known as jpenick17:44
*** yamahata has quit IRC17:45
openstackgerritDan Prince proposed openstack/python-tripleoclient: Use the messaging-websocket endpoint  https://review.openstack.org/34438417:46
openstackgerritMarius Cornea proposed openstack/tripleo-heat-templates: Fix gnocchi::storage::ceph::ceph_keyring  https://review.openstack.org/34438717:47
*** shardy has quit IRC17:50
*** electrofelix has quit IRC17:50
*** mcornea has quit IRC17:51
EmilienMtrown|lunch: weird issue :(17:53
openstackgerritwes hayutin proposed openstack/tripleo-quickstart: [WIP] Do not exit prior to the inventory of the deployment.  https://review.openstack.org/34150617:55
*** tosky has quit IRC17:56
weshaysshnaidm, my suggestion is to ping apevec in #rdo17:56
sshnaidmweshay, I do it from morning17:56
weshaysshnaidm, k.. you can also email him + ping in the morning.. thanks for going after it!!17:57
*** abehl has quit IRC17:57
*** gfidente is now known as gfidente|afk17:57
*** dsneddon has joined #tripleo17:59
sshnaidmweshay, priv17:59
*** paramite is now known as paramite|afk17:59
*** trown|lunch is now known as trown18:01
trown EmilienM: indeed, I just tested it with the tripleo-ci produced images and I get the same thing18:01
*** ebalduf_ has quit IRC18:01
EmilienMtrown: same error?18:01
trownEmilienM: I am pretty sure that patch will break RDO, but I am not even confident it will work for tripleo18:02
trownya same error18:02
trownsame behavior where there is a bunch of missing hiera18:03
EmilienMit's weird it works in ooo ci18:05
trownya, that is a bit concerning in itself as tripleo-quickstart devmode is supposed to be equivalent18:06
trownI wonder if there is something changed in tripleo-ci recently that is missing from tripleo-quickstart18:07
trownthe other possibility is that tripleo-ci is not actually testing that patch18:08
ayoungtrown, EmilienM, for Federation, I need to pass in a mapping file that is a JSON blob.  It seems wrong to embed this in the middle of a heat template.  Is there a way that I can specify a separate Metadata URL for the mapping file?  A simplistic one looks like this:  https://github.com/admiyo/rippowam/blob/master/roles/packstack/templates/mapping_sssd.json.j218:09
*** abehl has joined #tripleo18:09
trownEmilienM: uh oh... I think it is that tripleo-ci is not actually testing THT18:11
trownEmilienM: check yum log http://logs.openstack.org/16/341616/11/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/994d0f9/logs/undercloud/var/log/yum.txt.gz18:12
trownthe version of THT installed there is the one from 2016061421383118:12
trownie 5 days ago18:12
EmilienMtrown: WHAT18:13
EmilienMagain this bug?18:13
EmilienMayoung: it could be a template file in puppet-tripleo18:15
EmilienMtrown: we have a serious problem if you're right18:15
ayoungEmilienM, ok, when this fire is out, I might bug you more about the path to get there...I'll keep writing with the assumption that will work18:16
EmilienMI don't find 333fd2818:16
EmilienMayoung: ok np, not now, later plz18:16
EmilienMJul 19 11:26:19 Installed: openstack-tripleo-heat-templates-5.0.0-0.20160719112011.333fd28.el7.centos.noarch18:16
EmilienMI don't find the commit18:16
EmilienMlet's look another job18:17
EmilienMno it pick master18:18
trownhmm I must have been looking at the wrong line, because that is a different timestamp... still not the right commit18:18
EmilienMsee a recent job : http://logs.openstack.org/88/344188/2/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/459ab86/logs/undercloud/var/log/host_info.txt.gz18:18
trownoh I got tripped up by openstack-heat-templates...18:19
EmilienMand 970fcfcb58b2d0d26a39c2d9deab3a0301eb65ca is master18:19
EmilienMyes :)18:19
EmilienMdude you almost killed me18:19
EmilienMayoung: ok so back to you18:19
trownstill though where is 333fd28 coming from18:19
EmilienMis your file static / dynamic?18:19
trownwe cherry-pick instead of checkout?18:19
EmilienMayoung: is your file always the same during all deployments? Or not? Can it change after a deployment?18:20
EmilienMayoung: I need to know those 2 things18:20
ayoungEmilienM, no18:20
ayoungEmilienM, the file will be very likely to change18:20
EmilienMafter a deployment, right?18:21
ayoungEmilienM, yes18:21
ayoungEmilienM, like this18:21
ayoungstart by creaing IdP18:21
ayoungthis is a keystone call, will not change after the fact18:21
ayoungsecond create a mapping ,useing that file18:21
ayoung3 create a protocol, using IdP and mapping18:21
ayoungnow, as things come up, user will want to update the mapping file, and upload to Keystone18:22
ayoungIts a Keystone API call, and, since we have a shared database, does not necessarily need to be done in Heat18:22
EmilienMtrown: can you try again by cherry-picking?18:22
EmilienMtrown: because my patch is supposed to be rebased18:22
ayoungit might make sense to have all the rest of this done in Heat, and use a placeholder mapping file18:22
EmilienMtrown: the git cherry-pick doesn't rebase for you18:23
dprinceayoung: heat supports get_file. The keystone services (in t-h-t) could have get_file suck in an example federation mapping.json file I think18:23
ayoungthen let the sys admin modify it after the face18:23
EmilienMtrown: so you need to 1) rebase your tht on master and then cherry-pick my patch OR checkout my patch18:23
dprinceayoung: yes, I think this would work quite nicely18:23
ayoungdprince, so get_file would fetch the placeholder from metadata?18:23
dprinceayoung: yes, the file that the get_file references needs to live in tree though18:24
trownEmilienM: right, I am looking at what we do in tripleo.sh and I am not understanding how we get a different git hash18:25
ayoungAnd then we would say that the expected end state of the heat Federation deplopy would be a URL that would always return a 401 due to the mapping.  In order to get an unscoped token would require an updated mapping file via the Keystone API18:25
trownin my env I have http://chunk.io/f/23742e8cca104436be58e427a2d09b0d which matches the git hash on the patch18:26
trownbut in CI we are getting 333fd2818:26
trownin any case testing that patch rebased... didn't know CI did a rebase before building18:31
apetrichEmilienM, ok. will do. but I'm not sure that mitaka -> Newton Overcloud upgrade is working18:37
*** ayoung has quit IRC18:39
*** yamahata has joined #tripleo18:39
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: Remove ::nova::cron::archive_deleted_rows  https://review.openstack.org/34366118:45
EmilienMtrown: ok let me know!18:45
EmilienMtrown: I'm glad there is no dlrn bug again :)18:46
trownEmilienM: hmm, maybe it is only broken with master heat18:50
trownEmilienM: I have newer heat in my env than in CI18:50
trownwhich is a bug in quickstart dev mode... it should have the same packages as CI18:51
*** ebalduf_ has joined #tripleo18:57
*** egafford has quit IRC18:59
*** egafford has joined #tripleo19:00
*** abehl has quit IRC19:01
*** julim has quit IRC19:02
slagledid something recently change in how we configure galera.cnf? i'm seeing it has bind-address=
slagleis that new?19:07
slaglenvm, red herring19:08
*** r-mibu has quit IRC19:13
*** r-mibu has joined #tripleo19:13
*** abehl has joined #tripleo19:14
*** jcoufal_ has quit IRC19:22
EmilienMslagle: yes, I moved it to a composable role19:22
EmilienMbut nothing should have changed in the config except if I did something wrong19:22
*** florianf has quit IRC19:25
slagleEmilienM: yea i think something broke my multinode CI jobs19:25
slaglepuppet used /etc/my.cnf.d/server.cnf as the config file, where it has bind-address =
EmilienMit's a non ha job, right?19:26
EmilienMconfig should be https://github.com/openstack/puppet-tripleo/blob/master/manifests/profile/base/database/mysql.pp#L67-L7019:27
slaglebut even on nonha jobs i thought we always configured via galera.cnf19:27
EmilienMis enable_galera = true?19:27
slagleit should be, i don't change it19:28
*** ebalduf_ has quit IRC19:29
*** jubapa has quit IRC19:30
*** akrivoka has quit IRC19:36
EmilienMslagle: I'm looking logs19:39
EmilienMslagle: ah, we're missing lot of logs http://logs.openstack.org/77/324777/84/experimental/gate-tripleo-ci-centos-7-nonha-multinode/fcc869d/logs/etc/19:40
EmilienMshould I dl primary_node.tar.xz ?19:40
EmilienMok I got it19:41
slagleyes, all the normal tripleo logs should be there19:41
slagleEmilienM: i just kicked off a local run of this as well19:41
slagleEmilienM: don't let this distract you if you've got soemthign else to do :)19:42
EmilienMslagle: I can't continue by knowing I might have broken something19:42
EmilienMlet me dig a bit19:42
EmilienMdamn you're right19:44
EmilienMI checked a nonha job and we have /etc/my.cnf.d/galera.cnf19:44
EmilienMbut not in your multinode job19:44
slaglethe overcloud node logs are in http://logs.openstack.org/77/324777/84/experimental/gate-tripleo-ci-centos-7-nonha-multinode/fcc869d/logs/subnode-2/subnode-2.tar.xz19:44
EmilienMslagle: not subnode1?19:45
slaglethere is a /etc/my.cnf.d/galera.cnf there, it's just not managed by puppet19:45
slagleno, they start numbering at 119:45
slagleerr, 219:45
slaglei dunno why19:45
EmilienMk, a sec19:45
EmilienMslagle: subnode1 and subnode2 are 2 controllers? they both have mysql configured19:46
EmilienMits hyperconverged thing?19:47
slaglethere's only primary_node and subnode-219:47
slagleprimary_node is undercloud19:47
EmilienMah ok19:48
slaglesubnode-2 is aio overcloud19:48
slagleit does use pacemaker19:48
slagleeven though it is nonha19:48
EmilienMI see enable_galera to true19:48
openstackgerritMerged openstack/instack-undercloud: Fix ceilometer auth url on undercloud  https://review.openstack.org/34427919:48
EmilienMlet me check puppet logs19:48
EmilienMoh it use pacemaker19:48
EmilienMI don't see tripleo::profile::base::database::mysql resource in the puppet catalog19:51
EmilienMslagle: it fails early right? I don't see any pacemaker logs in /var/log/message19:53
slaglei believe it failed during ControllerOvercloudServicesDeployment_Step319:54
slagleit failed there19:55
EmilienMit's like puppet code to deploy mysql on ha is not deployed in your env19:55
openstackgerritDan Sneddon proposed openstack/tripleo-heat-templates: Add environment to preselect only VIP IP addresses  https://review.openstack.org/33674919:56
slagleEmilienM: should there be an include in puppet-tripleo/manifests/profile/pacemaker/database/mysql.pp to include the base profile?19:57
EmilienMit's in the tht service19:57
EmilienMslagle: https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/services/pacemaker/database/mysql.yaml#L2019:58
EmilienMI'm investigating your packages19:58
EmilienMyou're running on top of https://github.com/openstack/puppet-tripleo/commit/b613e7719:58
EmilienMwhich is good19:58
EmilienMlet's check tht19:58
EmilienMcurrent master , mhh19:59
*** akshai has quit IRC20:01
EmilienMslagle: found it20:02
*** julim has joined #tripleo20:02
EmilienMslagle: https://review.openstack.org/#/c/324777/84/test-environments/multinode.yaml20:02
EmilienMwe need to add MySQL20:02
slagleoh snap20:03
EmilienMdude you scared me20:03
slaglesorry man20:03
slaglei bet that tatoo jumped right off your skin!20:03
EmilienMI don't have it yet20:03
slagleeverytime we add a new service, i will have to remember to update this list20:03
EmilienMto be honest I didn't go yet, I was out all recent week ends20:04
slaglei know, i saw on twitter20:04
*** radek2 has quit IRC20:04
EmilienMslagle: we need to merge your patch next time it's passing20:04
EmilienMslagle: FYI the next services are Aodh; when they'll merge I'll let you know. Nothing else AFIK. Everything else is done20:05
openstackgerritJames Slagle proposed openstack-infra/tripleo-ci: Add mulitnode CI job support to tripleo-ci  https://review.openstack.org/32477720:05
slaglecool, thx20:05
*** rook has quit IRC20:07
EmilienMtrown: do you have more success?20:10
openstackgerritDan Prince proposed openstack/tripleo-heat-templates: Add 'service_name' to composable services  https://review.openstack.org/34444020:10
trownEmilienM: nope, I even made a local fix for quickstart dev mode so that I have the same packages as in CI20:11
trownand I still get the missing hiera issue with that patch20:11
*** shivrao has quit IRC20:11
dprinceEmilienM: ^^ that is the 'service_name' patch that will enable the firewall rule composability20:11
dprinceEmilienM: a few more patches and I will have the example in place20:12
*** dprince has quit IRC20:16
*** ebalduf has quit IRC20:16
trownEmilienM: I am sure that patch will break RDO at the very least, so if you don't mind it would be nice to get a promote there before merging if possible20:18
EmilienMsure thing20:18
trownEmilienM: we are 12 days behind as is, so merging that will be a major set back20:18
trownit is pretty confusing why tripleo-ci does not have an issue though20:19
*** paramite|afk is now known as paramite20:24
*** rbrady has quit IRC20:28
slaglei just saw a gnocchi db-sync failure in tripleo-ci20:29
*** shivrao has joined #tripleo20:29
*** abehl has quit IRC20:34
openstackgerritSumit Kumar proposed openstack/tripleo-heat-templates: Update NetApp configuration parameters.  https://review.openstack.org/34177820:34
*** athomas has quit IRC20:35
*** jprovazn has quit IRC20:36
openstackgerritBen Nemec proposed openstack-infra/tripleo-ci: Enable net-iso for ovb jobs  https://review.openstack.org/34301720:37
openstackgerritBen Nemec proposed openstack-infra/tripleo-ci: Pull all puppet modules from local sources  https://review.openstack.org/34445420:37
openstackgerritBen Nemec proposed openstack-infra/tripleo-ci: Add ovb to mirrored git repo list  https://review.openstack.org/34445520:37
openstackgerritBen Nemec proposed openstack-infra/tripleo-ci: Enable net-iso for ovb jobs  https://review.openstack.org/34301720:39
*** zeroshft has joined #tripleo20:42
openstackgerritBen Nemec proposed openstack-infra/tripleo-ci: Enable net-iso for ovb jobs  https://review.openstack.org/34301720:42
openstackgerritBen Nemec proposed openstack-infra/tripleo-ci: Pull all puppet modules from local sources  https://review.openstack.org/34445420:42
openstackgerritBen Nemec proposed openstack-infra/tripleo-ci: Add ovb to mirrored git repo list  https://review.openstack.org/34445520:42
*** dtrainor has quit IRC20:46
*** abehl has joined #tripleo20:47
*** egafford has quit IRC20:50
*** rcernin has joined #tripleo20:55
*** jayg is now known as jayg|g0n320:55
*** fultonj has quit IRC20:59
*** dtrainor has joined #tripleo21:00
*** trown is now known as trown|outtypewww21:01
*** paramite has quit IRC21:02
*** isq_ has quit IRC21:04
EmilienMslagle: oops21:10
EmilienMwe're missing Pacemaker I think21:10
EmilienMlet me check21:10
EmilienMyeah we need OS::TripleO::Services::Pacemaker21:11
slagleok i'll add it21:11
slaglei'll just resync the full list21:12
EmilienMslagle: what about OVERCLOUD_DEPLOY_ARGS ?21:12
EmilienMdon't you load puppet-pacemaker.yaml env?21:12
EmilienMslagle: you're missing Kernel role also21:13
EmilienMKernel is required for Swift21:14
openstackgerritBen Nemec proposed openstack/tripleo-docs: Update overcloud SSL docs  https://review.openstack.org/34447521:14
openstackgerritJames Slagle proposed openstack-infra/tripleo-ci: Add mulitnode CI job support to tripleo-ci  https://review.openstack.org/32477721:15
*** isq has joined #tripleo21:15
slagleEmilienM: it should have everything now, i pulled the full list over21:16
EmilienMerr, one more, you'll kill me21:16
*** jeckersb is now known as jeckersb_gone21:17
EmilienMceilocompute agent21:17
EmilienMa sec, what is name again21:17
slaglei'll get it21:17
*** lblanchard has quit IRC21:17
EmilienMmhh and I wonder if we need or not ComputeNeutronOvsAgent21:18
EmilienMdprince created it but I don't like the name21:18
slagleEmilienM: I guess I need the full list from compute too21:18
EmilienMit's very tight to compute role21:18
slaglei need the union of both lists21:18
EmilienMI'm not sure it will work21:18
EmilienMComputeNeutronCorePlugin and NeutronCorePlugin21:18
EmilienMdprince did that ^21:18
EmilienMlet me check if it will collide21:19
openstackgerritJames Slagle proposed openstack-infra/tripleo-ci: Add mulitnode CI job support to tripleo-ci  https://review.openstack.org/32477721:19
slagleok, just let me know21:19
EmilienMok you don't need ComputeNeutronOvsAgent and ComputeNeutronCorePlugin21:20
EmilienMthey are specific for vendors like plumgrid, etc21:20
EmilienMso we're good!21:20
openstackgerritJames Slagle proposed openstack-infra/tripleo-ci: Add mulitnode CI job support to tripleo-ci  https://review.openstack.org/32477721:20
EmilienMslagle: /me afk running now, let me know if any blocker21:21
*** isq has quit IRC21:32
*** isq has joined #tripleo21:36
*** abehl has quit IRC21:42
*** dtrainor has quit IRC21:43
openstackgerritwes hayutin proposed openstack/tripleo-quickstart: Do not exit prior to the inventory of the deployment.  https://review.openstack.org/34150621:45
*** weshay has quit IRC21:47
openstackgerritMerged openstack/diskimage-builder: Remove the escape in the centos7 README file  https://review.openstack.org/34399721:47
*** ayoung has joined #tripleo21:52
*** zeroshft has quit IRC22:04
*** [1]cdearborn has quit IRC22:14
*** pradk has quit IRC22:16
*** shivrao has quit IRC22:23
*** bfournie has quit IRC22:30
*** jcoufal has joined #tripleo22:32
*** rlandy has quit IRC23:08
*** ayoung has quit IRC23:11
*** yamahata has quit IRC23:14
*** yamahata has joined #tripleo23:24
*** ramishra has quit IRC23:26
*** ramishra has joined #tripleo23:27
*** shivrao has joined #tripleo23:32
*** rhallisey has quit IRC23:43
slagleEmilienM: seeing a lot of CI jobs failing with: Error: /Stage[main]/Gnocchi::Db::Sync/Exec[gnocchi-db-sync]: gnocchi-upgrade --config-file /etc/gnocchi/gnocchi.conf --skip-storage --create-legacy-resource-types returned 1 instead of one of [0]\u001b[0m\n",23:44
slaglerelated to https://review.openstack.org/#/c/343902/ i guess?23:44
EmilienMbut why do you have this bug?23:45
EmilienMthe patch is merged and should be in delorean23:45
slagleyea the patch is definnitely there, b/c it's running the command with --create-legacy-resource-types23:45
EmilienMhonestly I'm not expert enough in Gnocchi but there is definitly something wrong here23:46
slaglei'm getting a bug filed23:46
EmilienMif the error still occurs, I would suggest to disable gnocchi in the meantime it's get stable23:46
*** lblanchard has joined #tripleo23:46
openstackLaunchpad bug 1604624 in tripleo "CI: Error: /Stage[main]/Gnocchi::Db::Sync/Exec[gnocchi-db-sync]: gnocchi-upgrade --config-file /etc/gnocchi/gnocchi.conf --skip-storage --create-legacy-resource-types returned 1 instead of one of [0]\u001b[0m\n", " [Critical,Triaged]23:48
slaglei'll email prad, maybe he can take a look when gets in23:48
slaglethere's nothing in the gnocchi log23:50
*** penick has joined #tripleo23:50
*** jcoufal has quit IRC23:51
*** akshai has joined #tripleo23:51
*** penick has quit IRC23:52
*** jpenick has quit IRC23:52
EmilienMslagle: in gnocchi-manage logs either?23:52
*** dmacpher has quit IRC23:53
EmilienMsomething I noticed is that tripleo does not show logoutput on failure for exec23:53
EmilienMif you see what I mean23:53
EmilienMI'll investigate that tomorrow, it would be so helpful to have Exec output in the logs23:53
EmilienMwe could see TRACE etc23:53
slaglethat would be helpful23:53
*** gfidente|afk has quit IRC23:54
EmilienMslagle: ok i'm out a bit, ttyl23:55
*** akshai has quit IRC23:56

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!