Thursday, 2016-08-04

*** r-mibu has quit IRC00:10
*** rwsu has quit IRC00:14
EmilienMsshnaidm: I think we can land
*** fultonj has quit IRC00:28
openstackgerritMerged openstack/tripleo-heat-templates: Don't set selinux in 'permissive' mode on CephStorage nodes
openstackgerritMerged openstack/tripleo-heat-templates: Fixes type and description for NeutronL3HA
*** r-mibu has joined #tripleo00:35
openstackgerritMerged openstack/tripleo-heat-templates: Fixes nova host configuration on control nodes
*** limao has joined #tripleo00:41
*** limao has quit IRC00:41
openstackgerritGiulio Fidente proposed openstack/puppet-tripleo: Allow use of raw disks for Swift storage role
openstackgerritGiulio Fidente proposed openstack/tripleo-heat-templates: Permit usage of raw disks as backends for Swift storage
*** bana_k has quit IRC01:28
*** dmacpher has joined #tripleo01:40
*** weshay has quit IRC01:40
*** jeckersb_gone is now known as jeckersb01:42
*** hewbrocca-afk has quit IRC01:56
*** hewbrocca-afk has joined #tripleo01:56
*** jmiu has quit IRC01:57
*** jmiu_ is now known as jmiu01:57
*** jmiu_ has joined #tripleo01:58
*** bfournie has joined #tripleo02:16
openstackgerritOpenStack Proposal Bot proposed openstack/os-cloud-config: Updated from global requirements
openstackgerritOpenStack Proposal Bot proposed openstack/os-collect-config: Updated from global requirements
openstackgerritOpenStack Proposal Bot proposed openstack/tripleo-common: Updated from global requirements
*** ebalduf has joined #tripleo02:51
*** yamahata has quit IRC03:03
*** tzumainn has quit IRC03:22
*** coolsvap has joined #tripleo03:22
*** jrist has joined #tripleo03:34
openstackgerritSwapnil Kulkarni (coolsvap) proposed openstack/tripleo-quickstart: [WIP] undercloud reboot
*** ayoung has quit IRC04:08
*** julim has quit IRC04:16
*** links has joined #tripleo04:28
*** bana_k has joined #tripleo04:36
*** oshvartz has quit IRC04:40
*** cmorandin_ has joined #tripleo04:53
*** leanderthal|afk is now known as leanderthal04:54
*** cmorandin_ has quit IRC04:58
*** jraju has joined #tripleo05:04
*** links has quit IRC05:06
openstackgerritSwapnil Kulkarni (coolsvap) proposed openstack/tripleo-quickstart: [WIP] undercloud reboot
*** rcernin has joined #tripleo05:16
openstackgerritMerged openstack/diskimage-builder: Optionally remove portage files
*** TSCHAK has joined #tripleo05:41
*** bana_k has quit IRC05:49
openstackgerritJuan Antonio Osorio Robles proposed openstack/instack-undercloud: Enable SSL for ironic inspector
*** dmacpher has quit IRC05:49
openstackgerritJuan Antonio Osorio Robles proposed openstack/instack-undercloud: Refactor endpoint env variable population
openstackgerritJuan Antonio Osorio Robles proposed openstack/diskimage-builder: Nothing to see here
*** apetrich has quit IRC05:55
*** apetrich has joined #tripleo06:00
*** jtomasek has quit IRC06:09
*** pcaruana has joined #tripleo06:10
*** devvesa has joined #tripleo06:12
*** tremble has joined #tripleo06:15
*** tremble has joined #tripleo06:15
openstackgerritMerged openstack/tripleo-docs: Document additional NODE vars
*** jaosorior has joined #tripleo06:20
*** jbadiapa has joined #tripleo06:21
*** oshvartz has joined #tripleo06:22
*** pgadiya has joined #tripleo06:32
*** limao has joined #tripleo06:41
*** mcornea has joined #tripleo06:43
*** radek_ has joined #tripleo06:46
openstackgerritMichele Baldessari proposed openstack/tripleo-heat-templates: Next generation HA architecture work
*** bootsha has joined #tripleo06:52
*** tesseract- has joined #tripleo06:53
jaosoriorflaper87: Are you around?06:58
*** ebalduf has quit IRC07:06
*** ohamada has joined #tripleo07:09
gfidentebandini, :)07:10
*** florianf has joined #tripleo07:11
*** liverpooler has joined #tripleo07:11
*** dculik has joined #tripleo07:12
*** jpena|off is now known as jpena07:13
*** ccamacho has quit IRC07:16
*** yamahata has joined #tripleo07:18
*** ccamacho has joined #tripleo07:18
bandinigfidente: wasn't me!07:22
* bandini claims complete innocence07:22
*** abehl has joined #tripleo07:30
*** openstackgerrit has quit IRC07:33
*** openstackgerrit has joined #tripleo07:33
*** ebarrera has joined #tripleo07:44
*** zoli_gone-proxy is now known as zoliXXL07:44
*** radek_ has quit IRC07:45
*** zoliXXL is now known as zoli|wfh07:47
*** shardy has joined #tripleo07:48
jaosoriorccamacho: Morning, dude07:51
*** oshvartz has quit IRC07:51
*** fzdarsky has joined #tripleo07:52
*** milan has joined #tripleo07:57
*** jpich has joined #tripleo08:00
*** hewbrocca-afk is now known as hewbrocca08:05
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo: Add zaqar endpoint in HAProxy
jaosoriorshardy: Have you tried to build puppet-tripleo with lately?08:12
*** jaosorior has quit IRC08:13
shardyjaosorior: No, I just use upload-puppet-modules08:13
*** jaosorior has joined #tripleo08:13
jaosoriorccamacho: Which map?08:14
jaosoriorccamacho: regarding
ccamachojaosorior the endpoint_map **me messing it in the rebase, Im finishing to fix it08:14
*** Goneri has joined #tripleo08:15
*** dtantsur|afk is now known as dtantsur08:16
shardyjaosorior: Yeah it doesn't work08:16
openstackgerritCarlos Camacho proposed openstack/tripleo-heat-templates: Composable Mistral services
jaosoriorshardy: How are you testing puppet-* changes for the undercloud?08:17
shardyjaosorior: just do a git clone08:18
d0ugalshardy: btw, with temporary hard-coded keystone certs I get a CREATE_COMPLETE!08:18
shardythen run puppet manually08:18
* d0ugal hasn't seen one of those for a while08:18
shardyobviously we do need to fix the packaging too tho08:18
shardyd0ugal: woohoo!08:18
sshnaidmdoes anybody deal with new introspection failures?08:18
shardynext step, environments, then endpoints :)08:18
sshnaidmI mean in CI jobs08:18
shadowerobviously we need to introduce new bugs :-)08:18
d0ugalshadower: I think that is all I do08:19
*** bootsha has quit IRC08:19
jaosoriorshardy: So the plan now is to do keystone endpoints via mistral?08:19
jaosoriorsshnaidm: Which?08:19
shardyjaosorior: the plan is just to maintain the current os-cloud-config code and move it from tripleoclient to mistral08:19
shardyjaosorior: unless someone tells me we can actually land the puppet endpoint stuff08:19
openstackgerritDougal Matthews proposed openstack/python-tripleoclient: Migrate the deploy command to use the Mistral workflows
jaosoriorshardy: alright08:20
jaosoriorsshnaidm: Which patch?08:20
shardyjaosorior: that seems like the most low risk increment, everything the same as now, but via mistral08:20
sshnaidmjaosorior, it's in periodic job, but also in patches08:20
shardywe can work out further changes after that (the mistral endpoint stuff will be trivial to disable)08:20
d0ugalshardy: I am not sure how we are going to do it in Mistral :)08:21
d0ugalshardy: The deploy workflow ends when the Heat stack create/update starts08:21
shardyd0ugal: well we'll have to change the workflow ;)08:21
jaosoriorsshnaidm: that's funky... Have you been able to reproduce that?08:21
jaosoriorIt seems to be a mistral issue08:21
shardyso it polls for stack CREATE_COMPLETE then runs another action08:21
*** yamahata has quit IRC08:22
d0ugalshardy: we will hit a mistral bug with token timeouts - they don't renew them like Heat does. So deploys over 4 hours (default) will fail08:22
jaosoriorI ran into the same thing when trying to enable TLS for ironic-inspector. Since introspection is now driven by mistral08:22
shardye.g the same workflow we have hardcoded in tripleoclient?08:22
sshnaidmjaosorior, not yet, just wanted to know if anybody look at it08:22
openstackLaunchpad bug 1595084 in Mistral "Workflow execution lifespan is limited by auth token expire time" [High,Confirmed]08:22
d0ugalshardy: Do we might need a post-deploy workflow.08:22
shardyd0ugal: Ok, well we'll have to fix it (I fixed it in heat, so maybe I can help)08:22
shardybut yeah, a workaround will be enable running just the post-deploy steps08:23
d0ugalshardy: I thought you might have, I was going to ask you about it when things were quieter :)08:23
*** dtantsur is now known as dtantsur|bbl08:23
shardyso if the deploy command times out, you can still run the post-deploy08:23
d0ugalshardy: I think the big issue is that Mistral is stuck of keystone v2 for now.08:23
jaosoriorsshnaidm: Do you have access to any of the CI environments? We could debug it08:23
shardyd0ugal: Yeah, I'll admit it was me ;)  It involves some exciting abuse of trusts08:23
sshnaidmjaosorior, yeah, I think it would be reproducible in any nonha job, will run one now08:24
*** lucas-dinner is now known as lucasagomes08:24
jaosoriorsshnaidm: Sure. only thing we need to check is the output of mistral-db-sync. That usually gives the culprit08:24
d0ugalshardy: a post-deploy workflow sounds like a reasonable approach for now. if we can resolve the auth issue the deploy workflow could trigger it.08:24
sshnaidmthese error reports of mistral are completely unreadable..08:24
shardyd0ugal: Ok, I thought they were already using trusts for some reason08:24
d0ugalsshnaidm: The bugs?08:24
jaosoriorsshnaidm: For some reason it's not in the logs that CI persists though08:24
shardyd0ugal: ack08:24
d0ugaloh, the actual errors. nvm08:25
sshnaidmjaosorior, really? which log is this?08:25
shardyd0ugal: also, we can revive the discussion about having puppet do the post-deploy stuff08:25
jaosoriorI have no clue where mistral would put that. I just know that it's not in any log I've found. So I ended up running that command manually08:25
shardythat's only been going on for a year... ;)08:25
d0ugalshardy: +1, that would be even better08:25
hewbroccaMorning folks08:25
shardy'morning hewbrocca08:25
jaosoriorshardy, d0ugal: Any plans on moving mistral not to be extremely hardcoded for keystone v2?08:25
hewbroccadear god could we for the love of all that's good and holy, please, make puppet do the post-deploy stuff08:26
hewbroccaI hate os-cloud-config so much08:26
d0ugaljaosorior: There is some discussion in
openstackLaunchpad bug 1595084 in Mistral "Workflow execution lifespan is limited by auth token expire time" [High,Confirmed]08:26
jaosoriord0ugal: It's gonna be a LOT of work X_x. I checked that they even do manual keystone catalog checking of stuff (cause why use the keystone libraries?)08:26
jaosoriorand a bunch of other details that just make life harder08:26
d0ugaljaosorior: haha, oh dear. Maybe it is something I can look into once we have the deploy stuff working.08:27
d0ugalDoes Keystone v2 have an EOL?08:27
shardyUsing trusts really doesn't require mistral to be completely purged of v2.0 stuff08:27
jaosoriord0ugal: As far as I know it has passed already, that EOL08:27
shardyit just requires the v3 API to be actually accessible08:27
d0ugaljaosorior: o.O08:28
shardyfor years (like, since 2013) Heat used mostly v2.0 with v3 internally for trusts08:28
jaosoriord0ugal: it has been a LONG time the keystone team has been trying to get people to stop using v208:28
d0ugalOkay, cool. I think I shall come back to this one.08:28
* d0ugal makes a note.08:28
shardyI don't think it's even officially deprecated yet08:29
jaosoriorshardy haha yeah, cause when they tried to deprecated, people complaigned that they hadn't been able to move out of it yet08:29
shardybut I am aware of all the efforts trying to move away from it08:29
jaosorior*deprecate it08:29
jaosoriorthere was a pretty big backlash as far as i remember08:29
d0ugalMaybe I can make this a goal for O in Mistral.08:29
shardyd0ugal: sounds good - I'll also try to take a look, perhaps we can reuse some of the heat code08:30
openstackgerritDougal Matthews proposed openstack/tripleo-docs: Add TripleO UI to the components docs
*** chem has joined #tripleo08:35
*** cwolferh has quit IRC08:37
shardyd0ugal: I just commented, I may take a look later and see how hard it would be to fix08:42
shardylife would certainly be easier if the timeout behavior of heat and mistral was the same08:43
d0ugalshardy: It would indeed, thanks :)08:44
d0ugalunrelated: I think we are getting into a bit of a mess with our action names.08:45
*** bootsha has joined #tripleo08:46
shardythe class names or entry points?08:46
d0ugalshardy: entry points08:46
d0ugalThe class names are generally much more specific08:47
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci: Don't run undercloud idempotent on tempest jobs
shardyd0ugal: they seem OK to me, but you'd like a more crud for each item kind of interface?08:47
d0ugalshardy: for example:
d0ugalThe right side, which we will never see in a workflow, gives me much more info08:48
d0ugal(you just need to know it is a heat related thing)08:48
d0ugalI don't know what tripleo.deploy_config does.08:49
shardyI'm not sure I get it, both say update capabilities08:49
* shardy probably needs more coffee08:49
d0ugalMaybe I need less coffee08:49
d0ugalshardy: we have a heat namespace on the right, on the left we assume people will know it is heat related08:50
shardyd0ugal: Yeah, maybe that can be refined before we release, sure08:50
shardylet's focus on getting feature parity first tho :)08:50
shardyd0ugal: has any thought been given to how this switch to mistral will work on upgrade?08:51
d0ugalheh, sure08:51
*** apetrich has quit IRC08:51
shardylike, if an overcloud stack exists, and there is no stored plan?08:51
d0ugalshardy: lol, nope.08:51
* shardy puts head in hands08:51
shardyd0ugal: Ok, I added a new API to heat which means we can probably create the plan from heat08:52
d0ugalshardy: I mean, they would just deploy as normal from the CLI - we just need to allow a plan to work with an existing stack.08:52
d0ugal... or we can do that.08:52
shardyd0ugal: the update of any existing stack is going to expect a plan though right?08:52
shardyand we'd end up with a partial plan because we do PATCH updates08:53
d0ugalshardy: Yeah, but I assume users have the templates they deployed.08:53
*** jtomasek has joined #tripleo08:53
d0ugalshardy: ah, forgot about PATCH - dang08:53
shardyOk, well something to thing about, but I'll stop distracting you from fixing the basic deploy stuff :)08:53
d0ugaljtomasek: Do you know if upgrades have ever been discussed?08:57
d0ugaljtomasek: (Deploying, and then upgrading and having a deployment but no plan)08:58
jtomasekd0ugal, afaik, a deployment should be always tied to a plan08:59
d0ugaljtomasek: but what if they have deployed before plans existed?09:00
shardyjtomasek: Existing deployments have no plan09:00
jtomasekshardy: they should be able to import the template set that was used to deploy09:00
shardyjtomasek: our update workflow doesn't require you to have all of the original templates, so we'll have to reconstruct the plan from heat09:00
shardyWe'll need a new action for this, I'll take a look after we get the deploy fully working09:01
jtomasekshardy: hmm, I thought that customers were supposed to keep the templateset used for deployment09:01
shardyjtomasek: we use PATCH updates, so strictly speaking it is not required09:02
shardyprobably a good idea tho, I agree09:02
jtomasekshardy, how does the update currently work if user does not have the original templates?09:02
shardyjtomasek: for upgrade we probably do expect the original templates, but the upgrade deploy commmand isn't the same as the original deployment09:02
shardythe environment files used are all different09:03
shardyso there's some complexity here we'll have to work out09:03
shardywe can't just create a plan based on the upgrade deploy command AFAICS09:03
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo: Add zaqar API endpoint in HAProxy
jtomasekd0ugal, I totally agree, that update parameters action should take just a subset of all parameters (just those that are actually set by the user in that action call)09:07
jtomasekd0ugal, I am going to look at that patch today and test it with GUI09:07
shardyjtomasek: what's the status of integrating the UI with the undercloud?  Are we still blocked on packaging?09:08
jtomasekshardy, yeah, honza was not in yesterday, but packaging is progressing. Although I think that actual undercloud install integration work can be done in paralel. Here is the blueprint that lists things that GUI setup requires to be done09:10
*** numans has joined #tripleo09:11
jtomasekshardy: I know that mandre and florianf have been looking into it, mandre is there an actual patch that starts that implementation? you mentioned some review comments on separate puppet repositories yesterday09:12
shardyjtomasek: ack, thanks - seems there are some questions there, let me know if you need help with those09:12
jaosoriorsshnaidm: By the way, If you referred to this CR that error you talked about is to be expected09:12
jaosoriorit was an issue in mistral that I've fixed not too long ago. So that needs to be packaged still09:13
florianfshardy, jtomasek: That's the patch in question:
jtomasekflorianf, thx09:13
jaosoriornevermind, just saw it was in the periodic one09:14
*** jbadiapa has quit IRC09:15
jtomasekshardy: at some patch review, you mentioned generating password parameter values, can you please remind me which one was it?09:15
jaosoriorsshnaidm: have you had luck reproducing it?09:15
sshnaidmjaosorior, waiting..09:16
sshnaidmjobs take a long time now09:16
d0ugaljtomasek: cool, I'll change it to do an update. Maybe we don't need an action to set them all.09:16
*** dsneddon has quit IRC09:19
shardyjtomasek: Hmm, not sure which review tbh - there was where I see you commented09:21
shardyI need to try the capabilities stuff, I'm not sure it works for user provided environments09:21
shardythe passwords are currently generated in python-tripleoclient, so that needs to move into an action09:21
florianfjtomasek: This one:
florianfjtomasek: Is that it?09:22
* jtomasek looks09:23
hewbroccaand an hour later /me finishes reading the backchat from last night09:23
hewbroccayou guys talk too much09:23
jtomasekshardy, the capabilities stuff depends on capabilities-map, so user-provided environment needs to be added to capabilities map to get listed there09:24
jtomasekshardy, but if user wants to just enable the environment using update_capabilities, he can just send {<path_of_that_environment>.yaml: true} to it09:26
jtomasekshardy, (obviously that environment needs to be present in the plan)09:26
shardyjtomasek: Ok, well we have to hide all that complexity from the user, because that's not how user provided environments currently work09:28
shardythey don't exist in the capabilities map, we just add them to the list and pass them to heat09:28
*** stendulker has joined #tripleo09:28
shardyjtomasek: probably we'll have to push user environments to the plan e.g user_environments/foo.yaml09:28
shardythen add user_environments/foo.yaml to the capabilities map, and enable it09:29
lucasagomeshi all, if you have time mind taking a look at this instack-undercloud patch ? It's automating the configuration of UEFI for iPXE and is a small patch too09:29
shardyjtomasek: I was expecting the capabilities selection to output a list of environments, which we could just append to09:29
shardybut I guess we can do everything via the capabilities map if needed09:30
shardylucasagomes: Nice, I'll try to check it out - I have a uefi box locally which I have never got to boot via ironic/ipxe09:30
lucasagomesshardy, cool yeah that would be great09:30
jtomasekshardy, so capabilities map is currently used only for listing the environments (all of them that are part of plan) - considering that they are in sync09:30
jtomasekshardy: we could enhance the get_capabilities action to list environments from capabilities-map (as it does now) + one more special group that would include all remaining environments that are not in capabilities-map09:32
*** dsneddon has joined #tripleo09:32
jtomasekshardy: adding environment to existing plan is basically a plan update, we should be able to do it (not entirely sure it is implemented but there should not be any blocker for it)09:32
openstackgerritMartin André proposed openstack/instack-undercloud: Add tripleo deployment config file paths to hiera
openstackgerritMartin André proposed openstack/instack-undercloud: Introduce 'enable_validations' option
*** zoli|wfh is now known as zoli|brb09:33
jtomasekshardy: also, get_capabilities lists all environments that are in plan and for each environment it specifies whether the environment is enabled or not09:33
shardyjtomasek: Ok, well I guess we can work through this as d0ugal gets the CLI patch worked up, essentially we must fix this to enable feature parity with the CLI interfaces09:33
jtomasekshardy, ack09:34
*** electrofelix has joined #tripleo09:35
ccamachobandini I had the same issue than you with the aodh submission, so, I tried to isolate the problem, I did this, deploy as it is  (same error than you), remove the wsgi step and got
ccamachonow at least is not failing but still managing the service with systemd..09:37
shardyCan I get a review on please?09:37
shardypassing CI and already +2'd09:37
ccamachocant give +2 :P09:38
*** apetrich has joined #tripleo09:39
*** jbadiapa has joined #tripleo09:40
shadowershardy: would I be overstepping my boundaries if I +2/+A'd it? It looks good to me09:41
bandiniccamacho: so the reference to pacemaker is totally okay when a service is managed by pacemaker09:41
mandreI lost +A when rebasing, can I get a quick review?09:41
*** mgould|afk is now known as mgould09:41
bandiniccamacho: so it is wsgi screwing up things09:41
ccamachoso now the service is actually managed by pcm?09:41
shardyshadower: sure, thanks!09:42
bandiniccamacho: it always isin the HA deployment (until NG HA lands)09:42
ccamachoI think that was the reason why I remove it in first place.. Was breaking the deployment..09:42
ccamachoIll start checking this wsgi manifest..09:43
shadowershardy: yeah I wasn't sure because we should be using +A on the validations repo only, but this is a tiny patch09:43
bandiniccamacho: so we need to figure out why wsgi is deciding to manage httpd via systemd anyway09:43
bandiniccamacho: awesome, thanks!09:43
shardyshadower: Yup, I think it's fine for you to use your judgement and as you say it's a pretty simple patch09:44
openstackgerritMerged openstack/tripleo-heat-templates: Replace hard-coded regionOne with parameter references
*** dbecker has joined #tripleo09:46
*** dbecker has quit IRC09:50
*** dbecker has joined #tripleo09:50
socialEmilienM: I'm thinking about changing all package{'package':} in undercloud to ensure_resource09:51
hewbroccapimping for workflow +109:53
sociallucasagomes: ping -
sociallucasagomes: I'm working on dropping whole ipxe element from undercloud09:53
lucasagomessocial, hi there09:54
hewbroccaparamite spec, already received many +2 before last change09:54
lucasagomessocial, oh cool, so that patch will do part of the job that my patch is doing (copying the ipxe.efi image to /tftpboot) ?09:54
hewbroccafor ops-tools...09:55
*** limao has quit IRC09:55
sociallucasagomes: file { "${tftp_root}/ipxe.efi":09:55
hewbroccadie elements die09:56
marioshewbrocca: added to my review queue for tomorrow morning, fwiw, assuming it is still around then. i haven't seen it before and its a long read09:56
lucasagomessocial, what's the story with the -2 there? Are you working on breaking it up in smaller patches?09:56
socialhewbrocca: yeah it would be nice to remove elements and use puppet but we use puppet as shell script09:56
sociallucasagomes: yes, WIP09:56
hewbroccamarios: thank you09:57
hewbrocca... we use puppet as shell script...09:57
shardyhewbrocca: I had previously reviewed and now approved09:57
hewbroccashardy: thank you!09:57
lucasagomeshewbrocca, +109:57
shardymarios: feel free to review anyway and we can discuss any questions with paramite09:58
mariosthanks shardy sure will do09:58
jtomasekd0ugal, (shardy): So I've applied the templates processing and parameters actions patches and now when I run the action I get
mariospimping reviews aint for the faint hearted it can get rough hewbrocca stay strong09:59
lucasagomessocial, so just to understand, I don't think we should block that patch because it's adding one line to the ipxe element copying the ipxe.efi binary to the /tftpboot right?10:00
hewbroccamarios: LOL I know10:00
lucasagomessocial, once that patch to puppet-ironic merges we can remove the ipxe element all together10:00
jtomasekd0ugal, can you please try to reproduce by calling mistral run-action tripleo.get_parameters '{"container": "<your_plan_name>"}'10:00
openstackgerritMerged openstack/tripleo-specs: Availability monitoring support
d0ugaljtomasek: Yeah, I get the same error.10:04
d0ugaljtomasek: The CLI doesn't use that one at the moment.10:05
jaosoriorsshnaidm: I see the issue now. It's a mistral error. Will try to fix ASAP10:05
sshnaidmjaosorior, thanks, the reproducing stuck, trying again..10:06
openstackgerritMerged openstack/tripleo-specs: Update rendered specs URL
*** jbadiapa has quit IRC10:12
*** dtantsur|bbl is now known as dtantsur10:12
*** rasca has quit IRC10:16
*** rasca has joined #tripleo10:18
shardyjtomasek: I have reproduced, it's a heat issue10:19
shardyjtomasek: for now we'll have to put some dummy values into the action10:20
shardyheat template-validate --show-nested --template-file overcloud.yaml -e overcloud-resource-registry-puppet.yaml10:20
shardythat will reproduce it10:20
*** jbadiapa has joined #tripleo10:27
*** chlong has quit IRC10:28
d0ugaljtomasek: do you have any idea why the environments are a dictionary with only the path key?10:29
d0ugaljtomasek: why isn't it just a list of strings.10:29
*** tosky has joined #tripleo10:31
*** zoli|brb is now known as zoli|wfh10:33
openstackgerritDougal Matthews proposed openstack/tripleo-common: Adds parameters actions
openstackgerritDougal Matthews proposed openstack/tripleo-common: Adds Deployment Action & Workflow
openstackgerritDougal Matthews proposed openstack/tripleo-common: Adds action for template processing
openstackgerritDougal Matthews proposed openstack/tripleo-common: Add an action for adding envs to plans
d0ugalThe deploy command is a mess of weird things that we need to figure out how to support with Mistral :(10:38
jpichd0ugal: What kind of information are we using the Mistral environments for at the moment?10:39
d0ugaljpich: parameters, root template and a list of enable environments10:40
d0ugalI think that is all10:40
* shardy wonders if mistral encrypts the environments10:40
d0ugalIt isn't intended to be a secure store.10:40
shardyOk, then we'll have to move the password stuff into the deployment workflow10:40
shardyI commented on that yesterday for other reasons e.g plan reuse10:41
d0ugalbut swift doesn't encrypt them either?10:41
shardyd0ugal: heat does10:41
*** chlong has joined #tripleo10:41
d0ugalshardy: don't we need to store them somewhere else?10:41
jpichd0ugal: "enable environments"?10:41
d0ugalwill heat give me my paswords back?10:41
shardyd0ugal: If we can't do it semi securely, I'm not sure if we should10:42
d0ugalshardy: I completely agree.10:42
* jpich reading up on and trying to link all the uses of the term 'environment'10:42
d0ugalshardy: I have complained about this in the past :(10:42
shardyd0ugal: Ok, well lets get it working, then we can revisit the discussion10:43
d0ugaljpich: Yeah, sorry - I should say Heat environment.10:43
d0ugaljpich: They are used to enable and disable certain features.10:43
shardyjpich: they are kinda similar, except the syntax is different10:43
d0ugaljpich: For example: - this adds the network isolation envs via the CLI10:43
jpichd0ugal: Sorry, I'm starting from really the bottom. Thank you for the link and explanations10:45
d0ugaljpich: no problem - I hope I am answering the correct question!10:45
jpichshardy: Thank you. So we use them to do the same kind of thing, then?10:45
jpichd0ugal: I'll tell you when I have that figured out, too ;)10:45
*** apetrich has quit IRC10:45
shardyjpich: Yes, in both cases they provide data that influences the thing that is run10:45
shardyjpich: in mistral it's a workflow/action and in heat it's an orchstration template10:45
openstackgerritAthlan-Guyot sofer proposed openstack/tripleo-quickstart: Fix wrong naming of image's name in release.
jpichshardy: Cool! There's not much mistral documentation about environments so if I can adapt the heat docs to understand them I'm happy10:46
shardyjpich: it's a little confusing because in this case we're basically storing a definition on the heat environment inside the mistral environment10:46
jpichshardy: Is a Mistral environment associated with a workflow (one workflow = one env) or with an execution (different runs of a workflow can use a different environments)?10:47
*** r2bit has quit IRC10:47
d0ugaljpich: no, you refer to a environment by it's name.10:48
d0ugaljpich: so it can be shared across workflows. We use one at the moment that has the same name as the Heat stack (and swift container)10:49
sshnaidmshardy, can you please vote for ?10:50
sshnaidmpabelanger, ^^10:50
sshnaidmseems like we lost tempest job in all these movements10:50
shardysshnaidm: +1 - do you know if it will run within the infra timeout?10:51
jtomasekd0ugal: see discussion here:
sshnaidmshardy, to be sure I have another patch for it :)
jpichd0ugal: Ok, thank you! I can see in the Mistral docs how to call on environment variables from a workflow/task10:52
jpichIt's just not totally clear to me when/how a workflow gets linked with a particular environment but I'll figure that one out later10:52
openstackgerritDougal Matthews proposed openstack/tripleo-common: Add an action for adding envs to plans
sshnaidmshardy, in last month jobs took more and more time again, I need another 15 minutes for tempest, so removing the second undercloud install there10:53
*** ohamada has quit IRC10:53
*** ohamada has joined #tripleo10:53
d0ugaljpich: I am not sure if they are ever accessed from the workflow. We access it from the actions:
jpichd0ugal: Oh, thank you for the link! That helps a lot10:54
jtomasekjpich, mistral environment name = plan name10:54
d0ugalbut yeah, that is another reason to not store secure stuff in Mistral. Any action (or workflow maybe) can access it. I don't think it is tied to a user.10:54
d0ugalshardy: ^10:54
shardyd0ugal: ouch :(10:55
openstackgerritMerged openstack/instack-undercloud: Add tripleo deployment config file paths to hiera
jpichd0ugal: Scope seems to be same project_id or public -
d0ugaljpich: oh, good. better than I thought10:56
jtomasekd0ugal, shardy: regarding password generating, how do we find out there is a password to generate? We should not hardcode parameter names in workflows10:56
d0ugaljtomasek: There is no way :/10:57
d0ugal... that I know of.10:57
jtomasekd0ugal, shardy: ideal would be parameter type password, but there isnt such, so how about a constraint (custom constraint)10:57
shardyWell there are two ways I can think of - one is to introspect the templates via heat nested validation (which we do anyway, ref the bug jtomasek mentioned), then identify all *Passsword parameters that have no default (which should be nearly all of them)10:58
shardythe other (long term better) way would be to have a way to get heat to generate a random string as a default value10:58
d0ugaljtomasek, shardy: They are not all called "*Password" - i.e. NeutronMetadataProxySharedSecret10:59
shardyjtomasek: We could just use either a naming convention, or parameter_groups10:59
shardyjtomasek: as you say, a new parameter type would be a way to enable the auto-generated random password10:59
d0ugalAdminToken :P11:00
d0ugal(There is already an AdminPassword)11:00
* shardy throws something at d0ugal 11:00
*** bootsha has quit IRC11:01
*** stendulker has quit IRC11:02
*** r2bit has joined #tripleo11:02
*** r2bit has quit IRC11:03
*** r2bit has joined #tripleo11:06
openstackgerritDougal Matthews proposed openstack/python-tripleoclient: Migrate the deploy command to use the Mistral workflows
jtomasekshardy: yeah, if the parameter type password value is not provided, then autogenerate it...11:07
jtomasekshardy: do you know how I could workaround that 'ERROR: The Parameter (DeployIdentifier) was not provided.' bug?11:08
openstackgerritDougal Matthews proposed openstack/python-tripleoclient: Ignore the .eggs directory
*** akrivoka has joined #tripleo11:09
openstackgerritDougal Matthews proposed openstack/tripleo-common: Adds Deployment Action & Workflow
sshnaidmjaosorior, it's weird, I can't reproduce mistral issue, introspection was passed in my test job, can you send me your solution please when it's ready? i'll check it on my env11:14
*** bootsha has joined #tripleo11:14
jaosoriorsshnaidm: Will do11:15
jtomasekd0ugal: how is this command going to work? openstack overcloud deploy --templates /tmp/templates -e /tmp/templates/environments/enable_ceph.yaml -e ~/local_parameters.yaml11:17
openstackgerritDougal Matthews proposed openstack/tripleo-common: Adds Deployment Action & Workflow
openstackgerritDougal Matthews proposed openstack/tripleo-common: Add an action for adding envs to plans
jtomasekd0ugal, is it going to be creating plan from provided templates and setting those environments 'enabled'?11:17
jtomasekd0ugal, and then depoying?11:17
d0ugaljtomasek: that is what we are trying to figure out11:17
d0ugaljtomasek: but yeah, something like that.11:18
jtomasekd0ugal: ok, so the command is either going to use --templates or --plan11:19
d0ugaljtomasek: no, I don't think that will happen.11:19
d0ugalwell, it might11:19
d0ugalI don't know. This hasn't been discussed.11:19
d0ugal(other than in the last few days)11:19
jtomasekd0ugal, ok, yeah. I haven't gotten to deployment yet, I need to get parameters actions ready to be able to progress (that bug is blocking right now)11:20
*** lucasagomes is now known as lucas-hungry11:21
*** chlong has quit IRC11:27
*** chlong has joined #tripleo11:27
*** dsariel has joined #tripleo11:29
*** oshvartz has joined #tripleo11:30
*** sshnaidm is now known as sshnaidm|lnch11:31
*** rhallisey has joined #tripleo11:34
jaosoriorsshnaidm: This should work for mistral
jaosoriorsshnaidm: Are you able to test with that?11:43
jaosoriorsshnaidm: It seems to fix the issue in my environment. But of course, more reviewers/people testing should happen11:44
*** jpena is now known as jpena|lunch11:51
*** weshay has joined #tripleo11:56
*** bfournie has quit IRC11:56
*** lucas-hungry is now known as lucasagomes11:58
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci: WIP - Implement undercloud upgrade job - Mitaka -> Newton
*** egafford has quit IRC12:07
jaosoriorEmilienM: quick CR removing code :D12:07
*** sshnaidm|lnch is now known as sshnaidm12:08
EmilienMI love when we remove code12:08
sshnaidmjaosorior, yep, I'll give a try12:08
adarazsfolks, has anybody seen this recently? InputException: Invalid input [name=baremetal_introspection.wait_for_finish, class=NoneType, unexpected=[u'uuids']]12:10
adarazsthis started happening during introspection on the master branch, blocks promotion currently.12:11
jaosorioradarazs: Yep, trying to fix it here12:11
adarazsweshay: ^12:11
jaosoriorIt's a mistral issue12:11
EmilienMccamacho: hey, can I help on aodh composable role?12:11
weshayadarazs, k.. please add that to the etherpad12:12
ccamachowas going to ask you for help12:12
jaosorioradarazs, weshay: If you guys can reproduce the error and help test the fix. It would be great. Seems to work in my environment, but more eyes and testers are appreciated12:12
weshayadarazs, probably also want to run rdo-ci-check12:12
sshnaidmjaosorior, can you please mention in your commit?
openstackLaunchpad bug 1609688 in tripleo "CI: nonha jobs fails in introspection step because of mistral error" [Undecided,New]12:12
sshnaidmadarazs, ^^12:12
adarazsjaosorior: yep, I can test your fix if you think that works12:12
EmilienMccamacho: good, do you need me to test? review? tell me12:13
adarazsweshay: we could, but the error is not present in master, so I don't know if it will fix it :)12:13
ccamachoEmilienM We are having an issue when using apache wgsi L4112:13
*** apetrich has joined #tripleo12:13
adarazsweshay: only in the unpromoted master.12:13
*** jayg|g0n3 is now known as jayg12:13
dtantsurEmilienM, hi! when you have a second, could you please take a look at ? seems to work in my testing..12:14
*** dculik has quit IRC12:14
EmilienMdtantsur: sure12:14
EmilienMccamacho: ok12:14
ccamachoEmilienM If not used  then  httpd seems to start
jaosorioradarazs: Are you gonna test it in CI or will you try to reproduce it locally?12:14
sshnaidmEmilienM, I think we can proceed with this:  the error in job is timeout of postci, so everything succeeded12:15
weshayadarazs, oh.. you know what.. there's a thought.. I wonder if the 3rd party gate job should pull the testing image12:15
weshayadarazs, it probably should12:15
adarazsjaosorior: so, I can do both. but the last promoted master image is a bit old, so I hope it will work. :)12:15
weshayadarazs, know what I mean?12:15
EmilienMsshnaidm: ok, wait a sec, looking other things now12:15
adarazsweshay: we could maybe have a keyword to invoke it like that rdo-ci-check-test or something, but I would think it's better to use a promoted image by default that works.12:16
ccamachoEmilienM im testing the wsgi manifest but not finding any useful clue yet.12:16
sshnaidmEmilienM, sure, no hurry12:16
weshayadarazs, sure.. as long as we have the option.. ++12:16
adarazsweshay: should I make a card for it?12:16
weshayadarazs, aye12:16
weshaysshnaidm, morning :)12:16
EmilienMdtantsur: +A12:16
sshnaidmweshay, hey :)12:17
EmilienMccamacho: ok looking logs now12:17
*** Goneri has quit IRC12:17
EmilienMccamacho: why are we spending time on pacemaker/aodh12:18
EmilienMI thought we were removing it12:18
EmilienMccamacho: I'm sure we can sync with bandini and check if we really need Aodh with pacemaker12:18
openstackgerritMerged openstack/puppet-tripleo: Remove unnecessary certmonger manifest
ccamachowould be great if we just can remove it, but not sure if we need it at least for now12:19
EmilienMbandini: hey! in the new HA arch, do we still manage apache with pacemaker?12:20
EmilienMI think yes, look
EmilienMccamacho: ^12:20
EmilienMccamacho, bandini: I think we have a patch order problem on
EmilienMimho, we should rather push 314208 first and on top the aodh thing12:22
*** tbonds has joined #tripleo12:24
openstackgerritLukas Bezdicka proposed openstack/instack-undercloud: Use ensure_resource instead of package
*** Goneri has joined #tripleo12:25
*** jeckersb is now known as jeckersb_gone12:25
*** tzumainn has joined #tripleo12:25
EmilienMsocial: you could even declare them in an array :)12:26
*** zoli|wfh is now known as zoli|lunch12:27
*** dprince has joined #tripleo12:28
socialEmilienM: I'd like to drop them12:28
openstackgerritMerged openstack/puppet-tripleo: Fix Ironic dbsync ordering
shardyError: Could not find a suitable provider for mysql_datadir12:30
openstackgerritMerged openstack/puppet-tripleo: Add base nova-ironic profile
hewbroccaWhy the hell would we manage apache with pacemaker?12:30
*** bfournie has joined #tripleo12:31
hewbroccait's stateless...12:31
hewbroccaunless haproxy?12:31
jaosoriorshardy: if you go up there seem to be other errors12:31
openstackgerritDmitry Tantsur proposed openstack/tripleo-heat-templates: DO NOT MERGE: testing ironic
openstackgerritDmitry Tantsur proposed openstack/tripleo-heat-templates: Add composable service for configuring Nova compute with Ironic
*** ramishra has quit IRC12:31
jaosoriorshardy: For instance, this
jaosoriorNo such file or directory: '/sys/class/net/eth1/address'12:31
jaosoriorI remember there being a patch by slagle to check for eth1 in tripleo-ci. Might be related?12:32
sshnaidmI think we have a problem in CI with ha jobs, pingtest is always failing for various patches12:32
*** ramishra has joined #tripleo12:33
openstackgerritDmitry Tantsur proposed openstack/tripleo-heat-templates: DO NOT MERGE: testing ironic
openstackgerritDmitry Tantsur proposed openstack/tripleo-heat-templates: Add composable service for configuring Nova compute with Ironic
jaosoriorhewbrocca: No why we would be managing apache with pacemaker... not sure how haproxy would be involved either12:34
hewbroccabandini: ^^^ ?12:35
*** morazi has joined #tripleo12:35
*** pradk_ has quit IRC12:35
shardyjaosorior: Yeah, you're right, thanks12:35
*** dculik has joined #tripleo12:35
slaglejaosorior: i dont think that error is related to what i was fixing12:37
slaglethat error implies that there is no eth112:37
shardyYeah it seems to be a different issue12:37
jaosoriorI see12:38
slaglelooks like something wrong with how ovb setup the env12:38
slagleb/c the instance for the uc only has an eth012:38
EmilienMjaosorior, hewbrocca: I don't think we will manage apache with pacemaker, see
hewbroccaIt wouldn't make any sense if we did12:40
openstackgerritDougal Matthews proposed openstack/tripleo-common: Adds Deployment Action & Workflow
openstackgerritDougal Matthews proposed openstack/tripleo-common: Add an action for adding envs to plans
jaosoriordprince: By the way, the zaqar issue I was telling you about was an actual bug12:41
jaosoriordprince: can you check out this CR attempting to fix it?
openstackgerritBrent Eagles proposed openstack/tripleo-heat-templates: Add environment file to enable DVR
*** fultonj has joined #tripleo12:43
shardygfidente: Hey, you wrote
shardygfidente: and cschwede has referenced it on the ML12:44
shardygfidente: however, I don't think there is any way for a node to determine it's ironic UUID at runtime, is there?12:45
shardylast time I tried to find it I gave up and used the DMI system ID intead12:45
cschwedeshardy: seems it is - it’s working in my tests12:45
shardycschwede: That's good news, but where do you get the ironic ID on the node during deployment?12:46
dprincejaosorior: ack, glad you found it12:46
shardysomething may well have changed since I last investigated (or, I missed it at the time which is entirely possible :)12:46
cschwedeshardy: it’s not the ironic uuid, but the machine uuid12:46
shardycschwede: Ok, so the docs are wrong, cool12:47
* bandini back12:48
gfidenteshardy, cschwede right we don't want the ironic uuid12:48
gfidentewe want the dmi id there12:48
cschwedeshardy: i think the doc is right? i was irritate at first and used the ironic uuid, but then re-read the docs again and recognized that it is the machine uuid12:48
gfidentebecause that's how the per-node hieradata is distributed12:48
shardyYeah, it references the template I wrote that uses the DMI id12:48
shardyI'll post a patch to the docs12:48
*** jdob has quit IRC12:48
shardy"NODE-ID is the target node Ironic UUID"12:49
shardycschwede: ^^12:49
*** jdob has joined #tripleo12:49
shardyI think that's wrong12:49
cschwedeshardy: that references the command above12:49
cschwede„ The value returned by the above command will be a unique and immutable machine UUID which isn’t related to the Ironic node UUID. „12:49
gfidenteshardy, exactly12:49
pandapabelanger: you have time to chant in bj ?12:49
gfidenteit's the template which is describing it badly12:49
gfidentewhat we want is the machine uuid12:49
gfidentewhich is what the command gives12:49
bandiniEmilienM, ccamacho: (sorry just got back now) so no in the HA NG we do not run aodh nor apache under pacemaker.12:49
shardycschwede: gah, yeah, you're right, it's just confusingly worded12:49
shardyOk, carry on :)12:50
EmilienMbandini: it simplifies things for us12:50
EmilienMccamacho: we can remove all pacemaker things for aodh and rebase on top of bandini12:50
bandiniEmilienM, ccamacho: I can certainly push NG HA before aodh too, I was mainly doing it because "we did it for all the other composable services"12:50
bandinithat will mean that the aodh pacemaker profile will be broken, but hey not really a big deal12:50
bandinifor ocata I was going to propose to remove all the ones we do not use12:51
EmilienMbandini: yeah, we need both asap anyway12:51
ccamachoEmiliemM ack, then removing all the pacemaker logic from THT and PT12:51
bandiniEmilienM: ack. let me rebase without aodh, run a local test. and then I will submit and ping you and ccamacho12:51
bandinisounds good?12:51
EmilienMbandini: I think we can even remove them all in newton12:51
openstackgerritAttila Darazs proposed openstack/tripleo-quickstart: Improve readability of the overcloud failure output
bandiniEmilienM: ack, my only concern with removing them from newton is that if it turns out that a service does not cope well without pacemaker (because it needs some ordering constraints), we can still easily switch it to pacemaker12:52
bandinithat's why I'd let one release go with having them around12:53
bandinimaybe it is me just being extra cautious ;)12:53
openstackgerritCarlos Camacho proposed openstack/puppet-tripleo: Implement aodh profiles
*** dtantsur is now known as dtantsur|mtg12:54
*** dciabrin has quit IRC12:55
*** rlandy has joined #tripleo12:56
chemhi, is there a password defined by default for the heat-admin user, to log in using the console ?12:56
pabelangerpanda: not at the moment.  But I can here in IRC12:56
openstackgerritCarlos Camacho proposed openstack/tripleo-heat-templates: Add Aodh composable roles
*** jpena|lunch is now known as jpena12:58
*** mgould is now known as mgould|mtg12:58
ccamachoEmilienM, bandini deploying aodh locally now12:59
*** lblanchard has joined #tripleo13:00
bandiniccamacho: what changes did you make in the last iteration?13:01
openstackgerritJiri Stransky proposed openstack/tripleo-docs: Upgrade documentation
ccamachoremove the pacemaker profiles "EmilienM: ccamacho: we can remove all pacemaker things for aodh and rebase on top of bandini"13:02
*** jpich has quit IRC13:03
ccamachobandini ^13:03
bandiniccamacho: ah ok. got you13:03
ccamachojust testing all now13:03
*** pradk has quit IRC13:04
*** numans has quit IRC13:05
*** fultonj has quit IRC13:08
openstackgerritMichele Baldessari proposed openstack/tripleo-heat-templates: Next generation HA architecture work
bandiniccamacho: I am testing without the aodh stuff as well ^13:08
*** fultonj has joined #tripleo13:09
gfidenteshardy, cschwede sorry I was on a call but I think both the docs and the command are good, isn't it the case?13:09
shardygfidente: Yeah, I just misread it, sorry for the noise13:09
jaosoriorshardy, EmilienM, d0ugal: Now that we're using mistral more extensively, would it make sense to propose that tripleo's nonha job run on mistral commits?13:09
gfidenteshardy, cschwede I read the reply on the list13:10
EmilienMjaosorior: yes, the multinode non ha13:10
gfidenteI'm fine building on the undercloud13:10
EmilienMjaosorior: go for it13:10
jaosoriorEmilienM: Does the multinode nonha run introspection?13:10
shardyjaosorior: Maybe, but fwiw we don't do that for any other repos, even heat13:10
shardyso we mostly rely on the periodic promotion to fail and catch issues13:11
gfidenteit's the static mapping I was trying to avoid as well13:11
shardywe could definitely add an experimental job13:11
jaosoriorshardy: I see13:11
*** liverpooler has quit IRC13:14
EmilienMjaosorior: no, then you need ovb afik13:18
openstackgerritTomas Sedovic proposed openstack/tripleo-validations: Validation to verify the undercloud RAM reqs
d0ugalshardy: okay! I think I have something working13:19
*** egafford has joined #tripleo13:19
*** jeckersb_gone is now known as jeckersb13:20
shardyd0ugal: great! :)13:20
openstackgerritDougal Matthews proposed openstack/tripleo-common: Quick fixes
openstackgerritDougal Matthews proposed openstack/python-tripleoclient: Migrate the deploy command to use the Mistral workflows
d0ugalshardy: With those two patches ^13:22
d0ugalshardy: The first one I need to move into the parent commits, but wanted to push something up now13:22
shardylol @ "Quick fixes"13:22
d0ugalshardy: I'll sort that out when I get back from walking the dog. she is getting angry at being neglected :)13:22
shardydef deploy_hacks(self)13:23
shardyd0ugal: sounds good, I'll check it out after the deep-dive call13:23
openstackgerritLukas Bezdicka proposed openstack/instack-undercloud: Use ensure_resource instead of package
*** karthiks has quit IRC13:25
EmilienMthe undercloud upgrade job I've been working on is now failing because of Mistral db sync13:27
EmilienManyone saw it before?13:27
EmilienManyone able to upgrade undercloud to trunk now?13:28
hewbroccaI would be amazed if anyone has tried EmilienM13:28
pandapabelanger: ok, so I remember you saying that you could not upload tripleo-ci base image to some place public because it was too big. Do you think it's possible to build an alternate image that doesn't include project clones and put that somewhere public ? I think 5 of those 6 Gigabytes are project clones.13:29
*** gfidente has quit IRC13:29
openstackgerritBrent Eagles proposed openstack/tripleo-heat-templates: Add environment file to enable DVR
*** julim has joined #tripleo13:31
*** myoung|remote has joined #tripleo13:32
jaosoriorEmilienM: yeah, it hit RDO too. Trying to fix it here
pabelangerpanda: Ya, so after we talked the other day, I started looking into python-tripleoclient and seeing if we can create gate jobs that do just that. I'm testing with the overcloud-full at the moment, but if it works, we could do the same for the undercloud too13:34
EmilienMoh so the CI job I'm doing now is actually useful?13:34
jaosoriorthat might be another issue13:35
jaosoriornot sure13:35
*** gfidente has joined #tripleo13:36
*** gfidente has quit IRC13:36
*** gfidente has joined #tripleo13:36
EmilienMjaosorior: mistral dbsync fails !13:36
EmilienMI see it can be related to your patch13:36
jaosoriorEmilienM: where can I find the logs?13:36
jaosoriorfor the undercloud13:36
EmilienMI'm afraid there is no dbsync log but I can fix it13:36
EmilienMlet me a sec13:36
*** numans has joined #tripleo13:36
jaosoriorEmilienM: that would be great!13:37
pandapabelanger: great, thank! ping me if you need any help13:37
sshnaidmpabelanger, hi, maybe you know why does it happen? "No JSON object could be decoded" in
*** karthiks has joined #tripleo13:39
*** dciabrin has joined #tripleo13:40
*** pkovar has joined #tripleo13:40
EmilienMjaosorior: found it13:42
EmilienMin undercloud_install.txt13:42
openstackgerritAttila Darazs proposed openstack/tripleo-heat-templates: GATE TEST, please ignore
EmilienMjaosorior: ^13:42
*** dculik has quit IRC13:43
jaosoriorEmilienM: That actually doesn't seem related13:44
jaosoriorbut it seems that a dependency is missing13:44
openstackgerritTomas Sedovic proposed openstack/tripleo-validations: Validation to verify the undercloud CPU reqs
*** jpich has joined #tripleo13:45
EmilienMlooks like cachetools is missing13:45
EmilienMI think I found why13:46
*** jraju has quit IRC13:46
EmilienMin the logs, cachetools is updated at 12:40:16 but dbsync executed at 12:39:38.00013:46
EmilienMso I guess we have an orchestration issue here13:46
openstackgerritTomas Sedovic proposed openstack/tripleo-validations: Validation to verify the undercloud RAM reqs
EmilienMok it's a bug in
*** zoli|lunch is now known as zoli|wfh13:50
*** pgadiya has quit IRC13:51
EmilienMok the mistral upgrade problem is in packaging, I'll solve it here
*** dculik has joined #tripleo13:58
shardyTripleo Deep dive starting now14:00
*** milan has quit IRC14:01
*** milan has joined #tripleo14:01
openstackgerritBrad P. Crochet proposed openstack/tripleo-heat-templates: Composable Zaqar services
openstackgerritBrad P. Crochet proposed openstack-infra/tripleo-ci: Use tripleo-build-images for CI
*** pradk has joined #tripleo14:06
openstackgerritBrad P. Crochet proposed openstack/tripleo-docs: Switch docs to use tripleo-build-images
*** coolsvap has quit IRC14:11
*** apetrich has quit IRC14:12
*** ebalduf has joined #tripleo14:13
*** karthiks has quit IRC14:13
*** dtantsur|mtg is now known as dtantsur14:14
*** mgould|mtg is now known as mgould14:15
*** jaosorior has quit IRC14:17
*** apetrich has joined #tripleo14:20
openstackgerritDougal Matthews proposed openstack/tripleo-common: Adds parameters actions
openstackgerritDougal Matthews proposed openstack/tripleo-common: Adds Deployment Action & Workflow
openstackgerritDougal Matthews proposed openstack/tripleo-common: Add an action for adding envs to plans
d0ugalshardy: ^ tidied up a bit14:22
ccamachomichele the deployment doesn't fail, but after the deployment
*** karthiks has joined #tripleo14:30
*** Goneri has quit IRC14:35
*** dbecker has quit IRC14:38
openstackgerritTomas Sedovic proposed openstack/tripleo-validations: Validation to verify undercloud disk size reqs
hewbroccaEmilienM: very nice...14:39
*** jcoufal has joined #tripleo14:45
openstackgerritJiri Stransky proposed openstack/tripleo-common: Prevent replacement of Port, Net and Subnet
jistrafter a few months i finally had some bandwidth to revisit and test this ^^ :D14:47
*** dculik has quit IRC14:47
hewbroccajistr: back on trunk14:47
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci: WIP - Implement undercloud upgrade job - Mitaka -> Newton
pradkccamacho, the endpoint is really ?14:49
pradkccamacho, i wonder if the bind_host is not set correctly14:49
bnemecSigh.  The random bluejeans mute thing has been biting me a bunch lately.14:49
*** oshvartz has quit IRC14:49
pradkccamacho, whats the host in aodh.conf? is it still ?14:50
ccamachopradk :facepalm: for me.. Im running another test and I was testing against the undercloud.... So just a sec14:50
pradknot sure if bandini fixed the host issue14:50
pradkccamacho, so i see you removed all the pacemaker files from .. we dont need them anymore?14:52
pradki thought we havent fully made the switch to lightweight arch14:52
openstackgerritJiri Stransky proposed openstack/tripleo-common: Prevent replacement of Port, Net and Subnet
ansiwenbeaker tests can't be run with libvirt provider? seems there are only vagrant boxes for virtualbox and vmware14:53
*** dtrainor has quit IRC14:54
*** dtrainor has joined #tripleo14:55
*** beagles is now known as beagles_brb14:56
*** fpan has quit IRC14:59
bnemecSo rh1 was hosed this morning.14:59
bnemecIt looks like the db ran out of file handles, which caused basically everything to go down.15:00
*** dculik has joined #tripleo15:00
bnemecI've cleaned up the leftover heat stacks so nodepool actually has quota again.15:00
bnemecIt looks like we have some configuration tuning on the controller to do though.15:01
bnemecI'm going to keep an eye on it today.15:01
pradkbandini, EmilienM, , regarding apache issue wonder if it is because we're explicitly including aodh::wsgi::apache which is automatically loading apache via pacemaker?15:01
bandiniccamacho, pradk: nope I had not submitted the host fix yet, will do so shortly.15:02
d0ugalrbrady, shardy: - any idea how that could be possible?15:02
pradkthat would explain the 40415:02
bandinipradk: the reason for removing the pacemaker stuff, was because we weere thinking it is more efficient to first land HA next generation and then aodh and just skip the issue15:02
d0ugalrbrady, shardy: the line is different because i'm in a dev version - but it is here:
pradkbandini, ah ok.. so do we have to do anything explicit for aodh services to work on their own?15:03
pradkbandini, i.e via systemd15:04
d0ugalrbrady, shardy: and here:
bandinipradk: nope, I don't think so. but am testing aodh specifically in my next ha next generation test run (am redeploying now)15:04
ccamachopradk bandini Im running the deployment also again but taking ages..15:05
bandiniccamacho: I have added a comment what needs to be fixed for the aodh host issue. best if you just address it in your next iteration?15:06
*** numans has quit IRC15:06
pradkbandini, actually i already fixed the host issue15:06
pradkjust remembered .. see the right of the patch15:06
pradkbandini, thats what we want correct?15:07
*** akshai has joined #tripleo15:07
shardyd0ugal: is there any chance you're running an old os_cloud_config?15:07
bandinipradk: ah ops, right. I am blind. ccamacho ignore my comment15:08
bandinithen we need to check what else is wonky still15:08
*** apetrich has quit IRC15:09
*** rcernin has quit IRC15:09
pradkbandini, i'll do a new deploy now on top of your LW patches15:09
d0ugalshardy: damn, I am. Good spot15:10
d0ugalshardy: Looks like that version isn't released yet.15:10
bandinipradk: ack. the LW stuff is currently deploying correctly. I need to just smoketest vm creation15:10
shardyd0ugal: it's been released in both newton milestones15:11
d0ugalshardy: but not to PyPI I guess :)15:11
d0ugaloh, it had15:12
d0ugalah, I see - pip wont install a beta release15:12
*** akshai has quit IRC15:12
d0ugal(by default)15:12
pradkbandini, ccamacho, wonder if this is the issue .. see line 3115:14
pradkits set to {get_param: AodhApiVirtualIP} but AodhApiVirtualIP defaults to ' ' few lines above15:14
pradkthats explains my bind_host not configured correctly theory from above?15:15
dtrainorcan anyone tell me a little more about what happens behind the scenes with 'openstack baremetal configure boot'?  The docs say "Assign the kernel and ramdisk images to all nodes".  I'd like to know where that linkage is set etc.15:15
ccamachopradk If not defined yes will be '', can be, but remember that before we were running the pcm templates and that param were not defined, taking notes for when my deployment finish15:16
*** ayoung has joined #tripleo15:17
pradkccamacho, shouldnt the pcmk inherit from base?15:17
*** leanderthal is now known as leanderthal|afk15:17
d0ugaldtrainor: it is actually quite simple, it just sets a few properties on the nodes:
pradkccamacho, if not set in pcmk though that would actually be an issue to fix15:17
pradkccamacho, we need to tell apache what to bind on15:17
pradkaodh::wsgi::apache::bind_host:  would be required15:18
*** devvesa has quit IRC15:18
dtrainoroh, capabilities.15:18
dtrainorexcellent.  thank you.15:18
d0ugaldtrainor: np, hopefully that helps. I've not actually looked at the code in some time15:18
d0ugaldtrainor: You can blame me for the original version :)15:19
openstackgerritHonza Pokorny proposed openstack/tripleo-ui: Add packaging spec and config files
dtrainorhaha.  it makes sense, yes.15:19
pradkccamacho, interestingly we dont set that for other services15:21
*** ebarrera has quit IRC15:21
pradknor its set in mitaka.. i'll remove that and redeploy15:22
openstackgerritPradeep Kilambi proposed openstack/tripleo-heat-templates: Add Aodh composable roles
openstackgerritTomas Sedovic proposed openstack/tripleo-validations: Validation to check the undercloud processes
*** rwsu has joined #tripleo15:26
*** mcornea has quit IRC15:26
Sloweranyone else try building undercloud recently?15:29
EmilienMSlower: yes, a few min ago15:29
SlowerI'm getting mysql puppet errors when using pkgs and some repo error when using src15:29
EmilienMthere is a problem with mistral15:29
Slowerah ok15:29
EmilienMwhat is your trace?15:29
EmilienMis it mistral db sync?15:29
d0ugalseems like Mistral breaks us all the time :/15:30
d0ugalIs it our fault, or theirs?15:30
EmilienMwe need to improve their CI15:30
d0ugalEmilienM: What is missing?15:30
Slowerno different sounds like15:30
EmilienMd0ugal: ok not the same thing15:30
d0ugal(or what needs to be improved)15:30
EmilienMerr, Slower ^15:30
EmilienMd0ugal: tripleo jobs ;-)15:30
EmilienMSlower: sorry something I didn't hit15:31
SlowerEmilienM: or I can do puppet from source and I get something else15:31
openstackgerritGael Chamoulaud proposed openstack/tripleo-quickstart: Run dstat on the virthost and the undercloud
Slowerah there's a cache.. lemme remoev that15:31
*** bana_k has joined #tripleo15:32
SlowerEmilienM: from puppet src I get:
SlowerEmilienM: any thoughts on how to get around one of those? :)15:33
SlowerI'd really like to be able to do work again15:33
EmilienMSlower: sounds like a git issue15:33
Slowerwhich is odd15:33
EmilienM failed to get sha1 of fetch_master15:33
EmilienMI was able to deploy an undercloud like 1h ago15:34
SlowerI was seeing this last night too15:34
SlowerEmilienM: you use puppet src I presume?15:34
EmilienMSlower: now, I use packaging15:36
EmilienMand I do my modifications manually15:36
EmilienMcan you check if mysql is running?15:36
SlowerEmilienM: it's not even installed..15:37
openstackgerritSteven Hardy proposed openstack/instack-undercloud: Work around deletion of _member_ role assignments on upgrade
SlowerEmilienM: didn't know that was a requirement actually15:38
EmilienMSlower: mysql is deployed by puppet15:40
EmilienMyou don't have anything to deploy yourself15:40
EmilienMcan someone review to Add undercloud job in cistatus ?15:42
SlowerEmilienM: I'm just trying to build the undercloud image15:42
Slowereg openstack undercloud install15:43
d0ugalshardy: Okay, with a updated os-cloud-config, a deploy with no errors!15:44
shardyd0ugal: great, nice work! :)15:45
*** beagles_brb is now known as beagles15:45
d0ugalshardy: Now I think I need to integrate jpich's patch:
d0ugalshardy: So that users can specify their own templates, and don't need to pre-create a plan.15:45
*** dculik has quit IRC15:46
d0ugalrbrady: ^ update15:46
shardyd0ugal: sounds good15:46
shardyd0ugal: if you go ahead and do that, I'll pull everything tomorrow morning and test it15:46
shardyhopefully we can start landing things15:46
d0ugalshardy: Great, I'm not convinced I'll have that added by the end of today, but early tomorrow should be easy.15:47
d0ugalshardy: +1, it would be good to start landing the workflows so we can then have CI test the CLI and iron out all the remaining bits15:47
openstackgerritwes hayutin proposed openstack/tripleo-quickstart: remove overcloud roles from oooq
jpichd0ugal: You could upload stuff to Swift directly and call the create_plan action, if creating the JSON is too much? I didn't add a client action for that because I imagine folks will upload templates as files directly with jsonifying, but maybe there's a need for another workflow for that too...?15:48
d0ugaljpich: oh, is that how the create_plan workflow works.15:48
d0ugaljpich: Maybe I should do that? What is the use-case for your patch if we are just uploading to swift directly?15:49
jpichd0ugal: IIRC it takes a container and plan-ifyise it yeah :-)15:49
jpichd0ugal: Have a proper workflow to do this, I think uploading files as JSON was more relevant to the GUI directly15:49
jpichbut from a CLI perspective I can't see it making as much sense15:50
*** yamahata has joined #tripleo15:50
d0ugaljpich: hmm, okay. I'll try uploading directly. In some ways I'd rather us all using the same workflows... but I'm happy to not depend on another patch15:50
jpichd0ugal: which itself depends on another patch... ;)15:50
bandiniEmilienM, ccamacho, gfidente: I smoketested the last iteration of review 314208 and it looks okay. So I am okay with landing this now (we can land aodh on top afterwards)15:50
bandinilet's wait for CI in any case15:51
openstackgerritPradeep Kilambi proposed openstack/tripleo-heat-templates: Set Coordination url for central agent
gfidentebandini, yep now it looks good :)15:52
*** pkovar has quit IRC15:52
gfidenteI'd say the small reorg worked out15:52
gfidentewe'll see if it'll also pay or not15:52
bandinigfidente: yes I think it was worth it. thanks again for pushing me in that direction ;)15:53
gfidente(we had the full ha tested and are saying goodbye in style) :)15:53
rbradyjtomasek, d0ugal, jpich: "but from a CLI perspective I can't see it making as much sense".  I remember a time not too long ago we worried about cli divergence.  now it seems the GUI is diverging.15:53
gfidentebandini, honestly with marios talking bad about me all the times15:53
jpichrbrady: The workflow is there in tripleo-common, for the client to use if we want it?15:54
bandinigfidente: well marios is always right, so surely you did something evil15:54
gfidentebut it's not in tripleo15:55
bandiniahahah right15:55
bandinigfidente, marios: I'd like that we agree on "What happens in Barcelona, stays in Barcelona", just in case15:55
gfidenteI don't know, slagle is taking picutes15:56
slagleonly b/c i thought a felony was about to occur15:57
d0ugalrbrady: Indeed, but we are getting closer. Had you imagined the GUI would upload to swift directly?15:57
gfidenteslagle, I know I can count on you15:57
dprinced0ugal: I had, yes15:57
d0ugaldprince: :)15:57
* bandini chuckles15:58
rbradyd0ugal: previously discussed15:58
rbradyjpich: I'm not sure I understand what you're asking15:58
d0ugaldprince, rbrady: I don't know if there is a reason they can't do that? jtomasek jpich}15:58
* jtomasek reads back15:59
mariosbandini: honestly, i don't know what to do  i think i have to report gfidente to hr for the greater good15:59
mariosbandini: i think it is the right thing to do for all concerned15:59
mariosbandini: is anj going?15:59
jtomasekrbrady: where is that quote from?16:00
rbradyjtomasek: scroll up a bit.  it's from jpich16:00
d0ugaljtomasek: a bit further up - jpich said it :)16:00
bandinimarios: ANJ says hello, as usual ;)16:00
jpichrbrady: I actually wasn't around for the conversation around concerns related to CLI divergence - I assume it means code that lived in the CLI only? This one is in tripleo-common so it can be used by both CLI and GUI if wanted, but maybe your concern is different?16:00
*** liverpooler has joined #tripleo16:00
mariosbandini: y u so mean16:01
dprincejpich: simply putting code into tripleo-common isn't enough I think16:01
gfidenteI think I'm joining this marios is always right bus16:01
rbradyjpich: I think it's important we attempt to have both UI's utilize the same underlying code16:01
rbradyjpich: for CI and for consistency16:02
dprincejpich: using in both places would be important.16:02
gfidentewe need jistr to say that as well though16:02
dprincejpich: and if we can't get the CLI to use it in Newton, then I would argue the UI needs to try to adapt to do it the same way16:02
*** zoli|wfh is now known as zoli_gone-proxy16:02
dprincejpich: parity16:02
jtomasekyeah, I agree, that both CLI and GUI should use most of the same workflows. In cases where the 2 diverge, we could simply have an overlapping workflows16:03
*** cwolferh has joined #tripleo16:03
d0ugalAlright, so I should use the JSON plan uploader?16:04
jpichrbrady, dprince: I understand the idea behind it, and I agree with the principle. In this particular case it seems to me there is a way to do things (uploading files) that generally works differently depending on whether you're using a CLI to upload files or a web interface, and it's the more natural way to do it16:04
openstackgerritMerged openstack/tripleo-docs: Group advanced topics by type
EmilienMbandini: +116:05
jpichbut I don't have extremely strong feelings toward file uploading, whatever people would prefer we use for creating a new plan from templates I'm happy to help implement16:05
jtomasekjpich: so the problem is about CLI should not have to parse the files and send contents to a workflow as json?16:06
EmilienMbnemec, slagle: you aware about this CI issue?
*** apetrich has joined #tripleo16:06
EmilienMit looks like all CI is red16:06
bnemecEmilienM: Yeah, I'm looking into it.  There was a problem with the rh1 cloud this morning that broke everything, and it still looks like jobs aren't getting testenvs properly.16:07
jpichjtomasek: I suppose we could do it that way! Is that how the UI does it too? Means we have to add additional parsing code in Python only to the CLI command, but maybe it makes sense for that one16:07
bnemecAlthough the current failures may be teething problems from bringing the cloud back from the dead because I see jobs running right now.16:08
jtomasekjpich: yeah, we'll have to parse the files on the client side (GUI) - only way to get them to mistral16:09
jtomasekjpich, d0ugal: regarding adding files to plan, you use just the action you (d0ugal) created patch for recently16:10
d0ugaljtomasek: What parsing do you do?16:10
jpichjtomasek: Alright, I'll add a task to do it that way for the client as well. Thanks for the extra-background :-)16:10
d0ugaljtomasek: There is no chance I am going to add a full plan one file at a time :)16:11
mariosgfidente: ymmv16:11
d0ugaloh, actually, it accepts multiple files16:11
jpichd0ugal: File objects to json strings, I think16:11
jtomasekd0ugal: why one?16:11
d0ugaljpich: we almost wrote the same thing, lol16:11
* d0ugal face palms16:11
jpichd0ugal: Wait, what? Did I duplicate a patch?16:11
jtomasekcreate plan workflow is a workflow because it does 2 things - puts files into swift and if it succeeds, creates plan16:12
d0ugaljpich: Nah, it was me16:12
mandreshardy: what was name of the python module you used to debug the endpoints in the tripleo deep dive earlier again?16:12
*** ohamada has quit IRC16:12
jtomasekto add files to plan you only need to do single action -> add files to swift16:12
d0ugaljpich: I just wrote the action, it is very similar to yours.16:13
dprincejpich: in each case I'd like to consider using just a tarball16:13
shardymandre: entry_point_inspector16:13
d0ugaldprince: Can the GUI create tarballs?16:13
mandreshardy: thx :)16:13
jtomasekd0ugal: user provides a tarbal - that is how GUI currently gets the files16:13
d0ugaljtomasek: oh, cool.16:14
jtomasekd0ugal: but mistral is not able to accept file16:14
dprinced0ugal: I don't see why not16:14
d0ugalit is all coming back to me now16:14
dprincejtomasek: mistral doesn't need a file16:14
shardydprince: I think the debate is does it go via swift or direct to mistral16:14
dprincejtomasek: it needs to location of the file you uploaded elsewhere16:14
shardypersonally I'd be fine if we just always put the data directly into swift16:15
dprincejtomasek: whoever said Mistral needs to do this?16:15
jtomasekdprince: yes and that is exactly the problem16:15
d0ugalshardy: +1, I think that was the original plan - it has just gotten a bit confused.16:15
jtomasekdprince: the operation of creating a plan is 2 steps -> workflow16:15
dprincejtomasek: upload the templates16:15
dprincejtomasek: then create the plan16:15
dprinceI have no issue with that16:15
d0ugalSeems fine to me.16:16
jtomasekdprince: in this case we make client (GUI) handle the workflow -> errors such as a case when swift upload fails16:16
dprincejtomasek: if it fails you'll get back an HTTP error. What could possibly be better than that?16:16
shardydprince: is what sparked the debate I think16:16
*** dtantsur is now known as dtantsur|afk16:16
shardygiven the issues with messing with file limits there, I'd much prefer to just push the data to swift16:17
mandreshardy: thanks, this came in really handy16:17
shardyit can already handle it nicely16:17
jtomasekthis problem was actually the initial thing that lead to jpich and florianf looking into refactoring the plan creation workflow16:17
*** tremble has quit IRC16:18
d0ugalI think we have bigger issues to resolve. I'd vote for uploading to swift and then triggering a workflow.16:18
jtomasekshardy, dprince: so you say that client needs to know that files are stored in swift and handle the 'workflow' of creating the plan by itself16:18
jtomasek1. upload tarball to swift and 2. run create_plan action16:18
openstackgerritMerged openstack/instack-undercloud: Use ensure_resource instead of package
shardyjtomasek: yes16:19
shardyI think that's easier than having to parse a bunch of files and format a json blob16:19
shardydo you disagree?16:19
openstackgerritMerged openstack/instack-undercloud: Remove discover from test-requirements
d0ugalHaving the client know is kinda nice, maybe one day the workflow could support pulling from git16:19
d0ugal(from git or various other places that might be useful)16:19
shardyd0ugal: Yeah, that would be cool, and it could support an existing (non swift) http url16:20
jtomasekshardy: optimal solution would be to let workflow tool do the workflow stuff (mistral) and at the same time make the mistral accept files16:20
shardybut we're preoptimizing, lets get the deployment to work then refactor for adding interfaces :)16:20
d0ugaljtomasek: Okay, get Mistral to accept files and we can move to that ;)16:20
jtomasekd0ugal: hm16:20
florianfshardy, dprince, jtomasek: I should mention that it's not really just uploading to swift. There is some metadata that needs to be added to the container so tripleo-common can recognize it as a plan container.16:20
dprincejtomasek: yes that is fine I think. Never parse the files. The whole point of using the Swift bulk middleware was to get us out of the mess of per-file management16:20
*** weshay is now known as weshay_food16:21
dprinceflorianf: get rid of the metadata16:21
shardyflorianf: there is an action that can set that metadata AIUI16:21
dprinceflorianf: I didn't need it to deploy with the CLI
dprinceflorianf: which is arguable a simpler case, but store the data in the Mistral environment would be better16:22
ccamachohey, today is 2²/2³/2⁴16:22
jtomasekd0ugal: "Having the client know is kinda nice, maybe one day the workflow could support pulling from git" I don't follow16:22
jtomasekjtomasek: if the backend workflow was the one who interacted with git, that would be better, wouldn't it?16:23
gfidenteccamacho, 0.0312516:23
d0ugaljtomasek: Just a future thought, so don't worry about it. However, at the moment the workflow knows how to get it from swift, we could add more options later (like git)16:23
dprinced0ugal: git isn't openstack16:24
dprinced0ugal: git -> tarball -> swift is How I'd recommend you do that16:24
jtomasekd0ugal: yeah, optimal would be if workflow knew how to get files to swift and from it. but it doesnt16:24
d0ugaljtomasek: So having the client and the user know where the files live isn't a bad thing particularly16:24
d0ugaldprince: I didn't say it was, git was just an example of another place the files could live.16:24
d0ugalbut we are getting way off point16:24
dprinced0ugal: the sources can be anywhere I think16:25
d0ugaldprince: That's my point.16:25
dprinced0ugal: the format we accept is tarball16:25
*** maeca1 has joined #tripleo16:25
jtomasekdprince: ok, makes sense16:25
d0ugalSo I am going to have the CLI upload a tarball directly.16:25
dprinced0ugal: think of it like DIB creating a qcow. DIB sources are stored in Git but they generate a qcow16:25
d0ugalThanks all16:25
d0ugaldprince: Sure, I get it.16:25
ccamachogfidente, indeed if you assume left to right associativity :P16:26
dprinced0ugal: like 234 here works great FWIW:
dprinceline, not like16:26
jpichjtomasek, d0ugal, dprince: So, I should abandon the patches related to JSON file uploads then, is that correct?16:29
jtomasekjpich: seems so, basically we agreed to keep what we currently do in GUI16:29
jtomasekso the necessary workflows are in place16:29
jtomasekd0ugal: so how about that addition of environments? how are you going to handle it?16:30
jtomasekd0ugal: upload new files to plan in swift and then trigger an update_capabilities action?16:30
jtomasek(to enable them)16:30
jpichAll right then16:30
jpichflorianf ^ - I'll update the blueprint16:30
*** kro_focused is now known as krotscheck16:32
rbradyjtomasek: will additional environment data only be file-based or do you envision text as well?16:33
d0ugaljtomasek: Not sure yet. I have that patch that works, just need to figure out the best way now.16:33
florianfjtomasek, jpich: We can't keep what we currently do in the GUI if the tarball is the only accepted format16:33
d0ugaljtomasek: I don't think there was a plan for extra environments files like the CLI requires.16:34
jtomasekrbrady: file-based only16:34
d0ugalflorianf: What does the GUI do now?16:34
ccamachohey pradk quick question16:34
jpichflorianf: I think maybe the idea becomes "you do whatever you want to get the data to the Swift container"16:34
florianfjtomasek, jpich, d0ugal : We currently allow both tarballs and a JSON structure with filepath => content16:34
jtomasekflorianf: we can do both afaik16:35
openstackgerritPradeep Kilambi proposed openstack/tripleo-heat-templates: Add Aodh composable roles
florianfjpich: yeah, we're going to have to rewrite that so all files are added to swift16:35
florianfjtomasek: yes, we can. but we're going to have to rewrite that. and it's going to be quite a lot of swift calls :-)16:36
pradkccamacho, bandini, i think this should fix the apache host issue
*** oshvartz has joined #tripleo16:36
pradkwe keep that for other services16:36
d0ugaljtomasek, florianf: please don't upload to swift one file at a time.16:36
pradkin controller.yaml16:36
d0ugalthe reason we used a tarball originally was to avoid that.16:36
ccamachothe retry params for gnocchi they dont exist in puppet-gnocchi openstack/gnocchi or in the config files once the service its deployed. I did something to add the params in glance but they where in different manifest
jtomasekflorianf: we can't bundle the files into single json to send it to swift?16:36
ccamachopradk ^16:36
jtomasekd0ugal: haha16:37
pradkccamacho, oh so we dont use oslo::db i guess in this case16:37
*** Ryjedo has joined #tripleo16:37
florianfjtomasek: Not sure if swift can create multiple objects from a single json structure. I'd have to check...16:38
ccamachopradk, I think16:39
pradkccamacho, we actaully do use oslo db.. i guess puppet module needs to be updated16:39
*** lucasagomes is now known as lucas|dinner16:39
pradki'll submit a patch today16:39
*** myoung|remote has quit IRC16:40
ccamachoYou can use this as reference
pradkwe support both max retries and interval16:40
jpichflorianf: I'd be surprised but!16:40
d0ugaljpich: I would too, but I was surprised at the tarball extracting feature...16:40
*** weshay_food is now known as weshay16:43
*** florianf is now known as florianf|biab16:44
pabelangerslagle: EmilienM: panda: bnemec: here's the first pass at adding an experimental jobs to test overcloud-full.qcow2 builds.
pabelangerslagle: EmilienM: panda: bnemec: Tested it locally, diskimage-builder will build the image, but eventually fails becasue it is actually broken atm16:47
*** david-lyle has quit IRC16:53
*** david-lyle has joined #tripleo16:53
*** tosky has quit IRC16:55
*** jtomasek has quit IRC16:55
*** jpich has quit IRC16:57
*** mgould is now known as mgould|afk16:57
*** tzumainn has quit IRC16:59
slaglebandini: hey, i'm getting this error trying to start corosync:
slaglebandini: from what i can tell it's b/c of the hostname in corosync.conf:17:03
*** flaper87 has quit IRC17:03
openstackgerritDougal Matthews proposed openstack/python-tripleoclient: Migrate the deploy command to use the Mistral workflows
slagleif i add .localdomain to those hostnames, it starts fine17:03
slaglehow does corosync attempt to resolve the hostname?17:03
slagleb/c both ways are already in /etc/hosts17:04
slaglei see what the problem is, something else has added slagle-multinode-ha-4 to /etc/hosts17:07
slaglewhat would be doing that17:07
slagledevstack-gate of course17:10
*** tzumainn has joined #tripleo17:10
openstackgerritDougal Matthews proposed openstack/python-tripleoclient: Migrate the deploy command to use the Mistral workflows
pabelangerslagle: I've replied to your comments17:12
slaglepabelanger: that was bnemec17:14
pabelangerslagle: Ah, right. must get more coffee17:15
slaglebut i kinda agree with his point17:15
slaglewe document building images a different way17:16
pabelangerhow are they built?17:16
slagleand I guess this job is just testing that an image *can* be built?17:16
pabelangerbecause, today they can't be built17:16
slagleprobably b/c that's not the way we document to do it17:17
pabelangerwhere are the docs so I can see how it should be done17:17
slaglewe delorean rpms17:17
slaglethat assumes you're on an undercloud with the repos setup and the packages installed17:18
slaglewhich is doc'd earlier17:18
slaglepabelanger: how is it failing for you?17:19
pabelangerlooks like systemd17:19
pabelangeris failing:
slagleweird, os-collect-config must have not gotten fully installed17:20
pabelangeropenstack overcloud image build appears to be the command?17:20
slagleyes, it's the same command you have in the job17:21
pabelanger is what I am trying to do17:21
pabelangerfrom centos-717:21
pabelangerwhich is basically, what the job does17:21
*** bfournie has quit IRC17:21
slagleshould that be ELEMENT_PATH? or ELEMENTS?17:22
*** dbecker has joined #tripleo17:22
slagleoh i c17:22
slaglei'd need to go look at this tox env17:22
slaglemaybe the virtualenv is confusing things?17:23
pabelangerI don't think so, it fails outside virtualenv too17:23
pabelangerI suspect something in systemd is broken17:23
bandinislagle: am back. reading now17:23
bandinislagle: ah you solved it already?17:24
*** jcoufal_ has joined #tripleo17:24
slaglebandini: yea, nm. it was a bogus IP in /etc/hosts17:24
bandinislagle: ah ack17:24
openstackgerritBen Nemec proposed openstack-infra/tripleo-ci: Enable net-iso for ovb jobs
*** dprince has quit IRC17:24
bandinislagle: for the record corosync uses getaddrinfo() to resolve names17:25
slaglepabelanger: if you export break=after-error before you run the command, dib should drop you to a shell after an error17:26
*** jcoufal has quit IRC17:26
slaglei've never used that though when calling dib via tripleoclient, so not 100% sure it will work17:26
*** abehl has quit IRC17:27
pabelangerslagle: right, I haven't started debugging it yet. I wanted to land the experimental job first, then fix it and upload to gerrit to run the experimental job17:27
pabelangerslagle: ya, I've had to hack in DIB_DEBUG_TRACE into python-tripleo-client to expose move debug into17:28
*** myoung has quit IRC17:28
EmilienMpabelanger: awesome17:34
*** Tristitia has quit IRC17:34
slaglepabelanger: ok, i like where you're going with the job. but i'm not sure about building images this way17:34
slaglei guess there might be some utility in fixing that17:35
pabelangersure, how would you like the images to be build?17:35
slaglehow we do in tripleo-ci17:35
slaglei think if you enabled the delorean repos, could use --repo-setup for this17:35
slagleinstalled the needed packages17:36
*** myoung|remote has joined #tripleo17:36
slaglethen ran the command17:36
slaglethat would be equivalent to what we document/ci17:36
pabelangerwhy do we need dlrn to test if building an overcloud image works?17:36
*** jpena is now known as jpena|away17:37
slaglewe don't17:37
slaglei just dont know that testing it can be build a different way is all that meaningful17:37
slaglewhen we dont tell people to build that way, or know that those images work17:37
*** shardy has quit IRC17:39
pabelangerokay, so you want to make sure DIB is using packages from dlrn17:39
slagleyea, basically17:39
slaglei mean i'm not -1 on your patch17:39
slaglei just don't want you spend a lot of time getting it working17:40
*** myoung has joined #tripleo17:40
slagleand then we wouldnt use this image in tripleo-ci, b/c it's now how we say to build images17:40
slagle*not how17:40
*** Tristitia has joined #tripleo17:41
*** dprince has joined #tripleo17:41
pabelangerRight, I'm not saying use this image for tripleo-ci right now. This is a POC to see if we can build an image in the gate. Which right now, appears we cannot.17:41
pabelangernow, that could be because we are not using DRLN repos for DIB to install packages from, which I can make a change for17:41
slagleis the os-collect-config rpm installed in the image?17:44
*** apetrich has quit IRC17:45
*** apetrich has joined #tripleo17:45
pabelangerlooks like it: installing os-collect-config from os-collect-config17:45
pabelangerlet me try again exporting the delorean vars17:46
*** tesseract- has quit IRC17:53
*** bfournie has joined #tripleo17:56
*** ccamacho has quit IRC18:01
*** julim has quit IRC18:02
openstackgerritwes hayutin proposed openstack/tripleo-quickstart: [WIP] remove overcloud roles from oooq
*** julim has joined #tripleo18:05
*** sarath has joined #tripleo18:05
gfidenteleaving guys18:07
gfidenteslagle, <318:07
slaglecome back soon now!18:07
gfidentethe day you'll try to kill me too I quit okay?18:07
gfidenteweshay, curiosity, did we get ooo fixed with the centos repos?18:08
slaglei know better than to pull on a knife on someone named "giulio"18:08
weshaygfidente, I'm getting there :)18:09
gfidenteweshay, ack18:09
*** abehl has joined #tripleo18:09
beaglesslagle: are we experiencing time skewing occasionally in CI on the controller nodes?18:10
*** gfidente has quit IRC18:11
slaglei dunno :)18:11
beaglesI vaguely recall seeing something about it lately but didn't commit to memory where I saw it18:11
slaglebut the fact that you're asking makes me want to say yes18:11
beaglesyeah, just looking at some logs and they are a little suspicious... just doublechecking18:11
beaglesdoublechecking the logs that is.18:12
openstackgerritwes hayutin proposed openstack/tripleo-quickstart: [WIP] remove ci-scripts from the oooq repo
sarathdprince, bnemec: please help review
*** jdob has quit IRC18:22
*** myoung|remote has quit IRC18:22
*** jdob has joined #tripleo18:22
*** oshvartz has quit IRC18:23
*** jbadiapa has quit IRC18:23
beaglesare things like this: "error: Internal error: No action found for demote in redis:1" cause for concern or is this something we are living with?18:24
EmilienMbeagles: where?18:26
beaglesEmilienM, 1s lost tab18:28
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci: WIP - Implement undercloud upgrade job - Mitaka -> Newton
beaglesEmilienM, file is
*** florianf|biab is now known as florianf18:34
*** jpena|away is now known as jpena|off18:34
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci: WIP - Implement non-ovb overcloud update job - Newton -> Newton
pabelangerslagle: adding the DLRN repo was the trick, image built as expected18:36
EmilienMbeagles: looking18:37
EmilienMbeagles: it looks like a pacemaker thing18:37
EmilienMand since we're removing it for redis, I would say, ignore it now18:37
beaglesEmilienM, ack.18:37
*** ayoung has quit IRC18:38
*** florianf has quit IRC18:44
openstackgerritMerged openstack/tripleo-quickstart: Fix wrong naming of image's name in release.
*** milan has quit IRC18:46
*** Goneri has joined #tripleo18:47
*** milan has joined #tripleo18:47
*** mgarciam has joined #tripleo18:51
*** bootsha has quit IRC18:51
*** alhernan has joined #tripleo18:53
*** fultonj has quit IRC18:59
*** ccamacho has joined #tripleo19:05
*** Goneri has quit IRC19:08
*** fultonj has joined #tripleo19:09
openstackgerritFrank Zdarsky proposed openstack/tripleo-validations: Add jsonschema validation for os-net-config data
ccamachopradk bandini EmilienM my env went to... Had to reinstall it with less nodes with more RAM for each node.. Anyway after that, aodh works as expected :) finally
EmilienMccamacho: woot19:11
pradkccamacho, this with latest ps? mine is still in progress with my last fix .. i guess the bind_host helped?19:11
rhalliseyanyone else been seeing mistral errors thrown around during baremetal setup steps in the undercloud?19:12
ccamachopradk indeed :)19:12
pradkgreat, thx for confirming.. i'll let mine finish19:12
ccamachorhallisey yeahp, I saw a few some minutes ago when deploying my undercloud but didn't interfere with the  deployment19:13
*** radek_ has joined #tripleo19:13
ccamachopradk lets wait for CI19:13
rhalliseyyesterday it was failing with introspection.  Today it fails when I import instackenv19:13
rhalliseyto the logs19:13
*** chem has quit IRC19:14
rhalliseywonder if it's mistral or ironic19:15
pradkccamacho, did a recheck lets see19:15
*** bana_k has quit IRC19:16
*** dsneddon_ has joined #tripleo19:17
*** oshvartz has joined #tripleo19:20
rhalliseymistral db error19:24
*** akrivoka has quit IRC19:31
openstackgerritDan Prince proposed openstack/python-tripleoclient: Deploy the undercloud with Heat
openstackgerritDan Prince proposed openstack/python-tripleoclient: Deploy the undercloud with Heat
*** bana_k has joined #tripleo19:34
openstackgerritMerged openstack-infra/tripleo-ci: Remove pinning of delorean
openstackgerritHarry Rybacki proposed openstack/tripleo-quickstart: [WIP] Prototype tandem documentation generation
openstackgerritMerged openstack-infra/tripleo-ci: Add undercloud job in cistatus
*** dsneddon is now known as dsneddon_lunch19:37
*** dsneddon_ has quit IRC19:38
openstackgerritPaul Belanger proposed openstack/tripleo-quickstart: [WIP] Stop using data_files
openstackgerritRatnakar Kolli proposed openstack/tripleo-heat-templates: Adding optional aci-opflex hiera data
*** radek_ has quit IRC19:40
*** jrist has quit IRC19:43
*** lblanchard has quit IRC19:47
*** fultonj has quit IRC19:48
*** ayoung has joined #tripleo19:56
*** jrist has joined #tripleo19:56
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: Add Aodh composable roles
*** fultonj has joined #tripleo20:03
*** rasca has quit IRC20:04
dtrainoranything else i can do to monitor the progress of introspection or see the data coming in, short of 'ironic node-list' and 'ironic node-show ...' ?20:04
*** oshvartz has quit IRC20:05
*** mgarciam has quit IRC20:15
*** oshvartz has joined #tripleo20:20
*** dsneddon_lunch is now known as dsneddon20:21
*** bana_k has quit IRC20:24
*** jayg is now known as jayg|g0n320:25
*** bana_k has joined #tripleo20:31
*** jdob has quit IRC20:34
*** jdob has joined #tripleo20:35
EmilienMI got an overcloud update job working in CI using multinode nodes
EmilienMmy question is now: should I rename the CI job to be -updates instead of -upgrades ? (I'm doing Newton -> Newton now)20:36
EmilienMand keep -upgades for Mitaka to Newton20:36
*** jrist has quit IRC20:38
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: tripleo update job testing - do not merge
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci: test if overcloud update can actually fail
*** abehl has quit IRC20:44
*** alhernan has quit IRC20:46
*** jrist has joined #tripleo20:49
*** dsariel has quit IRC20:50
*** rhallisey has quit IRC21:01
*** tobias_fiberdata has joined #tripleo21:05
*** julim has quit IRC21:18
*** julim has joined #tripleo21:18
*** egafford has quit IRC21:19
*** julim has quit IRC21:23
*** jeckersb is now known as jeckersb_gone21:28
*** yamahata has quit IRC21:30
*** morazi has quit IRC21:33
*** fultonj has quit IRC21:40
*** fpan has joined #tripleo21:41
openstackgerritMerged openstack/tripleo-heat-templates: Remove some properties from overcloud.yaml Controller group
*** ayoung has quit IRC21:43
*** yamahata has joined #tripleo21:49
*** dprince has quit IRC21:59
*** tbonds has quit IRC22:01
*** rasca has joined #tripleo22:02
openstackgerritMerged openstack/puppet-tripleo: Next generation HA architecture work
*** jcoufal_ has quit IRC22:19
*** ayoung has joined #tripleo22:26
*** pradk has quit IRC22:34
*** tobias_fiberdata has quit IRC22:43
openstackgerritDan Sneddon proposed openstack/os-net-config: Enable setting of IPV6_AUTOCONF separately from DHCPV6C
*** dsneddon_ has joined #tripleo22:59
*** panda is now known as panda|zz23:01
*** ayoung has quit IRC23:06
*** bana_k has quit IRC23:09
*** dsneddon_ has quit IRC23:11
*** tzumainn has quit IRC23:32
*** bana_k has joined #tripleo23:39
*** weshay has quit IRC23:56
openstackgerritMerged openstack/tripleo-heat-templates: Next generation HA architecture work

Generated by 2.14.0 by Marius Gedminas - find it at!