14:07:18 #startmeeting tripleo 14:07:18 Meeting started Tue Oct 6 14:07:18 2015 UTC and is due to finish in 60 minutes. The chair is olap. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:07:19 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 14:07:21 The meeting name has been set to 'tripleo' 14:11:13 #stopmeeting tripleo 14:11:16 oops 14:14:36 Ryan Brown proposed openstack/python-tripleoclient: Skip validation on nodes in maintenance mode https://review.openstack.org/223615 14:36:16 Ryan Brown proposed openstack/python-tripleoclient: Move more functions to utils.py https://review.openstack.org/225246 14:36:17 Ryan Brown proposed openstack/python-tripleoclient: Consolidate module constants. https://review.openstack.org/225245 14:36:17 Ryan Brown proposed openstack/python-tripleoclient: Move common functions out into utils.py https://review.openstack.org/225244 15:00:18 Untriaged bugs so far: 15:00:19 https://bugs.launchpad.net/tripleo/+bug/1499298 15:00:19 Launchpad bug 1499298 in tripleo "Installers should use pymysql db driver as of Liberty" [Undecided,New] 15:00:20 https://bugs.launchpad.net/tripleo/+bug/1501378 15:00:22 https://bugs.launchpad.net/tripleo/+bug/1501296 15:00:31 Launchpad bug 1501378 in tripleo "pacemaker neutron resource constraints and cleanup" [Undecided,Triaged] 15:00:33 Launchpad bug 1501296 in tripleo "Stack updates fail due to a new resource type" [Medium,New] 15:02:35 jprovazn: Hey, FYI PATCH support for update preview landed for Liberty RC2 15:02:42 bug #1501207 15:02:42 bug 1501207 in python-heatclient "preview_update_stack inconsistent with update_stack" [High,Fix committed] https://launchpad.net/bugs/1501207 - Assigned to Steven Hardy (shardy) 15:03:27 jprovazn: we'll need to release the heatclient patch then you should be able to use it 15:05:39 shardy, cool! 15:10:16 shardy: hmm i just kicked off at least 5 deployments and neither of them hit the "resource not recognized" issue (i terminated them early once they got to puppet phase, i used to hit the issue before puppet config starts). I'll update my env from 3 weeks old repos to what's in current tripleo.sh and report the issue if i hit it. 15:15:52 * EmilienM 15:17:26 EmilienM: how is Portland? 15:17:37 jistr: Ok, thanks, let me know if you get a reproducer on recent heat and I'll look into it 15:17:42 slagle: I'm in Montreal waiting for the flight 15:17:49 slagle: thanks for asking! 15:18:10 EmilienM: oh. in that case, sounds like a good time to catch up on some Portlandia episodes 15:18:27 slagle: what's that? 15:19:03 EmilienM: you are going to portland without watching portlandia 15:19:05 EmilienM: just google it. and watch the shows if you can. it will prepare you for any trip to Portland 15:19:13 I thought it was required 15:19:28 EmilienM: it is still the 90's there 15:19:46 lol 15:20:03 it's my second time, first time was OpenStack Summit in 2013 15:20:11 but yeah, I'll watch it ok - thanks guys ahah 15:20:16 I really like Portland, but the show is pretty spot on 15:56:19 James Slagle proposed openstack/os-cloud-config: Remove admin role in admin tenant from ceilometer https://review.openstack.org/231465 15:57:59 * mkovacik__ is away: I'm busy 16:18:36 Merged openstack/tripleo-common: Fix CentOS detection https://review.openstack.org/230225 16:40:04 dprince, During deployment, what systemd unit is responsible for running os-net-config? I need to figure out how to run something else first (LLDP agent). 16:41:42 dsneddon: os-collect-config currently runs it 16:41:56 dsneddon: see the os-net-config element in tripleo-image-elements 16:42:19 dprince, That's what I thought, but I wasn't sure if it was os-collect-config or os-apply-config. Thanks 16:55:37 Nisha Agarwal proposed openstack/diskimage-builder: Reorder the script number of 'elements/dkms/post-install.d/99-dkms' https://review.openstack.org/220885 17:13:09 Nisha Agarwal proposed openstack/diskimage-builder: Add a new element hpdsa https://review.openstack.org/220888 17:29:42 James Slagle proposed openstack/os-cloud-config: Remove requirement of specifying hardware stats https://review.openstack.org/231613 17:36:02 Timothy Swanson proposed openstack/tripleo-heat-templates: Cisco Nexus ML2 MD: Need unique key for server level keys in hieradata https://review.openstack.org/231623 18:00:49 Merged openstack/python-tripleoclient: Skip validation on nodes in maintenance mode https://review.openstack.org/223615 18:39:22 John Trowbridge proposed openstack/instack-undercloud: Run db-sync as ironic-inspector user https://review.openstack.org/231656 20:15:41 James Slagle proposed openstack/python-tripleoclient: Install device-mapper-multipath on overcloud-full https://review.openstack.org/231690 20:28:56 dsneddon: thanks, I will see if I can implement that doc into the RDO CI 20:29:27 should we be mentioning RDO-Manager in tripleo docs? 20:29:50 trown, No. Where did I do that? 20:30:27 dsneddon: put a comment on the review 20:30:33 trown, Oh weird, it looks like the |project| got overwritten, probably by an errant rebase. I'll have to fix that. 20:55:01 Dan Sneddon proposed openstack/tripleo-docs: Updates to network documentation to accompany y1 https://review.openstack.org/221908 21:00:17 Untriaged bugs so far: 21:00:18 https://bugs.launchpad.net/tripleo/+bug/1501296 21:00:18 Launchpad bug 1501296 in tripleo "Stack updates fail due to a new resource type" [Medium,New] 21:02:26 dprince, I'm wondering if you think this scenario will work: deploy 3 controllers and 3 computes with 4x10Gb bonds, then modify compute.yaml and stack update with an additional 3 compute nodes that have 2x10Gb bonds. Will TripleO leave the first 3 computes alone and deploy the 3 others with an alternate NIC config? 21:03:19 dprince, I don't have a good handle on stack deploy vs. stack update behavior and how it applies to the network isolation. 21:19:46 dsneddon_errand: Ideally we'd want any updated os-net-config templates to get applied to all nodes 21:20:16 dsneddon_errand: so if update doesn't apply the changes to all nodes I think I'd call it a bug 00:28:37 Augustina Ragwitz proposed openstack/diskimage-builder: Update apt-conf elements README from free text to table formatting https://review.openstack.org/231770 01:45:02 OpenStack Proposal Bot proposed openstack/os-cloud-config: Updated from global requirements https://review.openstack.org/231788 03:00:18 Untriaged bugs so far: 03:00:19 https://bugs.launchpad.net/tripleo/+bug/1501296 03:00:19 Launchpad bug 1501296 in tripleo "Stack updates fail due to a new resource type" [Medium,New] 04:23:09 Ryan Brady proposed openstack/tripleo-common: [WIP]Initial workflow implementation of tripleo-common https://review.openstack.org/227297 04:34:46 Ryan Brady proposed openstack/tripleo-common: [WIP]Initial workflow implementation of tripleo-common https://review.openstack.org/227297 06:16:38 Bharat Kumar Kobagana (BharatK) proposed openstack/tripleo-heat-templates: Add Gluster integration to Manila https://review.openstack.org/230936 07:23:18 Merged openstack/tripleo-heat-templates: Enable Cisco N1KV driver https://review.openstack.org/201398 07:31:08 Dougal Matthews proposed openstack/python-tripleoclient: fix cisco networking rpm name https://review.openstack.org/225165 07:33:45 Merged openstack/python-tripleoclient: Replace generate password for alternative in passlib https://review.openstack.org/228412 07:35:04 Merged openstack/python-tripleoclient: Remove unused cli arguments from overcloud deploy https://review.openstack.org/229815 07:36:26 slagle, hi 08:04:43 Marios Andreou proposed openstack/tripleo-heat-templates: Enable Manila integration https://review.openstack.org/188137 09:00:18 Untriaged bugs so far: 09:00:19 https://bugs.launchpad.net/tripleo/+bug/1501296 09:00:19 Launchpad bug 1501296 in tripleo "Stack updates fail due to a new resource type" [Medium,New] 09:13:47 Yanis Guenane proposed openstack/tripleo-heat-templates: Manage keystone related settings directly in t-h-t manifests https://review.openstack.org/230357 09:13:47 Yanis Guenane proposed openstack/tripleo-heat-templates: Manage keystone related settings directly in t-h-t manifests https://review.openstack.org/230375 09:19:40 d0ugal: Hey, quick question about the post-deploy configuration in tripleoclient 09:20:07 d0ugal: I'd like to add an option to *only* do the post-deploy configuration on an existing, deployed overcloud 09:20:29 d0ugal: use-case is deploying directly via heatclient when debugging, then wanting to e.g create the endpoints after 09:21:30 Or, to re-configure a stack previously deployed via tripleoclient but broken at the end of the deploy, e.g a failure in a third-party post-deploy extraconfig step or whatever 09:22:25 d0ugal: question is what interface makes sense for that, a new overcloud post-deploy, overcloud deploy --nocreate, overcloud deploy --post-deploy-only, or ...? 09:24:21 shardy: post deploy stuff is being moved to puppet 09:24:31 shardy: I think that solves the problem in a better way :) 09:24:40 d0ugal: aha, cool! :) 09:25:02 shardy: I think thrash|g0ne was looking at doing that 09:25:12 d0ugal: I did a heat demo yesterday, and I decided to show creating a tripleo stack directly via heatclient 09:25:29 It actually works really well, except for the missing endpoint creation etc 09:27:03 shardy: Yeah, I can imagine. 09:27:29 shardy: Soon the deploy command will do almost nothing beyond a heat deploy - I wonder if we should eventually drop it 09:27:41 d0ugal, shardy hi 09:28:08 d0ugal: I think it's nice to have, as it's a good place to plug in additional validation checks etc, or wire in looking in known locations for templates 09:28:35 d0ugal: but, for developers, I think the direct-to-heat option would be nice to document too 09:28:40 d0ugal, shardy i am not sure if you guys are tripleo cores? 09:29:08 shardy: true, that makes sense 09:29:43 Nisha: Hi, yes, how can we help? 09:30:30 shardy, great, need reviews on https://review.openstack.org/220885 and https://review.openstack.org/220888 09:30:35 Bharat Kumar Kobagana (BharatK) proposed openstack/tripleo-heat-templates: Add Gluster integration to Manila https://review.openstack.org/230936 09:30:42 the first one has 3 +2 09:30:47 needs +A 09:37:07 Nisha: I've re-approved the first one, but can't really vote on the second one as I've got little experience of ubuntu or the driver it enables 09:39:43 shardy, thanks. but you could review it :) 09:40:19 shardy, the driver just makes the disk visible for the OS 09:43:06 spredzy: gave some feedback on the keystone init patches, going for lunch and then we can discuss on irc the details if you wish :) 09:43:43 jistr, I ll be more than happy 09:43:48 enjoy lunch 09:43:58 thx :) 10:08:45 hi shardy 10:09:11 devvesa: Hi! 10:09:28 Still struggling with image creations :) 10:09:50 I've been looking at the code in overcloud_image: 10:09:56 https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/v1/overcloud_image.py 10:10:05 (which is called by the ./tripleo.sh in tripleo-common= 10:10:22 I see that the overcloud-full is the only image created on the overcloud 10:10:41 I can not see any way to create overcloud-compute and overcloud-controller 10:10:48 Are they deprecated? 10:11:01 devvesa: Yes, that's correct, we use the same overcloud-full image for all roles now 10:11:37 So, what do you recommend me if I want to add a MidoNet image? 10:11:38 devvesa: the defaults in the heat template should probably be updated - we always pass in overcloud-full for all the image parameters via overcloud-deploy 10:12:01 Should I create a overcloud-midonet-full? 10:12:02 devvesa: I think you need to add an element which can be added in to the overcloud-full image 10:12:16 Oh, I see. 10:12:24 And also.. There is a way to remove packages, right? 10:12:42 Because this midonet image should not have the ml2 and openvswitch packages 10:13:54 devvesa: Hmm, I guess we can do that but I'll have to look at exactly how - most elements just specify what to install 10:14:13 devvesa: the other option is to take a pre-built overcloud-full image and use virt-customize to modify it 10:14:23 but that is less flexible than the image based approach 10:14:37 Well, having the packages installed without running the services is not bad at all 10:15:27 And... In what project should I work? I see the overcloud-full belongs to the instack-undercloud project 10:16:01 shardy, the patch doesnt seem to pass the jenkins gate :( 10:16:33 devvesa: https://github.com/openstack/tripleo-puppet-elements/tree/master/elements 10:16:55 ok. this is the one I'm working now. But the fact that overcloud-full was in the other one confused me 10:17:02 Thanks! 10:17:10 devvesa: I would suggest creating a new overcloud-network-midonet element there, or similar 10:17:17 shardy, does recheck helps in Jenkins gate ? 10:17:29 Yanis Guenane proposed openstack/tripleo-heat-templates: Manage keystone related settings directly in t-h-t manifests https://review.openstack.org/230375 10:17:32 devvesa: yeah, overcloud-full is a combination of several elements, some of which come from tripleo-puppet-elements 10:17:43 Ok. I'll name it like you said 10:18:40 devvesa: hmm, there are also elements in instack-undercloud - it could probably go in either tbh 10:18:48 https://github.com/openstack/instack-undercloud/tree/master/elements 10:19:22 Yes. This is what I was asking you. 10:19:30 I wasn't sure which one was the best 10:20:31 devvesa: honestly it probably doesn't matter much at this point, use whatever is most convenient to you 10:20:59 shardy: ok 10:21:01 devvesa: I've been thinking about how we can have "tripleo plugins" recently, so hopefully we can have that discussion and define/document this better 10:22:20 devvesa: the reason I said tripleo-puppet-elements was just because it's presumably going to be pulling in your puppet modules 10:24:26 shardy: yes. A plugin system will be great :) 10:24:54 tripleo-puppet-elements is good enough 10:25:06 according with what you said 11:17:55 shardy, just to be sure 11:18:42 the ServiceNetMap which is passed as json https://github.com/openstack/tripleo-heat-templates/blob/master/overcloud-without-mergepy.yaml#L565 11:19:12 we *can't* override a single item of the json from parameters: when deploying right? we have to pass the full structure 11:22:16 Juan Antonio Osorio Robles proposed openstack/python-tripleoclient: Improve Coverage for tripleoclient/utils.py https://review.openstack.org/228609 11:24:34 Juan Antonio Osorio Robles proposed openstack/python-tripleoclient: Improve Coverage for tripleoclient/utils.py https://review.openstack.org/228609 11:27:24 gfidente: right, you have to pass the full map 11:27:45 shardy, ack, I saw someone trying to do with single param 11:27:51 wanted to make sure 11:28:05 d0ugal: Hey, I noticed that --overcloud-full-dib-extra-args doesn't append elements to overcloud image build, is that a known issue? 11:28:22 it seems to stomp on all the internally defined ones, which was not what I expected 11:28:27 devvesa: ^^ 11:29:54 shardy: It isn't known by me :) 11:30:10 shardy: but I am probably least familiar with the image building. 11:30:20 thrash: ^ 11:30:39 d0ugal: Ok, thanks, I'll raise a bug and we can work out the best way to handle it 11:31:01 shardy: yeah.. We really need to move to the build-images code in tripleo-incubator 11:31:12 thrash: -common* 11:31:13 use-case is plugging in an additional element e.g for third-party integration, which it turns out isn't all that easy atm 11:31:27 Juan Antonio Osorio Robles proposed openstack/python-tripleoclient: Improve Coverage for tripleoclient/utils.py https://review.openstack.org/228609 11:31:29 d0ugal: I'm saying that the code is in incubator. :) 11:31:39 thrash: ohhh 11:31:41 it needs to move *into* common 11:31:43 :) 11:31:54 thrash: ah, cool - anyone looking at that atm? 11:32:31 devvesa needs this for building an additional element into overcloud-full - although we've got a workaround which is to specify all-the-elements in --overcloud-full-dib-extra-args 11:32:51 shardy: I was going to look at it 11:33:34 thrash: Ok, great, shall I raise a bug so we can track it? 11:33:43 Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Enable TLS deployment for the Overcloud https://review.openstack.org/231930 11:33:44 shardy: that would be great 11:39:34 thrash, devvesa: https://bugs.launchpad.net/tripleo/+bug/1503669 11:39:34 Launchpad bug 1503669 in tripleo "Including an additional element is inconvenient" [Undecided,New] 11:39:42 shardy: thanks. 11:39:49 did anybody figure out why the "InterpreterNotFound: python3.4" was happening every once in a while in the py34 checks of the python-tripleoclient? 11:40:32 shardy: https://review.openstack.org/229778 11:42:10 pino|work: awesome! Trying now, thanks! :) 11:42:10 shardy: although we might need something to append more paths for ELEMENTS_PATH 11:42:47 pino|work: Yeah, we probably need that too, but atm it's OK as I'm trying to add a new element to an existing elements repo 11:42:49 (since dib is sooooo flexible you need *%#! envvars to add new paths for elements) 11:43:35 pino|work: the other option is to have tripleoclient log the entire dib command, and/or have a --dry-run mode which dumps it out 11:43:43 then folks can wire in whatever they need 11:43:50 inflexible 11:44:05 having a simple way to just add an extra element without doing that seems like a good idea tho 11:44:56 I was actually wondering why we're using subprocess instead of oslo processutils, as I think we'd get that logging for free then 11:46:54 shardy: and btw, using virt-dib (see dependent patch) would allow easily to pass new element paths as arguments (eg '-p /foo myelementinfoo' as --builder-extra-args) 11:49:05 pino|work: sounds great. The next thing we need is much better docs around all of this in tripleo-docs ;) 12:01:59 Dougal Matthews proposed openstack/tripleo-common: Collect the common logic used when dealing with Heat https://review.openstack.org/228991 12:02:09 Dougal Matthews proposed openstack/tripleo-common: Collect the common logic used when dealing with Heat https://review.openstack.org/228991 12:11:33 Dougal Matthews proposed openstack/tripleo-common: Remove empty test file https://review.openstack.org/231949 12:24:14 jistr: I used your ironic-inspector workaround a couple of days ago, and it worked... 12:24:24 jistr: however, I'm getting 500's again. Are you seeing that? 12:24:38 shardy: thanks! 12:24:46 or anyone else for that matter seeing 500's on introspection? 12:25:00 with workaround? 12:26:16 I see that this bug has already been assigned. So quick! Is Brad Crochet here? 12:26:41 devvesa: yes 12:26:44 that's me. 12:27:03 thrash: Hi! Pleased to meet you :) 12:27:22 devvesa: you as well! 12:27:28 I was about to fix this bug this afternoon because I need a fix for my development 12:27:39 thrash: hi, yes, wrong permissions, one sec pls 12:27:39 Do you mind if I assign to myself? Or you are already working on it? 12:27:47 devvesa: feel free to 12:27:55 thrash: many thanks! 12:28:29 done 12:28:47 thrash: https://github.com/jistr/inlunch/blob/cce3a07a10a8c025319bff7a4fd2f48922bb7f0a/answers.yml.example#L111 12:29:03 jistr: Awesome. :) 12:30:01 devvesa: see https://review.openstack.org/229778 12:30:26 also, see the rest of the talk earlier between shardy and me 12:32:09 marios: if you used yesterday's delorean pin update, you'll need that ^ too. other than that, all worked out fine, now i'm in the middle of deploying overcloud 12:32:21 jistr: reading 12:34:38 jistr: yes indeed, i skipped introspection (vm env). i am having issues with ha/pacemaker deploy (simple deploy OK). I get like " Can't connect to local MySQL server through socket" and the puppet step 4 fails. am trying https://review.openstack.org/#/c/230488/ don't know if root yet 12:35:24 pino|work: gotcha 12:35:58 marios: hmm that's not going to be the cause probably. For the account removal race the error message something like "SQL command 'DELETE .......' failed." 12:36:27 jistr: ack thanks for info 12:40:06 pino|work: I don't see the difference between --overcloud-full-dib-extra-args and --builder-extra-args 12:40:40 Should I go on with the bug fixing or the --builder-extra-args already fixes this? 12:41:01 devvesa: --overcloud-full-dib-extra-args has a default value associated, so if you pass it you lose the default value 12:41:40 so either you specify the default on your own (and you have to know it exactly), or you are not really "adding new parameters" but also removing existing ones 12:41:46 That means bug: https://bugs.launchpad.net/tripleo/+bug/1503669 12:41:46 Launchpad bug 1503669 in tripleo "Including an additional element is inconvenient" [Undecided,New] - Assigned to Jaume Devesa (devvesa) 12:41:50 is invalid? 12:41:54 no 12:42:09 i mean that my patch could be a way to fix that 12:42:10 devvesa: I think it just means pino|work's patch fixes it 12:42:22 I'm testing now, making notes on how to build the review as a local delorean package 12:43:22 Ok. Then --overcloud-full-dib-extra-args will be reserved for current behavior and --builder-extra-args will add elements 12:43:58 devvesa: Yeah, which I think is OK provided we document the difference carefully 12:44:03 all of the existing parameters of `overcloud image` have some associated default non-empty value 12:44:09 Fair enough 12:44:25 imho that's problematic, but i was not the one doing/approving them... 12:45:22 pino|work: I think we should output a warning whenever an internal default is stomped on by the CLI 12:45:35 but yeah, that's the problem with having all the hard-coded stuff in the client 12:45:57 shardy: but still, what's the use case to allow that behaviour? 12:47:47 Lennart Regebro proposed openstack/python-tripleoclient: Update requirements https://review.openstack.org/231961 12:47:50 pino|work: any time you hard code anything inside a client-facing application, folks will sometimes want to override it, especially developers 12:48:27 hello, my bulk introspection is falling to dracut emergency with error "Multiple matching nodes found for attributes...", mentioning bmc_address and mac and a pair of nodes related 12:48:27 shardy: still is not a valid reason to allow break anything there 12:48:38 We had problems with the client tests failing on the requirements update before the rename. 12:48:44 any hints how to troubleshoot it? I can't find any duplicates 12:49:21 pino|work: I'm not saying to break anything, I'm saying give visibility of the hard-coded values, if you're going to expose CLI args to override them 12:49:40 otherwise it's a terrible user experience as you are forced to always look at the code 12:49:55 shardy: tbh, overriding default values should be done case-by-case 12:50:17 As a result, the requirements haven't been synced for ages (april in fact), 12:50:37 instead of "let's override the whole list of elements and element paths", rather "i want to use this set of stuff instead of the default one" 12:50:41 It would be very nice if we could get the above manual sync approved quickly. 12:51:04 pino|work: IMO that's wrong, and it's the whole problem with the various hard-coded values in tripleoclient, it ends up being fragile if you expose everything case-by-case and it's highly coupled to the implementation of e.g the templates, elements or whatever 12:51:09 shardy: imho the current way is... way too lax 12:51:21 it'd be far better to consume a (default) config file with all of that data IMHO 12:52:08 shardy: currently it is tie-ing directly to the implementation, which is bad 12:52:21 pino|work: Yeah, I think we're agreeing ;) 12:52:41 if i would have a way to print all the defaults... why would then i need to use the cli? 12:53:09 i could just run myself: disk-image-create `get default elements for overcloud image` etc 12:53:20 pino|work: because it's convenient and provides a more consistent user experience 12:53:37 I actually think we should also document how to use the underlying tools directly as well though 12:53:57 for example, you can easily create the overcloud directly via heatclient, all you need is a parameters file 12:54:03 yeah, but with the current parameter the consistency is not really done 12:54:11 *parameters 12:56:13 pino|work: we've been down this path before - look at the original instack shell scripts, they used the underlying tools directly and they were really quite complex 12:56:34 shardy: i'm not saying that a cli is not needed 12:56:40 the tripleoclient is better than that, although I completely agree some aspects of it need work 12:56:59 shardy: i'm saying that a cli that basically exposes you directly to the implementation does not make it worth use 12:57:38 pino|work: It's worth it because, by default you don't have to care about the implementation 12:58:04 shardy: yeah, but doing a very simple customization is prone to break everything 12:58:09 that isn't a good cli 12:59:02 pino|work: I am confused, your patches enable simple customization, I like it, it's a better CLI than what we have \o/ 12:59:35 shardy: my patch is not yet in, which means that using any of all the current arguments is prone to break 13:00:04 ... which is why I'm testing your patch so I can +2 it 13:02:30 cool, thanks 13:04:34 i have the heat issue with unrecognized resource reproduced 13:04:54 the deployment actually "succeeded", it ran all the puppet steps successfully 13:05:29 but VipDeployment is stuck in CREATE_IN_PROGRESS even though it finished, just the report back to heat didn't work 13:05:52 jistr: interesting - I have only ever seen this problem with VipDeployment 13:05:58 * shardy wonders why.. 13:07:44 important things break 13:08:26 Brad P. Crochet proposed openstack/python-tripleoclient: Remove parameter defaults https://review.openstack.org/226790 13:10:48 Bear with me because I am a newcomer and I can be easily wrong, but I feel like elements involved for TripleO can be better organized 13:11:16 I created a developer environment to build images (all the projects cloned, not installed from packages) 13:11:39 and this is my ELEMENTS_PATH env var for build the 'overcloud-full' image 13:11:43 export ELEMENTS_PATH=~/idev/tripleo-puppet-elements/elements:~/idev/diskimage-builder/elements:~/idev/instack-undercloud/elements:~/idev/tripleo-image-elements/elements:~/idev/heat-templates/hot/software-config/elements 13:12:19 devvesa: it's true, there is scope for consolidation 13:14:00 jistr: do you have the o-a-c and heat-api logs? 13:14:37 it is that heat can't find the resource, vs the curl from S99-refresh-completed can't signal heat? 13:14:38 shardy: walking throuh heat-api-cfn logs now, aside from the 400 response i don't see anything very useful 13:14:45 i can gather whatever info we need 13:15:01 i think heat can't find the resource 13:15:15 i'll paste o-c-c output 13:15:37 jistr: Ok, thanks, also please let me know what heat version you're working from 13:16:04 Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Enable TLS deployment for the Overcloud https://review.openstack.org/231930 13:20:12 hi, my bulk introspection is falling to dracut emergency with error "Multiple matching nodes found for attributes...", mentioning bmc_address and mac and a pair of nodes related 13:20:33 shardy: output of journalctl -u os-collect-config | grep -C20 'out-of-range value was supplied' 13:20:38 http://paste.fedoraproject.org/275867/44223990/raw/ 13:21:21 it tried to report the completion multiple times after being unsuccessful it seems, so the log is quite long 13:21:47 jistr: can you see the overcloud-VipDeployment-fjw47zvc4ryu stack in heat stack-list --show-nested? 13:22:07 if so then do a heat resource-list on it & check the status of the "0" resource 13:23:09 yes http://paste.fedoraproject.org/275868/44224172 13:24:03 shardy: hmm the resource-list is interesting http://paste.fedoraproject.org/275870/44422422 13:24:15 Brad P. Crochet proposed openstack/python-tripleoclient: Turn the orchestration client reference into a property https://review.openstack.org/224752 13:24:36 jistr: Hmm, that is interesting!! 13:25:09 jistr: can you see any errors in the engine log after it started creating the VipDeployment stack? 13:25:58 jistr: I think you've confirmed this is a heat bug, so if we can collect as much data as possible, I'll then attempt to reproduce in a more minimal way 13:33:26 devvesa: I tested the patch from pino|work and it looks good 13:33:28 http://paste.openstack.org/show/475605/ 13:34:00 That shows how to pull a couple of pending tripleo.sh packages and build the tripleoclient patches into a local version via delorean 13:34:30 s/tripleo.sh packages/tripleo.sh patches/ 13:34:31 I was about to test it right now 13:34:39 shadower: i don't see any errors there, the log is quite large though and i don't know what to look for exactly. But there's nothing marked 'ERROR' 13:34:43 shadower: sorry 13:35:09 shardy: here's a slice which fpaste still let me upload http://paste.fedoraproject.org/275875/42248161 13:35:34 that's grep -A100 VipDeployment-fjw /var/log/heat/heat-engine.log | head -n1000 13:37:47 shardy: i'll create a launchpad bug 13:39:28 jistr: Ok, thanks - that log may be enough actually - please add the resource-list output for the VipDeployment stack, and also the stack-show too please 13:39:37 ack will do 13:39:43 resource-list -n10 13:39:46 the log clearly shows us validating three resource, but then there's only one mentioned in the backup stack 13:39:49 Brad P. Crochet proposed openstack/python-tripleoclient: Turn the baremetal client reference into a property https://review.openstack.org/231998 13:40:20 jistr: No, I only need the resource-list and stack-show output for the overcloud-VipDeployment-fjw47zvc4ryu stack please 13:40:33 as I'm pretty sure that's where it's going wrong 13:40:39 ok 14:07:04 shardy: here is the bug https://bugs.launchpad.net/heat/+bug/1503694 -- do you think we need to collect anything more before i delete that env? 14:07:04 Launchpad bug 1503694 in heat "A bad or out-of-range value was supplied:The Resource (0) could not be found in Stack" [Undecided,New] 14:09:35 shardy_: not sure if you saw my message, pasting again 14:09:48 [16:07:03] shardy: here is the bug https://bugs.launchpad.net/heat/+bug/1503694 -- do you think we need to collect anything more before i delete that env? 14:09:48 Launchpad bug 1503694 in heat "A bad or out-of-range value was supplied:The Resource (0) could not be found in Stack" [Undecided,New] 14:33:20 Merged openstack/tripleo-common: Minimize walking through stack when removing a breakpoint https://review.openstack.org/219642 14:34:17 Merged openstack/tripleo-common: Catch and log all exceptions when removing a breakpoint https://review.openstack.org/222076 14:37:18 Ben Nemec proposed openstack/instack-undercloud: Add undercloud config wizard UI https://review.openstack.org/231212 14:41:31 James Slagle proposed openstack/python-tripleoclient: Handle images not being found https://review.openstack.org/232040 14:41:31 James Slagle proposed openstack/python-tripleoclient: Make hypervisor stat check a validation https://review.openstack.org/232041 14:42:42 Giulio Fidente proposed openstack/tripleo-heat-templates: Parametrize RabbitMQ FD limit https://review.openstack.org/201796 14:44:33 slagle, actually I'm updating a little the FIXME line as well, sec 14:52:33 Giulio Fidente proposed openstack/tripleo-heat-templates: Parametrize RabbitMQ FD limit https://review.openstack.org/201796 14:58:46 gfidente: thx 14:59:20 slagle, oh well, that's the purpose of the reviews! ;) 15:00:19 Untriaged bugs so far: 15:00:20 https://bugs.launchpad.net/tripleo/+bug/1503669 15:00:20 Launchpad bug 1503669 in tripleo "Including an additional element is inconvenient" [Undecided,New] - Assigned to Jaume Devesa (devvesa) 15:07:28 Yanis Guenane proposed openstack/tripleo-heat-templates: Allow a user to specify a comma separated list of ntp servers https://review.openstack.org/232051 15:13:44 Merged openstack/python-tuskarclient: Updated from global requirements https://review.openstack.org/194031 15:18:24 James Slagle proposed openstack/python-tripleoclient: Make hypervisor stat check a validation https://review.openstack.org/232041 15:27:26 we should set the topic back 15:39:00 jdob, ping 15:39:05 BharatK: pong 15:39:56 jdob, Good morning! Please review the patch https://review.openstack.org/#/c/230936/ (Gluster-Manila integration) which you have sometime. 15:40:28 s/which/when 15:40:45 will do 15:41:14 jdob, Thanks. 16:06:30 Yanis Guenane proposed openstack/tripleo-heat-templates: Allow a user to specify a comma separated list of ntp servers https://review.openstack.org/232051 16:12:41 Yanis Guenane proposed openstack/python-tripleoclient: Amend --ntp-server help message to add support for multiple servers https://review.openstack.org/232075 16:18:58 Steven Hardy proposed openstack/tripleo-heat-templates: Update Flavor defaults and descriptions https://review.openstack.org/232082 16:18:58 Steven Hardy proposed openstack/tripleo-heat-templates: Switch Image parameters to default to openstack-full https://review.openstack.org/232083 16:18:59 Steven Hardy proposed openstack/tripleo-heat-templates: Update overcloud template description and Count constraints https://review.openstack.org/232084 16:18:59 Steven Hardy proposed openstack/tripleo-heat-templates: Reorganize parameters per service and role https://review.openstack.org/232085 16:19:43 dprince: ^^ FYI that's a precursor to adding parameter_groups 16:21:17 shardy: ack, will have a look 16:21:45 shardy: I was just thinking about this... would it be appropriate to have a Deprecated params group? 16:21:52 dprince: thanks 16:21:57 shardy: or is there some other Heat mechanism we can use for this 16:22:18 dprince: actually I was thinking about the same thing - we don't have a good mechanism for that atm 16:22:43 I'm not sure even parameter_groups helps, as a parameter can only exist in one group (according to the docs, I've not yet checked the code) 16:22:50 shardy: okay, yeah... a deprecated params group could work perhaps then, and there are already several (network related) params we could stash in there 16:23:14 shardy: even as an organizational mechanism it might be somewhat useful 16:23:44 dprince: I was thinking something similar to what oslo does would be good, where you can make a thing deprecated, and even allow the old name to work with a warning when renaming (we've got quite a new naming inconsistencies atm) 16:23:56 but heat doesn't have a good way to make such warnings visible to the user 16:24:26 that's something we've wanted to solve for a while, perhaps we can add it to the Mitaka wish-list and I'll take a look into it 16:25:06 shardy: okay, sounds good 16:25:10 the problem is the create is an async API call, so it just returns, then all we have are events as a means to warn the users 16:47:02 Steven Hardy proposed openstack/tripleo-heat-templates: Allow ctlplane network or ID, and default to "ctlplane" https://review.openstack.org/232101 16:50:27 Any one knows if Brad P. Crochet is in this chan and what's his nick? 16:50:37 spredzy: meet thrash 16:50:48 pino|work, heh thx 16:50:52 thrash: meet spredzy, who is looking for you 16:51:01 * pino|work goes back to his cave 16:52:07 thrash, can I ask you to answer the comment here https://review.openstack.org/#/c/230949/ as I don't have the answer please 16:57:07 James Slagle proposed openstack/python-tripleoclient: Make hypervisor stat check a validation https://review.openstack.org/232041 16:57:07 James Slagle proposed openstack/python-tripleoclient: Make --validation-errors-fatal default to True https://review.openstack.org/232105 16:57:08 James Slagle proposed openstack/python-tripleoclient: Make node count check a validation https://review.openstack.org/232106 16:59:11 Yanis Guenane proposed openstack/python-tripleoclient: Amend --ntp-server help message to add support for multiple servers https://review.openstack.org/232075 17:20:32 James Slagle proposed openstack/python-tripleoclient: Make hypervisor stat check a validation https://review.openstack.org/232041 17:20:33 James Slagle proposed openstack/python-tripleoclient: Make --validation-errors-fatal default to True https://review.openstack.org/232105 17:20:34 James Slagle proposed openstack/python-tripleoclient: Make node count check a validation https://review.openstack.org/232106 17:23:25 spredzy: hey 17:24:54 spredzy: honestly, it was mainly for parity with existing code. 17:25:30 spredzy: But you don't want the whole auth thing. 17:25:56 I was going to just use keystone_endpoint and keystone_service directly 17:26:33 it might be better to just call it something like horizon::keystone::endpoint instead of horizon::keystone::auth 17:26:50 thrash, the auth thing is a common patttern we apply in the puppet modules so they all look alike. It does create the service and endpoint 17:27:00 Since configure_user is set to false it won't create a user 17:27:02 spredzy: ok. 17:27:09 that makes sense. 17:27:20 I think EmilienM wanted to know what is the actual use cases for this endpoint 17:27:39 spredzy: for reference: https://github.com/openstack/os-cloud-config/blob/master/os_cloud_config/keystone.py#L100-L106 17:27:51 spredzy: tbh, I have no idea. :) 17:27:57 Steven Hardy proposed openstack/tripleo-heat-templates: Update NeutronDhcpAgentsPerNetwork to align with ControllerCount https://review.openstack.org/232121 17:27:58 Steven Hardy proposed openstack/tripleo-heat-templates: Align NeutronAllowL3AgentFailover with NeutronL3HA https://review.openstack.org/232122 17:27:58 Steven Hardy proposed openstack/tripleo-heat-templates: Align remaining parameter defaults with tripleoclient https://review.openstack.org/232123 17:28:19 shardy, hello 17:28:21 I suppose it would mainly be used for discovery given a keystone endpoint 17:28:34 Nisha: hi 17:29:10 thrash, ack. Would you have a strong -1 if its not part of it ? ie. if we can't find why it is actually there/used for ? 17:29:23 spredzy: nope. 17:29:35 shardy, morning you had approved https://review.openstack.org/#/c/220885/ but that time https://bootstrap.pypa.io/get-pip.py ws not accessible which caused Jenkins gate to fail for element "ironic-agent" 17:30:00 now this link is accessible but recheck doesnt make Jenkins gate to rerun 17:30:45 shardy, is there any thing thru which your +A couldbe executed again 17:31:16 Nisha: I think you need "reverify" after the +A is added, but I've just re-approved, lets see if that helps 17:31:41 shardy, thanks 17:32:07 lets see else i will resubmit the patches :( 17:34:58 thrash, ack. Thank you 17:56:58 Dan Prince proposed openstack/tripleo-heat-templates: Update README.rst https://review.openstack.org/230675 18:11:40 Lokesh Jain proposed openstack/tripleo-heat-templates: Changes for configuring Nuage https://review.openstack.org/230116 18:37:45 Merged openstack/diskimage-builder: Reorder the script number of 'elements/dkms/post-install.d/99-dkms' https://review.openstack.org/220885 18:45:36 John Trowbridge proposed openstack/tripleo-docs: Clean up the basic install instructions https://review.openstack.org/232147 19:08:23 bnemec: is there a way to run the undercloud-config-wizard from the tox venv? 19:08:32 i can't seem to pip install PyQt4 19:09:53 slagle: Yeah, apparently PyQt4 is not pip-friendly. I did what is suggested here: http://stackoverflow.com/questions/1961997/is-it-possible-to-add-pyqt4-pyside-packages-on-a-virtualenv-sandbox 19:10:21 slagle: Note that pyqt is installed in /usr/lib64/python2.7 on fedora, not /usr/lib like most. 19:10:43 You also need the sip* files in that site-packages directory as well. 19:11:43 k, will try 19:11:54 Hm; where are Keystone endpoints being created now? After rebasing against master, my sahara (and trove) endpoints seem to have disappeared. Are we already in the post os-cloud-config world, and handling this in puppet within tripleo-heat-templates? (This would be great if true, though it will require jiggering on my end.) 19:11:57 i had looked at many stackoverflow links already, but not yet that one :) 19:17:33 egafford: endpoints are still created via os-cloud-config. so it must be something else 19:17:41 bnemec: works, thx 19:17:46 slagle: Ack; interesting. 19:19:04 slagle: bnemec, wdyt the chances are of getting the network iso docs and tempest docs merged by Mon next week? I am trying to implement both of those in RDO CI now, but there is an RDO test day Mon. 19:21:50 trown: probably high, if we review them :) 19:22:13 slagle: cool, I will review as I try to follow them 19:23:09 trown: https://review.openstack.org/221908 and https://review.openstack.org/221664 ? 19:23:13 i'll review 19:24:13 yep those are the 2 19:24:45 trown: do you think it makes sense to publish a tripleo-docs somewhere, that is RDO specific? 19:24:55 could add the rdo-manager logo back 19:25:08 and replace what is in repositories.txt, with how to use rdo liberty specifically 19:25:37 ya I was thinking that would make sense 19:26:10 not sure where they should go though 19:26:35 could also package tripleo-docs into an rpm and maintain the (hopefully small) patches to brand it as rdo-manager 19:26:58 ya, I would definitely hope there were not many patches... 19:27:24 yea, 99% of the content should be the same 19:27:44 right now it would just be repositories.txt (which I have totally forgot about when pointing RDO folks to tripleo) and branding... 19:28:33 hmm... that would actually be really good to have before the test day 19:28:58 right now people are installing some Mitaka Liberty frankencloud 19:29:53 slagle: that is true for tripleoci too actually... the delorean-deps repo is liberty versions 19:30:01 * bnemec still wants patches to add back in the internal repo details 19:30:29 RHEL testing is so much easier when you don't have to register with sub-man. :-) 19:30:47 trown: that's the only delorean-deps there is though isn't there? 19:31:17 slagle: ya, I do not have a better solution... just thinking that will break at some point during Mitaka 19:31:50 it is kind of part and parcel with using packages to validate trunk 19:49:46 slagle: I think this BZ actually highlights the need to fork RDO docs https://bugzilla.redhat.com/show_bug.cgi?id=1269610 19:49:46 bugzilla.redhat.com bug 1269610 in rdo-manager "Overcloud deployment fails - openvswitch agent is not running and nova instances end up in error state" [High,New] - Assigned to calfonso 19:50:19 following the tripleo docs gets you mitaka packages, and then RDO gets bugs filed against mitaka 19:50:31 neutron-8 is mitaka 20:13:56 hi, I'm having troubles running bulk introspection - ending in dracut emergency 20:14:12 seeing discoverd error "Multiple matching nodes found for attributes" ... 20:14:39 any experience with this error, or hints for troubleshooting? 20:42:28 Hi all. generate_overcloud_passwords doesn't generate new passwords in the case that tripleo-overcloud-passwords already exists and new password keys exist in the set (after an upgrade of tripleoclient.) I'm imagining this is a bug; it's easy to fix. Just confirming that it's not intentional for some reason. 20:59:14 James Slagle proposed openstack/python-tripleoclient: Add --skip-validation-checks cli arg https://review.openstack.org/232209 21:00:19 Untriaged bugs so far: 21:00:20 https://bugs.launchpad.net/tripleo/+bug/1503669 21:00:20 Launchpad bug 1503669 in tripleo "Including an additional element is inconvenient" [Undecided,New] - Assigned to Jaume Devesa (devvesa) 21:02:15 Augustina Ragwitz proposed openstack/diskimage-builder: Update apt-preferences element README from free text to table formatting https://review.openstack.org/232211 21:20:33 egafford: Ugh, no. We fixed that annoying behavior once, but apparently it came back in the move to Python. 21:20:38 Definitely a bug. 21:24:29 bnemec: Ack; cool. 21:31:03 bnemec: This is really odd; even after deleting the tripleo-overcloud-passwords file, I'm getting a file w/o Sahara and Trove passwords on running overcloud deploy. When I do "from os_cloud_config import utils; utils.generate_overcloud_passwords()", the passwords appear; when I run the deploy, no. Is tripleo using a secret venv now where the old, unmodified version of tripleoclient and/or os_cloud_config is hiding from me, or using an os_c 21:31:07 Deep weirdness. 21:44:07 hey.. can somebody please take a look at tripleo-heat-templates code reviews: https://review.openstack.org/#/c/229649/ and https://review.openstack.org/#/c/230116/ 21:49:54 Ryan Hallisey proposed openstack/tripleo-heat-templates: Docker compute role configured via Puppet https://review.openstack.org/209505 21:50:04 ^ works :) 21:50:41 was missing a dir for nova compute 21:52:29 rhallisey: could that be the cause of the weird thin-pool issue? 21:53:25 i'll verify once my deploy gets up again... 21:53:50 jpeeler, I don't think so 21:54:08 jpeeler, this fixed an issues that I was seeing when creating a vm 21:54:27 nova compute was looking for /var/lib/nova/instances 21:55:05 I'm still monitoring my setup.. I'll see if I hit a crash 21:55:13 jpeeler, I'll let it run for a while 21:55:15 well i found an atomicos volume group on the instack vm 21:55:24 that makes no sense 21:56:23 jpeeler, I'm curious to see if I crash again.. I haven't yet, but I may not have waited long enough 22:01:24 bnemec: Found the issue; somehow rdomanager had wormed its way into my instack install unbidden. 22:01:32 Merged openstack/tripleo-heat-templates: Drop examples https://review.openstack.org/230645 22:03:04 Merged openstack/tripleo-heat-templates: Drop stale base, nagios, debian-mirror templates https://review.openstack.org/230671 22:03:11 Merged openstack/tripleo-heat-templates: Move os-apply-config templates into a directory https://review.openstack.org/230672 00:09:51 OpenStack Proposal Bot proposed openstack/tripleo-common: Updated from global requirements https://review.openstack.org/232282 03:00:18 Untriaged bugs so far: 03:00:19 https://bugs.launchpad.net/tripleo/+bug/1503669 03:00:19 Launchpad bug 1503669 in tripleo "Including an additional element is inconvenient" [Undecided,New] - Assigned to Jaume Devesa (devvesa) 05:03:26 Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Enable TLS deployment for the Overcloud https://review.openstack.org/231930 05:18:36 Tzu-Mainn Chen proposed openstack/tripleo-specs: Spec for TripleO overcloud deployment REST API https://review.openstack.org/230432 07:06:02 xiexs proposed openstack/os-cloud-config: Add pxe_irmc driver support into os-cloud-config https://review.openstack.org/188717 07:23:23 Jan Provaznik proposed openstack/tripleo-docs: Add a doc note about updating changed templates https://review.openstack.org/229373 07:44:08 Marios Andreou proposed openstack/tripleo-heat-templates: Enable Manila integration https://review.openstack.org/188137 07:50:29 What's with all the ERROR: InterpreterNotFound: python3.4? 07:51:48 d0ugal: Haven't figured it out. That error happens quite sporadically 07:52:15 It's an odd one. 07:53:16 waiting for an hour or so and then doing a recheck usually works. But it's pretty annoying 07:53:57 jaosorior: heh, I keep doing rechecks too and felt guilty for it :) 07:54:08 (I should probably be hunting for the root of the problem) 09:00:18 Untriaged bugs so far: 09:00:20 https://bugs.launchpad.net/tripleo/+bug/1503669 09:00:20 Launchpad bug 1503669 in tripleo "Including an additional element is inconvenient" [Undecided,New] - Assigned to Jaume Devesa (devvesa) 09:03:16 Yanis Guenane proposed openstack/tripleo-heat-templates: Allow a user to specify a comma separated list of ntp servers https://review.openstack.org/232051 09:03:50 Merged openstack/tripleo-heat-templates: puppet: allow to configure any ceilometer param with Hiera https://review.openstack.org/230531 09:07:26 Yanis Guenane proposed openstack/python-tripleoclient: Amend --ntp-server help message to add support for multiple servers https://review.openstack.org/232075 09:07:39 Jan Provaznik proposed openstack/tripleo-specs: Spec for running overcloud nodes reboot https://review.openstack.org/224481 09:22:54 pino|work: I've -1'ed your yesterdays' patch with a humble request 09:23:09 devvesa: no, i won't add more magic envvars 09:23:25 oh 09:23:31 they are bad, and can change behaviour without you noticing that 09:24:10 parameters make invocations explicit, and you can easily spot the differences between invocations 09:24:35 ok. you are more experiencied than me on these executions, I'll remove the -1 09:25:11 I thought you missed it 09:54:25 Derek Higgins proposed openstack/diskimage-builder: Avoid transcending /proc with find https://review.openstack.org/230448 10:24:04 devvesa: Hey, context is we used to have a bunch of scripts (tripleo-incubator), where env vars were used heavily and it became quite hard to manage 10:24:34 lately we've been trying to move more to a parameter driven CLI approach, via tripleoclient, but there are still some env vars in there as you've discovered 10:26:29 It would be more comfortable with me, since I don't need to modify tripleo.sh and I thought it was a common practice, but yeah, fair enough :) 10:29:41 Bharat Kumar Kobagana (BharatK) proposed openstack/tripleo-heat-templates: Add Gluster integration to Manila https://review.openstack.org/230936 10:30:18 Yanis Guenane proposed openstack/python-tripleoclient: Amend --ntp-server help message to add support for multiple servers https://review.openstack.org/232075 10:55:35 shardy, question: I am trying to allow a user to specify multiple --ntp-server (https://review.openstack.org/#/c/232075/) in python-tripleoclient. For some reason no matter if I pass a single or several values it seems heat doesn't retrieved the value specified in there. Would you know why ? 10:56:14 I always end up with ntp::servers: [] but if I hardcode a value in overcloud-without-mergepy I do get the expected value for ntp::servers 11:02:17 spredzy 11:02:59 added a response in your CR 11:03:22 jaosorior, thx sorry was already thinking about the amend :p 11:03:55 Yanis Guenane proposed openstack/python-tripleoclient: Amend --ntp-server help message to add support for multiple servers https://review.openstack.org/232075 11:03:55 jaosorior, done 11:04:14 jaosorior, would you have any idea of the why of what I stated above ? 11:05:14 spredzy: Not from the top of my head 11:05:16 spredzy: I'm not sure tbh - I'd run tripleoclient with --debug then you can at least prove it's the client that's at fault 11:05:31 spredzy: have you considered that the template change is backwards incompatible? 11:05:40 if we leave the parameter name alone it wouldn't be 11:06:58 spredzy: on line 264 https://review.openstack.org/#/c/232075/4/tripleoclient/v1/overcloud_deploy.py there's a method which creates the parameter env file, you could add a debug print to have the whole file printed on stdout too, to see what went wrong 11:06:58 spredzy: the other option would be to document --ntp-server as taking a comma delimited string 11:07:21 I don't know why action=append isn't working tbh 11:07:35 spredzy: and i think that what jaosorior could be the actual cause of the trouble 11:07:46 *what jaosorior wrote 11:07:48 jistr, shadower it is actually working if I print parameters it is in there 11:07:59 but it doesn't end up in the hiera down the line 11:08:14 spredzy: can you paste how the line in the environment file looks? 11:08:16 jistr, yep typo on the patchset but I do have append on the test platform 11:08:24 jistr, sure 11:10:10 jistr, http://paste.openstack.org/show/475711/ 11:10:59 i think it should be 'NtpServers': '0.centos.pool.ntp.org,1.toto.com' 11:11:02 rather than 'NtpServers': ['0.centos.pool.ntp.org', '1.toto.com'] 11:11:57 jistr: Doesn't heat accept both representations? 11:12:41 jistr: At least that's how it appears in the examples http://docs.openstack.org/developer/heat/template_guide/hot_spec.html#parameters-section 11:12:52 shardy, I take the point for backward incompatiblity will switch back to NtpServer in next review 11:13:45 jaosorior, spredzy: hmm if the array syntax is valid too, then i don't know 11:15:30 spredzy: if you do `heat stack-show overcloud | grep NtpServers`, do you see the correct value? 11:16:47 Derek Higgins proposed openstack/diskimage-builder: Avoid transcending /proc with find https://review.openstack.org/230448 11:16:51 jistr, erf... I in fact do 11:17:06 hmm :/ 11:17:27 jistr, no wait... wrong pltfm 11:19:51 jistr, it is empty 11:20:01 "NtpServers": "" 11:21:45 spredzy: the paste you posted before is a print of a python data structure, right? (I don't see `parameters:` there) 11:22:23 jistr, yes its the print of the parameters variable 11:22:29 in python-tripleoclient 11:22:32 spredzy: if you can debug-print the actual env file as it gets written to tmp, it might uncover something 11:26:12 jistr, how do I do that ? 11:28:07 spredzy: i was thinking here https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/v1/overcloud_deploy.py#L268 11:28:09 print(json.dumps(parameter_defaults)) 11:28:34 but reading the code now, it seems pretty straightforward 11:29:44 i mean, i don't know how something could go wrong if it just uses json.dumps() 11:31:15 i wonder if the other settings from that environment file take effect 11:31:40 jistr, the value in paramter_defaults is correct 11:33:23 spredzy: Was that working when you had the string delimited by commas? 11:34:15 jaosorior, yes 11:34:52 Ryan Hallisey proposed openstack/tripleo-heat-templates: Docker compute role configured via Puppet https://review.openstack.org/209505 11:35:03 let me redeploy with the comma to confirm and get the output of heat stack-show resoure 11:36:08 Giulio Fidente proposed openstack/tripleo-heat-templates: Ensure Glance API reaches Registry using the service VIP https://review.openstack.org/232480 11:54:24 spredzy, jistr so in the hierarchy, the topmost preveals on the others? 11:54:38 gfidente: yes 11:54:46 tks ;) 12:11:18 Merged openstack/tripleo-docs: Add a doc note about updating changed templates https://review.openstack.org/229373 12:15:45 jistr, jaosorior ok I have a bit more of context. --ntp-server x,y,z is passed along fine while NtpServer is declared as a string. However as soon as NtpServer is changed from string to comma_delimited_list the value is not understood anymore resulting in "" 12:18:51 spredzy: so, if it's a comma_delimited_list it doesn't take the value when in the tripleoclient it's passed as a list? 12:19:36 jaosorior, this is what I am observing at the moment yes 12:20:28 funky 12:20:30 fair enough then 12:22:26 jaosorior, which sounds like I am doing something wrong as https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/v1/overcloud_deploy.py#L88-L89 are both declared as comma_delimited_list 12:22:37 in the overcloud-without-mergepy 12:24:29 spredzy: And you're sure you're using the correct template from your other commit and everything? (I mess that up often haha) 12:25:06 I am using master + all NtpServer switched from string to comma_delimited_list 12:25:23 in overcloud-without-mergepy + puppet/X.yaml 12:25:59 spredzy: comma_delimited_list will accept both a comma delimited string, and an actual list, as in NeutronTunnelIdRanges etc 12:26:22 spredzy: I'd check via openstack --debug overcloud deploy to see exactly what is getting passed to heat 12:26:44 spredzy: I can also provide an example environment that allows deploying directly via heatclient (for comparison) if that helps 12:29:33 shardy, that would help. I am also redeploying with --debug 12:38:56 shardy, "stack_status": "CREATE_IN_PROGRESS", "stack_owner": "admin", "parameters": {"NeutronPublicInterfaceRawDevice": "", "KeystoneSigningKey": "******", "NtpServer": "" this is what I see in --debug while few line above that one I have [...] -H 'X-Auth-Url: http://192.0.2.1:5000/v2.0' -d '{"stack_name": "overcloud", "environment": {"parameter_defaults": {"ControllerCount": 1, "NeutronNetworkVLANRanges": "datacentre:1:1000", "ComputeCount": 1, "NtpSer 12:38:56 ver": "0.centos.pool.ntp.org" 12:39:52 "NtpServer": "" 12:40:05 spredzy: its included twice 12:40:52 spredzy: Oh, I see it's two different log lines 12:41:01 they are 12:41:41 Still, that "" looks wrong, is it included twice, in both parameters and parameter_defaults for example? 12:44:46 shardy, no, just once Deploying environment: {u'parameter_defaults': {u'ControllerCount': 1, u'NeutronNetworkVLANRanges': u'datacentre:1:1000', u'ComputeCount': 1, u'NtpServer': u'0.centos.pool.ntp.org', [...] 12:48:15 Dougal Matthews proposed openstack/tripleo-common: Collect the common logic used when dealing with Heat https://review.openstack.org/228991 12:54:12 dtantsur: hi, I've been poking at IPA a bit. What is your take on when TripleO should switch to IPA as a deployment ramdisk? 12:54:36 dtantsur: I'm keen on it because it means one less ramdisk to build (faster CI potentiall) 12:56:35 dprince, yesterday is the best time, right now works for me as well :) 12:57:18 dprince: I am pretty sure lucasagomes has a patch up for that 12:57:25 * lucasagomes reads 12:57:51 dprince, ++ to switch to IPA 12:57:56 dtantsur: cool. most of my IP changes already landed... but I personally need this too: https://review.openstack.org/#/c/232157/ 12:58:05 lucasagomes: BOOM, lets do it 12:58:20 dprince, BOOM, yeah, we already deprecated the bash ramdisk in Ironic btw 12:58:46 dtantsur, lucasagomes: Yeah, I had several issues w/ IPA (testing on real baremetal) but I just got it working yesterday 12:58:53 it was on my list to switch the TripleO CI to IPA (I think I had a patch for it, but needed to be rebased with another patch by trown ) 12:59:09 trown: which patch? 13:00:30 dprince, I had this https://review.openstack.org/#/c/210461/ 13:00:46 but it was before because it was using devtest 13:01:25 dprince, https://review.openstack.org/#/c/222231/ (trown's patch, already merged) 13:01:38 lucasagomes: gotcha, I would have just gone and landed that BTW but it is probably fine to leave it now 13:01:47 we perhaps just need to change it to build the IPA by default on tripleoclient 13:02:44 lucasagomes: it is built by default now 13:02:56 lucasagomes: I think we just need to configure Ironic to use it 13:03:00 lucasagomes: yep, got it. I'd rather see us rip all of the 'overcloud image build' out of tripleoclient and use a YAML/JSON config to control what images get built. But cool. I get where it is at 13:03:06 trown, right, so we just need to set it to the nodes 13:03:39 dprince, ++ yeah needs some refactor, but I think that mostly, if not all, the plumbing is done 13:03:50 dprince, lemme know if you need some help setting it up 13:04:39 lucasagomes: cool, thanks. I'll actually have one more patch for the iboot PM to add a bit of sleep in the 'reboot' command. 13:04:57 lucasagomes: the agent deployment uses that power management command... and it fires off super fast 13:05:11 lucasagomes: I need a bit of sleep (probably a config setting too for it) 13:05:28 dprince, cool, sure 13:05:39 lucasagomes: just a minor difference from how the old/new deployment styles work... 13:05:57 lucasagomes: anyway, it works... but that ramdisk is massive 13:06:40 dprince, yeah... I hear ya, it can be shrunk we just need to work on that 13:07:57 lucasagomes: Awesome :D I've managed to get a tiny core IPA image nearly working I think its just missing some dependecies 13:07:59 dprince, ^ 13:08:18 it was today, so some people are working on slimmed versions of IPA ramdisks 13:09:11 lucasagomes: nice :) 13:09:18 lucasagomes: what is tiny core? is it something that can be packaged downstream? 13:09:38 Yanis Guenane proposed openstack/tripleo-heat-templates: Allow a user to specify a comma separated list of ntp servers https://review.openstack.org/232051 13:09:59 goodsquishy now has its own url and owl nice :) 13:10:08 trown, it's just a tiny linux distro 13:10:16 today we use a fedora cloud image to build IPA 13:10:26 and it comes with a lot of stuff in it 13:10:53 Yanis Guenane proposed openstack/python-tripleoclient: Amend --ntp-server help message to add support for multiple servers https://review.openstack.org/232075 13:10:57 lucasagomes: right, just wondering what the RDO implications of using a base other than fedora or centos are 13:11:09 lucasagomes: I am not sure we can do that 13:11:15 trown, yeah, probably we can't... we could clean up the fedora image tho 13:11:22 ya for sure 13:11:36 the problem I saw with fedora when I did some cleaning was the dependencies 13:11:51 e.g qemu-img depending on libX11 stuff for some reason 13:14:20 Yanis Guenane proposed openstack/tripleo-heat-templates: Allow a user to specify a comma separated list of ntp servers https://review.openstack.org/232051 13:18:19 i started a high priorty review etherpad https://etherpad.openstack.org/p/tripleo-review-priorities 13:18:21 trown: from an RDO standpoint we could still build a centos ramdisk... it'll just be bigger 13:18:29 for my own sanity if nothing else :) 13:20:14 slagle: I like it... all my reviews are high priority :p 13:23:09 ifarkas: do you think you could look at https://review.openstack.org/221664 I would like to get something merged for tempest for the RDO test day 13:35:09 jaosorior, shardy jistr I tried to sum up everything in here https://bugs.launchpad.net/tripleo/+bug/1504137 13:35:09 Launchpad bug 1504137 in tripleo "Parameter from tripleoclient not passed to Heat template" [Undecided,New] 13:35:38 Derek Higgins proposed openstack-infra/tripleo-ci: Test a update in trunk repository version https://review.openstack.org/229789 13:46:02 Marek Aufart proposed openstack/tripleo-common: [WIP]Add resource capabilities https://review.openstack.org/232534 13:58:15 Yanis Guenane proposed openstack/tripleo-heat-templates: Manage keystone related settings directly in t-h-t manifests https://review.openstack.org/230357 13:58:15 Yanis Guenane proposed openstack/tripleo-heat-templates: Add more components virtual ip mapping into controller. https://review.openstack.org/231361 14:00:51 Marek Aufart proposed openstack/tripleo-common: [WIP]Add resource capabilities https://review.openstack.org/232534 14:03:40 shardy: this would be a nice add https://review.openstack.org/#/c/213902/1 14:12:22 Jaume Devesa proposed openstack/tripleo-puppet-elements: Adding networking MidoNet element https://review.openstack.org/232551 14:15:07 Jaume Devesa proposed openstack/tripleo-puppet-elements: Adding networking MidoNet element https://review.openstack.org/232551 14:19:07 Hi! I have some issues with instack-virt-setup, was wondering if it was the right place to ask 14:19:22 Apparently my libvirt isn't configuring dnsmasq properly? 14:22:28 trown: to finish up the conversation earlier are you (or is anyone) working to finish the switch to IPA? 14:25:02 dprince: I am not currently working on that 14:25:19 I am pretty bogged down trying to get RDO liberty released 14:26:22 trown: okay, thanks I may poke at it a bit if I can... 14:36:35 thrash: around? I wanted to chat a bit about tripleoclient, and possibly using build-images/load-images from the incubator (or converting those to be part of tripleoclient) 14:36:48 Merged openstack/python-tripleoclient: Update requirements https://review.openstack.org/231961 14:37:04 dprince: here. 14:37:16 and yes. that's up for inclusion. 14:37:39 dprince: plan is to put them into tripelo-common 14:37:42 thrash: have you done work on it yet? 14:37:59 dprince: I was gonna start yesterday, but got side-tracked. 14:38:11 I think the biggest question I have about it... where should the default go? 14:38:21 aka, the base tripleo yaml template. 14:38:49 kinda bikeshedding but... 14:39:00 thrash: rather than just move them over I was thinking perhaps 'openstack diskimage build -c ' 14:39:29 thrash: and perhaps 'openstack diskimage load -c ' 14:40:02 thrash: as for the default YAML, I think tripleo-common would be fine. I want to see these split out into distro specific files though 14:40:56 dprince: I think the first step is to make it into somewhat of an api and put it in tripleo-common, then we can add the new commands to tripleoclient 14:41:32 dprince: I will work on it today. 14:41:43 xiexs proposed openstack/tripleo-incubator: Replace the hard-code home directory with a tilde. https://review.openstack.org/232569 14:42:12 dprince: work for you? :) 14:43:06 thrash: I'm still a bit fuzzy. Would you use something similar to this: http://git.openstack.org/cgit/openstack/tripleo-incubator/tree/scripts/overcloud_puppet_disk_images.yaml 14:43:28 that's the idea. were you thinking something else? 14:43:44 dprince, If I want to retrieve the value of PublicVirtualIP in puppet/controller.yaml how should I proceed since https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/controller.yaml#L419 is deprecated. Also when I look at https://github.com/openstack/tripleo-heat-templates/blob/master/overcloud-without-mergepy.yaml#L567-L594 I see that they all point to internal/storage/ctrl not public. Where can I retrieve the public ip value per serv 14:43:44 ice ? 14:44:12 thrash: no, I think that is much better than hard coding things into tripleoclient python, although we could improve build-images I think too 14:44:29 dprince: i would like to see the ability to 'layer' the yamls 14:44:34 thrash: having build-images support overlays (a mechanism) to layer would be idea 14:44:37 thrash: exactly 14:44:37 :) 14:44:54 I think something very similar was done in khaleesi-settings. 14:45:07 * dprince isn't familiar w/ khaleesi-settings 14:45:13 I will start with the naive port/implementation 14:45:28 dprince: it's really just about taking multiple yamls and merging them down 14:45:45 thrash: sure, that sounds simple enough 14:45:52 dprince: ok. cool. 14:46:16 dprince: Hey! Looking at that os-net-config patch 14:46:20 xiexs proposed openstack/tripleo-incubator: Replace the hard-code home directory with a tilde. https://review.openstack.org/232569 14:46:38 thrash: see also this docker patch I did a long time ago now https://review.openstack.org/#/c/199830/ 14:46:54 dprince: will do.. 14:47:14 dprince: also, wanted to chat about the templates defaults stuff 14:47:31 shardy: oh right, sorry I -1'd some of those 14:47:33 dprince: my motivation is to align more closely with both the docs, and tripleo.sh (which is based on the docs) 14:47:47 thrash: thanks, press on. Very anxious to have this :) 14:48:13 dprince: I know it won't work for everyone, but I've been using that environment (as have nearly all downstream users, and most developers), thus those defaults make more sense to me 14:48:26 dprince: also, I really thing having hard-coded (different) defaults in the client sucks 14:48:27 shardy: sure, my concern is mostly where those defaults don't align w/ upstream projects (something TripleO has traditionally strived for) 14:48:51 shardy: lioadm isn't the cinder iscsi_helper default for example 14:49:10 dprince: yeah, I was wondering why we choose lioadm in the client then 14:49:22 shardy: also, I consider tripleoclient to be an extremely prescriptive tool... almost unusable for some people (like me) who often don't use those defaults 14:49:34 dprince: the back-story to all this, is I've started deploying directly via heatclient 14:49:47 shardy: why not just a simple env file to set the tripleclient defaults for a distro (or product) 14:49:52 it actually works really nicely, but I need a huge environment file, which is inconsistent with the templates 14:49:54 shardy: and those can live whereever 14:49:58 shardy: https://review.openstack.org/#/c/226790/ 14:50:21 dprince: sure, but I still think it makes more sense for the template defaults to align with our docs 14:50:30 shardy: I deploy directly w/ heatclient sometimes too, I just have a custom.yaml where I set my defaults 14:50:42 dprince: If we had a "centos_defaults.yaml" in t-h-t, that would also be fine 14:51:05 shardy: sure, I think some of your defaults probably go into there. lioadm could for example 14:51:09 but then, we should remove all the conflicting defaults in the overcloud-without-mergepy template, because they're not helpful 14:51:22 shardy: but defaults for things like the images and flavors don't make sense to me I think 14:51:24 shardy: dprince that review is to remove all of those defaults... 14:51:36 thrash: ++ 14:51:43 except for the images and flavors. 14:52:07 dprince: but our docs define images, and flavors, which don't align *at all* with what's in the templates? 14:52:08 ugh...we moved templates under os-apply-config/ ? 14:52:10 that is horrible 14:52:18 I know it's historical, but how does that make sense? 14:52:32 considering puppet heavily relies on "os-apply-config" 14:52:37 shardy: right, and what about other implementations like docker? 14:52:51 dprince: that's fine, then we have an environments/docker.yaml 14:53:00 slagle: what would you call it? we can move it again to wherever? 14:53:08 dprince: but the template defaults, if we have any, should at least work with *a* documented implementation 14:53:23 dprince: honestly, i'd delete. or at least deprecate 14:53:26 and currently, we only have one which is landed, CI'd and actually works 14:53:26 shardy: they do, for flavors we just require that you set them 14:53:37 dprince: if we want a name, how about bash-elements or bash 14:53:46 dprince: that's the actual analogy to "puppet" 14:53:51 dprince: but we know that every developer and downstream user, if they follow the docs, or use tripleo.sh will have "baremetal" 14:54:00 slagle: I'm not sure it really matters, and I do think they are headed towards deprecation 14:54:13 why would we make them write 5 lines in a yaml file, when we know that nearly all new users will need that setting? 14:54:22 slagle: my goal was to leave them in place until we have at least one other implementation in t-h-t. So docker... 14:54:25 well, it matters if the point of the change was to add clarity 14:54:31 slagle: once we have that I think lets deprecate 14:54:39 dprince: for folks such as yourself your parameters file still works exactly the same 14:54:49 it's just lowering the bar (slightly) for new folks 14:54:51 Derek Higgins proposed openstack/tripleo-common: tripleo.sh Add option to locally build delorean package https://review.openstack.org/228157 14:54:52 slagle: name for this is perhaps a bike shed man... 14:54:52 Derek Higgins proposed openstack/tripleo-common: tripleo.sh Add option to create a local Delorean environment https://review.openstack.org/228156 14:55:08 Derek Higgins proposed openstack-infra/tripleo-ci: Use tripleo.sh to build packages https://review.openstack.org/229425 14:55:10 shardy: I don't use baremetal :) 14:55:10 dprince: e.g I had to cut/paste the parameters from a --debug from tripleoclient before I could get deployments via heat to work 14:55:19 shardy: I have 2 flavors FWIW 14:55:25 dprince: i'd wager it's not...but given you picked the name, i can see how you feel it might be 14:55:35 dprince: *You* don't, and that's fine, but 99% of other folks are 14:56:07 just the fact that we're saying os-apply-config templates are headed towards deprecation ought to set off alarm bells 14:56:07 reverse engineering from tripleoclient isn't a good experience, even for a developer 14:56:24 os-apply-config aint going anywhere, anytime soon 14:56:27 shardy: w/ regards to the flavors I think I could be talked into 'baremetal'. FOr the image names though I'm less convinced there is a good default 14:56:54 dprince: following our documentation results in exactly one image "openstack-full" 14:57:01 why wouldn't we use that as a default? 14:57:03 shardy: I agree, that is why I think a simple distro defaults heat environment is the best approach for these things 14:57:24 shardy: those docs are still very new, and don't (yet) support docker 14:57:40 dprince: that's fine, and setting these defaults has zero imact on docker 14:58:13 dprince: if we're saying the defaults can't default to one implementation and actually work, we may as well remove all of them IMO 14:58:25 at least then it's less confusing 14:58:41 slagle: sorry you don't like the name. I couldn't think of anything better... was just trying to clean house a bit 14:59:00 slagle: because new users typically find looking at t-h-t to be really confusing 14:59:31 shardy: For things like flavors, and images I'm not sure I like defaults... because nova doesn't have them 14:59:37 shardy: those are just required settings 15:00:12 dprince: Ok, we should remove all the defaults from the templates then 15:00:17 Untriaged bugs so far: 15:00:19 https://bugs.launchpad.net/tripleo/+bug/1503669 15:00:19 Launchpad bug 1503669 in tripleo "Including an additional element is inconvenient" [Undecided,New] - Assigned to Jaume Devesa (devvesa) 15:00:20 https://bugs.launchpad.net/tripleo/+bug/1504137 15:00:20 Launchpad bug 1504137 in tripleo "Parameter from tripleoclient not passed to Heat template" [Undecided,New] 15:00:29 and just maintain a defaults environment which is consumed by all CLI/UI tools 15:00:29 shardy: I think that is taking it too far 15:00:49 dprince: the whole point of a "default" is that it works for the most common case 15:01:04 which, in this case, is the only case we actually document, CI and support atm 15:01:46 In this case, the defaults work for exactly 0% of cases, because we do things like refer to legacy image names from old docs/scripts 15:02:41 shardy: so do you think a default for real hardware is prefered to virt testing then? Even though maybe 90% of the developers and users will try things out in virt land? 15:03:07 dprince: I did a Heat/TripleO demo at a conference this week, and it was bad - it was like "look here are the 35 different parameters you have to set to deploy TripleO" 15:03:11 shardy: I would argue there isn't a good virt default. I happen to like 'kvm' ... but in fact it will fail for most devs 15:03:43 shardy: right, I think a simple heat environment to set those for tripleo-docs makes the most sense here 15:03:45 dprince: Ok, maybe that's a bad setting - interestingly following the docs gets you kvm, even on VMs 15:04:03 shardy: yeah, well perhaps most people use nested virt? 15:04:11 shardy: anyways, just an example 15:04:33 dprince: Ok, let me see if I can rebase things to get the ones we actually agree on landed 15:04:38 shardy: I'm not -2 on these FWIW.... just a bit unconvinced that there *is* a good default for some of these things 15:04:39 then we can argue over the rest another day ;) 15:05:00 shardy: half of them are a pretty easy +2 from me :) 15:05:11 dprince: Yeah, I get why you say that, but I'm trying to improve the experience for new folks who just follow the docs 15:05:21 Thomas Herve proposed openstack/instack-undercloud: Don't use dnsmasq lease file to retrieve address https://review.openstack.org/232584 15:05:29 Yay first patch 15:05:29 shardy: ++ 15:05:33 more advanced folks can still set whatever they want, and they know how to do it :) 15:05:57 therve: Welcome! :) 15:06:07 shardy, Thanks :) 15:14:59 Dan Sneddon proposed openstack/tripleo-docs: Updates to network documentation for new features https://review.openstack.org/221908 15:16:39 hi all, second day I'm stucked running introspection with an strange error causing "dracut emergency" 15:16:46 may I ask for hints someone here? 15:22:07 Derek Higgins proposed openstack/tripleo-common: tripleo.sh Add option to locally build delorean package https://review.openstack.org/228157 15:22:07 Derek Higgins proposed openstack/tripleo-common: tripleo.sh Add option to create a local Delorean environment https://review.openstack.org/228156 15:24:00 Merged openstack/instack-undercloud: Run db-sync as ironic-inspector user https://review.openstack.org/231656 15:41:36 dprince: Do you think we can land these please? 15:41:42 https://review.openstack.org/#/q/status:open+project:openstack/tripleo-heat-templates+branch:master+topic:node_extraconfig,n,z 15:41:56 On aggregate all the tests are passing between the two patches in the series 15:43:22 Thomas Herve proposed openstack/instack-undercloud: Don't use dnsmasq lease file to retrieve address https://review.openstack.org/232584 15:49:20 Merged openstack/os-net-config: Support multiple addresses assignment with ifcfg https://review.openstack.org/213902 15:54:52 Derek Higgins proposed openstack/tripleo-common: tripleo.sh Add option to locally build delorean package https://review.openstack.org/228157 15:54:53 Derek Higgins proposed openstack/tripleo-common: tripleo.sh Add option to create a local Delorean environment https://review.openstack.org/228156 16:10:29 Ryan Brady proposed openstack/instack-undercloud: Adds Swift Versioning Support https://review.openstack.org/232634 16:16:01 Ryan Brady proposed openstack/instack-undercloud: Adds Swift Versioning Support https://review.openstack.org/232634 16:16:54 Dougal Matthews proposed openstack/tripleo-common: Collect the common logic in tripleo-common https://review.openstack.org/228991 16:19:55 Hi all. In my current dev flow, at `openstack overcloud image upload`, seeing 'ERROR: openstack Required file "./ironic-python-agent.initramfs" does not exist.' This is quite accurate; it definitely does not exist (and never has to date). Is this a known issue? 16:20:22 (Or is there a dependency chain among CRs that's partially merged?) 16:23:52 egafford: it should get built with `openstack overcloud image build` but probably doesnt due to docs 16:24:15 trown: Ah, so it's just not part of --all right now? 16:24:16 https://review.openstack.org/#/c/232147/ 16:24:25 Dan Prince proposed openstack/tripleo-heat-templates: Use same host for cinder volume active/passive https://review.openstack.org/225320 16:24:37 trown: Ran `openstack overcloud image build --all 2>&1 | tee image-build.log` 16:24:45 egafford: it is part of --all, but without the DIB_YUM_REPO_CONF in that doc patch the package can be found for it 16:25:04 and the image build silently fails to build it 16:25:09 trown: Nice. 16:25:13 ya 16:25:20 Thanks; sounds like all will be well. 16:25:56 `export DIB_YUM_REPO_CONF="/etc/yum.repos.d/delorean.repo /etc/yum.repos.d/delorean-deps.repo` is what is needed prior to image build 16:32:31 Merged openstack/diskimage-builder: Avoid transcending /proc with find https://review.openstack.org/230448 16:39:55 Dan Prince proposed openstack/tripleo-heat-templates: Update README.rst https://review.openstack.org/230675 16:41:28 Dan Prince proposed openstack/tripleo-heat-templates: Update README.rst https://review.openstack.org/230675 16:51:14 Dan Prince proposed openstack/tripleo-heat-templates: pacemaker: Set manage_service/enabled via Hiera https://review.openstack.org/221881 17:05:19 Dan Prince proposed openstack/tripleo-heat-templates: Update README.rst https://review.openstack.org/230675 17:50:20 dprince: just fyi, the instack kernel panic occurred again. so i'm disabling docker on the host and we'll see what happens then 17:57:40 jpeeler: sounds good 17:57:55 jpeeler: maybe even reboot to make sure all dm_thin enablement is gone 17:58:34 k 17:58:37 jpeeler: I can't remember if I just disabled docker, or if I disabled and rebooted 18:00:22 thrash: looks like this is needed for tripleoclient: https://review.openstack.org/232694 18:00:38 we got called out on the ML :) 18:08:30 slagle: ack 18:08:45 slagle: I had already handles that. 18:08:48 *handled 18:08:50 :) 18:08:58 https://review.openstack.org/#/c/232589/ 18:09:22 although your commit message looks better. :) 18:09:54 slagle: I will abandon in favor of yours. 18:10:31 slagle: did that break for *everybody*? 18:10:42 if so, that seems bad on their side. 18:12:03 Dan Prince proposed openstack/tripleo-heat-templates: Update README.rst https://review.openstack.org/230675 18:15:55 thrash: i think it's just a verification job they run 18:16:08 slagle: ok. good. :) 18:22:45 Ben Nemec proposed openstack/instack-undercloud: Nothing to see here https://review.openstack.org/232700 18:23:20 rhallisey: i'm trying out the docker patch, and getting: http://fpaste.org/276568/44328569/ from Heat after a while 18:23:35 rhallisey: the controller finished deploying fine. have you ever seen this before? 18:24:12 it's not obvious to me why Heat would error like this. i had no stacks present prior to starting the deployment 18:30:30 slagle: ive seen that once before I think. I don't think it is related to docker though 18:30:41 slagle: would be nice to have debug logs on it... 18:33:21 well, i have debug logs for heat, etc 18:35:02 I wonder if one of the containers just died or something 18:35:10 i think this is on the heat side 18:35:20 weird 18:35:27 dprince: got the panic again, no docker running on host and this was after a reboot. appears to be related to ironic since that's what i was doing 18:35:38 i have 2 cores on this undercloud, it's almost as if the request to create this nested stack got picked up by 2 different heat-engine processes 18:35:47 jpeeler, oh really hmm 18:35:55 jpeeler: oh, interesting 18:36:09 jpeeler: perhaps it is just a matter of time for me to hit it again too 18:36:21 jpeeler: can you still login to the VM? 18:36:26 not this time 18:37:03 did these kernel panics start after enabling nested kvm? 18:37:51 jpeeler: so Ironic is doing qemu-img operations on the fedora/centos-atomic image. I wonder if one of those is firing off the dm_thin code... 18:38:18 slagle: although i believe my setup does have nested vm enabled, no nested vms are in use 18:38:37 (haven't been deploying with --libvirt-type=qemu) 18:38:42 jpeeler: perhaps we need to understand whats sorts of IO operations activate this 18:39:09 jpeeler: sure, but if it is something within the VM that we think is causing it (ironic) I don't think nested virt matters 18:39:21 jpeeler: could be worth eliminating that as a concern though 18:40:14 i'll turn it off then 18:40:22 so ... I think we need a to reboot the undercloud in CI 18:40:34 slagle, the kernel panic is due to docker being present in the overcloud. For me, I got around it by using fedora-atomic, but not sure if that worked for jpeeler 18:40:52 there is an old kernel *we think* lying around 18:41:12 rebooting the undercloud breaks OVS in my virt env 18:42:19 trown: I think a reboot might be a bit expensive 18:42:25 trown: timewise 18:42:48 trown: probably good to track down why it isn't working though 18:43:09 dprince: ya, but it is a pretty bad result to have rebooting the undercloud break 18:43:39 dprince: but it maybe that is more of a downstream test 18:43:53 trown: yeah, infra has been pinging us about how long our jobs take 18:43:57 trown: especially the HA job 18:44:21 slagle, let me know if that happens again. I haven't hit that issue before 18:44:33 trown: perhaps when we get our walltime down we can entertain adding some coverage for undercloud, overcloud, and perhaps even tenant reboots 18:44:52 dprince: makes sense 18:45:09 dprince: it looks to me like 2 heat engines collided trying to create the same nested stack 18:45:13 https://bugs.launchpad.net/tripleo/+bug/1504266 18:45:13 Launchpad bug 1504266 in tripleo "Stack overcloud-ComputeNodesPostDeployment-dkq4fgg4lqk5-NovaComputeContainersDeployment-6zpj2rijh3lk already has an action (CREATE) in progress." [Medium,Triaged] 18:45:19 trown: the inability to reboot an undercloud was the initial use case that drove the creation of os-net-config FWIW 18:46:17 slagle: hmmm, interesting 18:46:27 slagle: add heat to that bug too? 18:47:44 i did 18:52:06 is there a way to force delete an ironic node since delete doesn't work with provision state deploy failed? 18:55:33 Ryan Brady proposed openstack/tripleo-common: [WIP]Initial workflow implementation of tripleo-common https://review.openstack.org/227297 18:58:45 jpeeler: this is the sledge hammer approach http://paste.openstack.org/show/475778/ 18:59:08 jpeeler: I hit this the other day... and I think ironic has some state transition issues 18:59:19 jpeeler: it should always be possible to recover using only the CLI 18:59:30 jpeeler: probably worth filing an Ironic bug on this... 18:59:49 or just look up the one i filed over a year ago :) 19:00:02 er wow 19:00:26 slagle: probably worth asking someone to have a closer look then 19:00:39 if it is annoying enough we should fix it 19:01:41 slagle: got a link? 19:02:52 i did, a while back 19:03:16 lucas looked into it, but it was all kind of blocked on the breaking lock discussions in ironic 19:03:22 i'm not sure what the status of that is now 19:10:49 dprince: jpeeler : https://bugs.launchpad.net/ironic/+bug/1461937 19:10:49 Launchpad bug 1461937 in Ironic "Node stuck on DEPLOYING (potentially all *ING) state(s)" [High,Fix released] - Assigned to Lucas Alvares Gomes (lucasagomes) 19:11:02 if it's in deploying, supposedly there is a timeout 19:14:02 slagle: I think I may be talking about a slightly different case 19:14:27 slagle: which is sort of that I'd like the equivalent of a 'nova reset-state' command for ironic that actually works... 19:14:43 slagle: to reset the power status to 'available' 19:15:25 jpeeler: it might be worth capturing exactly what you did to 'hang' the instance in a bug? 19:16:13 dprince: this was due to kernel panicking during a deploy 19:16:53 and then rebooting obviously 19:17:06 is that worth describing? 19:17:38 dprince, got the kernel panic 19:17:51 jpeeler: perhaps, only to the extend that ironic can get into a state that isn't recoverable without editing the database 19:17:56 I just rapidly redeployed the overcloud and that triggered it 19:18:32 dprince: i'm sure it'll happen again and then i'll collect some exact real details 19:19:55 rhallisey: yeah, based on what jpeeler said earlier I was thinking maybe this is just local to the ironic VM 19:20:28 rhallisey: because the ironic vm does run qemu-img commands on the fedora-atomic image... perhaps something there is tripping up the dm_thin code in the instack VM kernel? 19:21:07 rhallisey: basically one of the local "volume" operations that ironic does is causing this kernel bug to manifest itself 19:21:16 rhallisey: perhaps? 19:26:02 rhallisey: could we try tailing the ironic conductor log, watching the commands it runs 19:26:13 dprince, possibly. I'm definetly looking at ironic for this 19:26:15 rhallisey: and continually deploying just the fedora-atomic image to a VM 19:26:32 rhallisey: I think that would narrow it down quite a bit... 19:26:37 kk 19:29:10 rhallisey: I'm trying to look at this too BTW a bit more closely 19:29:17 ok cool 19:34:25 rhallisey: so even though the compute node is fedora, i can use my centos7 image for the controller? 19:34:40 i dont see why not, just figured i'd confirm :) 19:35:15 slagle, ya that should be fine 19:36:02 rhallisey, jpeeler: could we do the upcoming "config" rework for docker in a follow up patch 19:36:14 ya that's the plan 19:36:36 great, thanks 19:36:39 ack 19:36:48 I'd rather get the docker bits merged now and then move to stable liberty soon after 20:08:24 EmilienM: ping (re: Trove). 20:08:45 May be too late; if it is please do disregard. 20:11:17 he's on the west coast at puppetconf and drinking beer 20:11:50 slagle: Ah, cool. Not too late in general, then, just too late right now. :) 20:12:45 slagle: Was chatting with vkmc (Trove team) re: puppet Trove action items (changing rabbit namespace, etc.) The Trove integration card for director is a little light; is there an upstream card capturing the currently known action items? 20:14:10 egafford: not that i know of 20:14:34 slagle: Ack. Any sense of when EmilienM might be back about? 20:16:13 egafford: no...next week probably 20:17:03 slagle: Ok, next week isn't so far away. Thanks. 20:47:48 egafford: hey, sorry I was afk at puppetconf 20:47:52 slagle: not yet lol 20:48:22 egafford: there is not upstream card capturing the items, but we can create it on https://trello.com/b/4X3zxWRZ/on-going-effort 20:48:28 I would be happy to help 20:55:42 EmilienM: No problem at all! Thanks. What I'm seeing thus far is that the current trove.conf has a disconnect with the puppet modules in re: Rabbit config (puppet want the keys to be in oslo_messaging_rabbit; trove isn't having any of it and is keeping those keys in DEFAULT.) 20:56:10 that sucks 20:56:17 I'm working on a patch now to fix it, actually; not a puppet expert, but that much is in my power. :) 20:56:23 let me see though, I saw some patches like this 20:56:29 Ah, cool. 20:56:38 I didn't see anything in active review in puppet-trove. 20:57:45 egafford: wait, we already have oslo_messaging_rabbit stuff 20:57:54 It's also conceivable that it's the conf that's out of date; I have a bit more research to do before definitively defining the scope of the task (sahara guy, not trove guy, really.) 20:58:03 Yeah, you do, but trove doesn't, it seems. 20:58:32 https://github.com/openstack/puppet-trove/blob/master/manifests/api.pp#L344-L350 20:58:40 egafford: ^ 20:59:03 EmilienM: Yes, I know (I'm pulling overcloud modules from head of that repo.) 20:59:23 so what is missing? 20:59:27 Trove itself has those keys under DEFAULT/rabbit_userid, etc. in our current packages. 20:59:34 Not oslo_messaging_rabbit. 20:59:48 oh so you need to ask for an OPM update 21:00:01 What's that three-letter acronym mean? 21:00:06 you should ping imck8 (not on this channel, but on #rdo) 21:00:10 openstack puppet modules 21:00:15 it's an rpm that contains our modules 21:00:18 Untriaged bugs so far: 21:00:20 https://bugs.launchpad.net/tripleo/+bug/1504179 21:00:21 https://bugs.launchpad.net/tripleo/+bug/1503669 21:00:23 https://bugs.launchpad.net/tripleo/+bug/1504137 21:00:23 Ah, yes, got it. 21:00:25 https://bugs.launchpad.net/tripleo/+bug/1504263 21:00:25 Launchpad bug 1504179 in tripleo "instack-virt-setup can't use dnsmasq lease file anymore" [Undecided,In progress] - Assigned to Thomas Herve (therve) 21:00:26 Launchpad bug 1503669 in tripleo "Including an additional element is inconvenient" [Undecided,New] - Assigned to Jaume Devesa (devvesa) 21:00:27 Launchpad bug 1504137 in tripleo "Parameter from tripleoclient not passed to Heat template" [Undecided,New] 21:00:28 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 21:01:01 I don't think an opm update would help, though; that would pull master into OPM, which would still result in trove and puppet being out of sync, yes. 21:01:16 Ryan Brady proposed openstack/tripleo-common: [WIP]Initial workflow implementation of tripleo-common https://review.openstack.org/227297 21:03:00 The puppet modules are setting a fundamentally different config key than the packaged trove.conf, at master. Anyway, let me do a bit more investigation (genconfig on trove master) before I say firmly which one is out of date. 21:06:24 Tzu-Mainn Chen proposed openstack/tripleo-specs: Spec for TripleO overcloud deployment REST API https://review.openstack.org/230432 21:39:46 Emilien Macchi proposed openstack/puppet-tripleo: Drop fixtures https://review.openstack.org/232789 22:19:55 Emilien Macchi proposed openstack/puppet-tripleo: Try to use zuul-cloner to prepare fixtures https://review.openstack.org/232789 22:21:02 EmilienM: Figured out what's going on; it looks like we need to update trove.conf.sample in the trove package itself. Your team is off the hook entirely on this one. :) 22:21:21 egafford: ok 22:21:21 (Unless I'm wrong again, which one can never rule out.) 22:24:44 Ben Nemec proposed openstack/instack-undercloud: Update some discovery references to inspection https://review.openstack.org/232802 22:25:38 Ben Nemec proposed openstack/instack-undercloud: Update some discovery references to inspection https://review.openstack.org/232802 22:36:35 Hi, what is a rough time estimate for how long it takes to setup an undercloud on a Virtual environment ( http://docs.openstack.org/developer/tripleo-docs/environments/virtual.html )? 22:36:41 a few hours? 23:51:51 Augustina Ragwitz proposed openstack/diskimage-builder: Update rhel7 element readme https://review.openstack.org/222351 23:52:55 Augustina Ragwitz proposed openstack/diskimage-builder: Update apt-preferences element README from free text to table formatting https://review.openstack.org/232211 23:55:03 Augustina Ragwitz proposed openstack/diskimage-builder: Update apt-preferences element README from free text to table formatting https://review.openstack.org/232211 23:56:41 Augustina Ragwitz proposed openstack/diskimage-builder: Update rhel7 element readme https://review.openstack.org/222351 23:57:40 Augustina Ragwitz proposed openstack/diskimage-builder: Update apt-conf elements README from free text to table formatting https://review.openstack.org/231770 23:59:46 Augustina Ragwitz proposed openstack/diskimage-builder: Update rhel7 element readme https://review.openstack.org/222351 00:02:51 Augustina Ragwitz proposed openstack/diskimage-builder: Update rhel7 element readme https://review.openstack.org/222351 00:07:17 Augustina Ragwitz proposed openstack/diskimage-builder: Fix title of env vars section of redhat-common README to match template https://review.openstack.org/232817 00:10:34 Augustina Ragwitz proposed openstack/diskimage-builder: Fix title of env vars section of fedora README to match template https://review.openstack.org/232821 00:12:55 Augustina Ragwitz proposed openstack/diskimage-builder: Update apt-conf elements README from free text to table formatting https://review.openstack.org/231770 00:14:23 Augustina Ragwitz proposed openstack/diskimage-builder: Update apt-preferences element README from free text to table formatting https://review.openstack.org/232211 00:41:47 Merged openstack/diskimage-builder: Fix title of env vars section of fedora README to match template https://review.openstack.org/232821 00:51:22 Merged openstack/diskimage-builder: Fix title of env vars section of redhat-common README to match template https://review.openstack.org/232817 03:00:19 Untriaged bugs so far: 03:00:20 https://bugs.launchpad.net/tripleo/+bug/1504363 03:00:20 Launchpad bug 1504363 in tripleo "Updates to hiera configuration not propogating/making a puppet run" [Undecided,New] 03:00:21 https://bugs.launchpad.net/tripleo/+bug/1504179 03:00:21 Launchpad bug 1504179 in tripleo "instack-virt-setup can't use dnsmasq lease file anymore" [Undecided,In progress] - Assigned to Thomas Herve (therve) 03:00:23 https://bugs.launchpad.net/tripleo/+bug/1503669 03:00:23 Launchpad bug 1503669 in tripleo "Including an additional element is inconvenient" [Undecided,New] - Assigned to Jaume Devesa (devvesa) 03:00:25 https://bugs.launchpad.net/tripleo/+bug/1504137 03:00:26 Launchpad bug 1504137 in tripleo "Parameter from tripleoclient not passed to Heat template" [Undecided,New] 03:00:27 https://bugs.launchpad.net/tripleo/+bug/1504263 03:00:28 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 03:08:28 Ryan Hallisey proposed openstack/tripleo-common: Deploy Docker compute role with tripleo.sh https://review.openstack.org/232853 03:27:18 Ryan Hallisey proposed openstack/python-tripleoclient: Add option to deploy Docker compute role in the tripleoclient https://review.openstack.org/232854 04:54:28 Ian Wienand proposed openstack/diskimage-builder: Remove extra install of release pkgs in fedora-minimal https://review.openstack.org/227131 04:54:29 Ian Wienand proposed openstack/diskimage-builder: Add basic Fedora 22 support to yum-minimal https://review.openstack.org/227125 04:54:30 Ian Wienand proposed openstack/diskimage-builder: Move yum-based install into function https://review.openstack.org/227124 04:54:31 Ian Wienand proposed openstack/diskimage-builder: Cleanup the fedora-minimal yum/dnf path https://review.openstack.org/227127 04:54:32 Ian Wienand proposed openstack/diskimage-builder: Update default fedora-minimal to f22 https://review.openstack.org/227126 04:54:33 Ian Wienand proposed openstack/diskimage-builder: Reset yum/dnf cache to correct location https://review.openstack.org/232863 07:36:16 Dougal Matthews proposed openstack/python-tripleoclient: Add option to deploy Docker compute role in the tripleoclient https://review.openstack.org/232854 07:49:05 Ian Wienand proposed openstack/diskimage-builder: Reset yum/dnf cache to correct location https://review.openstack.org/232863 08:32:56 Merged openstack/tripleo-heat-templates: Update README.rst https://review.openstack.org/230675 08:37:28 Merged openstack/tripleo-heat-templates: Ensure Glance API reaches Registry using the service VIP https://review.openstack.org/232480 08:52:16 shardy: Given a bunch of files, is there a way to programatically determine the root template? 08:52:28 * d0ugal hopes that makes sense 08:52:37 s/files/templates/ 08:52:49 I guess not since there could be more than one 08:56:06 d0ugal: Not atm, that's part of what my heat template capabilites BP was designed to address 08:56:33 e.g annotate each template so that you can define it implements e.g OS::TripleO::Controller 08:56:38 or, in this case "overcloud" 08:57:01 Merged openstack/tripleo-heat-templates: Wire in NodeExtraConfig interface https://review.openstack.org/227225 08:57:04 d0ugal: so right now we have to rely on knowledge of the filename (or ask the user for it) 08:58:14 Martin André proposed openstack/diskimage-builder: Add missing six requirement for svc-map element https://review.openstack.org/232962 09:00:19 Untriaged bugs so far: 09:00:21 https://bugs.launchpad.net/tripleo/+bug/1504363 09:00:21 Launchpad bug 1504363 in tripleo "Updates to hiera configuration not propogating/making a puppet run" [Undecided,New] 09:00:22 https://bugs.launchpad.net/tripleo/+bug/1504179 09:00:22 Launchpad bug 1504179 in tripleo "instack-virt-setup can't use dnsmasq lease file anymore" [Undecided,In progress] - Assigned to Thomas Herve (therve) 09:00:24 https://bugs.launchpad.net/tripleo/+bug/1503669 09:00:25 Launchpad bug 1503669 in tripleo "Including an additional element is inconvenient" [Undecided,New] - Assigned to Jaume Devesa (devvesa) 09:00:26 https://bugs.launchpad.net/tripleo/+bug/1504137 09:00:27 Launchpad bug 1504137 in tripleo "Parameter from tripleoclient not passed to Heat template" [Undecided,New] 09:00:28 https://bugs.launchpad.net/tripleo/+bug/1504263 09:00:28 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 09:18:58 Merged openstack/tripleo-heat-templates: Rework pacemaker constraints from ovs and netns cleanup agents https://review.openstack.org/229466 09:20:38 Giulio Fidente proposed openstack/tripleo-heat-templates: Use same host for cinder volume active/passive https://review.openstack.org/225320 09:24:50 Jiri Stransky proposed openstack/tripleo-heat-templates: Support NFS backend for Glance (via Pacemaker) https://review.openstack.org/199152 09:51:58 Jiri Stransky proposed openstack/tripleo-heat-templates: Parametrize RabbitMQ FD limit https://review.openstack.org/201796 09:51:59 Jiri Stransky proposed openstack/tripleo-heat-templates: Remove forced rabbitmq::file_limit conversion to string https://review.openstack.org/232983 10:00:59 shardy: Thanks, forgot to reply above. That's what I thought 10:01:18 shardy: btw, I really like your idea with the shortcut syntax. 10:02:12 Steven Hardy proposed openstack/tripleo-specs: Add spec for release branch policy https://review.openstack.org/221811 10:03:44 d0ugal: cool, I may propose something to heatclient, then we can work together to ensure tripleoclient aligns 10:04:35 Javier Peña proposed openstack/puppet-tripleo: Set tunnel timeout for nova_novncproxy https://review.openstack.org/232986 10:05:30 Javier Peña proposed openstack/puppet-tripleo: Set tunnel timeout for nova_novncproxy https://review.openstack.org/232986 10:09:11 derekh: Hey, do you happen to know what the infra process is for proposing new branches to existing repos? 10:09:38 In the heat repos there is some magic that happens around release time to create them, but I don't really understand it tbh 10:09:48 shardy: I got no idea, part of me thinks it can be done via gerrit 10:10:01 derekh: Ok, no worries, thanks anyway! :) 10:10:38 derekh: btw you should add yourself as co-author to those tripleo.sh patches, given all the work you've done on them :) 10:11:38 shardy: I *think* (not sure at all), that it can be done on this screen, if people have the correct acl's https://review.openstack.org/#/admin/projects/openstack/tripleo-heat-templates,branches 10:12:07 shardy: I can't seem to do it, so I'm either wronge or don't have the access 10:12:35 derekh: Ok, thanks, I don't have the superpowers either 10:12:54 shardy: sounds will add them if doing another revision 10:13:03 I should probably wait for the spec to get approved, but I'm working out the steps to take when it hopefully is 10:16:53 Hey guys, is there a way to set up the fqdn somehow via the heat templates? 10:17:00 the fqdn of the overcloud 10:17:46 jaosorior: I think that's what the "CloudName" parameter is for 10:18:10 shardy, thanks, I hadn't noticed it 10:34:52 Lennart Regebro proposed openstack/python-tripleoclient: Validate the stack before deploying https://review.openstack.org/232999 10:44:28 jaosorior, keep in mind that names must be resolvable 10:45:16 jaosorior, pacemaker relies on the hostnames to figure which network to use for cluster communication, and the redis or galera resource agents will match those hostnames with cluster memebers 10:45:53 and this is pretty much a requirement from pacemaker we can't workaround just by using IPs 10:47:09 jaosorior, btw the domain suffix is assigned by the DHCP running on the undercloud, so you can edit that in the undercloud neutron 11:00:24 hi, what's a correct way to restart an unfinished overcloud deploy? 11:01:03 kukacz: provided it's in a failed state, you can just try openstack overcloud deploy again 11:01:08 the stack is in state CREATE_IN_PROGRESS but I know it will not complete 11:01:54 kukacz: ah, at present there's not a good way to force heat to just stop/abort the create 11:02:05 so you have to either wait for it to fail, or delete the stack and try again 11:02:11 e.g heat stack-delete overcloud 11:02:22 which will delete the in-progress stack, then you can re-try 11:02:29 shardy, ok, I'll delete it, thanks 11:09:03 marios, your comment here: https://review.openstack.org/#/c/210402/3/overcloud-without-mergepy.yaml,cm 11:09:10 you mean to leave it empty by default? 11:10:42 gfidente: looking 11:12:01 gfidente: yeah unset so we force its use 11:12:19 uhm 11:12:28 we don't have anything in the client which sets that yet though 11:13:10 I think we should also figure if we want the passwords being generated randomically 11:13:14 or provided by the user 11:13:26 gfidente: my point is it defaults to 'guest', which was fine, except we are now explicitly setting it. but either way the change still works, just a thought 11:14:47 gfidente: the ideal would be for heat to handle 'if set, use, otherwise, use randomstring' 11:15:06 gfidente: like a new default type 'default: random_string' or something 11:17:09 Giulio Fidente proposed openstack/tripleo-heat-templates: Ensure access to Redis is password protected https://review.openstack.org/210405 11:18:57 Giulio Fidente proposed openstack/tripleo-heat-templates: Ensure access to RabbitMQ is password protected https://review.openstack.org/210402 11:21:37 marios, merge https://review.openstack.org/#/c/211535/ 11:21:38 :) 11:22:17 * gfidente slowly getting back up to speed :( 11:23:58 gfidente: yes but is that crm_verify verbose enough? 11:24:10 i mean only -VVVVVV 11:24:19 I think jistr should have added a couple more -V indeed 11:24:49 :P 11:24:52 I'm sure derekh will be superhappy to archive all those infos 11:25:14 it was what spredzy asked me to add, i actually haven't run the command i think 11:25:26 he found it helpful when debugging some pcmk issue 11:26:06 Merged openstack/tripleo-heat-templates: Fix MariaDB account removal race condition https://review.openstack.org/230488 11:26:08 yup that was in order to see what was happening at the corosync/totem level 11:26:10 Merged openstack-infra/tripleo-ci: Use sudo for pacemaker status output https://review.openstack.org/211535 11:26:29 spredzy, is the meaning of every V documented somewhere that you know of? 11:26:50 gfidente, no that I know of 11:32:30 Giulio Fidente proposed openstack/tripleo-heat-templates: Ensure httpd is not enabled by puppet on system boot https://review.openstack.org/206463 11:32:41 marios, also, did I get ^^ right? 11:33:15 gfidente: looking 11:33:55 gfidente: Went to eat; And I jsut read what you just put, thanks for the tip 11:36:15 Steven Hardy proposed openstack/tripleo-docs: Update docs to show using ExtraConfig parameters https://review.openstack.org/233012 11:36:50 gfidente: yeah, the mod status was just rebase nit and the location of the pacemaker service doesn't need to change 11:36:57 gfidente: thanks, completely forgot about that one :) 11:36:58 Jiri Stransky proposed openstack/tripleo-heat-templates: Support NFS backend for Glance (via Pacemaker) https://review.openstack.org/199152 11:37:43 marios: thanks re ^, good catch :) 11:38:07 jistr: heh, was just looking. so i guess did it resolve correctly because they are in the same file or did it in fact error? 11:38:25 (I mean as get_param) 11:38:46 marios: i think it resolved correctly as it was, because i tested it, but the way with get_input follows the convention with the rest of the file 11:39:34 jistr: yeah k thanks. i thought it was like that (resolved ok) and so wasn't sure if it was actually something you missed, or just 'didn't matter' so left it :) 11:54:04 jistr, so I am back to the point where I want to build a valid hiera list from within the template 11:54:17 if I got it right I can use yaml syntax in the template now? 11:54:54 gfidente: do you have an example? 11:55:09 want to port this stuff https://review.openstack.org/#/c/212664/4/puppet/hieradata/ceph.yaml,cm 11:55:12 into the template 11:55:22 gfidente: i don't know what you guys are discussion, but sound like it might be like https://review.openstack.org/#/c/221881/3/puppet/controller-config-pacemaker.yaml 11:55:25 (setting hiera) 12:00:28 rhallisey: should i expect the docker compute patch to work with network isolation (single-nic-with-vlans)? 12:00:31 jistr, /me was thinking to put here a list_join 12:00:58 rhallisey: i realized i have this as part of my default deployment, but maybe it's not a good idea 12:01:08 slagle, try it 12:01:23 i have, it hangs on running /usr/local/bin/os-net-config 12:01:54 the network to the compute instance just dies apparently 12:02:11 ok I think Ian might still be working on that. I thought the networking bits were done 12:02:12 maybe not 12:02:14 i had to hack grub to set a root pw so i could login on the console 12:02:21 jistr, but that means building the string with the usual and not-very-nice trick of [' + string + ',' ... 12:02:41 slagle, so you could set a root pw in the first boot script 12:03:53 yes, if i had the forethought :) 12:04:03 gfidente: hmm so that depends if we have a way of constructing that list in Heat or not... if it's a list in heat, it will be a list in Hiera too. I'm still not 100% sure what is the attempted patch there, do you have some WIP? 12:04:06 i was trying to get into this failed node to figure out what went wrong 12:04:09 slagle, hehe you reminded me to remove it XD 12:04:12 lol 12:04:41 i know, well, we have it disabled in other places 12:04:42 slagle, I don't quite understand the issue since Ian was primarily dealing with it 12:04:44 jistr, yeah so list in heat is what I wanted to output, instead of list_join, let me try and I'll let you know 12:04:52 slagle, dprince would know thouhg 12:05:19 I remember we needed to change os-net-config though 12:06:10 gfidente: unfortunately the "data processing" functions are very limited in heat... i wonder if we'll have to use a combination of list_join and str_split 12:06:11 http://docs.openstack.org/developer/heat/template_guide/hot_spec.html#str-split 12:06:36 I'm going to try the - \n - yaml syntax first 12:06:37 which would be kinda ugly, but should produce a correct list 12:06:59 gfidente: that won't work as if it's a multiline string, it will get quoted 12:07:31 why would it get into a multiline string? 12:08:19 gfidente: well i still don't know what code you are about to write and what you meant exactly by "\n - yaml syntax" 12:08:19 dprince, hey 12:08:30 gfidente: but let's try and we'll see 12:08:36 dprince, slagle was just testing the net isolation stuff 12:08:53 rhallisey: hi, with docker you mean? 12:09:02 yes 12:09:13 should i even try it? 12:09:13 slagle: yeah, probably won't work FWIW 12:09:27 i can confirm the "probably" 12:09:37 slagle: Ian was working through some issues still with regards to os-net-config integration 12:09:45 k, got it. 12:09:48 will try w/o 12:09:58 slagle: turns out the various ifup/ifdown scripts have several 'systemd' among other calls 12:10:10 dprince, do you know where he left off? 12:10:13 dprince: rhallisey what is the modprobe ebtables for in start_docker_agents.sh? 12:10:32 and should that be reapplied after a reboot? b/c i dont think it is right now 12:10:33 derekh: could you point me to the fixes that are making HA work in tripleoci? I get dbsync errors for rdo-manager 12:10:39 slagle: that is until we can add that into os-net-config package requirements I think 12:11:01 it's on the atomic host though 12:11:23 that's was an issue Ian and I ran into months ago. 12:11:49 trown, sounds like you forgot 12:11:53 resource_registry: 12:11:53 OS::TripleO::ControllerConfig: /usr/share/openstack-tripleo-heat-templates/puppet/controller-config-pacemaker.yaml 12:12:08 gfidente: let me check, thanks 12:12:26 rhallisey: dprince : ok, was just curious, b/c on reboot, atomic didn't run dhcp 12:12:37 Atomic is missing a bunch of stuff 12:12:56 it still must run dhclient at some point 12:13:20 otherwise it will never get a working network 12:13:28 slagle: yeah, we've still got work and fine tuning to do on this 12:13:30 let me just try w/o network isolation 12:13:56 slagle: I'm still very much in favor of landing the initial patch as something we can iterate on though 12:14:15 dprince: sure. i just want to understand what works and what doesnt 12:14:18 dprince, may need help with net isolation stuff since Ian isn't around much over next few weeks 12:14:44 slagle: w/o network isolation will work. The network is still "wierd" though, I think there are IPs on both the bridge and the nic 12:15:32 i'm in favor of landing too. i just want to know what's landing 12:15:53 gfidente: so HA now requires passing in a custome environment to the deploy? it used to work just by setting control-scale>3 12:16:07 slagle: should just be on the bridge... but it functionally works fine. Ian was working on this also (on Centos) where network manager didn't like os-net-config trying to release the DHCP lease. It got hung up 12:16:12 trown, actually the upstream version of THT always required so 12:17:04 so I suspect that is surfacing in RDO now that we don't have the midstream deloran builds 12:17:40 one thing worth noting about the Docker implementation is that we are using kolla containers that require --privaleged access. Because of this I wouldn't recommend distro mixing as much. What I mean is Fedora containers w/ Fedora atomic... Centos containers w/ centos atomic, etc. 12:17:51 slagle, rhallisey ^^ 12:18:13 trown, the thing is that -pacemaker can do single and HA deployments, while the default one can only do (for now) single controller but does not rely on pacemaker 12:18:46 trown, it seemed more sensible to keep the default to the non-pacemaker version for upstream development 12:19:15 trown, the nonha job in the upstream gate uses the nonpacemaker config as well 12:19:27 dprince: is that what i'll get if i use fedora atomic? 12:20:23 slagle: I can't remember, I'm going to spin this up again too this morning and try it again. 12:20:34 trown, not sure if we can use a flag in the tripleoclient to select one of the implementations regardless of controller count 12:20:35 slagle: I think Fedora works better in this regard 12:20:51 d0ugal, ^^ ? (see last msh to trown) 12:21:16 gfidente: ya, trying to track down what tripleoci is setting for OVERCLOUD_DEPLOY_ARGS for HA 12:21:56 dprince, right now those containers are centos 12:22:14 we can't use centos atomic because neutron-agents doesn't work with that kernel 12:22:32 gfidente: yep, -e /usr/share/openstack-tripleo-heat-templates/environments/puppet-pacemaker.yaml is in there for the HA job, thanks! :) 12:22:44 rhallisey: okay, I spoke w/ Ian I think about switching to Fedora containers (at some point) 12:23:14 trown, exactly but I think we might want to try both -pacemaker for single and non-pacemaker for multiple as well (in the future) 12:23:25 gfidente: I don't know of any flags to do that, but I am not sure how you would do that with parameters either :) 12:23:27 regardless of the controllers count 12:23:39 dprince, I don't think it should matter tbh 12:23:51 d0ugal, we have this environment file environments/puppet-pacemaker.yaml 12:23:57 gfidente: ya makes sense, my main goal is to get rdo-manager liberty CI working :) 12:24:06 ok postponed :) 12:24:19 but then, I'd use -pacemaker for single controller as well FWIW 12:24:31 rhallisey: for some things it does appear to matter. ifdown'ing an interface can call nmcli (network manager cli) which tries to communicate w/ network manager on the atomic host 12:24:51 gfidente: cool, I will give that a try as well 12:24:52 rhallisey: if nmcli (in the container) and the version of network manager don't match it isn't happy 12:25:41 rhallisey: we've just got to rethink the networking, and bootstrapping a bit 12:25:46 gotcha 12:36:15 jistr, it worked! :) 12:36:30 Giulio Fidente proposed openstack/tripleo-heat-templates: Allow customization of Ceph pool names https://review.openstack.org/212664 12:42:29 Giulio Fidente proposed openstack/tripleo-heat-templates: Allow customization of Ceph pool names https://review.openstack.org/212664 12:46:12 Giulio Fidente proposed openstack/tripleo-heat-templates: Allow customization of Ceph client user https://review.openstack.org/212686 12:48:59 gfidente: cool :) yeah i didn't understand what you needed to do, i thought you needed something more complex than that 12:49:39 I'd never pick anything difficult to do! you should know by now :) 12:51:24 dprince: shardy : is the easiest way to pass in a mapping file for os-net-config just to use a firstboot script? 12:51:29 or maybe some cloud-init user-data 12:53:17 slagle: Yeah probably, a firstboot template is just cloud-init user-data 12:53:24 slagle: we could do it that way. I didn't follow the mapping-file thing too closely (I think it was modelled after Spinal stack). I had always invisioned the mapping file would show up early on during provisioning... like an ironic plugin creates it or something 12:53:58 ok 12:54:16 seems like people like to use it, if they know about it 12:54:28 might make sense to expose something at the top level or via parameter_defaults 12:54:47 slagle: I did orignally envisage a firstboot script which mounted the config drive and used the ironic introspection data to create the mapping 12:54:59 only we now no longer inject the instrospection data 12:55:11 shardy: is it in swift? 12:55:13 so, I guess you'd need some other logic in the script to derive the mapping 12:55:27 dprince: yeah, but we don't have a pre-built method to pass that to the instances 12:55:42 shardy: gotcha, what if heat were to grow swift capabilities? 12:55:49 like, we could pass a big map of swift tempurl's and node ID's or something 12:56:02 shardy: sure, a script deployment works too 12:56:16 shardy: yea, i recall that now. i think it's more of people just want to manually map 12:56:22 dprince: Heat already has swift resources 12:56:39 shardy: I thought just for containers, not for objects 12:56:40 dprince: maybe we could use that, but in this case the resources have already been created outside of heat 12:57:03 shardy: yeah, this would be an external resource for sure 12:57:14 dprince: honestly I think all we'd need is a mapping from dmidecode uuid to the swift tempurl where the data is 12:57:19 Giulio Fidente proposed openstack/tripleo-heat-templates: Allow customization of Ceph pool names https://review.openstack.org/212664 12:57:20 Giulio Fidente proposed openstack/tripleo-heat-templates: Allow customization of Ceph client user https://review.openstack.org/212686 12:57:48 https://review.openstack.org/#/c/215116/ 12:57:52 shardy: yeah, I'm not super happy w/ the usability of the dmidecode interface. very tedious 12:58:02 dprince: ^^ it could be passed via NodeDataLookup like that 12:58:18 shardy: yep 12:58:20 dprince: all we need is a unique identifier that's accessible at runtime 12:58:27 it could be anything really 12:58:45 but AFAIK the ironic uuid (which is what the introspection data is keyed by) isn't available to the node 12:58:56 we only get to see the nova uuid, which is different 12:59:02 (I could be wrong, but thats AIUI) 12:59:18 shardy: I think we need dmidecode somewhere, but I'd optionally like to use the ironic name of the instance (if set) 12:59:58 shardy: that could provide a more meaningful name for people who name their machines (like I do) 13:00:00 dprince: Ok, cool, the dmidecode ID was just the first thing I thought of, anything available to the firstboot script on any node wfm :) 13:01:00 dprince, slagle: where did we get to re the defaults patches? 13:01:16 I saw slagle commented on https://review.openstack.org/#/c/232082/, and it seems we mostly agree 13:01:35 I'm trying to decide if I need to rebase all-the-things or just wait for dprince to change his mind ;) 13:02:10 I can start a ML thread if that will help? 13:02:18 shardy: i'm am happy with a default environment instead of defaults in the templates...as long as it's overtly obvious you need to use that default environment 13:02:50 slagle: it sounded from your comments like you were +1 on just aligning the defaults in the templates with the docs? 13:03:01 that's still what I would prefer to do, if I can convince dprince 13:03:05 shardy: oh, yes, i am too 13:03:12 but it sounded like dprince was against that 13:03:22 so i can compromise on an environment :) 13:03:23 slagle: Ok, cool - you didn't vote on the patch so I wasn't sure if we needed to discuss it 13:03:47 alrighty, I'll leave them a bit longer and try to solicit some other opinions 13:03:48 shardy: sorry, i was trying to agree with your patch. i'm +2 on the patch 13:04:03 slagle: ok, excellent :) 13:04:21 we need to offer sane defaults 13:04:22 somewhere 13:05:12 shardy: I still lean towards flavors and images being in an environment. I really like what is becoming of tripleo-docs but I still veiw it heavily as being red hat influenced 13:05:44 shardy: my gut feeling is that perhaps someone else might not like 'baremetal' and 'overcloud-full' as the same defaults 13:06:36 shardy: we may not have that user/company today (besides myself). but should someone come along I really thing environment files work better for these sorts of non-critical defaults 13:07:02 shardy: perhaps non-critical isn't the right work, oppinionated is perhaps better 13:07:54 shardy: Nova doesn't have a default flavor right? Or image? But the nova docs often use m1.small and cirros. I don't see anyone arguing that nova should set cirros be the default image in nova... 13:08:52 shardy: I'm not -2 on these and I will certainly allow it to proceed if everyone else is convinced. My preference would be to use an environment for these settings I think... 13:11:33 does having a default take anything away from someone wanting to use something else? 13:15:34 trown: absolutely not, that's the whole point of the defaults 13:15:53 they are always overridden by either parameter_defaults or parameters (in that order) 13:16:44 shardy: right, so I dont really see the argument that some future users might want a different setting is a valid reason to not set defaults for the majority of current users 13:17:01 err, dprince ^ 13:17:01 dprince: all I'm looking for is a configuration encoded in the templates which allows you to stand up a basic environment without specifying loads of paramerters, or multiple environment files 13:17:17 trown: Yeah, that's basically where I'm coming from 13:17:20 brb, call 13:18:20 gfidente: so passing the pacemaker env, now gets me past all but ceilometer db-sync, whereas before all of them failed. http://fpaste.org/276965/14443965/ ring any bells? 13:19:13 trown: not having defaults doesn't prevent end users from setting these parameters differently, but it does put them at a disadvantage (that likely will only be documented in a footnote somewhere). 13:19:58 trown: say you add a new role, image, and flavor for that role too. The user who customizes may not pick up on that change immidiately 13:20:17 trown: it really comes down to how we deal w/ that case in the docs I guess. 13:20:44 trown: my preference would just be to have everyone clearly set their images and flavors 13:22:24 dprince: dumb question: was there some discussion about tripleoclient options like --neutron-flat-networks being a bad idea? 13:23:11 dprince, ie. things that would be otherwise configurable through the heat template etc. 13:23:13 beagles: I think so. I would like to see tripleoclient ditch all of those sorts of options 13:23:32 beagles: and only be involved in helping the actual deployment flow 13:23:47 +1 to dropping options that could go in environment files 13:24:09 dprince: okay cool.. I'm glad I wasn't imagining it :) 13:26:01 trown, no clue no 13:27:00 shardy: yo, so I've dropped the -1 on the flavors patch. I care less about that one. Still not the biggest fan of changing the image defaults (which are already wrong!). On that patch I'd rather see us just drop all image defaults. 13:28:06 shardy: and do note that I've +2'd already several patches in the latter half of the series :). Perhaps you rebase... we get those in quickly and we postpone the flavors/images patches while others chime in? 13:28:28 dprince: Ok, cool, thanks 13:29:14 dprince: I was just holding off rebasing because I unwisely put a big cosmetic reorg patch in the series so it's not going to reorder cleanly 13:29:22 nvm, I just need another coffee ;) 13:29:25 shardy: also, I still have some issues w/ the last patch in the series, in particular because I think the tripleoclient defaults are wrong in some of those cases 13:29:41 dprince: Ok, cool, that's fine and a good discussion to have 13:30:08 shardy: I'm all for aligning these things, I just prefer to align w/ upstream projects like Cinder, Nova, etc. rather than tripleoclient 13:30:12 dprince: I'm hoping we can work out those sort of issues and stop having defaults in tripleoclient at all 13:30:34 dprince: sure, I'm just aligning with it because I know it works ;) 13:30:50 if they are wrong, we can default to something different and fix tripleoclient :) 13:35:54 shardy: I think the right solution is to just take tripleoclient out of this equation entirely and focus on tripleo-heat-templates, which is exactly where your patch series steps towards. Along with that though (for some of these things) we are going to need a distro specific environment. I think that is the best place for non-default lioadm settings. 13:36:20 and probably several other settings too.... 14:15:52 thrash: any particular reason to not +1 workflow on https://review.openstack.org/#/c/232040/? 14:15:53 John Trowbridge proposed openstack/tripleo-docs: Clean up the basic install instructions https://review.openstack.org/232147 14:15:55 Guys, I had an issue where I had to restart the virtual machines from my tripleo deployment, and some ironic nodes were left on state "deleting" and linked to an instance 14:16:15 I tried running agani stack-delete, and that actually deleted the stacks but the ironic nodes are still in that state 14:16:17 oh, CI wasn't finished then 14:16:20 :) 14:16:28 was waiting for that 14:16:29 any idea how to remove them? 14:18:03 *stack not stacks 14:18:44 jaosorior: you can try sthing like ironic node-update $i remove instance_uuid 14:19:01 jaosorior: ultimately, i have to do that and then delete/recreate nodes etc 14:20:30 jistr: have you had luck deploying ha/pacemaker with your upstream setup? i am about to give inlunch a go (i mean actually try it, not copy/paste some workarounds) 14:20:38 John Trowbridge proposed openstack/tripleo-docs: Clean up the basic install instructions https://review.openstack.org/232147 14:20:52 dprince: is that a hard -1 on https://review.openstack.org/#/c/232105/2 ? 14:20:56 Dan Prince proposed openstack/diskimage-builder: Use ironic-agent for source-repositories https://review.openstack.org/233088 14:21:26 marios: Thanks, trying that out 14:21:43 marios: yes, except i'm very often hitting this now https://bugs.launchpad.net/heat/+bug/1503694 14:21:43 Launchpad bug 1503694 in heat "A bad or out-of-range value was supplied:The Resource (0) could not be found in Stack" [High,Triaged] - Assigned to Steven Hardy (shardy) 14:21:48 Merged openstack/python-tripleoclient: Handle images not being found https://review.openstack.org/232040 14:22:17 jistr: ok thanks for info, will see if i get it 14:22:45 By the way, any worksflows for https://review.openstack.org/#/c/229768/ ? Would be pretty useful for debugging the client 14:23:01 marios: so i have a created stack except one resource, and because of that the overcloud initialization will not run. Otherwise all seems ok. 14:23:38 marios: since the node was left on deleting the node-update returned an error :/ 14:23:44 Node 79325546-c0f4-4cad-9213-5e4d6ec09578 can not be updated while a state transition is in progress. (HTTP 409) 14:24:16 thrash: that was a let me double check some things -1. If you are confident this won't break anyone then go for it I think 14:24:22 jaosorior: could try ironic node-set-provision-state a213b51e-ee11-480e-8b91-4414486fbd8a deleted 14:24:27 jaosorior: at this point... :/ 14:25:09 marios tried, and got the following 14:25:24 The requested action "deleted" can not be performed on node "79325546-c0f4-4cad-9213-5e4d6ec09578" while it is in state "deleting". (HTTP 400) 14:26:06 thrash: looking at the patch again we've got a bit of a double negative going on with this flag 14:26:23 jaosorior: that's all i got. are they powered on? could try setting power off as well 14:26:44 thrash: would --validation-errors-fatal = True make better sense (which is what the commit message says) 14:27:01 jaosorior: ironic node-set-power-state $node off 14:27:50 marios: They were powered off before 14:27:58 :( 14:28:30 dprince: I think he changed that... 14:28:56 dprince: oh, not the commit message 14:28:59 thrash: yeah, I'm actually confused because the commit message subject doesn't match what is happening in this patch 14:29:33 dprince: i was initially confused by patchset 1. There was actually no way to turn it off. :) 14:29:59 thrash, slagle: still confused, why isn't just setting the previous option --validation-errors-fatal == True by default? 14:30:06 okay? 14:30:15 dprince: I think I see what you're saying... Make the defautl True, and the action be 'store_false' 14:30:26 thrash: yep 14:30:46 but then again, that seems to be the same behaviour... 14:30:55 no 14:32:45 dprince: slagle the validation-warnings-fatal should probably also have that same treatment. 14:33:19 thrash: agree, avoid the double negatives if we can 14:37:18 dprince: slagle it makes the check at L959 read more naturally. 14:41:25 jistr: Hey, looking at https://review.openstack.org/#/c/224815/, is there any reason we can't just deploy the pacemaker hieradata always in controller.yaml? 14:44:40 shardy: we can for the bits which will not negatively affect non-pacemaker deployments. However we wanted to start moving more things to hiera. I submitted that patch after discussion with dprince some time ago, as an attempt to refactor the approach of using direct inputs here https://review.openstack.org/#/c/221881/ 14:45:15 shardy: the issue with using direct inputs like that is that they are top priority, so they cannot be overriden by ExtraConfig 14:46:51 jistr: actually they can be overridden by both extraconfig and controller_extraconfig 14:47:04 it's only direct inputs to the deployment which take precedence over them 14:47:25 not the datafiles (which can always be overridden by ExtraConfig) 14:48:02 shardy: these result in a datafile? i thought they were direct inputs https://review.openstack.org/#/c/221881/3/puppet/controller-config-pacemaker.yaml 14:48:12 https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/controller.yaml#L942 14:48:57 shardy: right, but the inputs i linked above ^^ won't go into controller.yaml, no? 14:49:00 jistr: ah, yeah, those are direct inputs 14:49:28 I would suggest that we don't do static mappings in that manner unless they are never to be overridden 14:49:50 is there a reason those aren't in a hieradata file? 14:50:43 looking at dprince's patch, it seems like we could move them from the manifest into a hieradata file, mapped via datafiles, then they could be overridden? 14:50:58 shardy: i think having those hiera keys in controller.yaml would be fine, as long as we can somehow achieve that they are present there only in the pacemaker scenario and not in the non-pacemaker scenario 14:51:39 shardy: as setting the service_manage / enable values to 'false' for non-pacemaker scenario would break the deployment 14:52:03 jistr: ack, Ok, let me think some more about how we could do that 14:55:28 shardy: thanks :) 15:00:16 Untriaged bugs so far: 15:00:17 https://bugs.launchpad.net/tripleo/+bug/1504363 15:00:17 Launchpad bug 1504363 in tripleo "Updates to hiera configuration not propogating/making a puppet run" [Undecided,New] 15:00:18 https://bugs.launchpad.net/tripleo/+bug/1504179 15:00:18 Launchpad bug 1504179 in tripleo "instack-virt-setup can't use dnsmasq lease file anymore" [Undecided,In progress] - Assigned to Thomas Herve (therve) 15:00:20 https://bugs.launchpad.net/tripleo/+bug/1503669 15:00:20 Launchpad bug 1503669 in tripleo "Including an additional element is inconvenient" [Undecided,New] - Assigned to Jaume Devesa (devvesa) 15:00:22 https://bugs.launchpad.net/tripleo/+bug/1504137 15:00:23 Launchpad bug 1504137 in tripleo "Parameter from tripleoclient not passed to Heat template" [Undecided,New] 15:00:24 https://bugs.launchpad.net/tripleo/+bug/1504263 15:00:24 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 15:26:33 * bnemec fixed his squeaky office chair 15:26:42 Half tempted to send that to successbot. ;-) 15:49:25 slagle, bnemec, could you review/approve https://review.openstack.org/#/c/221702/ please? it was lost during migration to tripleo-docs (yes, back then!) 15:51:39 Dmitry Tantsur proposed openstack/tripleo-docs: Disk type for virtual environment must be SATA https://review.openstack.org/233141 16:13:17 dprince: thrash : if we just make --validation-errors-fatal store_true by default, then how do you set it to false? 16:13:32 on the cli, what is the syntax? 16:13:40 slagle: keep it --validation-errors-nonfatal 16:13:56 but True is the default, and action='store_false' 16:14:12 ok 16:14:26 and do the same for --validation-warnings-fatal 16:14:46 then the check makes much more sense. 16:14:48 k, i think i understand. will revisit it 16:14:54 slagle: cool 16:18:07 Merged openstack/tripleo-docs: Add documentation on ironic drivers https://review.openstack.org/221702 16:21:08 Steven Hardy proposed openstack/tripleo-docs: Split node configuration and customization docs https://review.openstack.org/233168 16:21:08 Steven Hardy proposed openstack/tripleo-docs: Add docs for per-node (pre-deploy) config interfaces https://review.openstack.org/233169 16:21:24 dsneddon: ^^ The ExtraConfig docs updates I promised 16:43:37 I'm trying to build overcloud images and I'm encountering a weird error 16:43:47 Using "openstack overcloud image build --type discovery-ramdisk" 16:44:47 It fails looking for a "ironic-discoverd-ramdisk" binary 16:45:07 That dep seems to come from ironic-discoverd-ramdisk-instack 16:45:33 But indeed I don't see this binary anywhere (something replaced by ironic-inspector?) 16:50:42 Merged openstack/tripleo-docs: Updates to network documentation for new features https://review.openstack.org/221908 16:51:36 therve: what docs are you following... the ironic-discoverd-ramdisk has been replaced by ironic-python-agent 16:52:02 trown, http://docs.openstack.org/developer/tripleo-docs/basic_deployment/basic_deployment_cli.html 16:52:21 I'm not referencing discoverd directly, something under the hood seems to be 16:52:52 therve: hmm those are the correct docs 16:53:13 therve: where do you see the --type command? 16:53:29 trown, I run "openstack overcloud image build --all" 16:53:36 But didn't get all the necessary bits 16:53:49 So I tried to see what was missing but I didn't get any obvious error message 16:53:59 Thus the --type discovery-ramdisk 16:54:28 ah, therve did you get an ironic-python-agent.kernel? 16:54:47 trown, No 16:54:56 therve: I think this might help https://review.openstack.org/#/c/232147/ 16:55:43 That looks relevant :) 16:56:03 trown, Doesn't tripleoclient requires an update though? 16:56:47 therve: no, all the bits are there, it is just a doc patch 16:57:19 that DIB_YUM_REPO_CONF line is why you did not get an agent ramdisk 16:57:49 Hum 16:58:04 therve: the comments are about how this is really a workaround of a bug in tripleoclient, but it is a workaround we are using elsewhere too 16:58:18 trown, https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/v1/overcloud_image.py#L507 I'm refering to this 16:58:39 The agent-ramdisk should be in here AFAIU 16:58:45 therve: look at 509 16:59:04 Yeah? 16:59:15 Oh ramdisk_agent, hum 16:59:26 It doesn't build discovery_ramdisk indeed 17:00:37 Thanks for the pointer :). So I guess if I run --type agent-ramdisk I ought to get a different failure fixed by your DIB_YUM_REPO_CONF ? 17:01:09 therve: yep 17:01:18 trown, Awesome, thanks a ton 17:01:22 np 17:06:54 EmilienM: ping 17:07:01 egafford: bonjour 17:07:10 sorry, I miss speaking french sometimes. 17:07:21 EmilienM: Why hello! I found an actual piece of work, which is almost certainly real. 17:07:37 It'd be great to update ::trove::api to use auth_uri and identity_uri 17:07:50 indeed 17:08:08 Atm it's composing keystone credentials from fragments, and our heat templates don't even have a standalone auth_host (nor should they.) 17:08:15 dprince did all the work for the other modules, I'll look at how he did 17:08:51 EmilienM: Cool. I can do it as well; I'm just concerned that I'd end up with a different deprecation flow. 17:09:19 I have trove responding to calls in my overcloud, though, so things are looking saneish. 17:10:36 EmilienM: Is that a task you/your team can pick up (and if so, any incredibly vague sense of timeline?) 17:11:03 egafford: you can create a card in our backlog: https://trello.com/b/2uRnGtZ1/mgmt-puppet 17:15:49 trown: hi, a really minor style suggestion here and then I'm +2 https://review.openstack.org/#/c/228190/5 17:16:15 trown: your puppet-ironic changes is in now BTW so really close on this... 17:16:25 dprince: I know, thanks for landing that 17:17:07 dprince: I will put a patch up with your suggestion 17:17:37 dprince: I am also fine with only enabling inspection on the non-ha job... I dont really see that it would fail differently on the different jobs 17:18:29 has anyone seen issues with ceilometer when deploying HA? 17:18:59 for me it is failing on ceilometer dbsyc, and when I go onto a controller and manually run it I see why: http://fpaste.org/277168/44441093/ 17:19:46 trown: re, the ci job. I agree. I don't think HA would affect inspection so saving the wall time (and skipping inspection) on the HA and Ceph jobs would be ideal 17:20:05 trown: we are already getting occasional pings from infra about how long our jobs are running 17:20:15 ya, less to troubleshoot when they fail too 17:39:50 John Trowbridge proposed openstack/instack-undercloud: Use puppet to configure ironic-inspector https://review.openstack.org/228190 18:12:03 Dan Sneddon proposed openstack/os-net-config: Fix kwarg assignment in objects.py https://review.openstack.org/231247 18:22:50 Dan Sneddon proposed openstack/os-net-config: Add sample for Linux bridge, fix Linux bond sample https://review.openstack.org/233208 18:27:21 Does anyone else frequently see their heat engine go down on deleting stacks in their undercloud? 18:28:07 It may be a peculiarity of my dev env, but it's reasonably frequent, and reproducible between iterations from instack_virt_setup down. 18:29:10 Lokesh Jain proposed openstack/tripleo-heat-templates: Added libvirt_vif_driver, ovs_bridge and security_group_api parameters https://review.openstack.org/229649 18:47:33 hello, is there some method how to make overcloud deploy a node with software mirrored disk system ? 19:07:25 dprince: I +2'd https://review.openstack.org/#/c/209505 19:07:46 shall I merge? do you want to do the honor? 19:08:40 slagle: sure, thanks :) 19:09:03 Merged openstack/tripleo-heat-templates: Docker compute role configured via Puppet https://review.openstack.org/209505 19:09:06 I do not think this happens often: https://review.openstack.org/#/c/228190/ 19:09:12 rhallisey, jpeeler, Slower_ : t-h-t gets docker \o/ ^^ 19:09:13 passed HA and failed the other 2 19:09:18 BOOM 19:09:27 yay! 19:09:39 nice work on that, a long time coming... 19:11:45 EmilienM: Do puppet bugs live in launchpad/tripleo, or somewhere more specific? 19:12:03 oh nice! 19:12:18 The puppet wiki page seemed only to have links to overall bug flow. 19:12:41 Ah, got it. 19:20:36 Ethan Gafford proposed openstack/os-cloud-config: Sahara Integration https://review.openstack.org/220860 19:20:36 Ethan Gafford proposed openstack/os-cloud-config: Trove Integration https://review.openstack.org/233230 19:24:15 Ethan Gafford proposed openstack/puppet-tripleo: Sahara integration https://review.openstack.org/220859 19:24:16 Ethan Gafford proposed openstack/puppet-tripleo: Trove integration https://review.openstack.org/233232 19:29:40 FYI the CentOS deps repos are down ... so all CI is going to fail 19:30:05 trown: ouch, any idea when they are coming back? 19:30:06 Ethan Gafford proposed openstack/tripleo-puppet-elements: Sahara integration https://review.openstack.org/220861 19:30:06 Ethan Gafford proposed openstack/tripleo-puppet-elements: [WIP] Trove Integration https://review.openstack.org/233236 19:30:11 hah, i was just wondering what was going on with that 19:30:32 i rechecked something, then realized that the link was 404'ing for me too 19:30:49 dprince: we are contacting centos-devel 19:35:25 Ethan Gafford proposed openstack/tripleo-heat-templates: Sahara Integration https://review.openstack.org/220863 19:35:26 Ethan Gafford proposed openstack/tripleo-heat-templates: [WIP] Trove Integration https://review.openstack.org/233240 19:37:06 Ethan Gafford proposed openstack/python-tripleoclient: Sahara integration https://review.openstack.org/221418 19:37:07 Ethan Gafford proposed openstack/python-tripleoclient: [WIP] Trove integration https://review.openstack.org/233241 19:46:44 dprince: slagle, centos repo should be back 19:47:07 trown: nice 20:11:20 Ben Nemec proposed openstack/instack-undercloud: Nothing to see here https://review.openstack.org/232700 20:28:20 trown: i'm still getting 404 for http://cbs.centos.org/repos/cloud7-openstack-common-testing/x86_64/os/repodata/repomd.xml 20:28:25 should that work? 20:28:34 slagle: they went down again :( 20:28:43 probably b/c i tried it again! 20:28:54 no one use it, it will stay up 20:32:26 pinging in centos-devel again 20:43:20 slagle: dprince, repos are back 20:43:34 maybe they will stay that way until I leave for the weekend 20:48:20 i'm afraid to try 20:48:32 it works! 20:51:21 dprince: To be clear, I stand reasonably well behind the Sahara patches, but the Trove ones are significantly more WIP (the service responds like a boss, but I need to get the Trove team to check my work in a significant way.) 20:51:34 Thanks for the reviews though; much appreciated. :) 20:52:02 egafford: I'm going to have a closer look at the t-h-t one soon too 20:52:12 dprince: Cool! 20:52:26 What was the eventual decision about methodology for disabling services? 20:52:27 egafford: but the os-cloud-config ones are much simpler 20:52:32 dprince: ...yes. 20:52:53 egafford: I'd like to tinker w/ it a bit 20:53:32 os-cloud-config and tripleo-puppet-elements are nice Friday afternoon review fodder, for sure. puppet-tripleo too really. 20:53:37 egafford: we want something that scales better for sure, but we can press ahead w/ what we've got 20:54:09 dprince: Yeah: onward, upward, all that. 20:54:57 slagle: You might give this a glance https://review.openstack.org/#/c/228190/ 20:55:08 slagle: once CI passes I'd like to get it in 20:55:47 slagle: if jistr was still online I'd ping him... but you've got at least 5 minutes of work left in you I'm sure of it 20:59:18 dprince: fwiw the CI that is failing there is not even excercising that code 20:59:25 since inspector is not enabled 20:59:26 trown|outttypeww: oh you're still here :) 20:59:30 trown|outttypeww: for teh docs patch... 20:59:34 lol 20:59:36 https://review.openstack.org/#/c/232147/3 20:59:44 tripleo.sh is still enabling epel 20:59:54 but did you test without out it? 21:00:02 trown|outttypeww: trying to slow down your own patch are you? 21:00:16 Untriaged bugs so far: 21:00:17 https://bugs.launchpad.net/tripleo/+bug/1504137 21:00:17 Launchpad bug 1504137 in tripleo "Parameter from tripleoclient not passed to Heat template" [Undecided,New] 21:00:18 https://bugs.launchpad.net/diskimage-builder/+bug/1504632 21:00:18 Launchpad bug 1504632 in diskimage-builder "elements README documentation environment variables should be clearer" [Undecided,In progress] - Assigned to Augustina Ragwitz (missaugustina) 21:00:21 https://bugs.launchpad.net/tripleo/+bug/1504179 21:00:22 https://bugs.launchpad.net/tripleo/+bug/1503669 21:00:22 Launchpad bug 1504179 in tripleo "instack-virt-setup can't use dnsmasq lease file anymore" [Undecided,In progress] - Assigned to Thomas Herve (therve) 21:00:23 Launchpad bug 1503669 in tripleo "Including an additional element is inconvenient" [Undecided,New] - Assigned to Jaume Devesa (devvesa) 21:00:24 https://bugs.launchpad.net/tripleo/+bug/1504263 21:00:25 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 21:00:27 https://bugs.launchpad.net/tripleo/+bug/1504363 21:00:27 Launchpad bug 1504363 in tripleo "Updates to hiera configuration not propogating/making a puppet run" [Undecided,New] 21:00:53 slagle: it is on my list to remove epel as a req, but it is still needed... I just moved it around to make the update doc use the same repositories.txt 21:00:54 https://review.openstack.org/#/c/232147/3/doc/source/installation/installing.rst 21:01:18 ah, i see 21:01:25 i just noticed the removal from repositories.txt 21:01:28 ok, sounds good 21:05:00 Merged openstack/tripleo-docs: Clean up the basic install instructions https://review.openstack.org/232147 21:06:59 I am trying to install undercloud on Virtual Environment setup using the latest ci . However I am facing this error again and again : 21:06:59 "Error: Could not find resource 'Keystone_domain[heat_domain]' for relationship from 'Class[Keystone::Roles::Admin]' on node instack" in the file /etc/puppet/manifests/puppet-stack-config.pp:295 21:06:59 Any idea regarding this? 23:50:43 Merged openstack/instack-undercloud: Use puppet to configure ironic-inspector https://review.openstack.org/228190 00:22:25 Merged openstack/diskimage-builder: Move yum-based install into function https://review.openstack.org/227124 03:00:15 Untriaged bugs so far: 03:00:16 https://bugs.launchpad.net/tripleo/+bug/1504137 03:00:16 Launchpad bug 1504137 in tripleo "Parameter from tripleoclient not passed to Heat template" [Undecided,New] 03:00:18 https://bugs.launchpad.net/diskimage-builder/+bug/1504632 03:00:18 Launchpad bug 1504632 in diskimage-builder "elements README documentation environment variables should be clearer" [Undecided,In progress] - Assigned to Augustina Ragwitz (missaugustina) 03:00:20 https://bugs.launchpad.net/tripleo/+bug/1504179 03:00:20 Launchpad bug 1504179 in tripleo "instack-virt-setup can't use dnsmasq lease file anymore" [Undecided,In progress] - Assigned to Thomas Herve (therve) 03:00:22 https://bugs.launchpad.net/tripleo/+bug/1503669 03:00:23 Launchpad bug 1503669 in tripleo "Including an additional element is inconvenient" [Undecided,New] - Assigned to Jaume Devesa (devvesa) 03:00:24 https://bugs.launchpad.net/tripleo/+bug/1504263 03:00:24 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 03:00:26 https://bugs.launchpad.net/tripleo/+bug/1504363 03:00:27 Launchpad bug 1504363 in tripleo "Updates to hiera configuration not propogating/making a puppet run" [Undecided,New] 09:00:16 Untriaged bugs so far: 09:00:17 https://bugs.launchpad.net/tripleo/+bug/1504137 09:00:17 Launchpad bug 1504137 in tripleo "Parameter from tripleoclient not passed to Heat template" [Undecided,New] 09:00:19 https://bugs.launchpad.net/diskimage-builder/+bug/1504632 09:00:19 Launchpad bug 1504632 in diskimage-builder "elements README documentation environment variables should be clearer" [Undecided,In progress] - Assigned to Augustina Ragwitz (missaugustina) 09:00:21 https://bugs.launchpad.net/tripleo/+bug/1504179 09:00:21 Launchpad bug 1504179 in tripleo "instack-virt-setup can't use dnsmasq lease file anymore" [Undecided,In progress] - Assigned to Thomas Herve (therve) 09:00:23 https://bugs.launchpad.net/tripleo/+bug/1503669 09:00:24 Launchpad bug 1503669 in tripleo "Including an additional element is inconvenient" [Undecided,New] - Assigned to Jaume Devesa (devvesa) 09:00:24 https://bugs.launchpad.net/tripleo/+bug/1504263 09:00:25 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 09:00:26 https://bugs.launchpad.net/tripleo/+bug/1504363 09:00:27 Launchpad bug 1504363 in tripleo "Updates to hiera configuration not propogating/making a puppet run" [Undecided,New] 15:00:16 Untriaged bugs so far: 15:00:17 https://bugs.launchpad.net/tripleo/+bug/1504137 15:00:17 Launchpad bug 1504137 in tripleo "Parameter from tripleoclient not passed to Heat template" [Undecided,New] 15:00:19 https://bugs.launchpad.net/diskimage-builder/+bug/1504632 15:00:19 Launchpad bug 1504632 in diskimage-builder "elements README documentation environment variables should be clearer" [Undecided,In progress] - Assigned to Augustina Ragwitz (missaugustina) 15:00:21 https://bugs.launchpad.net/tripleo/+bug/1504179 15:00:21 Launchpad bug 1504179 in tripleo "instack-virt-setup can't use dnsmasq lease file anymore" [Undecided,In progress] - Assigned to Thomas Herve (therve) 15:00:23 https://bugs.launchpad.net/tripleo/+bug/1503669 15:00:24 Launchpad bug 1503669 in tripleo "Including an additional element is inconvenient" [Undecided,New] - Assigned to Jaume Devesa (devvesa) 15:00:24 https://bugs.launchpad.net/tripleo/+bug/1504263 15:00:25 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 15:00:26 https://bugs.launchpad.net/tripleo/+bug/1504363 15:00:26 Launchpad bug 1504363 in tripleo "Updates to hiera configuration not propogating/making a puppet run" [Undecided,New] 21:00:15 Untriaged bugs so far: 21:00:16 https://bugs.launchpad.net/tripleo/+bug/1504137 21:00:16 Launchpad bug 1504137 in tripleo "Parameter from tripleoclient not passed to Heat template" [Undecided,New] 21:00:18 https://bugs.launchpad.net/diskimage-builder/+bug/1504632 21:00:18 Launchpad bug 1504632 in diskimage-builder "elements README documentation environment variables should be clearer" [Undecided,In progress] - Assigned to Augustina Ragwitz (missaugustina) 21:00:20 https://bugs.launchpad.net/tripleo/+bug/1504179 21:00:21 Launchpad bug 1504179 in tripleo "instack-virt-setup can't use dnsmasq lease file anymore" [Undecided,In progress] - Assigned to Thomas Herve (therve) 21:00:22 https://bugs.launchpad.net/tripleo/+bug/1503669 21:00:22 Launchpad bug 1503669 in tripleo "Including an additional element is inconvenient" [Undecided,New] - Assigned to Jaume Devesa (devvesa) 21:00:24 https://bugs.launchpad.net/tripleo/+bug/1504263 21:00:25 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 21:00:26 https://bugs.launchpad.net/tripleo/+bug/1504363 21:00:26 Launchpad bug 1504363 in tripleo "Updates to hiera configuration not propogating/making a puppet run" [Undecided,New] 03:00:17 Untriaged bugs so far: 03:00:19 https://bugs.launchpad.net/tripleo/+bug/1504137 03:00:20 https://bugs.launchpad.net/diskimage-builder/+bug/1504632 03:00:20 Launchpad bug 1504137 in tripleo "Parameter from tripleoclient not passed to Heat template" [Undecided,New] 03:00:21 https://bugs.launchpad.net/tripleo/+bug/1504179 03:00:21 Launchpad bug 1504632 in diskimage-builder "elements README documentation environment variables should be clearer" [Undecided,In progress] - Assigned to Augustina Ragwitz (missaugustina) 03:00:22 Launchpad bug 1504179 in tripleo "instack-virt-setup can't use dnsmasq lease file anymore" [Undecided,In progress] - Assigned to Thomas Herve (therve) 03:00:23 https://bugs.launchpad.net/tripleo/+bug/1503669 03:00:24 Launchpad bug 1503669 in tripleo "Including an additional element is inconvenient" [Undecided,New] - Assigned to Jaume Devesa (devvesa) 03:00:25 https://bugs.launchpad.net/tripleo/+bug/1504263 03:00:25 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 03:00:27 https://bugs.launchpad.net/tripleo/+bug/1504363 03:00:28 Launchpad bug 1504363 in tripleo "Updates to hiera configuration not propogating/making a puppet run" [Undecided,New] 09:00:17 Untriaged bugs so far: 09:00:18 https://bugs.launchpad.net/tripleo/+bug/1504137 09:00:18 Launchpad bug 1504137 in tripleo "Parameter from tripleoclient not passed to Heat template" [Undecided,New] 09:00:20 https://bugs.launchpad.net/diskimage-builder/+bug/1504632 09:00:20 Launchpad bug 1504632 in diskimage-builder "elements README documentation environment variables should be clearer" [Undecided,In progress] - Assigned to Augustina Ragwitz (missaugustina) 09:00:22 https://bugs.launchpad.net/tripleo/+bug/1504179 09:00:22 Launchpad bug 1504179 in tripleo "instack-virt-setup can't use dnsmasq lease file anymore" [Undecided,In progress] - Assigned to Thomas Herve (therve) 09:00:24 https://bugs.launchpad.net/tripleo/+bug/1503669 09:00:24 Launchpad bug 1503669 in tripleo "Including an additional element is inconvenient" [Undecided,New] - Assigned to Jaume Devesa (devvesa) 09:00:26 https://bugs.launchpad.net/tripleo/+bug/1504263 09:00:27 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 09:00:28 https://bugs.launchpad.net/tripleo/+bug/1504363 09:00:28 Launchpad bug 1504363 in tripleo "Updates to hiera configuration not propogating/making a puppet run" [Undecided,New] 11:48:29 I remember deva telling me about a deployment tool I can use for tripleo without having to deploy a full undercloud first 11:48:34 but I don't remember its name 12:10:49 Bharat Kumar Kobagana (BharatK) proposed openstack/tripleo-heat-templates: Add NetApp integration to Manila https://review.openstack.org/188138 13:27:48 Dan Prince proposed openstack/python-tripleoclient: Use ironic-agent for deployment and introspection https://review.openstack.org/233449 15:00:15 Untriaged bugs so far: 15:00:17 https://bugs.launchpad.net/tripleo/+bug/1504137 15:00:17 Launchpad bug 1504137 in tripleo "Parameter from tripleoclient not passed to Heat template" [Undecided,New] 15:00:18 https://bugs.launchpad.net/diskimage-builder/+bug/1504632 15:00:18 Launchpad bug 1504632 in diskimage-builder "elements README documentation environment variables should be clearer" [Undecided,In progress] - Assigned to Augustina Ragwitz (missaugustina) 15:00:20 https://bugs.launchpad.net/tripleo/+bug/1504179 15:00:20 Launchpad bug 1504179 in tripleo "instack-virt-setup can't use dnsmasq lease file anymore" [Undecided,In progress] - Assigned to Thomas Herve (therve) 15:00:22 https://bugs.launchpad.net/tripleo/+bug/1503669 15:00:22 Launchpad bug 1503669 in tripleo "Including an additional element is inconvenient" [Undecided,New] - Assigned to Jaume Devesa (devvesa) 15:00:24 https://bugs.launchpad.net/tripleo/+bug/1504263 15:00:25 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 15:00:26 https://bugs.launchpad.net/tripleo/+bug/1504363 15:00:26 Launchpad bug 1504363 in tripleo "Updates to hiera configuration not propogating/making a puppet run" [Undecided,New] 21:00:17 Untriaged bugs so far: 21:00:19 https://bugs.launchpad.net/tripleo/+bug/1504137 21:00:19 Launchpad bug 1504137 in tripleo "Parameter from tripleoclient not passed to Heat template" [Undecided,New] 21:00:21 https://bugs.launchpad.net/diskimage-builder/+bug/1504632 21:00:22 Launchpad bug 1504632 in diskimage-builder "elements README documentation environment variables should be clearer" [Undecided,In progress] - Assigned to Augustina Ragwitz (missaugustina) 21:00:22 https://bugs.launchpad.net/tripleo/+bug/1504179 21:00:23 Launchpad bug 1504179 in tripleo "instack-virt-setup can't use dnsmasq lease file anymore" [Undecided,In progress] - Assigned to Thomas Herve (therve) 21:00:25 https://bugs.launchpad.net/tripleo/+bug/1503669 21:00:25 Launchpad bug 1503669 in tripleo "Including an additional element is inconvenient" [Undecided,New] - Assigned to Jaume Devesa (devvesa) 21:00:26 https://bugs.launchpad.net/tripleo/+bug/1504263 21:00:27 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 21:00:28 https://bugs.launchpad.net/tripleo/+bug/1504363 21:00:28 Launchpad bug 1504363 in tripleo "Updates to hiera configuration not propogating/making a puppet run" [Undecided,New] 22:11:36 can someone point me in the correct location to change the fallback IP assignment? Currently things fall back to the provisioning network, but I'd like everything to fall back to the internal_api network. 01:46:28 Graeme Gillies proposed openstack/tripleo-docs: Add documentation on how to backup and restore the Undercloud https://review.openstack.org/233468 02:10:40 Graeme Gillies proposed openstack/os-cloud-config: Add wol ironic driver support https://review.openstack.org/213956 02:45:51 Ian Wienand proposed openstack/diskimage-builder: Remove extra install of release pkgs in fedora-minimal https://review.openstack.org/227131 02:45:52 Ian Wienand proposed openstack/diskimage-builder: Reset yum/dnf cache to correct location https://review.openstack.org/232863 02:45:53 Ian Wienand proposed openstack/diskimage-builder: Add basic Fedora 22 support to yum-minimal https://review.openstack.org/227125 02:45:54 Ian Wienand proposed openstack/diskimage-builder: Cleanup the fedora-minimal yum/dnf path https://review.openstack.org/227127 02:45:55 Ian Wienand proposed openstack/diskimage-builder: Update default fedora-minimal to f22 https://review.openstack.org/227126 03:00:16 Untriaged bugs so far: 03:00:17 https://bugs.launchpad.net/tripleo/+bug/1504137 03:00:17 Launchpad bug 1504137 in tripleo "Parameter from tripleoclient not passed to Heat template" [Undecided,New] 03:00:19 https://bugs.launchpad.net/diskimage-builder/+bug/1504632 03:00:19 Launchpad bug 1504632 in diskimage-builder "elements README documentation environment variables should be clearer" [Undecided,In progress] - Assigned to Augustina Ragwitz (missaugustina) 03:00:21 https://bugs.launchpad.net/tripleo/+bug/1504179 03:00:21 Launchpad bug 1504179 in tripleo "instack-virt-setup can't use dnsmasq lease file anymore" [Undecided,In progress] - Assigned to Thomas Herve (therve) 03:00:24 https://bugs.launchpad.net/tripleo/+bug/1503669 03:00:25 https://bugs.launchpad.net/tripleo/+bug/1504263 03:00:25 Launchpad bug 1503669 in tripleo "Including an additional element is inconvenient" [Undecided,New] - Assigned to Jaume Devesa (devvesa) 03:00:26 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 03:00:27 https://bugs.launchpad.net/tripleo/+bug/1504363 03:00:27 Launchpad bug 1504363 in tripleo "Updates to hiera configuration not propogating/making a puppet run" [Undecided,New] 06:57:26 Giulio Fidente proposed openstack/tripleo-heat-templates: Ensure access to RabbitMQ is password protected https://review.openstack.org/210402 07:10:22 what's the way to set the CloudName parameter from the templates? I've tried to set it from an environment file, but it seems it didn't go through. I suspect it's because it's one of the parameters coming from the tripleoclient; so it might not be passing through 07:18:27 parameters: 07:18:27 CloudName: mycloud 07:18:43 with a -e /path/to/mylfie option to tripleoclient 07:18:49 that should work jaosorior ^ 07:18:59 spredzy: That's what I had done 07:20:02 jaosorior, ack. I am rebuilding my platform atm, I'll try when I have it backup 07:20:12 spredzy: Thanks, I would appreciate that 07:20:33 but yeah, If I understood correctly, that way it should set the fqdn for the overcloud nodes, right? 07:27:54 Yanis Guenane proposed openstack/tripleo-heat-templates: Allow one to specify horizon ALLOWED_HOSTS https://review.openstack.org/221255 07:31:15 Yanis Guenane proposed openstack/tripleo-heat-templates: Add more components virtual ip mapping into controller. https://review.openstack.org/231361 07:37:27 hi 07:38:01 after running "openstack undercloud install" command on the instack machine 07:38:04 i got this error 07:38:22 Error: Could not find class ::ironic::inspector for instack on node instack 07:38:23 Error: Could not find class ::ironic::inspector for instack on node instack 07:40:12 any ideas? 07:45:14 Morning all 07:45:28 https://review.openstack.org/#/c/227226/2 is all green and needs a final +A if anyone has a moment 07:46:24 shardy: Got a moment to check this one out https://review.openstack.org/#/c/229768/ ? Only missing +A, would be really useful for debugging the client 07:49:09 jaosorior: nice, thanks! :) 07:50:02 Did anyone had this issue : Parse error: Invalid string: control characters from U+0000 through U+001F must be escaped at line 32, column 30 (while running instack-virt-setup) ? 07:50:03 shardy: Thanks Mr. :) 07:53:42 Merged openstack/python-tripleoclient: Add debug environment to tox https://review.openstack.org/229768 07:54:25 spredzy: i saw this when i had mixed up/downstream repos (there is also a BZ for it, is because of jq https://bugzilla.redhat.com/show_bug.cgi?id=1265403#c3 07:54:25 bugzilla.redhat.com bug 1265403 in instack-undercloud "instack-undercloud fails to setup seed vm, parse error while creating ssh key" [High,Closed: notabug] - Assigned to jslagle 07:55:34 marios, thanks for the link. Unfortunately if I didn't enable EPEL it complained also. Will look into it 07:56:34 EPEL repository is required to install python-pip for CentOS. 07:57:39 Merged openstack/os-net-config: Add support for Linux Bonding to os-net-config ifcfg https://review.openstack.org/218134 08:03:00 Merged openstack/tripleo-heat-templates: Ensure httpd is not enabled by puppet on system boot https://review.openstack.org/206463 08:04:03 spredzy: np - just looked familiar as was relatively recently i hit that :) 08:05:07 Merged openstack/tripleo-heat-templates: Move RHEL (un)registration to NodeExtraConfig https://review.openstack.org/227226 08:08:06 spredzy: I think it's not getting the parameter I gave it because it's getting the default parameters that are specified in the tripleoclient. CloudName is set as overcloud and that's what it's getting regardless of what I give it in a separate environment file 08:10:28 Merged openstack/tripleo-heat-templates: Allow enabling debug mode for config management (Puppet) https://review.openstack.org/202465 08:10:29 jaosorior, if you try directly with heat client does it take the paramter ? 08:11:12 spredzy: I'll try that out. Been trying only with tripleoclient 08:12:51 jaosorior: are you using parameters or parameter_defaults in your environment file? 08:13:20 jaosorior: "parameters" should overwrite any hard-coded tripleoclient value, otherwise we still have bugs 08:13:45 (I previously raised a downstream bug about rdomanager-oscplugin due to this very issue) 08:14:16 shardy, marios gfidente Can I have reviews on those two please ? https://review.openstack.org/#/c/221255/ https://review.openstack.org/#/c/231361/ 08:14:24 it's another reason why we really want all those hard-coded defaults to die, things will work much more intuitively if we maintain them in the heat templates, or maybe an environment file in tripleo-heat-templates 08:15:00 +1 08:15:35 shardy: would it be out of the ordinary here to open a bug for hard coded defaults ? 08:17:16 cmyster: I already raised https://bugzilla.redhat.com/show_bug.cgi?id=1245737 downstream 08:17:16 bugzilla.redhat.com bug 1245737 in python-rdomanager-oscplugin "hard-coded parameters can't be overridden" [Urgent,New] - Assigned to jason.dobies 08:17:30 if tripleoclient still does it, we need an upstream bug 08:18:07 cmyster: FWIW I have a whole series up for t-h-t which fixes the defaults as a precursor to ripping them out of the client 08:18:33 unfortunately that has met more resistance than I expected, but when that's resolved hopefully we can remove the client-side defaults for good 08:19:43 shardy: rgr 08:23:59 d0ugal: Hey, any idea what happened to "openstack overcloud validate" ? 08:24:13 AFAICS it's not in the tripleoclient code, but it's mentioned in the docs? 08:24:34 shardy: hmm, odd. 08:24:49 Maybe I'm looking in the wrong place, or just need more coffee ;) 08:25:14 shardy: Checking. 08:30:08 shardy_: https://github.com/openstack/python-tripleoclient/commit/a8c069c6e794f0846bafa348defe48500701da56 08:31:06 shardy_: So I remember it being changed so that it creates the tempest config only, I didn't notice that it got merged into the deploy command 08:31:16 so on every deploy it creates the tempest config it seems 08:56:09 shardy: I'm using parameters not parameter_defaults. But it's from downstream 09:00:15 Untriaged bugs so far: 09:00:17 https://bugs.launchpad.net/tripleo/+bug/1504137 09:00:17 Launchpad bug 1504137 in tripleo "Parameter from tripleoclient not passed to Heat template" [Undecided,New] 09:00:18 https://bugs.launchpad.net/diskimage-builder/+bug/1504632 09:00:18 Launchpad bug 1504632 in diskimage-builder "elements README documentation environment variables should be clearer" [Undecided,In progress] - Assigned to Augustina Ragwitz (missaugustina) 09:00:20 https://bugs.launchpad.net/tripleo/+bug/1504179 09:00:20 Launchpad bug 1504179 in tripleo "instack-virt-setup can't use dnsmasq lease file anymore" [Undecided,In progress] - Assigned to Thomas Herve (therve) 09:00:22 https://bugs.launchpad.net/tripleo/+bug/1503669 09:00:22 Launchpad bug 1503669 in tripleo "Including an additional element is inconvenient" [Undecided,New] - Assigned to Jaume Devesa (devvesa) 09:00:24 https://bugs.launchpad.net/tripleo/+bug/1504263 09:00:25 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 09:00:26 https://bugs.launchpad.net/tripleo/+bug/1505129 09:00:26 Launchpad bug 1505129 in tripleo "Docs describe non-existent validation options" [Undecided,New] 09:00:28 https://bugs.launchpad.net/tripleo/+bug/1504363 09:00:28 Launchpad bug 1504363 in tripleo "Updates to hiera configuration not propogating/making a puppet run" [Undecided,Confirmed] 09:12:26 Steven Hardy proposed openstack/tripleo-docs: Add note that qemu libvirt-type is needed for virt environment https://review.openstack.org/233535 09:29:26 derekh: Hey, I was wondering where we are wrt moving from the docker to mock based delorean for tripleo CI/.sh? 09:30:07 derekh: I'm looking to do a docs patch showing how the tripleo.sh works for building on-review changes, but I assume some of the initial steps will change when we move from docker 09:39:51 shardy: we're close. I'm not quite sure why the ci switchover didn't work https://review.openstack.org/#/c/229425/ 09:40:16 shardy: I'm yting to spin up and CI env manually to reproduce 09:40:21 ramishra. d0ugal: Turns out there's already a docs patch up: https://review.openstack.org/#/c/221664/ 09:40:33 shardy: but have hit a timeout problem on our CI cloud and have to figure that out first 09:40:34 oh, neat 09:40:37 * d0ugal reads 09:42:22 derekh: Ok, cool, I'll hold off documenting anything yet then, thanks! 09:42:26 shardy: Looks pretty good other than trown|outttypeww's comments 09:42:59 d0ugal: Cool, I've assigned the bug to ifarkas so we can hopefully get that updated/landed 09:43:36 shardy: the 2 patches up for tripleo.sh should be extremely close (if not complete) 09:44:30 derekh: Yeah, it was just stuff like having to mess with the docker group, which I assume won't be needed when we move to mock 09:45:55 shardy: the switch from docker to mock is done in PS11 https://review.openstack.org/#/c/228156/11 09:46:11 shardy: the docker stuff never merged so I've switched it to mock 09:48:19 derekh: Oh, thanks, I hadn't spotted that and was still using the old docker version 09:49:01 Marios Andreou proposed openstack/tripleo-image-elements: Grab passthrough horizon.allowed_hosts https://review.openstack.org/233549 09:49:10 I've been running out of memory on the undercloud building stuff there, so was going to refresh some steps to run builds on the host and export them to the undercloud as a local repo 09:49:42 shardy: sounds good 09:54:08 Marios Andreou proposed openstack/tripleo-image-elements: Grab passthrough horizon.allowed_hosts https://review.openstack.org/233549 09:56:53 Yanis Guenane proposed openstack/tripleo-heat-templates: Allow one to specify horizon ALLOWED_HOSTS https://review.openstack.org/221255 10:32:22 derekh: Hmm, one disadvantage of the mock based approach is it appears to use a fixed --installroot 10:32:46 I'm running short on /var space and I can't see an easy way to redirect the (large) mock output to e.g $HOME 10:33:01 * shardy starts grepping delorean tree 10:34:43 shardy: not sure either off the top of my head, but ya there are probably places where we can make improvements. 10:35:50 shardy: What do you want to do? 10:37:47 jaosorior: I want mock, driven via delorean, to use a different build root instead of /var/lib/mock 10:38:16 obviously I can symlink it, but I was looking for a way to easily influence it via a config option 10:40:28 Jan Provaznik proposed openstack/tripleo-heat-templates: Add a resource for node reboot https://review.openstack.org/233573 11:00:25 Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Set shared secrets, keys and passwords as hidden https://review.openstack.org/233578 12:15:17 shardy: If you have time, could you take a quick look at this CR? https://review.openstack.org/#/c/231930/ started a basic implementation of TLs enabling in the overcloud 12:25:45 derekh: hey, so this takes 2 image builds out of the CI job https://review.openstack.org/#/c/233449/. The nonha job ran in 66 minutes. 12:28:39 spredzy: did you somehow solve the issue when setting a value to comma_delimited_list parameter via parameter_defaults didn't work? 12:28:49 marios, regarding the rabbit fd limit, I've fixed the spelling but the original problem mentioned in bug about 12:29:06 spredzy: i'm trying to set a value for a different parameter, which is a comma_delimited_list too, and i can't get it working 12:29:09 standard limits.conf from systemd is transparently fixed by the puppet module now 12:29:16 other params are fine 12:29:53 jistr, I hit that https://bugs.launchpad.net/tripleo/+bug/1504137 12:29:53 Launchpad bug 1504137 in tripleo "Parameter from tripleoclient not passed to Heat template" [Undecided,New] 12:30:34 jistr, via parameters: it does work, the issue is when I specify it from the tripleoclient 12:30:45 somehow its not passed to the heat stack 12:31:34 Giulio Fidente proposed openstack/tripleo-heat-templates: Parameterize RabbitMQ FD limit https://review.openstack.org/201796 12:31:49 spredzy: hmm that sounds like it could be the same issue, as tripleoclient uses parameter_defaults instead of parameters... maybe the problem is not really in tripleoclient, but in Heat not processing parameter_defaults for comma_delimited_list properly 12:32:03 shardy, ^ 12:32:12 jistr: it's not, we investigated and it works fine direct to heat 12:32:34 hmm do you have somewhere the syntax you used? 12:32:42 jaosorior: are you using parameters or parameter_defaults in your environment file? 12:32:42 jaosorior: "parameters" should overwrite any hard-coded tripleoclient value, otherwise we still have bugs 12:32:59 jistr, the above was discussed this morning 12:33:01 spredzy: yeah, bugs in tripleoclient :\ 12:33:02 well yeah, 'parameters:' works for me too 12:33:12 but 'parameter_defaults:' doesn't 12:33:23 and i'm not feeding it through tripleoclient 12:33:42 jistr: Ok, I can investigate, what parameter are you changing? 12:34:28 gfidente: sec, reading 12:35:14 gfidente: thanks man that's fine, was just an early monday morning rant :) 12:35:32 shardy: CinderNfsServers. i want to switch storage_environment.yaml from parameters to parameter_defaults, because some of the parameters will no longer be top-level 12:35:37 shardy: https://gist.github.com/jistr/b2c921e1e02cc8405a92 12:35:53 jistr: FWIW, I've been through this "parameter_defaults don't work" cycle several times now, and it's always been bugs in the client, where stuff is either merged in the wrong order, or some other value is passed via parameters, which overrides parameter_defaults 12:37:12 jistr: Ok, do you have debug output of the actual call to heat? 12:37:43 First thing to check is that the value in the POST to heat is as expected, and that there's not a duplicate value passed e.g in parameters 12:37:49 * shardy tries a test 12:40:36 marios, hey thank you for checking it, it's the purpose of the reviews :) 12:40:53 shardy: in the debug output of heat-api the parameter value is received from client already wrong. I don't see any parameter_defaults there at all though. Is parameter_defaults a client thing, rather than a Heat API thing? 12:41:39 jistr: no, it's a heat thing, but obviously if you're passing wrong and/or missing values from the client, heat can't do what you want ;) 12:42:19 jistr: the client confusion is that heat (API) expects exactly one environment, with exactly one parameters map, and exactly one (or zero) parameter_defaults map 12:42:51 jistr: the tripleoclient bugs have typically been around merging things before making the call to heat, where you have to resolve multiple environment files into one environment for heat 12:46:21 shardy: ack, thanks for the info. i'll try to look around a bit more to find where the problem is 12:47:11 dprince: ^^ FYI this is why I'm pushing to make it much easier to create an overcloud directly via heatclient 12:47:40 it'd be great to have an easy binary test which proves immediately heat vs client bugs 12:49:57 np gfidente thanks for response 12:52:05 shardy: I'm sold on the idea, just not a fan of some of the tripleoclient defaults 12:52:38 dprince: cool, if I were to refactor into an environments/foo_defaults.yaml, what would I call it? 12:52:51 centos7_ci_defaults.yaml? 12:53:00 shardy: you are talking to someone who actually misses devtest_overcloud.sh in this regard... because it was less hard coded and just used heatclient 12:53:48 shardy: tripleo-docs.yaml? 12:53:51 dprince: ack - well I'm essentially trying to reassemble that workflow for myself ;) 12:54:10 shardy: you said you were aiming to accomidate the tripleo-docs defaults right? 12:54:55 dprince: Yeah, I want a new contributor to be able to either follow the docs, or run tripleo.sh (which should amount to the same thing), then create an overcloud direct via heat with the absolute minimum of complexity 12:55:29 heat stack-create overcloud -f overcloud.yaml -e overcloud_env.yaml 12:57:17 shardy: sure, I always have my own -e $HOME/custom.yaml in there too 12:57:58 dprince: yeah, I think that's great, it just shouldn't be required for basic virt day-1 users 12:58:35 shardy: as to what you call it either be super descriptive (and have a nice long env name) or perhaps just name it all-in-one-defaults.yaml 12:59:01 tripleo-virt-defaults.yaml? 12:59:40 shardy: that works for me 12:59:49 Or maybe tripleo-docs-defaults.yaml as only one arg is virt specific 12:59:49 shardy: you might even throw the word 'test' in there too 13:00:23 tripleo-docs-maybe-virt-test-proven-by-ci-use-these-please-defaults.yaml ;) 13:00:23 shardy: sure, the only thing super virt about it is what we choose for the KVM setting default 13:01:07 dprince: https://review.openstack.org/#/c/233535/ that's already catching folks out actually 13:01:15 as we don't document it at all, fixed by that patch 13:04:40 shardy: right, this is a case where I'd argue a default should not get set (like we have it already in the Heat templates) 13:05:00 shardy: because people just need to choose the right value for what they want 13:05:24 shardy: so if your environment has this setting set to qemu, then I'd name it virt-testing 13:05:40 shardy: otherwise it sort of targets testing on real hardware 13:06:44 btw i created overcloud with heatclient (not successfully finished, just triggered the stack creation to see what params it will get) and i'm getting the same result. Default for a string parameter applied, default for comma_delimited_list didn't 13:06:45 http://fpaste.org/278353/65502714/ 13:06:50 http://paste.fedoraproject.org/278351/54801144/ 13:07:39 i still wonder if i'm doing something wrong... tried to double-check the param spelling at least 13:12:02 dprince: ack, maybe removing the default for the libvirt type is a good idea 13:12:18 but it's not unset in the templates, it's "" 13:12:27 which means heat won't ever give you a good validation error 13:12:37 shardy: probably all of the '' are wrong, or most of them 13:13:19 dprince: Cool, I'll take a look and see which may be better removed, or abstracted into a developer-orientated environment file 13:14:44 anyone having trouble with using tripleo.sh righjt now? 13:14:51 Error: Could not find class ::ironic::inspector for instack on node instack 13:15:47 rhallisey: your instack is missing puppet modules? 13:15:47 rhallisey: need to use puppet from source 13:15:50 rhallisey: i think you need to export DIB_INSTALLTYPE_puppet_modules=source before installing undercloud 13:16:07 yes, do that ^^^ 13:16:11 lol thanks 13:16:11 we should add that to tripleo.sh 13:16:15 rhallisey: the puppet-ironic change did not make it into the openstack-puppet-modules rpm 13:16:25 ah ok 13:16:26 thx guys 13:17:52 and here's a full debug output of the heat stack-create http://paste.fedoraproject.org/278359/55741144/ 13:18:47 shardy: hey, while we are on the topic of parameters would it make sense to outline our "guidelines" in a HACKING doc for t-h-t? 13:18:49 the correct CinderNfsServers value is right beside the only occurence of "retry=1" value (that one did get applied) 13:19:08 shardy: starts to look like a heatclient bug maybe? ^ 13:19:29 shardy: it occurs to me that perhaps we should agree on that first, and then I think it'll make reviewing your patch series much more clear 13:19:49 jistr: Hmm, heatclient is OK AFAICS, ""CinderNfsServers": "192.0.2.1:/export/cinder"" 13:20:01 Derek Higgins proposed openstack-infra/tripleo-ci: [WIP] Use tripleo.sh to build packages https://review.openstack.org/229425 13:20:04 so, it is being passed 13:20:08 John Trowbridge proposed openstack/tripleo-docs: EPEL needed for instack-virt-setup https://review.openstack.org/233624 13:20:10 jistr: let me try to reproduce 13:21:03 dprince: +1, sounds like a good idea, and we can also cover stuff like preferred use of parameters vs parameter_defaults etc 13:22:55 John Trowbridge proposed openstack/tripleo-docs: Workaround for missing puppet-ironic patch in OPM https://review.openstack.org/233627 13:23:12 that is the patch I applied downstream ^ 13:29:14 trown, slagle: this saves us a good bit of CI time https://review.openstack.org/#/c/233449/ 13:30:55 dprince: nice, I am surprised it was so easy :) 13:31:25 thanks to IPA folks for making it support pxe_ssh driver 13:31:57 trown: yeah, I just got IPA working on my bare metal last week 13:33:11 Yanis Guenane proposed openstack/tripleo-heat-templates: Start the nova-cert service in HA setup https://review.openstack.org/223027 13:39:23 Giulio Fidente proposed openstack/tripleo-specs: Support the use of an external load balancer as frontend for the Overcloud https://review.openstack.org/233634 13:45:06 thrash: https://review.openstack.org/#/c/233449/ is the patch to remove deploy ramdisk I said would come later when you were looking at the patch to build the agent ramdisk 13:46:10 trown: ack 13:51:32 dprince: you havn't changed anything on rh1 recently have you ? 13:51:51 derekh: no 13:51:54 derekh: whats up? 13:52:41 dprince: I'm having trouble accessing the overcloud api url (nova list timing out etc...) 13:53:09 Giulio Fidente proposed openstack/tripleo-heat-templates: external loadbalancer: Add net_vip_map_external https://review.openstack.org/222189 13:53:10 Giulio Fidente proposed openstack/tripleo-heat-templates: Make load balancer deployment optional via template param https://review.openstack.org/222305 13:53:11 Giulio Fidente proposed openstack/tripleo-heat-templates: Allow for usage of pre-allocated IPs for the Controller nodes https://review.openstack.org/222497 13:53:26 derekh: locally? from your local machine to the API? 13:54:04 dprince: yup, looks like nodpool may be having similar problems, jobs starting to queue up a little 13:57:39 derekh: I did a list --all-tenants on the seedhost and it didn't take too long 13:58:48 dprince: ack, is it working for you from outside the rack ? 13:59:02 derekh: no, it hangs for me locally 14:00:29 derekh: when you were working on getting HA working in tripleoci, did you come across an issue with ceilometer db-sync failing? 14:00:42 dprince: I was looking in this a few hours ago and ended up restarting some of the controller services which seemed to help, but now its seem to be back 14:00:46 dprince: brb 14:10:12 Merged openstack/tripleo-heat-templates: Allow one to specify horizon ALLOWED_HOSTS https://review.openstack.org/221255 14:12:54 spredzy: doing the 3 reviews you requested. if we merge this, the passing of NtpServer param will stop working, right? (parameter_defaults + comma_delimited_list) 14:13:36 spredzy: i'm inclined to give it +2 -A, because if we merge it we won't have a way to set even a single NTP server until the bug you mentioned earlier today is fixed 14:14:00 spredzy: so the patch is good imo, we just need to wait with it 14:14:05 https://review.openstack.org/#/c/232051/5 14:14:10 jistr, you're right. It would break it I think 14:14:37 ok let me comment 14:29:55 jistr, last one if you still have some time :) https://review.openstack.org/#/c/223027/ 14:30:30 marios, ^ I added the constraint does it look ok to you ? 14:34:00 Rob Pothier proposed openstack/tripleo-heat-templates: Cisco Nexus ML2 MD: Add Support for mulitple UCSMs https://review.openstack.org/233664 14:34:22 spredzy: reading back 14:35:22 spredzy: which review, the nova-cert one? 14:35:49 marios, yes 14:35:54 spredzy: yeah sorry was readying thx 14:44:17 spredzy: so i don't see anything from poke at https://github.com/beekhof/osp-ha-deploy/search?utf8=%E2%9C%93&q=nova-cert&type=Code about nova-cert - unless i missed it. oterwise sounds sane enough to depend on nova-conductor 14:46:43 marios, they are nothing there. But based on https://bugzilla.redhat.com/show_bug.cgi?id=1245740 they are no strong constraint. So just making one to make sure the service is colocated with the other nova services 14:46:43 bugzilla.redhat.com bug 1245740 in openstack-tripleo-heat-templates "openstack-nova-cert is installed, but not running on the controller nodes" [Unspecified,On_dev] - Assigned to yguenane 14:47:03 spredzy: cool thanks for confirm 14:48:40 marios, thx to you for the review :) 14:51:07 slagle: dprince, I wonder if we could consider merging https://review.openstack.org/#/c/228034/ without CI... it is causing issues with jq 1.5 and CI is only failing because it cant build tripleo-incubator 14:55:21 i thought fixing the project name map was going to fix that? 14:55:51 ya, me too, but it is a bit of a rabbit hole 14:56:10 slagle: or are you talking about fixing it in delorean proper? 14:56:41 wherever the mappings are maintained 14:56:58 what map-project-name uses 14:58:38 map-project-name just looks at rdoinfo/rdo.yml for the name 15:00:15 Untriaged bugs so far: 15:00:16 https://bugs.launchpad.net/tripleo/+bug/1504137 15:00:16 Launchpad bug 1504137 in tripleo "Parameter from tripleoclient not passed to Heat template" [Undecided,New] 15:00:17 https://bugs.launchpad.net/diskimage-builder/+bug/1504632 15:00:17 Launchpad bug 1504632 in diskimage-builder "elements README documentation environment variables should be clearer" [Undecided,In progress] - Assigned to Augustina Ragwitz (missaugustina) 15:00:21 https://bugs.launchpad.net/tripleo/+bug/1504179 15:00:22 https://bugs.launchpad.net/tripleo/+bug/1503669 15:00:23 https://bugs.launchpad.net/tripleo/+bug/1504263 15:00:25 https://bugs.launchpad.net/tripleo/+bug/1504363 15:00:26 Launchpad bug 1504179 in tripleo "instack-virt-setup can't use dnsmasq lease file anymore" [Undecided,In progress] - Assigned to Thomas Herve (therve) 15:00:27 Launchpad bug 1503669 in tripleo "Including an additional element is inconvenient" [Undecided,New] - Assigned to Jaume Devesa (devvesa) 15:00:28 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 15:00:29 Launchpad bug 1504363 in tripleo "Updates to hiera configuration not propogating/making a puppet run" [Undecided,Confirmed] 15:03:13 slagle: I wonder actually if we can change the project to tripleo-incubator and add openstack-tripleo name and it will "just work" https://github.com/redhat-openstack/rdoinfo/blob/master/rdo.yml#L281-L285 15:04:37 trying it locally... we will need to revert my hack in tripleoci too https://github.com/openstack-infra/tripleo-ci/commit/3db00edc76113cfd71e152b804c21cff94b07e0c 15:05:49 trown: sorry, missed your message earlier, yes, if you hit the same thing I did, here is the fix https://review.openstack.org/#/c/229183/ 15:07:20 derekh: odd, in RDO we are using puppet from source in the overcloud, so I should have that 15:07:43 trown: yup 15:27:11 John Trowbridge proposed openstack-infra/tripleo-ci: Revert "There is no tripleo-incubator package" https://review.openstack.org/233686 15:50:06 dprince: i have this bash script to generate the JSON config files http://paste.fedoraproject.org/278420/64942144/ is this the best place to ask a few things about it? 16:10:17 URGENT TRIPLEO TASKS NEED ATTENTION 16:10:17 +--------------------------+-------------------------------+-------------------+-------------+ 16:10:17 | Title | URL | Members | Last Active | 16:10:17 +--------------------------+-------------------------------+-------------------+-------------+ 16:10:17 | Tripleo ci cloud is DOWN | https://trello.com/c/ORiaa8pz | **NEEDS MEMBERS** | 5 min | 16:10:18 +--------------------------+-------------------------------+-------------------+-------------+ 16:10:35 o.O 16:20:54 Jiri Stransky proposed openstack/tripleo-heat-templates: Support NFS backend for Glance (via Pacemaker) https://review.openstack.org/199152 16:51:52 Steven Hardy proposed openstack/tripleo-heat-templates: Allow ctlplane network or ID, and default to "ctlplane" https://review.openstack.org/232101 16:51:53 Steven Hardy proposed openstack/tripleo-heat-templates: Update overcloud template description and Count constraints https://review.openstack.org/232084 16:51:53 Steven Hardy proposed openstack/tripleo-heat-templates: Align some neutron defaults with CI https://review.openstack.org/232123 16:51:54 Steven Hardy proposed openstack/tripleo-heat-templates: Align some defaults with with ControllerCount default https://review.openstack.org/232122 16:53:27 dprince: The controller has been restarted (details of what I've done are on this etherpad https://etherpad.openstack.org/p/snqXyvqubi ) 16:53:58 dprince: api access from outside of rack is now working, I'll come back to continue in a few hours 16:54:20 dprince: next we need to being up the compute nodes, 16:55:03 dprince: I've added a iptable rule to block infra/nodepool intol we're back up and running 16:55:34 derekh: thanks 16:56:50 derekh_afk: our ssl cert might have regressed since running os-refresh-config, we'll need to put back in place the updated one for 2015 17:03:31 Steven Hardy proposed openstack/tripleo-docs: Add note that qemu libvirt-type is needed for virt environment https://review.openstack.org/233535 17:04:51 thrash: i dont get what you're asking for on https://review.openstack.org/#/c/232105 17:05:11 thrash: you want validation-errors-nonfatal defaulted to True? 17:06:19 in that case, i'd just leave the arg as validation-errors-fatal, and default that to False 17:06:24 slagle: yes, but then the 'action' would be store_false 17:06:35 true is the default. 17:06:48 then the check at line 959 would make more sense. 17:07:02 especiailly if you flip validation-warnings-fatal 17:07:18 it'd be the same check though (modulo the warnings change) 17:07:33 you'd still have to "if not parsed_args.validation_errors_nonfatal" 17:07:50 then that check at 959 becomes 'it not parsed_args.validation_warnings_fatal and parsed_args.validation_errors_nonfatal' 17:08:06 *if 17:08:37 ok, so why i reveresed the arg was so that the existing behavior would stay the same 17:09:01 I thought you *wanted* the existing behaviour to change? 17:09:13 You wanted the errors to become fatal, right? 17:09:30 by default... 17:10:07 they already were fatal by default, just not covered by the cli arg 17:10:14 URGENT TRIPLEO TASKS NEED ATTENTION 17:10:14 +--------------------------+-------------------------------+-------------------+-------------+ 17:10:14 | Title | URL | Members | Last Active | 17:10:14 +--------------------------+-------------------------------+-------------------+-------------+ 17:10:14 | Tripleo ci cloud is DOWN | https://trello.com/c/ORiaa8pz | **NEEDS MEMBERS** | 65 min | 17:10:15 +--------------------------+-------------------------------+-------------------+-------------+ 17:10:46 thrash: there was no way to say "not fatal" before for the errors that i made validations in the subsequent patches 17:11:53 but, if we don't care about those staying fatal by default, we can just keep the existing arg as-is 17:12:46 the node count check for instance 17:12:58 there's no way to disable that right now 17:13:47 slagle: I think I'm mostly concerned with the statement at 959 17:14:05 reading it "naturally" it doesn't make sense to me. 17:14:57 slagle: I may be misreading that chunk of code tho 17:16:16 slagle: I see it now. The return is actually exiting the command... 17:17:15 slagle: I am convinced. I will remove my -1 if you fix L901. :) 17:18:45 thrash: i guess what it comes down to is...do we want warnings and errors fatal by default? 17:19:05 slagle: I think errors should be fatal by default. It's an error. 17:19:09 that implies which arg we need...fatal or nonfatal 17:20:05 ok, i'll go with that. and give warnigns the same treatment, just for consistency sake 17:20:42 warnings on the other hand I think makes sense to say non-fatal by default 17:21:14 as you have it seems fine to me. 17:21:27 ok 17:21:37 just fix the help string on L901. :) 17:21:46 you have the inconsistent cli args that way, but that's ok i suppose 17:22:25 slagle: but it makes sense at least. Warnings are non-fatal by default, so an arg to make them fatal. Errors are fatal by default, so an arg to make it non-fatal 17:40:59 looks like http://cbs.centos.org is out again 17:44:19 trown|lunch, slagle: who do we reach out to for the cbs.centos.org failures? 17:54:40 Ben Nemec proposed openstack-infra/tripleo-ci: Fix cherry-pick syntax https://review.openstack.org/225342 17:54:41 dprince: most likely #rdo or #centos-devel 17:57:51 Ben Nemec proposed openstack-infra/tripleo-ci: Fix cherry-pick syntax https://review.openstack.org/225342 18:04:31 Dan Sneddon proposed openstack/os-net-config: Add Linux Bridge capability to os-net-config ifcfg https://review.openstack.org/219927 18:07:55 Dan Sneddon proposed openstack/os-net-config: Fix kwarg assignment in objects.py https://review.openstack.org/231247 18:10:16 URGENT TRIPLEO TASKS NEED ATTENTION 18:10:16 +--------------------------+-------------------------------+-------------------+-------------+ 18:10:16 | Title | URL | Members | Last Active | 18:10:16 +--------------------------+-------------------------------+-------------------+-------------+ 18:10:16 | Tripleo ci cloud is DOWN | https://trello.com/c/ORiaa8pz | **NEEDS MEMBERS** | 125 min | 18:10:17 +--------------------------+-------------------------------+-------------------+-------------+ 18:24:19 Ben Nemec proposed openstack/python-tripleoclient: Move common functions out into utils.py https://review.openstack.org/225244 18:25:59 Ben Nemec proposed openstack/python-tripleoclient: Consolidate module constants. https://review.openstack.org/225245 18:26:29 Ben Nemec proposed openstack/python-tripleoclient: Move more functions to utils.py https://review.openstack.org/225246 18:40:56 James Slagle proposed openstack/python-tripleoclient: Make hypervisor stat check a validation https://review.openstack.org/232041 18:40:56 James Slagle proposed openstack/python-tripleoclient: Add --skip-validation-checks cli arg https://review.openstack.org/232209 18:40:57 James Slagle proposed openstack/python-tripleoclient: Make validation errros fatal by default https://review.openstack.org/232105 18:40:58 James Slagle proposed openstack/python-tripleoclient: Make node count check a validation https://review.openstack.org/232106 18:53:20 bnemec: OHAI. I was looking at https://review.openstack.org/#/c/222351/8 and a tangent thought occured to me - do the centos7 and rhel7 elements need to be very different? I wonder if we could make one element which does both so we can test the code (since theres no real hope of testing an element which uses publicly unavailable images) 18:54:46 Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Set shared secrets, keys and passwords as hidden https://review.openstack.org/233578 19:10:14 URGENT TRIPLEO TASKS NEED ATTENTION 19:10:14 +--------------------------+-------------------------------+-------------------+-------------+ 19:10:14 | Title | URL | Members | Last Active | 19:10:14 +--------------------------+-------------------------------+-------------------+-------------+ 19:10:14 | Tripleo ci cloud is DOWN | https://trello.com/c/ORiaa8pz | **NEEDS MEMBERS** | 185 min | 19:10:15 +--------------------------+-------------------------------+-------------------+-------------+ 19:23:54 Merged openstack/diskimage-builder: Update apt-conf elements README from free text to table formatting https://review.openstack.org/231770 19:24:02 dprince: ack, will figure out the ssl certs now and then hopfully start bringing things back 19:26:15 Merged openstack/diskimage-builder: Update apt-preferences element README from free text to table formatting https://review.openstack.org/232211 19:28:26 James Slagle proposed openstack/python-tripleoclient: Add --skip-validation-checks cli arg https://review.openstack.org/232209 20:00:24 James Slagle proposed openstack/os-cloud-config: Remove requirement of specifying hardware stats https://review.openstack.org/231613 20:15:14 Merged openstack/diskimage-builder: Adds debian support to iso element https://review.openstack.org/211976 20:19:18 dprince or derekh any chance you could look at a small patch to tripleo-ci, I think I fixed the issue with tripleo-incubator not building in CI, but we need to revert my previous hack https://review.openstack.org/#/c/233686/ 20:20:02 trown: looking 20:21:37 trown: +2, will send once we get CI back online... 20:21:58 awesome, thansk 20:35:58 greghaynes: There's actually almost nothing in the rhel7 element at this point. It's almost all been factored out into redhat-common and rhel-common. 20:36:14 bnemec: perfect! 20:36:18 We do need to set up third party CI to run tests with rhel, but ENOTIME. 20:40:58 Ben Nemec proposed openstack/diskimage-builder: Update rhel7 element readme https://review.openstack.org/222351 20:55:05 should ceilometer be trying to run dbsync on the controller in a HA setup? 21:00:16 Untriaged bugs so far: 21:00:17 https://bugs.launchpad.net/tripleo/+bug/1504137 21:00:17 Launchpad bug 1504137 in tripleo "Parameter from tripleoclient not passed to Heat template" [Undecided,New] 21:00:18 https://bugs.launchpad.net/diskimage-builder/+bug/1504632 21:00:18 Launchpad bug 1504632 in diskimage-builder "elements README documentation environment variables should be clearer" [Undecided,In progress] - Assigned to Augustina Ragwitz (missaugustina) 21:00:20 https://bugs.launchpad.net/tripleo/+bug/1504179 21:00:21 Launchpad bug 1504179 in tripleo "instack-virt-setup can't use dnsmasq lease file anymore" [Undecided,In progress] - Assigned to Thomas Herve (therve) 21:00:22 https://bugs.launchpad.net/tripleo/+bug/1503669 21:00:23 Launchpad bug 1503669 in tripleo "Including an additional element is inconvenient" [Undecided,New] - Assigned to Jaume Devesa (devvesa) 21:00:24 https://bugs.launchpad.net/tripleo/+bug/1504263 21:00:24 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 21:00:26 https://bugs.launchpad.net/tripleo/+bug/1504363 21:00:27 Launchpad bug 1504363 in tripleo "Updates to hiera configuration not propogating/making a puppet run" [Undecided,Confirmed] 21:04:10 Jon Schlueter proposed openstack/tripleo-docs: Add step to basic deployment cli to verify images were built https://review.openstack.org/233817 22:10:18 URGENT TRIPLEO TASKS NEED ATTENTION 22:10:18 +--------------------------+-------------------------------+---------------+-------------+ 22:10:18 | Title | URL | Members | Last Active | 22:10:18 +--------------------------+-------------------------------+---------------+-------------+ 22:10:18 | Tripleo ci cloud is DOWN | https://trello.com/c/ORiaa8pz | Derek Higgins | 160 min | 22:10:19 +--------------------------+-------------------------------+---------------+-------------+ 22:14:58 Merged openstack/instack-undercloud: Update README to include tags recognized by release-tools https://review.openstack.org/221671 23:10:18 URGENT TRIPLEO TASKS NEED ATTENTION 23:10:18 +--------------------------+-------------------------------+---------------+-------------+ 23:10:18 | Title | URL | Members | Last Active | 23:10:18 +--------------------------+-------------------------------+---------------+-------------+ 23:10:18 | Tripleo ci cloud is DOWN | https://trello.com/c/ORiaa8pz | Derek Higgins | 220 min | 23:10:19 +--------------------------+-------------------------------+---------------+-------------+ 00:03:18 James Slagle proposed openstack/python-tripleoclient: Add --skip-validation-checks cli arg https://review.openstack.org/232209 00:10:19 URGENT TRIPLEO TASKS NEED ATTENTION 00:10:19 +--------------------------+-------------------------------+---------------+-------------+ 00:10:19 | Title | URL | Members | Last Active | 00:10:19 +--------------------------+-------------------------------+---------------+-------------+ 00:10:19 | Tripleo ci cloud is DOWN | https://trello.com/c/ORiaa8pz | Derek Higgins | 280 min | 00:10:20 +--------------------------+-------------------------------+---------------+-------------+ 00:43:37 James Slagle proposed openstack/instack-undercloud: Make 98-undercloud-setup rerunnable https://review.openstack.org/222227 01:10:19 URGENT TRIPLEO TASKS NEED ATTENTION 01:10:19 +--------------------------+-------------------------------+---------------+-------------+ 01:10:19 | Title | URL | Members | Last Active | 01:10:19 +--------------------------+-------------------------------+---------------+-------------+ 01:10:19 | Tripleo ci cloud is DOWN | https://trello.com/c/ORiaa8pz | Derek Higgins | 340 min | 01:10:20 +--------------------------+-------------------------------+---------------+-------------+ 01:27:02 Ci is back running, seems to be ok but I wont be here to see if thing are passing or not 02:16:20 Emilien Macchi proposed openstack/tripleo-heat-templates: Implement Workers parameters https://review.openstack.org/229471 02:18:24 Emilien Macchi proposed openstack/tripleo-heat-templates: neutron: enable nova-event-callback by default https://review.openstack.org/225962 02:28:30 Emilien Macchi proposed openstack/tripleo-heat-templates: puppet: run keystone in wsgi https://review.openstack.org/213175 02:29:12 Emilien Macchi proposed openstack/tripleo-heat-templates: Puppet / Compute: allow to run Ephemeral only storage with RBD https://review.openstack.org/202206 02:31:45 Emilien Macchi proposed openstack/tripleo-heat-templates: Implement Advanced Firewalling support https://review.openstack.org/191195 03:00:16 Untriaged bugs so far: 03:00:17 https://bugs.launchpad.net/tripleo/+bug/1505430 03:00:17 Launchpad bug 1505430 in tripleo "Triggering puppet runs with config_identifier is error prone and complicates templates" [Undecided,New] - Assigned to Steve Baker (steve-stevebaker) 03:00:18 https://bugs.launchpad.net/tripleo/+bug/1504137 03:00:18 Launchpad bug 1504137 in tripleo "Parameter from tripleoclient not passed to Heat template" [Undecided,New] 03:00:21 https://bugs.launchpad.net/diskimage-builder/+bug/1504632 03:00:21 Launchpad bug 1504632 in diskimage-builder "elements README documentation environment variables should be clearer" [Undecided,In progress] - Assigned to Augustina Ragwitz (missaugustina) 03:00:23 https://bugs.launchpad.net/tripleo/+bug/1504179 03:00:23 Launchpad bug 1504179 in tripleo "instack-virt-setup can't use dnsmasq lease file anymore" [Undecided,In progress] - Assigned to Thomas Herve (therve) 03:00:25 https://bugs.launchpad.net/tripleo/+bug/1503669 03:00:26 Launchpad bug 1503669 in tripleo "Including an additional element is inconvenient" [Undecided,New] - Assigned to Jaume Devesa (devvesa) 03:00:27 https://bugs.launchpad.net/tripleo/+bug/1504263 03:00:28 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 03:00:28 https://bugs.launchpad.net/tripleo/+bug/1504363 03:00:29 Launchpad bug 1504363 in tripleo "Updates to hiera configuration not propogating/making a puppet run" [Undecided,Confirmed] 07:45:20 Rabi Mishra proposed openstack/tripleo-docs: change config option to run benchmarks when inspecting nodes https://review.openstack.org/234016 08:00:17 Merged openstack/python-tripleoclient: fix cisco networking rpm name https://review.openstack.org/225165 08:07:49 zaneb: hey, have you had any issues with etherpad.openstack.org lately? right now it is down for me (was just poking at mail and wanted to look at https://etherpad.openstack.org/p/tripleo-kilo-to-liberty-upgrades) 08:09:13 marios: yeah the whole etherpad is down for me too 08:09:30 reeeeeeeealy slow (am starting to get _something_ but not usable). thanks shardy 08:10:02 gerrit was reeeeeeeealy slow yesterday too for some reason 08:16:29 correction, gerrit is still really slow :( 08:57:07 marios: etherpad is working for me now 09:00:16 Untriaged bugs so far: 09:00:17 https://bugs.launchpad.net/tripleo/+bug/1505430 09:00:17 Launchpad bug 1505430 in tripleo "Triggering puppet runs with config_identifier is error prone and complicates templates" [Undecided,New] - Assigned to Steve Baker (steve-stevebaker) 09:00:19 https://bugs.launchpad.net/diskimage-builder/+bug/1504632 09:00:19 Launchpad bug 1504632 in diskimage-builder "elements README documentation environment variables should be clearer" [Undecided,In progress] - Assigned to Augustina Ragwitz (missaugustina) 09:00:21 https://bugs.launchpad.net/tripleo/+bug/1504179 09:00:21 Launchpad bug 1504179 in tripleo "instack-virt-setup can't use dnsmasq lease file anymore" [Undecided,In progress] - Assigned to Thomas Herve (therve) 09:00:23 https://bugs.launchpad.net/tripleo/+bug/1503669 09:00:23 Launchpad bug 1503669 in tripleo "Including an additional element is inconvenient" [Undecided,New] - Assigned to Jaume Devesa (devvesa) 09:00:24 https://bugs.launchpad.net/tripleo/+bug/1504263 09:00:25 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 09:00:26 https://bugs.launchpad.net/tripleo/+bug/1504363 09:00:26 Launchpad bug 1504363 in tripleo "Updates to hiera configuration not propogating/making a puppet run" [Undecided,Confirmed] 09:01:28 shardy: thanks, yeah here too 09:07:18 shardy, that email regarding upgrades and passwords, that also seems to be one of those changes we don't want to 'backport by default' into the stable branch 09:08:31 gfidente: Yeah, anything which modifies an existing parameter interface can't be backported, and anything which adds new parameters will have to be backwards compatible 09:08:53 Yanis Guenane proposed openstack/tripleo-heat-templates: Allow a deployer to specify HAProxy syslog server address https://review.openstack.org/222602 09:08:54 gfidente: we also need to work out a deprecation policy, similar to other projects, so we don't have to support parameter interfaces indefinitely 09:09:25 gfidente: the main problem with that is there's currently no template annotation which warns the user when they use a derecated interface 09:09:37 I proposed a heat summit session to discuss how we might fix that 09:12:49 shardy, /me likes, thanks :) 09:13:36 * mkovacik is away: lunch 09:17:07 shardy, regarding the 'immediate' problem instead, have ideas? 09:18:05 shardy, spredzy: i have a fix for the "parameter_defaults not working for comma_delimited_list" issue https://review.openstack.org/234055 09:18:38 it may not pass CI, i didn't do any test amendments (i don't have a proper heat dev env set up, it's my first patch to heat i think) 09:18:49 but it did fix the issue locally for me 09:20:39 gfidente: we need to have a wider discussion - I was originally thinking we would only use parameter_defaults for optional ExtraConfig stuff, e.g third party additions, but I can see you're going in the opposite direction 09:23:09 jistr, testing in few mins 09:24:17 jistr: thanks for digging into it! 09:24:34 spredzy: cool 09:24:39 shardy: np :) 09:24:44 * jistr -> lunch 09:30:28 marios, hey thanks for checking the spec, updating 09:34:01 gfidente: np thanks will check it later let's give others a chance too 09:45:15 marios, so for the glory the changes (excluding the validation script) are up for review already, linked from the LP blueprint 09:45:24 tx 09:45:38 that means you're volunteering to review, mind you! 09:46:26 sure but not right now because i'm done with gerrit for now (slow and frustrating) 09:46:29 gfidente: ^ 09:46:31 Merged openstack-infra/tripleo-ci: Revert "There is no tripleo-incubator package" https://review.openstack.org/233686 09:47:10 gfidente: gimme the lp link 09:47:32 gfidente: nm got it 09:47:42 https://blueprints.launchpad.net/tripleo/+spec/tripleo-mitaka-external-load-balancer 09:49:01 jistr, I applied the patch but still have the issue NtpServer: "" 09:49:33 Giulio Fidente proposed openstack/tripleo-specs: Support the use of an external load balancer as frontend for the Overcloud https://review.openstack.org/233634 09:49:37 gfidente: added to https://etherpad.openstack.org/p/tripleo-review-priorities 09:50:00 I think we should add the ceph pools naming there as well 09:50:09 * gfidente doing 09:51:24 spredzy: did heat-engine definitely get restarted? 09:51:36 spredzy: I'm testing locally via unit tests and it seems to work OK 09:52:27 shardy, daah sorry I missed that 09:52:36 jistr, shardy yes its working 09:56:31 Yanis Guenane proposed openstack/tripleo-heat-templates: Allow a deployer to specify HAProxy syslog server address https://review.openstack.org/222602 09:57:23 Jan Provaznik proposed openstack/python-tripleoclient: Change RHEL subscription template path https://review.openstack.org/229847 10:02:00 Gael Chamoulaud proposed openstack/tripleo-common: Fix bashate errors https://review.openstack.org/234108 10:17:20 shardy, jistr can I get your reviews back on https://review.openstack.org/#/c/222602/ (just rebased it) 10:18:42 jistr: Hey, I just pulled and tested your patch - it shows that actually dprince's https://bugs.launchpad.net/heat/+bug/1460063 was not invalid 10:18:42 Launchpad bug 1460063 in heat "parameter_defaults only apply to nested stacks" [Undecided,Invalid] 10:19:11 jistr: sorry I didn't catch this sooner, I wasn't expecting a type-specific error like this, it turns out only comma_delimited_list and json are broken 10:19:17 because they both do the caching thing 10:19:49 jistr: I've written some unit tests, and have some ideas how we could refactor to solve for both, let me know if you're happy for me to push to your review with those changes 10:22:52 Giulio Fidente proposed openstack/tripleo-heat-templates: Parameterize RabbitMQ FD limit https://review.openstack.org/201796 10:23:46 Derek Higgins proposed openstack-infra/tripleo-ci: [WIP] Use tripleo.sh to build packages https://review.openstack.org/229425 10:33:38 Gael Chamoulaud proposed openstack/tripleo-heat-templates: Make puppet manifests compliant with Puppet 4.x https://review.openstack.org/225664 10:42:51 shardy: hi, sure thing, please feel free to amend the patch 10:49:57 hmm, unhappy Jenkins? https://review.openstack.org/#/q/status:open+project:openstack/python-tripleoclient,n,z 10:50:51 Dougal Matthews proposed openstack/python-tripleoclient: Use ironic-agent for deployment and introspection https://review.openstack.org/233449 10:51:07 Dougal Matthews proposed openstack/python-tripleoclient: Validate the stack before deploying https://review.openstack.org/232999 10:51:45 Dougal Matthews proposed openstack/python-tripleoclient: Do not initialize/manage keystone in python-tripleoclient https://review.openstack.org/231395 10:51:59 Dougal Matthews proposed openstack/python-tripleoclient: Make root directory configurable https://review.openstack.org/230275 10:52:07 Dougal Matthews proposed openstack/python-tripleoclient: Improve Coverage for tripleoclient/utils.py https://review.openstack.org/228609 10:52:13 Dougal Matthews proposed openstack/python-tripleoclient: Also disallow tunnel types without network type https://review.openstack.org/228445 10:52:26 spredzy: thanks for verifying the patch on the NtpServer issue 10:52:29 Gael Chamoulaud proposed openstack/tripleo-heat-templates: Add Puppet 4.x lint checks https://review.openstack.org/225664 10:53:03 Dougal Matthews proposed openstack/python-tripleoclient: Turn the orchestration client reference into a property https://review.openstack.org/224752 10:53:19 Dougal Matthews proposed openstack/python-tripleoclient: Sahara integration https://review.openstack.org/221418 11:10:10 Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Enable TLS deployment for the Overcloud https://review.openstack.org/231930 11:10:38 jistr, thanks for looking into it :p 11:10:38 Marek Aufart proposed openstack/tripleo-common: [WIP]Add resource capabilities https://review.openstack.org/232534 11:11:10 * mkovacik is back (gone 01:57:35) 11:18:11 Gael Chamoulaud proposed openstack/tripleo-heat-templates: Make puppet manifests compliant with Puppet 4.x https://review.openstack.org/225664 11:18:12 Gael Chamoulaud proposed openstack/tripleo-heat-templates: Add Puppet 4.x lint checks https://review.openstack.org/234150 11:32:58 James Slagle proposed openstack/python-tripleoclient: Make hypervisor stat check a validation https://review.openstack.org/232041 11:32:59 James Slagle proposed openstack/python-tripleoclient: Add --skip-validation-checks cli arg https://review.openstack.org/232209 11:32:59 James Slagle proposed openstack/python-tripleoclient: Make validation errros fatal by default https://review.openstack.org/232105 11:33:00 James Slagle proposed openstack/python-tripleoclient: Make node count check a validation https://review.openstack.org/232106 11:37:18 Jon Schlueter proposed openstack/tripleo-docs: Add step to basic deployment cli to verify images were built https://review.openstack.org/233817 11:41:27 James Slagle proposed openstack/python-tripleoclient: Install device-mapper-multipath on overcloud-full https://review.openstack.org/231690 11:45:50 nova baremetal-node-list <--- I have to be the last person on the planet still tying that command 11:46:03 *typing 11:46:32 slagle: fyi https://review.openstack.org/#/c/234129 11:48:56 Gael Chamoulaud proposed openstack/instack-undercloud: Fix bashate errors https://review.openstack.org/234163 11:51:21 EmilienM: cool, where can i see what the jobs do? 11:52:21 slagle: https://github.com/openstack-infra/project-config/blob/master/jenkins/jobs/puppet-module-jobs.yaml#L177-L221 11:52:27 you need to look in macros.yaml also 11:53:32 EmilienM: hi, where can I put it in non-voting? first time changing stuff project-config ... 11:57:17 gchamoul: look at zuul/layout.yaml and grep "voting: false" 11:57:22 you'll see a lot of examples 11:57:35 yes! 12:12:51 Yanis Guenane proposed openstack/tripleo-heat-templates: Manage keystone related settings directly in t-h-t manifests https://review.openstack.org/230357 12:12:51 Yanis Guenane proposed openstack/tripleo-heat-templates: Add more components virtual ip mapping into controller. https://review.openstack.org/231361 12:13:47 Yanis Guenane proposed openstack/tripleo-heat-templates: Manage keystone related settings directly in t-h-t manifests https://review.openstack.org/230375 12:15:25 imain proposed openstack/os-net-config: Add ip addr flush on ifdown to ensure IP addresses are removed. https://review.openstack.org/234176 12:17:28 dprince slagle ^ I think should work with net isolation 12:19:19 rhallisey: cool! will have to give it another go 12:19:31 dprince: yeah so that fixes the IP being stuck on eth0 12:19:47 slagle: what were you seeing with network isolation not working? 12:20:22 Slower: loss of networking on the compute node when os-net-config ran 12:20:37 oh fun :) 12:21:38 slagle: well hopefully this fixes it otherwise we're going to have to get a terminal going and take a look see 12:24:49 Yanis Guenane proposed openstack/tripleo-heat-templates: Manage keystone related settings directly in t-h-t manifests https://review.openstack.org/230375 12:28:06 shardy: hi, has there been any discussion about the desire to deploy multiple overclouds (and potentially different versions) from the same undercloud might affect the stable branch proposal? 12:28:14 Yanis Guenane proposed openstack/tripleo-heat-templates: Manage keystone related settings directly in t-h-t manifests https://review.openstack.org/230375 12:28:54 shardy: supposing we have a mitaka undercloud, and we needed to support deploying a mitaka overcloud and liberty overcloud from that single undercloud 12:29:11 is that valid or something we should take into consideration? 12:30:50 slagle: Yes good point, IIRC it has been discussed but not in great detail 12:30:50 Mike Burns proposed openstack/python-tripleoclient: Install bigswitch networking plugin by default https://review.openstack.org/222312 12:31:30 slagle: I was originally thinking from a t-h-t perspective we could just support having both branches packaged, e.g /usr/share/openstack-tripleo-heat-templates/liberty or whatever 12:31:46 shardy: right, i think that will be required regardless 12:31:50 but then there's how to handle the image building etc too & the puppet dependencies at that stage 12:32:07 the question becomes....do we need that same model for ever single other repo? 12:32:18 which would be kind of unmanageable 12:32:40 slagle: Yeah, I was hoping not, but that implies a degree of backwards compat of those non-branched repos 12:32:45 it's a question of backwards compatibility vs stable branches 12:33:00 it's kinda difficult because tripleo has so many different repos compared to most other projects :( 12:33:32 ok, well right now i'm just asking if anyone has asked for this use case :) 12:33:55 i heard some mention of such a thing a while back, but we should probably clarify this 12:35:35 slagle: yes, agreed, I see it as something we want to support so wider discussion of the challenges involved would be good 12:40:50 EmilienM: is that better? https://review.openstack.org/#/c/234129/2 12:41:56 slagle: hey - were you planning any docs update for tripleo.sh? 12:42:11 I may do a patch now if not, as I'm just setting up a new environment 12:42:52 Mike Burns proposed openstack/python-tripleoclient: remove default Neutron Network and Tunnel types https://review.openstack.org/224870 12:43:33 gchamoul: I don't think so 12:43:50 shardy: beyond what I already did? not specifically 12:43:51 the job name is not good 12:44:00 EmilienM: and the patchset #3? 12:44:04 shardy: if you want to make it better, or do something different, go4it 12:44:04 gchamoul: you need to look in puppet-check-jobs jobgroup what will be the job names 12:44:33 gchamoul: maybe, but it's very general. 12:44:43 slagle: Ok, cool, I was thinking about a "developer environment" page in the advanced section of tripleo-docs, including stuff like how to build local deloran packages, and use that repo from the undercloud node 12:44:46 gchamoul: have you tested it? There are testing tools in project-config repo 12:44:51 in tools/run_tests.sh iirc 12:44:57 shardy: sounds good to me 12:45:09 EmilienM: nope .. I will test it 12:47:39 Yanis Guenane proposed openstack/tripleo-heat-templates: Manage keystone related settings directly in t-h-t manifests https://review.openstack.org/230375 12:49:02 Jon Schlueter proposed openstack/tripleo-docs: Add step to basic deployment cli to verify images were built https://review.openstack.org/233817 12:50:25 Lennart Regebro proposed openstack/python-tripleoclient: If neutron tunneling is disabled, do not throw an error on validation https://review.openstack.org/228446 12:52:39 jistr|biab, is this patchset looking closter to what you expected in term of VIP https://review.openstack.org/#/c/230357/ ? 12:54:55 Slower, rhallisey: nice on the os-net-config patch 12:55:50 Emilien Macchi proposed openstack/tripleo-heat-templates: puppet: run keystone in wsgi https://review.openstack.org/213175 12:55:57 Let me ponder this a bit. Would a one time 'cleanup' perhaps be better here? Or do we think flushing each time you ifdown is the right approach? 12:58:00 Yanis Guenane proposed openstack/tripleo-heat-templates: Manage keystone related settings directly in t-h-t manifests https://review.openstack.org/230375 12:58:58 dprince, Ian should be at the meeting at 1. He's asleep now :p 12:59:47 rhallisey: cool, I guess I'll go back to bed too then 12:59:55 loll 12:59:57 :D 13:03:42 Yanis Guenane proposed openstack/python-tripleoclient: Do not initialize/manage keystone in python-tripleoclient https://review.openstack.org/231395 13:09:18 Ryan Brady proposed openstack/tripleo-common: [WIP]Initial workflow implementation of tripleo-common https://review.openstack.org/227297 13:15:05 Steven Hardy proposed openstack/tripleo-docs: Update docs to show using ExtraConfig parameters https://review.openstack.org/233012 13:15:05 Steven Hardy proposed openstack/tripleo-docs: Add docs for per-node (pre-deploy) config interfaces https://review.openstack.org/233169 13:15:06 Steven Hardy proposed openstack/tripleo-docs: Split node configuration and customization docs https://review.openstack.org/233168 13:17:36 dprince: Hey btw, it turns out you were right about bug #1460063, jistr discovered the root cause, apologies for not correctly reproducing when you originally reported 13:17:36 bug 1504137 in heat "duplicate for #1460063 environment's parameter_defaults not applied some parameter types" [High,In progress] https://launchpad.net/bugs/1504137 - Assigned to Steven Hardy (shardy) 13:17:53 https://review.openstack.org/#/c/234055/ fixes 13:18:11 * dprince looks 13:18:53 dprince: the problem is parameter_default was only broken for json/comma_delimited_list, and I tried other types 13:19:35 shardy: oh, I see. Thanks for fixing, jistr too 13:19:49 shardy: there was an old (duplicate) bug for this too FWIW 13:20:03 perhaps slighly different (less descriptive) 13:20:34 dprince: yeah you raised 1460063, which I've now marked duplicate of 1504137 which has a fix 13:20:59 or is there another dupe we need to tag? 13:23:20 shardy: great, I think that is it 13:32:52 spredzy: i think the public URL is good, however i wonder if we can have the classes included in step 3? 13:33:00 spredzy: keystone isn't started there yet 13:34:52 spredzy: keystone starts in step 4, and i guess the `admin` and `endpoints` classes have a dependency on Service['keystone'] if any, and that probably wouldn't do the job in HA anyway because we start the service with pacemaker::resource::... and not with service {'...': } 13:36:23 jistr, ack for HA ill follow the same pattern as here https://review.openstack.org/#/c/230375/17/puppet/manifests/overcloud_controller_pacemaker.pp,cm 13:36:45 Imre Farkas proposed openstack/instack-undercloud: Revert "Disable tempest" https://review.openstack.org/234216 13:37:46 Imre Farkas proposed openstack/instack-undercloud: Revert "Disable tempest" https://review.openstack.org/234216 13:40:44 spredzy: ack that looks fine to me :) in case we'd have problems like "keystone reported to pacemaker that it started, but when puppet tried to create endpoints, keystone wasn't ready yet to process requests", we can move it to step 5 anytime 13:41:57 Hey guys, if anybody has time to check out this CR https://review.openstack.org/#/c/231930/ I would appreciate the input 13:42:40 Florian Fuchs proposed openstack/tripleo-common: [WIP]Adds deployment configuration methods https://review.openstack.org/230315 13:43:01 jistr, ack. Although on the top review out of the 3 are you ok with the assignment of the IP (ie. public, internal, admin) ? 13:44:14 Ryan Brady proposed openstack/tripleo-common: [WIP]Initial workflow implementation of tripleo-common https://review.openstack.org/227297 13:47:11 spredzy: hmm i'm not sure we should use KeystoneAdminApiVirtualIP for all services... (not saying we shouldn't, just really not sure). Do you know what the CLI does presently? Maybe thrash or regebro would know? 13:47:34 spredzy: public/internal are fine i think 13:47:59 jistr, ack will wait for feedback, I used KyestonAdmin as this was on the only ip and the ctl_network (based oon SericeNetMap) 13:50:19 spredzy: jistr we use the KeystoneURL output for the public keystone endpoint. 13:51:07 spredzy: or is the question for the other endpoints? 13:57:03 thrash|biab: yeah public should be fine i think, but we're wondering about admin endpoints 13:57:35 dprince: do you have everything you need to schedule the tripleo sessions at the summit? 13:58:01 thrash|biab: do you know how admin endpoints are set presently by the CLI? 13:58:25 thrash|biab: are they the same as the internal endpoints, with the exception of the CLI 13:58:31 slagle: I think it is all on the etherpad right? 13:58:40 thrash|biab: duh, i meant to say "with the exception of Keystone" 13:59:01 slagle: https://etherpad.openstack.org/p/tripleo-mitaka-proposed-sessions 13:59:02 dprince: those are the proposals, and then the actual details of each session get filled out in cheddar 13:59:11 dprince: did you get an email from ttx about cheddar? 13:59:17 slagle: not yet 13:59:33 dprince: see if you can login here http://design-summit-prep.openstack.org/ 13:59:49 it's telling me i'm not a track admin, so he may have already set you up 14:00:10 slagle: boom, got it 14:00:19 k, cool 14:00:21 slagle: I will try to fill these out today, thanks 14:00:29 you should be able to refine the detail as we get closer too 14:00:39 slagle: I see 5 slots, (2 fish, 1 workroom, 2 meetups) 14:01:00 yes, that's what we got 14:01:05 and is the same as https://www.openstack.org/summit/tokyo-2015/schedule/design-summit 14:01:45 spredzy: IIRC admin endpoints are currently set to the same value as internal ones, except Keystone (hopefully thrash|biab can ack that info when he's back). I'm inclined to keep it that way for now, even though it might not be the "correct final state". I'm thinking if we want to change it, we should probably have and InternalApiVirtualIP for each service, instead of using the Keystone one everywhere. I think that could/should be done in a 14:01:45 different patch though. 14:02:02 slagle: yes, the tbd's will go away when I fill those in I guess 14:02:20 meeting time? 14:02:29 trown: next week I think 14:02:47 dprince: it is biweekly? 14:02:50 trown: we are doing alternate meetings I think 14:02:50 yeah is still scheduled as biweekly dprince, was wondering if we should revisit 14:03:02 spredzy: i think keeping the heat_admin_url *inputs* is ok though 14:03:19 marios: well, perhaps we should, because 3 people have asked me about it this morning already ;) 14:03:30 dprince: it is just the much more convenient time 14:03:32 oh... was really hoping we could hash out the stable branch spec, as it is now getting a bit urgent for RDO 14:03:53 spredzy: otoh if they're the same as heat_internal_url etc. we could also get rid of them and add them later... so wrt inputs i'm fine either way 14:03:55 ie, if we do not have stable/liberty upstream, we kind of need to maintain something downstream soon 14:06:08 trown: other than minor tweaks to the spec I think it would be safe to start working towards that 14:06:25 trown: was there something specific you wanted to clarify in the spec? 14:07:32 dprince: no, more just wanted to see us move forward with it, THT is especially urgent because the workaround for https://bugs.launchpad.net/bugs/1505669 would break master ceilometer 14:07:32 Launchpad bug 1505669 in Ceilometer "empty replica_set from config can clobber url-based replica set for mongodb" [Undecided,New] 14:07:50 dprince: and that bug is blocking HA for RDO-Manager 14:08:01 * spredzy reads backlog 14:08:22 jaosorior, actually I think it's pretty cool 14:08:28 jaosorior, have you tested it? 14:11:47 jistr, spredzy aren't glance-api and swift-proxy VIPs on the storage network? 14:12:33 gfidente, spredzy: i think they are, that should get handled by the ServiceNetMap correctly, right? 14:13:20 gfidente: i mean the GlanceInternalApiVirtualIP received by the controller resource should point to storage network i think 14:13:22 jistr, for the VIPs yes, but I'm not sure from which data are the internal keystone endpoints written 14:13:33 jistr, yes exactly 14:14:17 gfidente: https://review.openstack.org/#/c/230375/17/puppet/controller.yaml 14:14:18 jistr, gfidente https://github.com/openstack/tripleo-heat-templates/blob/master/overcloud-without-mergepy.yaml#L576 14:15:05 gfidente: what we were discussing is how to treat the 'admin_url' endpoints in keystone 14:15:31 gfidente: currently i *think* they point to the same as 'internal_url', except for keystone, which is on ctlplane 14:15:44 gfidente: ^^ that is when initialized by the CLI 14:15:58 gfidente: i'd probably keep that behavior when we move to puppet 14:17:12 gfidente: if we want to set the 'admin_url' for all services to something else than their 'internal_url', that can be done by a later patch i think 14:19:47 admin_url can very well remain set to ctlplane 14:19:54 if you ask me :) 14:20:26 gfidente, and for the glance-api and swift-proxy what is your take ? You'd like me to change them ? 14:20:45 no no, it's fine the way it is now in the review 14:20:58 realying on the VIP sounds good to me 14:21:02 *relying 14:24:06 gfidente: see line 709 please https://review.openstack.org/#/c/230375/17/puppet/controller.yaml 14:24:54 gfidente: what do you think about setting heat_admin_url by using {get_param: KeystoneAdminApiVirtualIP} ? 14:25:45 gfidente: it's kinda double bladed imho... maybe we should have all the admin_urls on ctlplane, but then perhaps we shouldn't use a keystone-specific variable for services which aren't keystone? 14:27:03 ^ if that's the expectation, then given the choice of 1) add those per-service variables vs. 2) set admin_urls to the values which are in internal_urls like the CLI does 14:27:30 i'm inclining towards option no. 2, because i think we shouldn't force spredzy to fix the world in a single patch 14:27:54 :) 14:28:33 and also no. 2 remains compatible with what CLI did 14:28:41 gfidente: +1 was just typing that 14:28:46 jistr: why wouldn't we? 14:28:56 I mean if he *can* fix it in a single patch... :)... 14:29:18 ahah hewbrocca you wish it was that simple ;) 14:29:25 gfidente: if someone for whatever reason says "oh no we have to put the admin_urls back where they were", it would be better to have it isolated in a single patch 14:29:43 git commit -m "World is fixed" 14:29:45 :) 14:29:53 yeah and I also +1 the doubts about overloading KeystoneAdminWhatever 14:31:03 jistr, gfidente so we agree on admin_url == internal_url in a first iteration ? 14:31:26 except for keystone that already provide a ctrl_pane wired parameter 14:31:31 spredzy: yes, i'm just typing it into gerrit comment so that it's visible 14:33:20 jistr, ack, will update on my side 14:33:43 spredzy, except keystone? keystone is not in the catalog 14:34:15 gfidente, I mean we have already access to keystone admin vip 14:34:31 so for the case of keystone we can use it directly, no ? 14:35:20 gfidente: i have keystone in catalog on my deployment 14:35:42 keystone is itself in the catalog? 14:35:50 gfidente: yes. i guess it can be useful for cases like "i have admin_url of keystone and i'd like to know the public one" or vice versa 14:35:51 identity I see 14:36:32 spredzy: yes keystone's admin_url should be using the keystone admin vip i think, that's probably the only exception 14:37:17 i think they added that exception to the CLI because in some net iso deployments the undercloud couldn't reach the overcloud keystone 14:37:27 to do the init 14:37:27 jistr, yeas was going to say the same 14:37:44 which also means, if we'll be doing the init from puppet 14:37:50 so ideally we don't need it anymore 14:38:35 we'll probably not need that exception anymore. but anyway i think it's good to keep backward compat with what CLI did 14:38:56 so my vote is +1 for keystone admin_url using keystone admin vip 14:39:20 gfidente: you're taking words from my mouth today more than usual 14:39:32 :P 14:39:40 bleah 14:39:46 oh cute :) 14:39:53 you guys spent too much time online together :p 14:40:12 I'm trying to figure where keystone_internal comes from now 14:40:42 keystone_auth_uri 14:42:05 that needs "fixing" though? 14:44:17 John Trowbridge proposed openstack/tripleo-docs: EPEL needed for instack-virt-setup https://review.openstack.org/233624 14:44:24 gfidente: you mean another input `keystone_internal_url` even though it will be the same value as `keystone_auth_uri`? 14:45:45 ah because that's called KeystonePublic but it is not the external net 14:46:07 should have been KeystoneApi 14:46:29 yeah 14:46:34 but we can't rename those params now 14:47:21 so it's good as is 14:48:08 I think 14:51:32 jaosorior, you around? 14:52:42 yeah i think it's good. If rename, then a separate patch i think. I'm thinking we could rename KeystonePublicApiVirtualIP (maybe?), but we probably can't rename the KeystonePublicApiNetwork entry in the ServiceNetMap 14:53:20 yeah that Public is a bit confusing but spredzy can't fix everything in a single patch 14:53:24 :P 14:56:21 keystone is confusing in general :) i think the distinction what was meant by that name was that port 5000 is considered public interface and 35357 is considered admin (even though the public 5000 api is used for internal usage within the cloud too). so there are multiple meanings of 'public' here i think :/ 14:56:44 Yanis Guenane proposed openstack/tripleo-heat-templates: Manage keystone related settings directly in t-h-t manifests https://review.openstack.org/230357 14:58:41 Merged openstack/tripleo-heat-templates: Add more components virtual ip mapping into controller. https://review.openstack.org/231361 15:00:16 Untriaged bugs so far: 15:00:18 https://bugs.launchpad.net/tripleo/+bug/1505430 15:00:18 Launchpad bug 1505430 in tripleo "Triggering puppet runs with config_identifier is error prone and complicates templates" [Undecided,New] - Assigned to Steve Baker (steve-stevebaker) 15:00:19 https://bugs.launchpad.net/diskimage-builder/+bug/1504632 15:00:19 Launchpad bug 1504632 in diskimage-builder "elements README documentation environment variables should be clearer" [Undecided,In progress] - Assigned to Augustina Ragwitz (missaugustina) 15:00:22 https://bugs.launchpad.net/tripleo/+bug/1504179 15:00:22 Launchpad bug 1504179 in tripleo "instack-virt-setup can't use dnsmasq lease file anymore" [Undecided,In progress] - Assigned to Thomas Herve (therve) 15:00:23 https://bugs.launchpad.net/tripleo/+bug/1503669 15:00:23 Launchpad bug 1503669 in tripleo "Including an additional element is inconvenient" [Undecided,New] - Assigned to Jaume Devesa (devvesa) 15:00:25 https://bugs.launchpad.net/tripleo/+bug/1504263 15:00:25 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 15:00:28 https://bugs.launchpad.net/tripleo/+bug/1504363 15:00:28 Launchpad bug 1504363 in tripleo "Updates to hiera configuration not propogating/making a puppet run" [Undecided,Confirmed] 15:04:18 spredzy: here's `keystone endpoint-list` from my deployment. It's not net-iso so we won't see anything interesting about the VIPs, but what caught my eye is that i have /v2.0 prefixes there for keystone too. Not sure if that's needed or not though. http://paste.fedoraproject.org/278744/48551144/raw/ 15:04:52 (and those prefixes are not in the patch) 15:05:16 jistr, I removed them because they are added later in the modules. In a previous patch set I had them in place causing :5000/v2.0/v2.0 entry in the db 15:05:29 causing any keystone client action to fail 15:05:30 spredzy: ah ok, thanks 15:06:12 jistr, cf. https://review.openstack.org/#/c/230357/6/puppet/controller.yaml,cm 15:08:28 +2 15:09:09 Florian Fuchs proposed openstack/tripleo-common: [WIP]Adds deployment configuration methods https://review.openstack.org/230315 15:22:43 Yanis Guenane proposed openstack/tripleo-heat-templates: Manage keystone related settings directly in t-h-t manifests https://review.openstack.org/230375 15:22:51 jistr, thanks ! 15:23:42 Ben Nemec proposed openstack/instack-undercloud: Remove some legacy stuff https://review.openstack.org/224125 15:23:53 jistr, gfidente comments addressed ^ 15:26:17 Merged openstack/tripleo-docs: EPEL needed for instack-virt-setup https://review.openstack.org/233624 15:31:45 +2 15:32:08 jistr, :) thanks. Testing w/o and w/ ha let's see what it gives us 15:51:28 Ben Nemec proposed openstack/diskimage-builder: Enable decimal value for $DIB_IMAGE_SIZE. https://review.openstack.org/190026 15:59:44 Merged openstack/tripleo-heat-templates: Parameterize RabbitMQ FD limit https://review.openstack.org/201796 16:03:45 Florian Fuchs proposed openstack/tripleo-common: [WIP]Adds deployment configuration methods https://review.openstack.org/230315 16:04:58 Giulio Fidente proposed openstack/tripleo-heat-templates: Ensure access to Redis is password protected https://review.openstack.org/210405 16:08:40 Ryan Hallisey proposed openstack/tripleo-heat-templates: Update docker compute environment to use json config https://review.openstack.org/234313 16:10:20 Ethan Gafford proposed openstack/tripleo-heat-templates: [WIP] Trove Integration https://review.openstack.org/233240 16:10:20 Ethan Gafford proposed openstack/tripleo-heat-templates: Sahara Integration https://review.openstack.org/220863 16:27:28 Emilien Macchi proposed openstack/instack-undercloud: neutron: enable nova-event-callback by default https://review.openstack.org/234325 16:30:18 Emilien Macchi proposed openstack/tripleo-heat-templates: Implement Workers parameters https://review.openstack.org/229471 16:31:34 Giulio Fidente proposed openstack/tripleo-heat-templates: Wire Neutron ML2 plugin and OVS agent settings as arrays https://review.openstack.org/187506 16:36:10 Giulio Fidente proposed openstack/tripleo-heat-templates: Wire Neutron ML2 plugin and OVS agent settings as arrays https://review.openstack.org/187506 16:41:14 dprince: should I abandon https://review.openstack.org/222002 ? 16:42:32 EmilienM: yeah, lets just go w/ https://review.openstack.org/#/c/223182/ 16:43:04 EmilienM: really good ideas on yours though which I think perhaps we might revisit someday when things mature 16:43:36 Merged openstack/instack-undercloud: Drop consoleauth and novncproxy https://review.openstack.org/226713 17:05:07 dprince: wfm 17:12:22 Emilien Macchi proposed openstack/instack-undercloud: neutron: enable nova-event-callback by default https://review.openstack.org/234325 17:13:39 Emilien Macchi proposed openstack/tripleo-heat-templates: neutron: enable nova-event-callback by default https://review.openstack.org/225962 17:15:23 lol 17:15:28 knew it 17:15:52 haha 17:15:57 i guess we were done anyway 17:16:25 shardy: the heat standalone work was done a while back and needs to essentially be reimplemented due to how kolla has evolved with config file handling 17:49:11 I'm trying to debug https://review.openstack.org/#/c/234325/ but I don't see much useful logs in undercloud, am I missing something? 17:53:08 Jeff Peeler proposed openstack/tripleo-heat-templates: Update docker compute environment to use json config https://review.openstack.org/234313 17:54:19 Steven Hardy proposed openstack/tripleo-specs: Add spec for release branch policy https://review.openstack.org/221811 18:08:06 https://review.openstack.org/228034 has been waiting a long time to pass CI, but I was finally able to fix tripleo-incubator to build in delorean 18:11:42 bnemec: shardy ^ 18:11:58 you just have to bother people directly :) 18:18:10 Jeff Peeler proposed openstack/tripleo-heat-templates: Update docker compute environment to use json config https://review.openstack.org/234313 18:18:20 EmilienM: you mean debug the CI failure? or locally? 18:22:39 slagle: the CI failure 18:22:43 slagle: ya I feel bad bothering the same people, but thanks 18:22:52 slagle: I looked in the logs and I don't find anything useful 18:23:24 shardy: sorry for the late -1 on the release branch policy even after I was the one saying we need this ASAP 18:24:02 EmilienM: there is a ~/.instack directory on the undercloud that has the install log in it. if that's not being saved off by CI, we need to fix that 18:24:03 shardy: if we do not maintain a branch for tripleo-docs though it will definitely have to be maintained downstream 18:41:59 slagle: trown: +A 18:42:22 trown: Also, good point about the docs. I completely missed that one, but it's also required as far as I'm concerned. 18:45:13 bnemec: ya I was hoping not to have to maintain a fork of the docs, because that will be painful I think 18:46:52 trown: No, you shouldn't have to. Stable branches are useless without matching docs. How would anyone know how to use them? 18:47:12 right 18:47:19 I mean, there have already been significant changes in how things like net-iso work in this release, and you'd be completely lost trying to apply the current docs to the old code. 18:48:11 slagle: I see https://review.openstack.org/#/c/229471/ is failing but my patch is not related to the undercloud and I don't see overcloud logs 18:48:44 Merged openstack/tripleo-incubator: jq 1.5 support https://review.openstack.org/228034 18:51:50 EmilienM: the previous review you posted when you asked for help was related to the undercloud, that's why i mentioned the undercloud logs 18:51:59 EmilienM: in 229471, the error is in the consoel output: 18:52:05 damn 18:52:13 2015-10-13 17:55:48.908 | | overcloud | a32707ac-42a8-4561-9f51-e44e6e9247e6 | Resource CREATE failed: resources.Controller: resources[0]: Parameter 'GlanceWorkers' is invalid: could not convert string to float: | CREATE_FAILED | 2015-10-13T17:54:49 | 18:52:26 ah damn, I'm dumb 18:52:44 slagle: for the first patch (undercloud), I haven't seen errors, except in neutron ovs agent 18:52:50 I'll look at it more 18:53:54 ok. it looks like the undercloud install logs are not saved in CI (needs to be fixed), but it's in the console output 18:54:12 2015-10-13 17:37:01.986 | [1;31mError: Failed to connect to Keystone server at http://192.0.2.1:35357/v2.0/tokens: Connection refused - connect(2)[0m 18:54:15 2015-10-13 17:37:01.986 | [1;31mError: /Stage[main]/Neutron::Server::Notifications/Nova_admin_tenant_id_setter[nova_admin_tenant_id]/ensure: change from absent to present failed: Failed to connect to Keystone server at http://192.0.2.1:35357/v2.0/tokens: Connection refused - connect(2)[0m 18:54:28 no idea if that is related to your change or not 18:54:36 at http://logs.openstack.org/25/234325/2/check-tripleo/gate-tripleo-ironic-overcloud-f21puppet-nonha/580206d/console.html 18:54:38 slagle: yeah 18:54:40 great... 18:54:47 slagle: thanks a lot man 19:06:35 Emilien Macchi proposed openstack/tripleo-heat-templates: Implement Workers parameters https://review.openstack.org/229471 19:06:57 * bnemec really needs to figure out how to fix building images in 8 GB VMs. 19:07:13 4 works, 12 works, but at 8 I run into OOMs. 19:07:40 bnemec: have you reduced workers? 19:09:44 EmilienM: The problem is that at 8 GB the image build happens in a ramdisk, and that seems to eat up too much memory. The reason it works at 4 GB is we skip the ramdisk on low memory systems. 19:10:50 slagle: I found why https://review.openstack.org/#/c/234325/ and https://review.openstack.org/#/c/225962/ are failing 19:11:14 slagle: because we're not managing keystone resources during the puppet run (service, endpoints, users, etc) 19:11:26 Although I have to say the situation has improved for some reason. It used to kill the image build, now it just kills all the OpenStack services. :-) 19:11:54 slagle: so for both undercloud & overcloud, puppet-neutron queries keystone to get some tenant ID, etc but it can't work since our manifests do not manage keystone resources 19:12:18 slagle: that's what is doing spredzy https://trello.com/c/lMKsj2Uo/245-tripleo-manage-keystone-resources-with-puppet 19:12:27 which means we are stuck I think 19:12:33 yea, i've seen the overcloud patches 19:12:50 we just need to do the same for the undercloud right? 19:12:58 slagle: yes 19:13:07 maybe he already did it, let me check 19:13:15 i don't recall seeing a patch 19:13:28 EmilienM: do the puppet modules handle endpoint updates and user password changes? 19:13:48 slagle: https://review.openstack.org/#/c/231395/ 19:14:13 slagle: endpoint updates -> yes, password change -> let me check 19:14:56 slagle: yes it supports it 19:15:19 we haev a param for that "replace_password" 19:15:26 default to True 19:16:04 ok, cool 19:36:33 Ian Wienand proposed openstack/diskimage-builder: Remove extra install of release pkgs in fedora-minimal https://review.openstack.org/227131 19:36:34 Ian Wienand proposed openstack/diskimage-builder: Reset yum/dnf cache to correct location https://review.openstack.org/232863 19:36:35 Ian Wienand proposed openstack/diskimage-builder: Add Fedora 22 support to yum-minimal https://review.openstack.org/227127 19:36:36 Ian Wienand proposed openstack/diskimage-builder: Update default fedora-minimal to f22 https://review.openstack.org/227126 19:49:21 Lokesh Jain proposed openstack/tripleo-heat-templates: Changes for configuring Nuage https://review.openstack.org/230116 20:47:34 investigating https://review.openstack.org/#/c/213175/ - I see no logs on overcloud/controller in /var/log/httpd 20:47:43 slagle: any thought? 20:49:01 I see why it fails http://logs.openstack.org/75/213175/6/check-tripleo/gate-tripleo-ironic-overcloud-f21puppet-nonha/7dd7f9d/console.html#_2015-10-13_14_24_28_162 20:49:05 but no chance to see apache logs 20:55:38 EmilienM: all of /var/log/ should be included in the CI output 20:55:47 if httpd is empty, apache probably didnt create any logs 20:56:04 slagle: weird, it could not start 20:56:18 puppet was not able to start it, I'm trying to reproduce 20:57:56 all i see in the systemd journal is Oct 13 10:14:00 overcloud-controller-0 systemd[1]: httpd.service: main process exited, code=exited, status=1/FAILURE 21:00:18 Untriaged bugs so far: 21:00:19 https://bugs.launchpad.net/tripleo/+bug/1505815 21:00:19 Launchpad bug 1505815 in tripleo "Docs refer to ironic-discoverd instead of ironic-inspector" [Undecided,New] 21:00:20 https://bugs.launchpad.net/tripleo/+bug/1504363 21:00:21 Launchpad bug 1504363 in tripleo "Updates to hiera configuration not propogating/making a puppet run" [Undecided,Confirmed] 21:00:23 https://bugs.launchpad.net/tripleo/+bug/1504179 21:00:23 Launchpad bug 1504179 in tripleo "instack-virt-setup can't use dnsmasq lease file anymore" [Undecided,In progress] - Assigned to Thomas Herve (therve) 21:00:25 https://bugs.launchpad.net/tripleo/+bug/1504263 21:00:25 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 21:00:26 https://bugs.launchpad.net/tripleo/+bug/1505430 21:00:26 Launchpad bug 1505430 in tripleo "Triggering puppet runs with config_identifier is error prone and complicates templates" [Undecided,New] - Assigned to Steve Baker (steve-stevebaker) 21:40:57 Emilien Macchi proposed openstack/instack-undercloud: puppet: drop some warnings https://review.openstack.org/234446 21:40:58 dprince: ^ 21:42:54 EmilienM: ack, thanks 21:43:31 dprince: I can't find in logs why apache does not start on https://review.openstack.org/#/c/213175/ 21:43:50 dprince: the error is http://logs.openstack.org/75/213175/6/check-tripleo/gate-tripleo-ironic-overcloud-f21puppet-nonha/7dd7f9d/console.html#_2015-10-13_14_24_28_162 21:43:55 but I can't see why httpd fails 21:45:06 EmilienM: gotta run, will look into this in a bit :) 21:45:09 np 00:36:05 Zane Bitter proposed openstack/tripleo-heat-templates: Prototype SW deployments for Controller upgrades https://review.openstack.org/234485 00:53:07 Colleen Murphy proposed openstack/diskimage-builder: Fix devuser pubkey defaults https://review.openstack.org/234490 02:08:29 Juan Antonio Osorio Robles proposed openstack/python-tripleoclient: Make root directory configurable https://review.openstack.org/230275 03:00:16 Untriaged bugs so far: 03:00:27 https://bugs.launchpad.net/tripleo/+bug/1505815 03:00:27 Launchpad bug 1505815 in tripleo "Docs refer to ironic-discoverd instead of ironic-inspector" [Undecided,New] 03:00:28 https://bugs.launchpad.net/tripleo/+bug/1504363 03:00:29 https://bugs.launchpad.net/tripleo/+bug/1504263 03:00:29 https://bugs.launchpad.net/tripleo/+bug/1505430 03:00:29 Launchpad bug 1504363 in tripleo "Updates to hiera configuration not propogating/making a puppet run" [Undecided,Confirmed] 03:00:30 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 03:00:31 Launchpad bug 1505430 in tripleo "Triggering puppet runs with config_identifier is error prone and complicates templates" [Undecided,New] - Assigned to Steve Baker (steve-stevebaker) 07:51:38 What is our deprecation policy for the CLI interface. Do we have an official one? 07:51:47 Looking at this change - https://review.openstack.org/#/c/232105/ 07:52:14 shardy: You mentioned deprecation the other day, what do you think? 07:54:04 d0ugal: Most projects would not allow that sort of change, you'd have to support the old option with a warning for a while 07:54:22 d0ugal: In heat we normally observe at least 2 cycles (e.g a year) 07:54:27 I think that is typical 07:54:45 Just ripping out options is not cool at all IMHO 07:56:32 d0ugal: has that option been shipped in either a tagged pypi release, or downstream packages? 07:57:06 shardy: Yeah, that makes sense - I guess we just need to figure out our poilicy and go with it 07:57:13 I am not sure if it is released anywhere :) 07:57:23 Not knowing what is and isn't released is part of the issue 07:57:56 d0ugal: that is a good reason to start releasing regularly via pypi and rebasing downstream on those releases 07:58:05 that's what most other clients do AFAIK 07:59:09 shardy: Makes sense, I guess I might need to take this to openstack-dev :) 07:59:51 d0ugal: sounds good, I'll reply there 08:00:40 shardy: Do you happen to know if there is an online reference I can use for other projects? like Heat? 08:05:06 d0ugal: http://governance.openstack.org/reference/tags/assert_follows-standard-deprecation.html is one reference 08:05:20 I think that's derived from some previous docs but I'm failing to find them atm 08:05:30 * shardy notes Heat needs to be added there 08:06:16 shardy: Thanks 08:06:17 d0ugal: for the most part it's common sense tho, e.g don't break your users without fair warning ;) 08:07:19 Sure, that is nice in theory :) 08:20:05 Derek Higgins proposed openstack/tripleo-common: tripleo.sh Add option to locally build delorean package https://review.openstack.org/228157 08:20:05 Derek Higgins proposed openstack/tripleo-common: tripleo.sh Add option to create a local Delorean environment https://review.openstack.org/228156 08:21:36 Derek Higgins proposed openstack-infra/tripleo-ci: Use tripleo.sh to build packages https://review.openstack.org/229425 08:25:14 Yanis Guenane proposed openstack/tripleo-heat-templates: Manage keystone related settings directly in t-h-t manifests https://review.openstack.org/230357 08:42:27 Has any of you already encountered "ERROR: InterpreterNotFound: python3.4" in python-tripleoclient py34 tox job ? 08:42:49 (link. http://logs.openstack.org/95/231395/4/check/gate-python-tripleoclient-python34/a71fadb/console.html) 08:44:01 spredzy: yes but with tripleo-common! 08:44:57 spredzy: http://logs.openstack.org/08/234108/1/check/gate-tripleo-common-python34/d05120f/console.html 08:45:52 gchamoul, any workaround ? 08:47:24 spredzy: on the gate, nope ... but is py34 tox job working locally? 08:49:49 gchamoul, locally I have gcc issues (cffi related), so I was expecting the gate to let me know :) 09:00:17 Untriaged bugs so far: 09:00:18 https://bugs.launchpad.net/tripleo/+bug/1505815 09:00:18 Launchpad bug 1505815 in tripleo "Docs refer to ironic-discoverd instead of ironic-inspector" [Undecided,New] - Assigned to Ruchi Parmar (ruchiparmar143) 09:00:19 https://bugs.launchpad.net/tripleo/+bug/1504363 09:00:19 Launchpad bug 1504363 in tripleo "Updates to hiera configuration not propogating/making a puppet run" [Undecided,Confirmed] 09:00:21 https://bugs.launchpad.net/diskimage-builder/+bug/1505900 09:00:22 Launchpad bug 1505900 in diskimage-builder "Failed to create 'wily' release with DIB_RELEASE" [Undecided,New] 09:00:23 https://bugs.launchpad.net/tripleo/+bug/1504263 09:00:23 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 09:00:25 https://bugs.launchpad.net/tripleo/+bug/1505430 09:00:25 Launchpad bug 1505430 in tripleo "Triggering puppet runs with config_identifier is error prone and complicates templates" [Undecided,New] - Assigned to Steve Baker (steve-stevebaker) 09:03:36 Hi. What should I do when I find a DELETE_FAILED status when removing the overcloud stack? 09:03:41 Imre Farkas proposed openstack/instack-undercloud: Mostly revert "Disable tempest" https://review.openstack.org/234216 09:05:40 shardy: ^ Can you help me? 09:10:09 devvesa: Hi, there is occasionally a race between nova and ironic which can cause this - the best thing to do is to try to delete the stack again 09:10:18 devvesa: does nova list show instances in ERROR state? 09:10:49 nova instances are in shutoff state 09:11:08 I'll try to start again and run stack-delete again 09:12:07 fails again: DELETE_FAILED status on stack-list 09:12:44 and nova instances are ACTIVE 09:24:33 devvesa: what reason is given in heat stack-show overcloud? 09:24:57 there is a stack status_reason field which may tell us why the delete failed 09:26:38 http://paste.openstack.org/show/476229/ 09:27:21 It complains about the user: Could not find user: 080ff33bd1dc455c86c4aae723d180d1 09:27:34 But this user is not in the `keystone user-list` response 09:27:38 Merged openstack/tripleo-heat-templates: Set shared secrets, keys and passwords as hidden https://review.openstack.org/233578 09:31:40 devvesa: are you running the latest upstream TripleO? 09:32:43 shardy: you mean the tripleo-heat-templates project? 09:32:46 devvesa: also can you check the heat-engine log on the undercloud and see if there's an associated backtrace with that failed delete 09:33:14 devvesa: well everything, but particularly is the heat on the undercloud up to date? 09:33:34 I created the instack virtualmachine where I am working on last Friday 09:33:51 devvesa: Ok, so it should be close to lastest master, thanks 09:35:49 shardy: I find the same stack trace on the heat engine log about invalid user 09:36:36 I'm wondering from where it gets this user 09:37:44 ok. I'll create the instack machine again 09:38:17 devvesa: sec, I'm trying to reproduce now 09:38:21 devvesa: http://hardysteven.blogspot.co.uk/2014/04/heat-auth-model-updates-part-2-stack.html 09:38:31 the users are created internally by heat, in the heat_stack domain 09:38:39 so you won't see them via keystone user-list 09:38:58 http://fpaste.org/278997/44815459/ 09:39:07 that shows how to list them via openstackclient 09:39:24 if deleting one of those users causes the stack delete to fail, we have a heat regression 09:41:45 shardy: I don't know if I already gave you feedback. But your blog posts have been extremely useful! 09:42:02 jaosorior: thanks! I really need to find time to re-start them 09:42:25 specially the one about debugging heat templates helped me a lot 09:42:35 * hewbrocca notes that we're all so busy fighting off alligators, we've forgotten the mission was to drain the swamp 09:42:47 great, that is good to hear :) 09:42:52 shardy: http://fpaste.org/278999/14448157/ 09:43:01 I think much of that info found it's way into the tripleo-docs now too 09:43:13 hewbrocca: heh, very true ;) 09:43:44 shardy: Still don't see the user that causes the error here 09:43:51 shardy: reading you blog post, btw :) 09:47:25 devvesa: ouch. I've just reproduced, it's a heat bug 09:47:34 devvesa: looking into a fix 09:47:44 shardy: ouch 09:48:13 shardy: can I help you in any way? 09:48:36 devvesa: not atm, sorry for the inconvenience, you can help test the fix when I work it out :) 09:48:54 shardy: sure 09:49:09 shardy: where this user comes from, then? 09:49:10 devvesa: it does raise the question of why the user was deleted - can you see anything earlier in the heat logs referencing that user ID? 09:49:34 shardy: which log? the heat-engine one? 09:49:42 devvesa: it's all in that blog post - each SoftwareDeployment resource creates a special user, inside the heat_stack domain 09:49:57 shardy: Ok. I'll keep reading :) 09:50:02 that user is the identity used for signalling back to heat after the deployment create/update completes (or fails) 09:50:12 devvesa: yup, the heat-engine one 09:50:40 I recreated by manually deleting the user, but it'd be good to understand why it happened in your case, because really that user should still exist 09:53:05 shardy: http://fpaste.org/279014/44816375/ 09:53:43 In the beggining of the log, it returns data about the user, but suddenly (line 14), it seems it has been deleted 09:53:48 I'm sorry if this has been asked many times before; But is there somewhere I could find documentation for the HA implementation? Tried searching in the docs but wasn't very successful 09:55:00 devvesa: did the DELETE line happen during a previous attempt at a stack-delete? 09:56:03 * devvesa is diving in logs 09:56:07 devvesa: e.g line 15 is definitely a bug, but there may be another one, e.g why did we delete the user, and not fully persist the delete of the associated resource 09:56:26 jaosorior, like informations on the general architecture? 09:56:37 jaosorior, or on the templates/manifests bits? 09:59:10 gfidente: Well, I could do with the general architecture, but if you have links to the specific templates/manifests bits that would be very helpful 09:59:53 jaosorior, so regarding the architecture I think best source is https://github.com/beekhof/osp-ha-deploy/blob/master/ha-openstack.md ; this pretty much describes what the templates/manifests should be implementing 10:00:19 devvesa: https://bugs.launchpad.net/heat/+bug/1505981 10:00:19 Launchpad bug 1505981 in heat "SoftwareDeployment delete fails if stack domain user is missing" [High,Triaged] - Assigned to Steven Hardy (shardy) 10:00:26 you can add any further clues there :) 10:00:31 shardy: http://fpaste.org/279033/44481681/ 10:01:02 This is the first attempt to delete the overcloud. A minute before the deletion of the user 10:01:21 (heat-api.log) 10:02:05 devvesa: Ok, thanks, and nothing unexpected happened during that delete, for example restarting heat-engine? 10:02:56 No. I was tracing constantly executing 'heat stack-list' to check out when it was deleted 10:03:14 but regarding the actual implementation I don't think we have much, the only hint I can give is that in addition to the regular puppet dependencies, it gets deployed in steps to have some hard-stops (which we should at some point validate) 10:04:39 gfidente: So, only thing I knew is that deploying 3 controllers was the requirement. And I see there is pacemaker and HAProxy already running in the controllers. However, how the load balancing is actually working there is still pretty vague for me :/ 10:05:24 instack-virt-setup is stuck in a loop for me displaying "instack never got an IP address from the libvirt default network." 10:05:28 Is this a known issue? 10:06:19 d0ugal: did the instack VM boot OK? 10:06:49 I hit a similar issue yesterday and it failed to boot, re-running instack-virt-setup after destroying/underfining fixed it 10:07:45 shardy: Thanks, I'll try that 10:10:16 jaosorior, the thing is that we have two archs, one uses pacemaker and the other doesnt, but only the one which uses pacemaker is currently capable of deploying multiple controllers 10:10:28 10.35.7.32/w 10 10:10:30 oops 10:10:31 the haproxy part is in common though, so it's in a shared module 10:10:50 jaosorior, https://github.com/openstack/puppet-tripleo/blob/master/manifests/loadbalancer.pp 10:11:24 the upstream nonha job is actually deploying the non-pacemaker arch 10:12:16 gfidente: Thanks for the link. that's the manifest I was checking out when I did the TLS stuff. there's where I add the path so HAProxy gets the service certificate. But I don't have it very clear how the controllers and their endpoints behave with the pacemaker arch 10:13:41 jaosorior, oh you mean how we configure the endpoints in keystone? 10:14:58 gfidente: Well, I would like to understand how it works in general; But ultimately what I need is to know how the endpoints will be configured and how they will behave in case of failure of a node 10:15:53 Yanis Guenane proposed openstack/instack-undercloud: Manage keystone related configuration via Puppet https://review.openstack.org/234704 10:16:08 they point to the haproxy vips, we used to do this as a post-deployment step from a tool called os-cloud-config but this is moving into puppet now, spredzy working on it, check this series : https://review.openstack.org/#/c/230375/ 10:16:21 jaosorior, ^^ 10:19:11 jaosorior, as for now everything happens in here https://github.com/openstack/os-cloud-config/blob/master/os_cloud_config/keystone.py 10:20:18 devvesa: http://paste.openstack.org/show/476232/ 10:20:30 that's the fix - you can manually add those two lines to work around the issue 10:20:38 proper heat patch in-progress 10:20:43 thanks! 10:21:03 devvesa: it's a regression caused by some internal heat refactoring during liberty, unfortunately 10:21:19 shardy: Good to find it before the release then :) 10:21:52 heh, yeah, although a few hours ago we confidently declared there was no need for an RC3 in the heat meeting 10:21:56 * shardy sighs 10:22:07 gfidente, spredzy: Alright, I'll take a look. Thanks for the info :D 10:27:48 Merged openstack/python-tripleoclient: Move common functions out into utils.py https://review.openstack.org/225244 10:32:06 Juan Antonio Osorio Robles proposed openstack/python-tripleoclient: Make root directory configurable https://review.openstack.org/230275 10:37:01 shardy: hi, i thought about https://review.openstack.org/#/c/224815/ -- if we didn't want another resource registry entry, i think we could generalize it from controller_pacemaker.yaml to controller_variant.yaml and have something like ControllerVariantHieraConfig emitted from controller-config-pacemaker.yaml via an output, and then have ControllerVariantHieraDeployment be in controller-post.yaml. This would keep the abstraction of the 10:37:01 controller-config*.yaml files just providing configs without having to accept the `servers` param. However, i'm not sure how would we no-op that. Even controller-config.yaml (the non-pacemaker one), which doesn't need special hiera, would probably have to contain ControllerVariantHieraConfig resource, which would contain no hiera keys 10:38:41 another registry entry is a possibility too i think, but the drawback there is that we'd be switching two registry entries always together in sync, which gives a hint that it should really be a single resource switch perhaps 10:41:21 ... given that both of those resources would essentially specify controller configuration, just a different part of it. One would configure hiera, the other one would switch the puppet manifest file -- sounds like they should be a single resource. 10:45:54 shardy: hi 10:46:21 jistr: Yeah, I've been thinking about it too, and I think there may be a simpler way - since liberty heat can join multiple lists, so I think we can conditionally join the additional hieradata inside controller.yaml 10:46:39 jistr: I'm about to go catch a train, but I'll pull your patch and work on an example which I'll share later 10:46:50 jprovazn: hi! 10:47:04 shardy: tricky heat question... 10:47:16 here I define resource dependency: https://review.openstack.org/#/c/233573/1/puppet/compute.yaml 10:47:22 shardy: ok, sounds good, thanks :) 10:47:38 where RebootDeployment depends on UpdateDeployment (intention is to run reboot after update) 10:48:47 shardy: but in real life, this dependency is not respected on when heat-config runs deployment scripts - it evaluates JSON (deployments key in JSON) in order it's passed to it, and this order looks like this: 10:48:52 http://paste.openstack.org/show/476234/ 10:49:29 reboot script is in this JSON before update - line ~634 10:49:51 shardy: am I missing some important bit about defining dependencies? 10:50:08 marios, it was unfair to ask about os-apply-config though! 10:50:14 jprovazn: that doesn't sound right - the RebootDeployment should not exist in the metadata at all until UpdateDeployment is CREATE_COMPLETE 10:50:29 marios, what if I do extract it with list_join like you did for vni ranges but don't test it? 10:50:30 shardy: this is UPDATE action 10:50:34 jprovazn: we use that pattern everywhere, e.g all the steps of puppet deployments 10:51:02 jprovazn: Hmm, it should work the same, if it's not then please raise a heat bug 10:51:35 jprovazn: the tricky thing will be if UpdateDeployment doesn't need an update, then you will go directly to RebootDeployment 10:51:52 gfidente: reading 10:52:52 gfidente: well gfidente, you're welcome. seriously though, the issue has come up recently on other reviews. do we even bother trying to maintain parity with the os-apply-config templates? our ci doesn't even use them anymore 10:54:43 jprovazn: have to go afk now, I'd suggest creating a minimal template example, then using heat resource-signal to step through each deployment 10:55:02 then you should be able to prove/disprove if it's working as expected or if there's a bug 10:55:08 gfidente: you know, i also only just notice dyour comment at jun 16 there. so, you could also drop that part altogether i think? (i.e. the os-apply-config/)? maybe this is something to bring up at weekly too 10:55:26 Florian Fuchs proposed openstack/tripleo-common: [WIP]Adds deployment configuration methods https://review.openstack.org/230315 10:56:23 marios, indeed we know it doesn't work with oac as-is because we had CI back in june, but we don't anymore so breakage would go unnoticed and we have to decide what to do with those I think at this point 10:56:39 kill-with-fire 10:57:10 shardy_afk: ok, thanks 10:58:13 marios, /me -1 the review and adding as topic for the weekly 10:59:39 gfidente: ok sounds good thanks 11:00:23 to be honest I am not sure if instack can get us an overcloud with OAC even if we wanted to 11:00:38 shardy: i've created a new environment using instack-virt-setup and then running the tripleo.sh scripts 11:00:47 it fails the deployment of the overcloud now 11:01:16 Looking at the heat-engine I see A LOT of ImportErrors 11:01:36 gfidente: i *think* we still use a couple of tripleo image elements if not mistaken. other than that, i have no idea. plus now with no more devtest.sh not easy to setup test env there 11:03:06 shardy: http://fpaste.org/279045/44482056/ <- is this normal? 11:05:10 But actually, I think the trace is this one: http://fpaste.org/279046/82067514/ 11:05:12 marios, actually I was thinking, maybe this is time to *not* paste template updates for the puppet implementation into the oac implementation anymore? 11:05:19 marios, so we don't break it either? 11:05:42 marios, yet we don't need to deprecate it as in theory it continues to work as-is ? 11:06:38 jistr, ^^ ? 11:06:47 gfidente: yeah exactly, this is what i meant before. just drop it. we did same for a review spredzy had recently. 11:07:15 oh okay so I was just late to the party, I'll update the review and mention this in the weekly anyway then 11:07:43 tks :) 11:08:29 +1 on dropping it 11:08:56 devvesa: how did you deploy the overcloud? (cli command? 1/3 control? x compute. any params/env files? is this vm setup i think?) 11:10:48 marios: ./tripleo-common/scripts/tripleo.sh --overcloud-deploy 11:11:14 from the instack/undercloud machine 11:11:20 devvesa: hm ok. those import errors are they from the controller? 11:11:42 I found them in the undercloud's heat-engine.log file 11:12:20 devvesa: ok, you could try logging onto the compute/controller and seeing if there is osmething interesting in there (like journalctl -fn 10000 | grep -ni error for ex) 11:12:29 devvesa: import errors in the heat-engine log are normal 11:12:38 devvesa: ssh heat-admin@node_ip then sudo su 11:12:41 (nova list) 11:13:01 dereckh: Ok. Then I'll look for another cause 11:13:03 devvesa: they just mean certain extensions can't be used, (if they are at the top of the log during startup 11:13:15 devvesa: also, fyi, heat resource-list overcloud | grep -ni fail (for ex) will show you which heat resource failed 11:13:49 marios: handy! I've deleted the stack (to see if it happenend again the previous error) and I am deploying again the overcloud 11:14:05 now with the `openstack overcloud deploy --templates` command 11:14:33 failed again: 11:14:44 [stack@instack ~]$ heat resource-list overcloud | grep -ni fail 11:14:44 20:| ComputeNodesPostDeployment | 1fc35e96-7562-40a2-9f4d-1bf2661999d6 | OS::TripleO::ComputePostDeployment | CREATE_FAILED | 2015-10-14T11:09:33 | 11:15:15 devvesa: ok at that point i'd look on the compute for more info 11:15:27 * devvesa in the compute right now 11:15:39 just the compute postdeploy failed? 11:15:43 yes 11:16:04 Giulio Fidente proposed openstack/tripleo-heat-templates: Wire Neutron ML2 plugin and OVS agent settings as arrays https://review.openstack.org/187506 11:16:05 And that's becoming interesting :) 11:16:58 http://fpaste.org/279048/44821403/ 11:17:02 marios, ^^ we need to hiera neutron::plugins::ml2::mechanism_drivers for the cisco ifs though; would need testing as well 11:20:43 devvesa: it is failing for package_manifest https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/manifests/overcloud_compute.pp#L105 11:20:55 devvesa: sounds like the images you have don't have https://review.openstack.org/#/c/203803/ 11:20:55 yes 11:22:03 I built the image based on the overcloud full plus my midonet elements: https://review.openstack.org/#/c/232551/ 11:22:18 something like: 11:22:48 Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Enable TLS deployment for the Overcloud https://review.openstack.org/231930 11:23:03 openstack overcloud image build --type overcloud-full --builder-extra-args overcloud-network-midonet 11:23:28 using Pino's patch: https://review.openstack.org/#/c/229778/ 11:23:54 devvesa: sec, trying to confirm something 11:25:29 puppet-tripleo in my compute host does not have this patch 11:25:32 devvesa: so, i think using export DIB_INSTALLTYPE_puppet_modules=source for image build like http://docs.openstack.org/developer/tripleo-docs/basic_deployment/basic_deployment_cli.html#get-images would probably fix that (so you'll get latest vrsion of say puppet-tripleo which is where the issue in this case) 11:26:08 marios: that could be the reason. I understand that package_manifest is not released yet in last puppet-tripleo puppetforge module 11:26:56 devvesa: i also think you are in rights to file abug "it don't work!" since tripleo.sh doesn't do that. perhaps we can document it there... though in time, this particular issue won't exist (i.e. the deployed puppet-tripleo will eventually have that) 11:27:09 devvesa: so not sure if bug worthy at this point 11:27:41 knowing the issue, i'll build the image again 11:27:41 devvesa: but simple matter remains ' i tried triplso.sh and it didn't work because of missing dependency' 11:30:04 maybe tripleo.sh works 11:30:22 because I built the image using `openstack overcloud image build` command 11:30:33 to specify my element 11:37:32 Yanis Guenane proposed openstack/instack-undercloud: Manage keystone related configuration via Puppet https://review.openstack.org/234704 11:41:21 Dmitry Tantsur proposed openstack/diskimage-builder: Fix tests/test_functions.bash https://review.openstack.org/229334 11:41:22 Dmitry Tantsur proposed openstack/diskimage-builder: Add a tox target to run functional tests locally https://review.openstack.org/229335 12:05:11 Dougal Matthews proposed openstack/tripleo-common: Collect the common logic in tripleo-common https://review.openstack.org/228991 12:10:15 Martin Mágr proposed openstack/tripleo-heat-templates: Remove deleted Cinder rows https://review.openstack.org/209019 12:12:25 Martin Mágr proposed openstack/tripleo-heat-templates: Remove deleted Nova rows https://review.openstack.org/207909 12:13:37 Martin Mágr proposed openstack/tripleo-heat-templates: Keystone domain for Heat https://review.openstack.org/180566 12:27:37 Martin Mágr proposed openstack/tripleo-heat-templates: Ceph cluster validation https://review.openstack.org/225248 12:37:20 imain proposed openstack/os-net-config: Add ip addr flush on ifdown to ensure IP addresses are removed. https://review.openstack.org/234176 12:39:36 Jan Provaznik proposed openstack/tripleo-heat-templates: Add a resource for node reboot https://review.openstack.org/233573 12:40:03 gfidente: hi, so you added an agenda item for this to next weeks meeting "OAC templates breakage, better to just leave those as they are now that we don't have CI?". What breakage are you referring too? 12:40:41 dprince, yeah maybe it's not necessary to discuss it at the meeting and it's just me behind 12:41:41 gfidente: I'm fine to discuss it, just wanted to know more what we are discussing? 12:41:45 but the idea is, shall we continue to update os-apply-config when puppet files gets new parameters or a switched for example from string to another type? 12:42:11 *or are switched 12:42:25 gfidente: oh, right. So I think we'll be moving os-apply-config to deprecated soon 12:42:31 gfidente: i think is worth having the discussion. for example, tht change that adds foo config item to puppet/controller and sthing in in controller/pacemaker. same change also updates the os-apply-config/controller 12:42:48 dprince: yeah that 12:43:26 gfidente: but sure, lets bring these things up next week 12:43:35 one more thing is, do we know if instack can deploy with oac? 12:44:02 gfidente: from an overcloud standpoint I don't see why not 12:44:16 gfidente: the requirement there is unchanged (Heat w/ Ironic) 12:44:20 yeah should be just the images which need appropriate build scriptds 12:45:02 dprince: did you comment on that patch? I don't see it.. 12:45:37 Slower: the os-net-config patch? not yet... haven't had a chance to kick the tires yet 12:45:56 dprince: I cant beleive all that f**kadoo with dhclient/nm etc resulted in such a simple patch.. 12:46:46 dprince: well voice your concerns damnit! :) 12:47:12 dprince: I hear your the new PTL? 12:47:43 Slower: sssh, tis a secret 12:47:51 uh oh 12:48:03 sorry! also sorry I'm canadian! 12:48:16 Slower: wait, you're Canadian? 12:48:34 hewbrocca: yes, sorry! 12:50:53 Dougal Matthews proposed openstack/tripleo-common: Collect the common logic in tripleo-common https://review.openstack.org/228991 12:52:15 Daisuke Fujita proposed openstack/os-cloud-config: Fix typos in usage.rst https://review.openstack.org/234771 12:58:54 Slower: commented on https://review.openstack.org/#/c/234176/2 12:59:12 d0ugal: thrash: hi, I'd like to get this merged https://review.openstack.org/#/c/230964/ + dependency: https://review.openstack.org/#/c/230969/, could you please take a look? 12:59:18 dprince: cool thx 12:59:31 jprovazn: looking 12:59:39 thanks 13:00:54 jprovazn: can you add a Depends-On to the tripleoclient patch? 13:01:04 jprovazn: nm 13:01:07 I missed it. :P 13:03:31 thrash: d0ugal: and this one too: https://review.openstack.org/#/c/226682/ 13:04:13 jprovazn: Looks like valid tests failures on https://review.openstack.org/#/c/230969/? 13:04:44 d0ugal: there is that cross repo dependency 13:04:59 once the first one is in, I can re-run CI 13:05:03 oh, I see 13:05:18 I thought CI was more intelligent :) 13:05:29 :) 13:05:47 d0ugal: same 13:09:10 thrash: d0ugal: thanks 13:10:50 Merged openstack/tripleo-common: Explicitly unset package update hooks when deleting a node https://review.openstack.org/230964 13:11:34 Merged openstack/tripleo-common: Fix wrong node count when deleting specific nodes https://review.openstack.org/226682 13:26:52 shardy_afk, dprince question re heat templates 13:28:06 suppose I want to make significant changes to a role 13:28:17 add/remove a service, etc. via editing the templates themselves 13:28:57 can I do that via a stack update? 13:29:09 without redeploying the world 13:29:28 hewbrocca: should be able to yes, puppet apply would apply those changes just fine I think 13:29:39 Right 13:29:43 that's what I was hoping 13:29:45 thanks dprince 13:46:13 Jan Provaznik proposed openstack/python-tripleoclient: Explicitly unset package update hooks when re-deploying https://review.openstack.org/230969 13:48:09 hewbrocca: yup that's correct, but you'd need to be careful not to change anything like firstboot scripts, as when certain properties of a node/server change heat will replace it 13:48:25 if it's only puppet manifests/hieradata tho it should be fine 13:57:57 Emilien Macchi proposed openstack/instack-undercloud: neutron: enable nova-event-callback by default https://review.openstack.org/234325 14:02:43 Yanis Guenane proposed openstack/instack-undercloud: Manage keystone related configuration via Puppet https://review.openstack.org/234704 14:46:46 Merged openstack/instack-undercloud: Don't use dnsmasq lease file to retrieve address https://review.openstack.org/232584 15:00:17 Untriaged bugs so far: 15:00:21 https://bugs.launchpad.net/tripleo/+bug/1505815 15:00:21 Launchpad bug 1505815 in tripleo "Docs refer to ironic-discoverd instead of ironic-inspector" [Undecided,Confirmed] - Assigned to Ruchi Parmar (ruchiparmar143) 15:00:21 https://bugs.launchpad.net/tripleo/+bug/1504363 15:00:22 https://bugs.launchpad.net/diskimage-builder/+bug/1505900 15:00:22 Launchpad bug 1504363 in tripleo "Updates to hiera configuration not propogating/making a puppet run" [Undecided,Confirmed] 15:00:24 https://bugs.launchpad.net/tripleo/+bug/1504263 15:00:24 Launchpad bug 1505900 in diskimage-builder "Failed to create 'wily' release with DIB_RELEASE" [Undecided,New] 15:00:26 https://bugs.launchpad.net/tripleo/+bug/1505430 15:00:26 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 15:00:28 Launchpad bug 1505430 in tripleo "Triggering puppet runs with config_identifier is error prone and complicates templates" [Undecided,New] - Assigned to Steve Baker (steve-stevebaker) 15:20:21 Vinay Rao proposed openstack/tripleo-heat-templates: This change makes the following parameters user configurable: https://review.openstack.org/230764 15:54:19 Emilien Macchi proposed openstack/instack-undercloud: neutron: enable nova-event-callback by default https://review.openstack.org/234325 16:03:23 Ben Nemec proposed openstack/python-tripleoclient: Set --min-tmpfs for overcloud-full build https://review.openstack.org/234860 16:04:44 marios: shardy: available? 16:11:58 devvesa: barely wasup 16:12:18 I still have problems deploying the overcloud... and these ones even weirder 16:13:12 The ./tripleo.sh scripts create the two 'barmetal vms' and I can see them on `nova list` 16:13:27 the controller has the ip : 192.0.2.20 16:13:50 When I log into it and I try to follow the puppet executions 16:14:01 I see the following when execution db_syncs: 16:14:35 2015-10-14 16:03:38.528 4384 TRACE cinder DBConnectionError: (OperationalError) (2003, "Can't connect to MySQL server on '192.0.2.18' (113)") None None 16:14:52 No node belongs the address 192.0.2.18 16:15:49 devvesa: this should be ok is the vip... eventually the dbconnection should succees 16:16:14 devvesa: do you see any other errors like journalctl -fn 10000 | grep -ni error 16:16:27 Ben Nemec proposed openstack/python-tripleoclient: Set --min-tmpfs for overcloud-full build https://review.openstack.org/234860 16:16:47 marios: only the dbconnections one 16:17:02 devvesa: what about the compute? 16:17:25 I have to say that the cinder database (for instance) has been created but the tables are not there... 16:18:23 devvesa: yeah for example at this point probably 'mysql' works fine 16:19:09 lots of errors on compute 16:19:31 ERROR oslo_messaging._drivers.impl_rabbit [req-736d10dc-2861-436d-a35d-b97a308eb12a - - - - -] AMQP server on 192.0.2.20:5672 is unreachable 16:19:47 devvesa: id say grab those errors to fpaste for now in case you need to file a bug, i have to run momentarily 16:19:56 devvesa: what about heat, which resources fail? 16:20:02 they are stuck 16:20:25 devvesa: yeah grab like heat resource-list overcloud at the very least 16:20:55 http://fpaste.org/279161/48396491/ 16:20:59 Ben Nemec proposed openstack/python-tripleoclient: Don't download fedora-user as part of --all build https://review.openstack.org/234870 16:21:35 devvesa: oh so it didn't fail yet? 16:21:35 I have to leave as well. Keep talking tomorrow if you don't mind 16:21:52 devvesa: sure thing 16:38:26 has anyone ever seen "/usr/bin/instack-virt-setup: line 175: ip: command not found" when running instack-virt-setup even though iproute is definitely installed and the ip command works? 16:39:53 trown: i haven't, is it $PATH related? 16:40:19 Merged openstack/diskimage-builder: Add Fedora 22 support to yum-minimal https://review.openstack.org/227127 16:40:31 slagle: that is my only thought as well, this just started happening this afternoon 16:40:46 does sudo ip work? 16:41:18 slagle: yep 16:42:35 slagle: it must be khaleesi related, I will take it up with those folks 16:43:04 oh you didnt say khaleesi 16:43:14 probably an ansible thing munging $PATH 16:43:16 Ben Nemec proposed openstack/python-tripleoclient: Don't parse redhat-release for distro info https://review.openstack.org/234885 16:47:11 does anyone know what could be messing with $PATH in the rdo-manager playbooks? I just started getting "/usr/bin/instack-virt-setup: line 175: ip: command not found" when doing instack-virt-setup 16:47:23 but if I log on to the virthost and run it, it works fine? 16:47:44 that last bit is not actually a question...it does work fine 16:47:55 trown: ip is in /usr/sbin. It might not be in the ansible user's path? 16:48:15 arg, wrong channel ... so much fail 16:48:21 thanks bnemec 16:49:23 trown: In general I try to stay out of khaleesi discussions though. That's a whole can of worms, and my plate is already full of worms. /mixed metaphors ;-) 16:50:00 bnemec: fair, luckily worms are full of protein 16:50:22 but you can only eat so many never the less 16:50:30 trown: I almost typed mixed meatphors. Yummy, yummy worm meat. :-D 17:07:42 Ben Nemec proposed openstack/python-tripleoclient: Don't parse redhat-release for distro info https://review.openstack.org/234885 17:29:36 James Slagle proposed openstack/python-tripleoclient: Install device-mapper-multipath on overcloud-full https://review.openstack.org/231690 17:32:17 Merged openstack/os-cloud-config: Add SSL support for Ironic registration https://review.openstack.org/195333 17:33:07 Does anybody know if the missing python3.4 interpreter problem has been reported to infra? 17:33:13 I keep seeing failures like this: http://logs.openstack.org/85/234885/2/check/gate-python-tripleoclient-python34/b24c4af/console.html 18:02:39 Ben Nemec proposed openstack/instack-undercloud: Add check for sufficient memory to undercloud install https://review.openstack.org/224852 18:03:41 Ryan Hallisey proposed openstack/tripleo-heat-templates: Update docker compute environment to use json config https://review.openstack.org/234313 18:22:10 Merged openstack/tripleo-docs: Add note that qemu libvirt-type is needed for virt environment https://review.openstack.org/233535 18:34:32 hi, I'm fighting with error "DRAC driver failed to create config job for node..." on one of our Dell servers during overcloud deploy 18:34:54 I've tried to restart conductor service, which didn't help 18:35:13 neither deleting the stack and attempting to recreate found the error on same server 18:35:35 any hints where to look for the cause? 18:37:55 kukacz: sounds like an issue on the DRAC itself, I have seen that where a config job gets stuck there and has to be manually cleared using iDRAC 18:38:30 trown: I did an iDRAC reset which did not help 18:38:41 is there anything else to make the job cleared? 18:39:26 trown: or what was the procedure you've applied? 18:41:14 kukacz: I have not actually done the clearing myself, so I am not sure exactly what that looks like, but the colleague I was working with went into the iDRAC console and cleared the jobs 18:41:57 trown: thanks a lot! I'll look for Dell's docs related 18:42:26 np. kukacz what gen are the servers do you know? 18:42:54 they are R630s, not sure what gen it is 18:43:01 iDRAC version is 8 18:45:29 kukacz: hmm, I think those are gen13, which I dont think we have seen that issue on 18:46:40 I've just cleared the job queue and am updating the stack again 19:03:41 Merged openstack/diskimage-builder: Update rhel7 element readme https://review.openstack.org/222351 19:10:45 EmilienM: can I use $::fqdn in a hiera file? 19:11:05 I'm not sure, last time I tried it did not work. Let me check again 19:11:09 i'm trying to specify a value for horizon::allowed_hosts 19:11:18 horizon::allowed_hosts: - $::fqdn 19:11:26 err 19:11:28 horizon::allowed_hosts: 19:11:30 yeah, I tried that last time, I had an issue 19:11:32 let me check 19:11:33 - $::fqdn 19:11:39 - '192.1.1.1' 19:11:43 something like that 19:12:11 it should be: horizon::allowed_hots: "%{::fqdn}" 19:12:13 for example 19:12:30 ok, and i can make that value a list? 19:12:33 yes 19:12:36 thx 19:13:24 Where do I go to add a new dependency to the instack-undercloud package now? 19:14:21 delorean 19:14:37 or... 19:15:09 https://review.gerrithub.io/#/q/project:openstack-packages/instack-undercloud 19:15:14 is that what you meant? 19:15:55 Ah, openstack-packages. I kept looking for the wrong name. :-) 19:16:06 slagle: Thanks, I think that's what I'm looking for. 19:18:22 trown|mtg: thanks, your hint helped! clearing the jobqueue resolved my issue 19:21:19 James Slagle proposed openstack/instack-undercloud: Add ip addresses to horizon allowed_hosts https://review.openstack.org/234939 20:05:35 Jeff Peeler proposed openstack/tripleo-heat-templates: Update docker compute environment to use json config https://review.openstack.org/234313 20:05:52 Ben Nemec proposed openstack/instack-undercloud: Add check for sufficient memory to undercloud install https://review.openstack.org/224852 20:21:29 Rohit Pagedar proposed openstack/tripleo-heat-templates: Added nova parameteres to controller: instance_name_template, use_forwarded_for After this change, the above parameters will be configurable via heat templates. https://review.openstack.org/234965 20:27:03 Rohit Pagedar proposed openstack/tripleo-heat-templates: Changes for configuring Nuage Added ExtraConfig templates and environment files for Nuage specific parameters. Modified overcloud_compute.pp to conditionally include nuage-metadata-agent. https://review.openstack.org/234970 20:29:16 Ian Wienand proposed openstack/diskimage-builder: Remove unused RELEASE_RPMS variable https://review.openstack.org/234971 21:00:18 Untriaged bugs so far: 21:00:19 https://bugs.launchpad.net/tripleo/+bug/1505815 21:00:19 Launchpad bug 1505815 in tripleo "Docs refer to ironic-discoverd instead of ironic-inspector" [Undecided,Confirmed] - Assigned to Ruchi Parmar (ruchiparmar143) 21:00:20 https://bugs.launchpad.net/tripleo/+bug/1504363 21:00:20 Launchpad bug 1504363 in tripleo "Updates to hiera configuration not propogating/making a puppet run" [Undecided,Confirmed] 21:00:22 https://bugs.launchpad.net/diskimage-builder/+bug/1505900 21:00:22 Launchpad bug 1505900 in diskimage-builder "Failed to create 'wily' release with DIB_RELEASE" [Undecided,New] 21:00:24 https://bugs.launchpad.net/tripleo/+bug/1504263 21:00:25 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 21:00:27 https://bugs.launchpad.net/tripleo/+bug/1505430 21:00:27 Launchpad bug 1505430 in tripleo "Triggering puppet runs with config_identifier is error prone and complicates templates" [Undecided,New] - Assigned to Steve Baker (steve-stevebaker) 21:53:07 Rohit Pagedar proposed openstack/tripleo-heat-templates: Changes for configuring Nuage Added ExtraConfig templates and environment files for Nuage Networks specific parameters. Modified overcloud_compute.pp to conditionally include nuage-metadata-agent. https://review.openstack.org/234970 21:57:42 Rohit Pagedar proposed openstack/tripleo-heat-templates: Adding nova parameteres to controller: instance_name_template use_forwarded_for https://review.openstack.org/234965 00:08:18 OpenStack Proposal Bot proposed openstack/os-cloud-config: Updated from global requirements https://review.openstack.org/231788 00:10:06 OpenStack Proposal Bot proposed openstack/tripleo-common: Updated from global requirements https://review.openstack.org/232282 00:35:27 Steve Baker proposed openstack/python-tripleoclient: Log all resource events during overcloud deploy https://review.openstack.org/235032 00:38:53 Steve Baker proposed openstack/python-tripleoclient: Log all resource events during overcloud deploy https://review.openstack.org/235032 01:19:21 Steve Baker proposed openstack/python-tripleoclient: Log all resource events during overcloud deploy https://review.openstack.org/235032 01:36:17 Vinay Rao proposed openstack/tripleo-heat-templates: Fixed syntax error https://review.openstack.org/235050 02:11:24 Emilien Macchi proposed openstack/instack-undercloud: neutron: enable nova-event-callback by default https://review.openstack.org/234325 02:13:09 Emilien Macchi proposed openstack/tripleo-heat-templates: Implement Workers parameters https://review.openstack.org/229471 02:27:27 Emilien Macchi proposed openstack/tripleo-heat-templates: neutron: enable nova-event-callback by default https://review.openstack.org/225962 03:00:15 Untriaged bugs so far: 03:00:17 https://bugs.launchpad.net/tripleo/+bug/1505815 03:00:17 Launchpad bug 1505815 in tripleo "Docs refer to ironic-discoverd instead of ironic-inspector" [Undecided,Confirmed] - Assigned to Ruchi Parmar (ruchiparmar143) 03:00:18 https://bugs.launchpad.net/tripleo/+bug/1505430 03:00:18 Launchpad bug 1505430 in tripleo "Triggering puppet runs with config_identifier is error prone and complicates templates" [Undecided,New] - Assigned to Steve Baker (steve-stevebaker) 03:00:20 https://bugs.launchpad.net/diskimage-builder/+bug/1505900 03:00:21 Launchpad bug 1505900 in diskimage-builder "Failed to create 'wily' release with DIB_RELEASE" [Undecided,New] 03:00:22 https://bugs.launchpad.net/tripleo/+bug/1504263 03:00:22 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 03:00:24 https://bugs.launchpad.net/diskimage-builder/+bug/1506275 03:00:25 Launchpad bug 1506275 in diskimage-builder "$DIB_OFFLINE and $DIB_DEBIAN_USE_DEBOOTSTRAP_CACHE are redundant" [Undecided,New] 03:00:26 https://bugs.launchpad.net/tripleo/+bug/1504363 03:00:26 Launchpad bug 1504363 in tripleo "Updates to hiera configuration not propogating/making a puppet run" [Undecided,Confirmed] 04:01:08 Merged openstack/diskimage-builder: Update default fedora-minimal to f22 https://review.openstack.org/227126 04:01:10 Merged openstack/diskimage-builder: Remove extra install of release pkgs in fedora-minimal https://review.openstack.org/227131 07:46:08 Yanis Guenane proposed openstack/tripleo-heat-templates: Allow a user to specify a comma separated list of ntp servers https://review.openstack.org/232051 07:52:24 Yanis Guenane proposed openstack/python-tripleoclient: Amend --ntp-server help message to add support for multiple servers https://review.openstack.org/232075 08:00:48 Yanis Guenane proposed openstack/python-tripleoclient: Do not initialize/manage keystone in python-tripleoclient https://review.openstack.org/231395 08:05:49 Dmitry Tantsur proposed openstack/python-tripleoclient: Do not try to introspect nodes in maintenance mode and associated ones https://review.openstack.org/235148 08:13:24 Dmitry Tantsur proposed openstack/python-tripleoclient: Do not try to introspect nodes in maintenance mode and associated ones https://review.openstack.org/235148 08:26:12 Dmitry Tantsur proposed openstack/python-tripleoclient: Support new ironic "enroll" state https://review.openstack.org/235158 09:00:17 Untriaged bugs so far: 09:00:19 https://bugs.launchpad.net/tripleo/+bug/1505815 09:00:19 Launchpad bug 1505815 in tripleo "Docs refer to ironic-discoverd instead of ironic-inspector" [Undecided,Confirmed] - Assigned to Ruchi Parmar (ruchiparmar143) 09:00:20 https://bugs.launchpad.net/tripleo/+bug/1505430 09:00:20 Launchpad bug 1505430 in tripleo "Triggering puppet runs with config_identifier is error prone and complicates templates" [Undecided,New] - Assigned to Steve Baker (steve-stevebaker) 09:00:22 https://bugs.launchpad.net/diskimage-builder/+bug/1505900 09:00:23 Launchpad bug 1505900 in diskimage-builder "Failed to create 'wily' release with DIB_RELEASE" [Undecided,New] 09:00:24 https://bugs.launchpad.net/tripleo/+bug/1504263 09:00:24 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 09:00:26 https://bugs.launchpad.net/diskimage-builder/+bug/1506275 09:00:26 Launchpad bug 1506275 in diskimage-builder "$DIB_OFFLINE and $DIB_DEBIAN_USE_DEBOOTSTRAP_CACHE are redundant" [Undecided,New] 09:00:28 https://bugs.launchpad.net/tripleo/+bug/1504363 09:00:28 Launchpad bug 1504363 in tripleo "Updates to hiera configuration not propogating/making a puppet run" [Undecided,Confirmed] 09:01:47 Merged openstack/tripleo-image-elements: Add contrail-repo element https://review.openstack.org/203967 09:03:55 Merged openstack/tripleo-puppet-elements: Support for the Opencontrail controller https://review.openstack.org/206343 09:19:12 Jan Provaznik proposed openstack/tripleo-heat-templates: Add a resource for node reboot https://review.openstack.org/233573 09:49:51 Merged openstack/python-tripleoclient: Change RHEL subscription template path https://review.openstack.org/229847 09:51:51 spredzy: hey, you there? 09:52:00 hewbrocca, yes 09:52:28 EmilienM says you have patches upstream under review that will allow director to configure keystone v3? 09:52:43 and that that is the last required piece for v3 support? 09:53:17 Is anyone else getting regular kernel crashes on centos7? 09:53:28 kernel crashes 09:53:29 ick 09:53:32 I've just oopsed my undercloud for the second time while building images 09:53:52 spredzy: if you could either confirm that or tell me I'm crazy, I'd appreciate it 09:53:55 shardy, me there before yes! 09:53:56 hewbrocca, he was referring to https://review.openstack.org/#/c/234704/ for undercloud and https://review.openstack.org/#/c/230357/8 for overcloud 09:54:07 shardy, it's OOM 09:54:27 hewbrocca, The reviews per-se make overcloud and undercloud use puppet upstream modules to manage keystone related stuff 09:54:29 shardy, I think the tripleo.sh instead is setting some env var to not use TMPFS for image building 09:54:45 spredzy: OK then 09:54:46 gfidente: Hmm, I bumped my memory up to 8G already, but you're right there is no swap 09:54:58 shardy, 8GB and how many cores? 09:55:04 gfidente: 8G and 2 cores 09:55:06 hewbrocca, since upstream modules are about/or are already managing keystone v3, this will confirm his sentence 09:55:13 excellent 09:55:29 do you think there is anything else we would need for keystone v3? 09:55:30 shardy, yeah that didn't work for me either 09:56:14 hewbrocca, have a round check upstream if everything is ready (to confirm), but nothing in t-h-t as long as those reviews are in 09:56:24 I'd say ... :) 09:56:33 spredzy: many thanks 09:56:44 yw 09:58:44 shardy, try https://github.com/jistr/inlunch/blob/master/answers.yml.example#L126 09:59:05 shardy, 12G and 2cpus passed for me though 10:02:29 shardy: yeah what gfidente linked should help i think. I hit issues on 6G RAM undercloud and that solved it for me. 10:02:55 there's some check in DIB whether to use tmpfs or not https://github.com/openstack/diskimage-builder/blob/3972510d20a85932da925411be4edf9790757274/lib/common-functions#L17-L30 10:03:05 so to be fair it isn't tripleo.sh but inlunch :) 10:03:05 based on the RAM size 10:03:23 however it doesn't know how big will be the image being built 10:03:58 so... it's not a very useful check imo 10:04:12 gfidente, shardy jistr what is the rule, when one should decide or not to add a parameter here https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/v1/overcloud_deploy.py#L779-L921 10:04:24 never add a parameter 10:04:32 gfidente: +1 :D 10:04:40 ok that is straight forward thanks ;) 10:05:13 +1, IMHO adding all those parameters was a mistake, we should consider deprecating almost all of them and documenting using the environment files instead 10:05:52 ack that 10:06:04 gfidente, jistr: thanks for the tips /me looks around for some more RAM ;) 10:07:09 shardy, I think the problem is tmpfs is in memory but if you disable it with the env it's going to be a little slower maybe but won't consume that much ram 10:07:23 gfidente: ack, trying that now, thanks 10:07:45 we should probably do that by default in tripleoclient considering we don't enable any swap 10:08:08 or, perhaps just in tripleo.sh, but I hit this when attempting to only rebuild openstack-full 10:25:02 Rico Lin proposed openstack/python-tripleoclient: improve readme contents https://review.openstack.org/235234 10:29:09 Dmitry Tantsur proposed openstack/diskimage-builder: Add kmod to package-installs of ironic-agent https://review.openstack.org/235236 11:01:33 ~/query gfidente Well, Yanis doesn't seem too keen on changing the approach 11:02:22 aaah Yanis is spredzy, spredzy, are you around? 11:05:19 spredzy: I guess we could do with the changes you made. But I think in the end we would need to do some significant changes to acommodate the TLS parts; So I was hoping the creation of the URLs which is currently done with the list_join in the templates could be done in a separate module... or in another way that would make it easier to work with 11:17:11 jistr, currently in rdo-manager we use Ceph release 'giant' right? 11:17:42 hmm i don't know 11:18:01 i think we use what CentOS 7.1 has 11:49:03 can i get eyes on this again please :) https://review.openstack.org/#/c/199152/ 11:49:14 i've been rebasing it since July :)) 11:50:40 Marios Andreou proposed openstack/tripleo-incubator: Adds LIBVIRT_VOL_POOL_TARGET to tripleo create-nodes https://review.openstack.org/235278 11:52:29 hewbrocca: no, I said yanis had patches to manage keystone resources with Puppet 11:52:51 hewbrocca: like we said yesterday, v3 is already available 11:56:07 * mkovacik is away: coffee 11:59:33 EmilienM: but I don't think director can deploy it without those patches, can it? 12:00:33 hewbrocca: directory already deploys v3 12:00:48 those patches are just changing the way we create endpoints/users/services/tenants 12:00:56 today we use some python & bash 12:01:02 tomorrow we'll use puppet 12:01:10 that's all about it 12:03:55 v3 is definitely deployed, but, the endpoint isn't created, and all the services are configured to still use v2 12:08:08 Brad P. Crochet proposed openstack/python-tripleoclient: Remove parameter defaults https://review.openstack.org/226790 12:10:26 can anyone review https://review.openstack.org/#/c/228445/? d0ugal has a +2 on it, but obviously I can't do the same. :) 12:11:32 thrash: Maybe add it to https://etherpad.openstack.org/p/tripleo-review-priorities? 12:11:41 d0ugal: thx 12:14:57 thrash, d0ugal: +A'd 12:15:36 slagle, hewbrocca: I had this debate recently with lsmola_ - AIUI keystone folks have been trying to move away from versioned endpoints, and have support for discovering the endpoint for a particular API version 12:15:41 jistr: thanks 12:16:05 however everyone still uses versioned endpoints for everything, and it's also what's in all the docs for every service AFAICS 12:16:06 d0ugal: about https://review.openstack.org/#/c/231998/... 12:16:28 shardy: yeah I will do support for Fog for next release, if it will nto be there ealready 12:16:30 it's been a problem for several years, we had to hack around it in heat when we wanted to start using v3 features 12:16:34 d0ugal: the ironicclient stuff is post-liberty, so it's not quite ready yet. 12:16:49 thrash: cool, thanks for the clarification. I meant to ping you about it but forgot 12:16:56 Merged openstack/python-tripleoclient: Also disallow tunnel types without network type https://review.openstack.org/228445 12:17:06 shardy: ok. but all our services are still hardcoded to use v2, so that likely needs to still be fixed 12:17:16 thrash, d0ugal: btw usually we try not to reference RH BZs from commits in tripleo as the upstream bug tracking tool is launchpad 12:17:19 thrash: will re-review properly today 12:17:26 jistr: ack 12:19:31 thrash: jistr beat me to it :) one comment that is general 12:19:46 d0ugal: do we not have a https://bugs.launchpad.net/python-tripleoclient 12:19:54 noticed when looking at thrash review 12:20:18 seems like we do have https://bugs.launchpad.net/python-tuskarclient though 12:20:28 jistr: heh, yeah :D 12:20:33 marios: we are using tripleo 12:20:41 marios: ... or we are supposed to be using tripleo :) 12:20:53 d0ugal: i see ok thanks (no don't think there are rules. though you certainly could do) 12:21:04 d0ugal: probably right now there aren't that many bugs cos its brilliant 12:21:10 lol 12:22:02 ha 12:22:25 Pino Toscano proposed openstack/python-tripleoclient: image build: add support for virt-dib https://review.openstack.org/222541 12:22:43 Pino Toscano proposed openstack/python-tripleoclient: image build: add --builder-extra-args https://review.openstack.org/229778 12:22:46 * mkovacik is back (gone 00:26:40) 12:23:15 Brad P. Crochet proposed openstack/python-tripleoclient: If neutron tunneling is disabled, do not throw an error on validation https://review.openstack.org/228446 12:23:52 Brad P. Crochet proposed openstack/python-tripleoclient: Do not throw an error on validation when tunneling is disabled https://review.openstack.org/228446 12:24:01 maybe someone could also review my patches...? 12:24:52 slagle: yeah, agreed - it'd be good to seek clarification from keystone folks about exactly how this is supposed to work before doing that - I've seen some folks just flipping the endpoint to v3, which AIUI is wrong, but that's not yet reflected in any docs I could find 12:28:01 we should probably remove the v2 from the rc files too 12:28:09 or, whatever the right thing is 12:37:47 pino|work: you have unittest failures. 12:38:02 thrash: eh, i see that, thanks... 12:42:36 Pino Toscano proposed openstack/python-tripleoclient: image build: add support for virt-dib https://review.openstack.org/222541 12:49:23 Pino Toscano proposed openstack/python-tripleoclient: image build: add support for virt-dib https://review.openstack.org/222541 12:53:54 bnemec ping 12:56:35 Pino Toscano proposed openstack/python-tripleoclient: image build: add --builder-extra-args https://review.openstack.org/229778 12:56:39 jaosorior: btw for some reason i also had some problem with injecting the CA trust and overcloud didn't get initialized by the CLI, but i cannot say with certainty that your patch is the cause. I see the firstboot cloud-init bits being on the instance, but somehow the CA certificate wasn't created. I'll try to re-do my env and get Heat with the latest fix. 12:57:18 jaosorior: after i manually did the CA injection, things seemed to work fine 12:57:44 jistr: I tested the certificate and the CA cert injection with the downstream code and it was working 12:58:07 ok perhaps a problem specific to my env then 12:58:39 but it could be that some things are being handled differently downstream 12:58:55 heat is a different version, I assume 12:59:15 yea, kilo vs. liberty 12:59:47 yeah... since I had set up the downstream version already, I was using that for development. 12:59:54 probably I should switch to using upstream soon 13:02:44 Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Enable TLS deployment for the Overcloud https://review.openstack.org/231930 13:03:10 jistr: I fixed the path issues you pointed out in the CR 13:03:29 cool, thx 13:03:55 Derek Higgins proposed openstack-infra/system-config: Add Fedora 22 images to the tripleo cloud https://review.openstack.org/235319 13:03:55 Derek Higgins proposed openstack-infra/project-config: Promote the tripleo ha and ceph jobs to voting https://review.openstack.org/235320 13:03:55 Derek Higgins proposed openstack-infra/project-config: Add tripleo F22 periodic jobs https://review.openstack.org/235321 13:06:46 thrash: both patches fixed 13:08:58 shardy: I believe because 2 of us worked on https://review.openstack.org/#/c/228157/15 , one of us can add a +2 (if you want to) 13:11:27 derekh: Ok, +2'd both of them 13:11:40 landing those would be good as I've started a docs patch showing how to use them 13:12:50 derekh: nice on getting HA stable enough for voting 13:17:27 trown: thanks, although I think something else has changed in addition to what I've been doing, somebody else probably fixed something that was causing some of the errors aswell, but I don't know what..... 13:18:26 hehe 13:19:19 derekh: RDO-Manager HA is still failing because of a liberty only ceilometer bug, that is -2 to be fixed before GA :( 13:19:37 https://bugs.launchpad.net/bugs/1505669 13:19:37 Launchpad bug 1505669 in Ceilometer "empty replica_set from config can clobber url-based replica set for mongodb" [Undecided,New] 13:20:12 Marios Andreou proposed openstack/tripleo-docs: Adds note about the LIBVIRT_VOL_POOL_TARGET used by create-nodes https://review.openstack.org/235332 13:22:01 Merged openstack/diskimage-builder: Reset yum/dnf cache to correct location https://review.openstack.org/232863 13:22:19 trown: bum 13:27:09 * mkovacik is away: I'm busy 13:38:12 slagle: shardy: lazy_prince: should i go ahead and make a review at irc-meetings to update tripleo (weekly?) 13:38:28 anyone else ? ( http://lists.openstack.org/pipermail/openstack-dev/2015-October/076966.html ) 13:38:52 I am pro weekly 13:39:27 Rob Pothier proposed openstack/tripleo-heat-templates: Cisco Nexus ML2 MD: Add Support for mulitple UCSMs https://review.openstack.org/233664 13:40:04 shardy, jistr looks like this needs backport to liberty? https://review.openstack.org/#/c/234055/ 13:41:49 gfidente: Yes it will need backporting, it's tagged liberty-backport-potential 13:43:10 marios: sure 13:43:20 fyi, lazy_prince isn't dprince :) 13:43:27 slagle: oh 13:43:35 still , i like your assumption there :) 13:43:37 slagle: thanks. assumed it was a bot. sorry lazy_prince 13:44:35 (i mean another login that he has when away) 13:44:43 hahaha 13:44:59 as long as there are at least 2 people who want to meet every week, and at least 1 person willing to chair, i say go4it 13:45:19 well, last week some people were even expecting a meeting 13:45:28 we could maybe rotate the chair responsibility so it doesnt fall all on dan. he might not mind though 13:46:19 shardy, jistr ack so I sent it to make sure we don't break things when switching from _defaults to parameters: in storage-environment 13:48:19 gfidente: Yeah, well it'll be OK for upstream tripleo but we will need to be careful with any downstream cherry-picks or rebases 13:49:03 gfidente: liberty heat GA has happened but I can propose it to the stable branch, thanks for the reminder 13:50:08 shardy, I think having it in liberty will turn useful when doing upgrades 13:50:18 jistr: maybe we should mention the heat commit required in the commit message for https://review.openstack.org/#/c/199152/6 to ensure anyone doing a backport knows it is needed? 13:50:32 gfidente: agreed, just unfortunate we missed the cut for GA 13:51:39 shardy: yeah thanks, i'll mention it 13:53:39 Jiri Stransky proposed openstack/tripleo-heat-templates: Support NFS backend for Glance (via Pacemaker) https://review.openstack.org/199152 14:26:20 Yanis Guenane proposed openstack/tripleo-heat-templates: Secure the mongodb server https://review.openstack.org/235385 14:40:20 slagle: shardy: A couple of links for switching to keystone v3: http://adam.younglogic.com/2015/08/template-for-a-keystonev3-rc/ http://adam.younglogic.com/2015/05/rdo-v3-only/ 14:40:45 Although I think it's worth noting that both times I have tried to do this (for LDAP integration in Kilo) I have failed to get a 100% functional environment. 14:41:07 I don't know if that would be improved by removing the LDAP part from the equation though. 15:00:19 Untriaged bugs so far: 15:00:20 https://bugs.launchpad.net/tripleo/+bug/1505815 15:00:20 Launchpad bug 1505815 in tripleo "Docs refer to ironic-discoverd instead of ironic-inspector" [Undecided,Confirmed] - Assigned to Ruchi Parmar (ruchiparmar143) 15:00:21 https://bugs.launchpad.net/tripleo/+bug/1505430 15:00:22 https://bugs.launchpad.net/diskimage-builder/+bug/1505900 15:00:22 Launchpad bug 1505430 in tripleo "Triggering puppet runs with config_identifier is error prone and complicates templates" [Undecided,New] - Assigned to Steve Baker (steve-stevebaker) 15:00:23 https://bugs.launchpad.net/tripleo/+bug/1504263 15:00:23 Launchpad bug 1505900 in diskimage-builder "Failed to create 'wily' release with DIB_RELEASE" [Undecided,New] 15:00:25 https://bugs.launchpad.net/diskimage-builder/+bug/1506275 15:00:27 https://bugs.launchpad.net/tripleo/+bug/1504363 15:00:30 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 15:00:31 Launchpad bug 1506275 in diskimage-builder "$DIB_OFFLINE and $DIB_DEBIAN_USE_DEBOOTSTRAP_CACHE are redundant" [Undecided,New] 15:00:32 Launchpad bug 1504363 in tripleo "Updates to hiera configuration not propogating/making a puppet run" [Undecided,Confirmed] 15:08:33 Giulio Fidente proposed openstack/tripleo-heat-templates: Set Django cache backend to Memcached instead of LocMemCache https://review.openstack.org/235418 15:12:21 Derek Higgins proposed openstack-infra/tripleo-ci: Removing cellspacing from the tripleo ci report https://review.openstack.org/235419 15:12:22 Derek Higgins proposed openstack-infra/tripleo-ci: Use the new shortened tripleo ci job names https://review.openstack.org/235420 15:12:23 Derek Higgins proposed openstack-infra/tripleo-ci: Add the new periodic jobs https://review.openstack.org/235421 15:17:00 Dan Sneddon proposed openstack/tripleo-heat-templates: Add IPv6 Support to Isolated Networks https://review.openstack.org/235423 15:19:05 shardy, ^^ First crack at IPv6 support for Net Iso 15:28:22 dsneddon: nice, will check it out! :) 15:29:16 shardy, It looks too simple, so I need you to tell me if I'm missing something. ;-) 15:36:18 Yanis Guenane proposed openstack/tripleo-heat-templates: Manage keystone related settings directly in t-h-t manifests https://review.openstack.org/230375 15:36:57 Yanis Guenane proposed openstack/tripleo-heat-templates: Manage keystone related settings directly in t-h-t manifests https://review.openstack.org/230357 15:38:12 Yanis Guenane proposed openstack/tripleo-heat-templates: neutron: enable nova-event-callback by default https://review.openstack.org/225962 15:53:39 derekh: hi, do you know if the purpose of openstack-heat and openstack-ironic being listed here https://github.com/openstack/tripleo-common/blob/ed2713dc14ec157e3b63e4b069a0f657d8f07285/scripts/tripleo.sh#L172 15:53:49 was to fetch all heat and ironic RPMs from -current? 15:53:57 it doesn't seem to work that way 15:54:28 jistr: ya, its broken https://review.openstack.org/#/c/229906/ 15:55:32 jistr: so we're currently pulling heat ind ironic from the /current-triple repo 15:55:40 *o 15:57:10 derekh: I think we need to rename the toci jobs in Jenkins. 15:57:29 Heh, just as you jump in on the infra discussion. :-) 15:57:38 derekh: yeah, thanks for the link. i updated heat and ironic from -current and i can't spawn nova instances (backed by fake baremetal). I'll try to use exactly what you have in your patch, as it seems the non-ha job passed there. 15:58:03 i didn't pull python2-ironic-python-agent 16:03:51 Emilien Macchi proposed openstack/tripleo-heat-templates: puppet: run keystone in wsgi https://review.openstack.org/213175 16:17:52 Timothy Swanson proposed openstack/tripleo-heat-templates: Cisco Nexus ML2 MD: Need unique key for server level keys in hieradata https://review.openstack.org/231623 16:32:23 Giulio Fidente proposed openstack/tripleo-heat-templates: Sync httpd vhost settings in between pcmk and non-pcmk scenarios https://review.openstack.org/235484 16:47:30 Merged openstack/tripleo-common: Fix bashate errors https://review.openstack.org/234108 17:21:41 Did we remove the delay between starting introspection on nodes? I just had 3 out of 6 nodes fail to start introspection due to the DHCP conflict issue. 18:00:15 Giulio Fidente proposed openstack/tripleo-heat-templates: Allow customization of Ceph pool names https://review.openstack.org/212664 18:05:08 slagle, dprince: Hey just FYI I spent some time today looking into allowing heat to reauthenticate for very long running creates that exceed the keystone token expiry ref the long-standing bug #1306294 18:05:08 bug 1306294 in tripleo "Heat fails to re-authenticate when faced with authentication failure during stack operations" [High,In progress] https://launchpad.net/bugs/1306294 18:05:46 I just posted https://review.openstack.org/#/c/235516/ which is a rework of https://review.openstack.org/#/c/226384/8 18:06:10 tl;dr - I successfully tested a TripleO deployment with keystone token expiry set to only two minutes 18:06:42 there are issues on delete stil TODO, but it could be useful e.g for long-running updates etc 18:06:45 zaneb: ^^ 18:07:18 shardy: nice, thanks for the update 18:07:28 will be good to have this fixed 18:07:34 nice 18:24:52 bnemec: are you able to look at https://review.openstack.org/#/c/235418/ it is one of two blocking issues for RDO-Manager GA 18:25:06 or rather it fixes one of two blocking issues 18:29:30 trown: Done 18:30:22 bnemec: woot! thanks, one ceilometer bug to go 18:30:23 Merged openstack/tripleo-heat-templates: Set Django cache backend to Memcached instead of LocMemCache https://review.openstack.org/235418 18:31:06 Derek Higgins proposed openstack/diskimage-builder: Define $YUM for centos-minimal https://review.openstack.org/235539 18:41:52 Derek Higgins proposed openstack/diskimage-builder: Define $YUM for centos-minimal https://review.openstack.org/235539 18:51:35 Rob Pothier proposed openstack/tripleo-heat-templates: Cisco Nexus ML2 MD: Add Support for mulitple UCSMs https://review.openstack.org/233664 19:13:53 James Slagle proposed openstack/tripleo-docs: Remove GUI documentation https://review.openstack.org/235552 19:22:38 Merged openstack/tripleo-docs: Add documentation on how to backup and restore the Undercloud https://review.openstack.org/233468 20:13:27 Brad P. Crochet proposed openstack/tripleo-common: Build image files from definitions in yaml https://review.openstack.org/235569 20:14:10 dprince: slagle d0ugal first pass at pulling build-images from incubator. Do not be gentle. :) 20:14:24 trown: you might want to look at that too. 20:14:48 I am going to work on a new command in tripleoclient now that uses this. 20:14:57 oh shiny 20:14:57 then we can deprecate 'overcloud image build' 20:15:19 can we branch stable/liberty before deprecating overcloud image build :) 20:32:37 John Trowbridge proposed openstack/tripleo-docs: Modify instructions for using delorean to build images https://review.openstack.org/235575 20:35:38 thrash: cool! 20:36:06 Resource CREATE failed: resources.ObjectStorageAllNodesDeployment: Stack overcloud-ObjectStorageAllNodesDeployment-335j7oqrhill already has an action (CREATE) in progress. 20:36:29 I vaguely recall seeing discussion of a similar issue. Is there a bug for it yet? 20:36:35 Emilien Macchi proposed openstack/instack-undercloud: puppet: drop some warnings https://review.openstack.org/234446 20:36:41 bnemec: I think slagle filed a bug on that, yes 20:37:33 dprince: Okay, thanks. 20:37:36 * bnemec tries again 20:37:46 bnemec: https://bugs.launchpad.net/heat/+bug/1504266 20:37:46 Launchpad bug 1504266 in tripleo "Stack overcloud-ComputeNodesPostDeployment-dkq4fgg4lqk5-NovaComputeContainersDeployment-6zpj2rijh3lk already has an action (CREATE) in progress." [Medium,Triaged] 20:38:13 bnemec: you might comment on the ticket so heat knows we are seeing this more often now 20:38:33 yea ramishra thinks it might already be fixed 20:38:37 it was a recent fix 20:38:45 slagle: ah, cool 20:38:56 slagle: so probably just us using an old DELOREAN_URL? 20:39:07 could be 20:43:57 Yeah, my openstack-heat-engine package is dev44. It looks like the latest is dev181 (!) 20:45:43 * trown agreed to look into automating current-tripleo link, but has been pretty swamped trying to get liberty RDO-Manager out the door 20:45:47 I have not forgot though 20:48:44 trown: derek is working on adding some periodic jobs that I think were related to that: https://review.openstack.org/#/c/235321/ 20:50:19 bnemec: I think it is this part that is still an open question "trigger a promotion of trunk repositories used by tripleo" 20:50:41 although if derekh has solved that one that would be awesome 20:50:56 Grr, stupid server just froze again. 20:51:13 trown: Right, but step one will be having a periodic job running. 20:51:46 ya 21:00:17 Untriaged bugs so far: 21:00:18 https://bugs.launchpad.net/tripleo/+bug/1505815 21:00:18 Launchpad bug 1505815 in tripleo "Docs refer to ironic-discoverd instead of ironic-inspector" [Undecided,Confirmed] - Assigned to Ruchi Parmar (ruchiparmar143) 21:00:19 https://bugs.launchpad.net/tripleo/+bug/1505430 21:00:19 Launchpad bug 1505430 in tripleo "Triggering puppet runs with config_identifier is error prone and complicates templates" [Undecided,New] - Assigned to Steve Baker (steve-stevebaker) 21:00:21 https://bugs.launchpad.net/diskimage-builder/+bug/1505900 21:00:21 Launchpad bug 1505900 in diskimage-builder "Failed to create 'wily' release with DIB_RELEASE" [Undecided,New] 21:00:23 https://bugs.launchpad.net/tripleo/+bug/1504263 21:00:23 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 21:00:25 https://bugs.launchpad.net/diskimage-builder/+bug/1506275 21:00:26 Launchpad bug 1506275 in diskimage-builder "$DIB_OFFLINE and $DIB_DEBIAN_USE_DEBOOTSTRAP_CACHE are redundant" [Undecided,New] 21:00:27 https://bugs.launchpad.net/tripleo/+bug/1504363 21:00:27 Launchpad bug 1504363 in tripleo "Updates to hiera configuration not propogating/making a puppet run" [Undecided,Confirmed] 21:00:30 https://bugs.launchpad.net/tripleo/+bug/1506546 21:00:30 Launchpad bug 1506546 in tripleo "Cisco Nexus ML2 MD: Need unique key for server level keys in hieradata" [Undecided,In progress] - Assigned to Timothy Swanson (tiswanso) 21:10:45 Emilien Macchi proposed openstack/instack-undercloud: puppet: restart mysql if config changes https://review.openstack.org/235584 21:23:44 dprince: I'm trying to see why https://review.openstack.org/#/c/230375/ fails 21:23:50 dprince: logs in overcloud look fine 21:23:55 http://logs.openstack.org/75/230375/19/check-tripleo/gate-tripleo-ci-ironic-overcloud-f21puppet-nonha/39b4bea/console.html#_2015-10-15_19_19_33_666 21:24:04 but it seems like something with logs script 21:24:25 http://logs.openstack.org/75/230375/19/check-tripleo/gate-tripleo-ci-ironic-overcloud-f21puppet-nonha/39b4bea/console.html#_2015-10-15_19_20_49_578 21:24:30 and http://logs.openstack.org/75/230375/19/check-tripleo/gate-tripleo-ci-ironic-overcloud-f21puppet-nonha/39b4bea/console.html#_2015-10-15_19_20_58_047 21:24:53 slagle: ^ 21:35:56 2 of my 6 development boxes are now hanging randomly. 21:35:59 Awesome. 21:39:33 bnemec: can you help with my failures ^ ? 21:41:27 EmilienM: It looks like it's still trying to run init-keystone, when keystone should already have been init'd by Puppet (IIUC). 21:42:42 EmilienM: That change needs to depend on one to remove the keystone.initialize call in the client: https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/v1/overcloud_deploy.py#L460 21:43:46 And actually all of that keystone code there. The endpoint registration is probably going to conflict too. 21:43:51 bnemec: https://review.openstack.org/231395 ? 21:44:18 EmilienM: Looks like it. 21:45:05 I thought he put it in DdependsOn 21:45:34 bnemec: it's actually a chicken and egg problem 21:45:48 look https://review.openstack.org/231395 - it already depends on the overcloud issue 21:46:13 EmilienM: Yeah, I don't know if Depends-On handle circular dependencies, but I'd guess not. 21:46:35 it does not 21:46:37 This may be a case where we have to merge without a CI pass on one of the patches, and use the one depending on it as the test. 21:46:38 we're doomed here 21:46:46 we need to merge them together 21:46:53 I would ask spredzy to invert the order 21:46:59 so we can validate both patches 21:47:20 Well, if one passes CI it means the other must be fine too, right? 21:47:37 bnemec: right 21:47:44 bnemec: I'm blond you know? 21:47:46 Those CI jobs aren't gating, so we can just merge both at the same time, even though one will be "failing". 21:55:34 Marius Cornea proposed openstack/tripleo-docs: Add documentation on how to depoy HA overcloud https://review.openstack.org/235597 22:18:41 Ignacio Bravo proposed openstack/tripleo-docs: Updated /etc/hosts file definition https://review.openstack.org/235599 23:16:08 Marius Cornea proposed openstack/tripleo-docs: Add documentation on how to depoy HA overcloud https://review.openstack.org/235597 23:23:09 Marius Cornea proposed openstack/tripleo-docs: Fixed indentation which caused RHEL Satellite Registration to always show in step 7 of Virtual Environment page https://review.openstack.org/235614 23:34:50 OpenStack Proposal Bot proposed openstack/tripleo-common: Updated from global requirements https://review.openstack.org/232282 02:25:39 Martin André proposed openstack/diskimage-builder: Fix uniqueness check of initrd in fedora-minimal https://review.openstack.org/235705 03:00:17 Untriaged bugs so far: 03:00:18 https://bugs.launchpad.net/tripleo/+bug/1505815 03:00:18 Launchpad bug 1505815 in tripleo "Docs refer to ironic-discoverd instead of ironic-inspector" [Undecided,Confirmed] - Assigned to Ruchi Parmar (ruchiparmar143) 03:00:20 https://bugs.launchpad.net/tripleo/+bug/1506651 03:00:20 Launchpad bug 1506651 in tripleo "Definition of /etc/hosts not correct in docs" [Undecided,New] - Assigned to Ignacio Bravo (ibravo) 03:00:22 https://bugs.launchpad.net/tripleo/+bug/1505430 03:00:23 Launchpad bug 1505430 in tripleo "Triggering puppet runs with config_identifier is error prone and complicates templates" [Undecided,New] - Assigned to Steve Baker (steve-stevebaker) 03:00:24 https://bugs.launchpad.net/diskimage-builder/+bug/1506692 03:00:24 Launchpad bug 1506692 in diskimage-builder "Broken test in fedora-minimal/install.d/99-ramdisk" [Undecided,In progress] - Assigned to Martin André (mandre) 03:00:26 https://bugs.launchpad.net/diskimage-builder/+bug/1505900 03:00:26 Launchpad bug 1505900 in diskimage-builder "Failed to create 'wily' release with DIB_RELEASE" [Undecided,New] 03:00:28 https://bugs.launchpad.net/tripleo/+bug/1504263 03:00:29 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 03:00:30 https://bugs.launchpad.net/diskimage-builder/+bug/1506275 03:00:31 Launchpad bug 1506275 in diskimage-builder "$DIB_OFFLINE and $DIB_DEBIAN_USE_DEBOOTSTRAP_CACHE are redundant" [Undecided,New] 03:00:32 https://bugs.launchpad.net/tripleo/+bug/1504363 03:00:32 Launchpad bug 1504363 in tripleo "Updates to hiera configuration not propogating/making a puppet run" [Undecided,Confirmed] 03:00:34 https://bugs.launchpad.net/tripleo/+bug/1506546 03:00:34 Launchpad bug 1506546 in tripleo "Cisco Nexus ML2 MD: Need unique key for server level keys in hieradata" [Undecided,In progress] - Assigned to Timothy Swanson (tiswanso) 03:05:33 OpenStack Proposal Bot proposed openstack/os-collect-config: Updated from global requirements https://review.openstack.org/232883 03:07:25 OpenStack Proposal Bot proposed openstack/tripleo-common: Updated from global requirements https://review.openstack.org/232282 03:07:30 OpenStack Proposal Bot proposed openstack/tuskar: Updated from global requirements https://review.openstack.org/233887 03:21:24 hey look, logged progress from an openstack overcloud deploy run http://logs.openstack.org/32/235032/3/check-tripleo/gate-tripleo-ironic-overcloud-f21puppet-ha/bcc94c4/console.html#_2015-10-15_02_13_28_328 03:29:04 Ignacio Bravo proposed openstack/tripleo-docs: Updated references from ironic-discovered to ironic-inspector https://review.openstack.org/235714 03:38:26 Tzu-Mainn Chen proposed openstack/tuskar: [WIP] Tuskar API v3 https://review.openstack.org/235716 04:22:14 Lokesh Jain proposed openstack/tripleo-heat-templates: Changes for configuring Nuage https://review.openstack.org/230116 04:26:00 Lokesh Jain proposed openstack/tripleo-heat-templates: Changes for configuring Nuage https://review.openstack.org/230116 04:56:18 stevebaker, that is awesome :) 04:56:52 gfidente: https://review.openstack.org/#/c/235032/ 05:01:48 is nested 2 by default to catch something important? 05:03:21 I think 1 is good enough honestly but I can imagine we all have different opinions on which value that should be :P 05:07:39 stevebaker, I'm not sure I understood how to use the marker? 05:08:27 stevebaker, you can just point it to any arbitrary past event and it'll log status for the resources modified from that point on? 05:09:43 gfidente: its how paging works in the client libs. the marker is the id of the entity where you want every one *after* that one 05:10:09 oh, I need to update that change to handle update as well as create 05:10:37 yeah was going to say that 07:12:35 Yanis Guenane proposed openstack/instack-undercloud: Manage keystone related configuration via Puppet https://review.openstack.org/234704 07:12:44 Yanis Guenane proposed openstack/instack-undercloud: Manage keystone related configuration via Puppet https://review.openstack.org/234704 07:16:48 Yanis Guenane proposed openstack/tripleo-heat-templates: Manage keystone related settings directly in t-h-t manifests https://review.openstack.org/230375 07:16:55 Yanis Guenane proposed openstack/tripleo-heat-templates: Manage keystone related settings directly in t-h-t manifests https://review.openstack.org/230357 07:17:49 Yanis Guenane proposed openstack/tripleo-heat-templates: Manage keystone related settings directly in t-h-t manifests https://review.openstack.org/230375 07:22:08 Yanis Guenane proposed openstack/tripleo-heat-templates: Manage keystone related settings directly in t-h-t manifests https://review.openstack.org/230375 07:31:54 Gael Chamoulaud proposed openstack/puppet-tripleo: Introduce 'log_aggregation_source' resource https://review.openstack.org/179848 07:41:16 Yanis Guenane proposed openstack/python-tripleoclient: Do not initialize/manage keystone in python-tripleoclient https://review.openstack.org/231395 07:41:16 Yanis Guenane proposed openstack/python-tripleoclient: Fix chicken and eggs problem with tripleo-heat-templates https://review.openstack.org/235769 07:43:18 Yanis Guenane proposed openstack/tripleo-heat-templates: Manage keystone related settings directly in t-h-t manifests https://review.openstack.org/230357 07:49:21 Merged openstack/python-tripleoclient: Explicitly unset package update hooks when re-deploying https://review.openstack.org/230969 07:52:26 Yanis Guenane proposed openstack/python-tripleoclient: Fix chicken and eggs problem with tripleo-heat-templates https://review.openstack.org/235769 07:54:32 Giulio Fidente proposed openstack/tripleo-heat-templates: Allow customization of the Ceph pool names https://review.openstack.org/212664 07:56:01 spredzy: nice commit title :) 07:56:29 spredzy, d0ugal: Have we considered how we handle backwards compatibility, e.g what if someone uses a new tripleoclient with some older t-h-t/puppet? 07:56:36 shardy, hehe. Naming is hard :p 07:57:00 or do we just document the requirement carefully? 07:58:47 shardy, as part of OSP-d, isn't the version of t-h-t some kind guaranteed to work with a specific version of python-tripleoclient ? Also, we could have only the chicken and eggs commit in - for some time - with a depreciation warning and in few weeks/months merge the other one 07:59:03 this way it remains compatible with both ways of managing keystone 08:01:59 spredzy: Yeah, I think this is a really nice step towards decoupling the templates and tripleoclient, but I was pointing out that we probably shouldn't switch immediately from the chicken/eggs approach to https://review.openstack.org/#/c/231395/ without somehow warning users 08:03:15 shardy, ack, let me reshape the review 08:03:23 spredzy: e.g maybe when we remove the endpoint config from tripleoclient, perhaps we can replace it with some validation, which checks the endpoints exist, and warns that you may need a newer tht/puppet version if they don't exist on CREATE_COMPLETE 08:04:11 do we do pypi releases for tripleoclient yet? 08:04:29 that was my main concern, e.g someone pip installs a new version on an older downstream environment or something 08:04:44 shardy you mean add the test + warning message here https://review.openstack.org/#/c/231395/6/tripleoclient/v1/over4cloud_deploy.py,cm 08:05:43 spredzy: yeah, there we could have a simple endpoint check, with a warning if none exist 08:06:17 shardy, https://pypi.python.org/pypi/python-tripleoclient/0.0.10 08:06:42 shardy ok will had that 08:07:40 spredzy: cool, so we probably need two pypi releases, one containing the chicken/eggs workaround, and another with the endpoint setup removed and replaced with a warning 08:07:52 then we can correlate package/pypi versions with t-h-t 08:08:09 sounds like a plan 08:10:06 jaosorior, ping - https://review.openstack.org/#/c/230375/. Sorry I missed your ping yesterday 08:12:01 Their was a first attemps doing what you proposed during PS6 https://review.openstack.org/#/c/230375/6/puppet/controller.yaml,cm, but then we agreed on that it was going to be done in a later patchset as the scope of the change was much broader than just keystone itself 08:15:41 Derek Higgins proposed openstack/diskimage-builder: Define a default $YUM for yum-minimal https://review.openstack.org/235539 08:20:32 Is image building broken? 08:20:41 I seem to just run into a new issue every day at the moment :) 08:27:07 shardy, have a weird request 08:27:51 shardy, do you think there is any chance we can change the type of a resource via param? 08:29:00 gfidente: No, that's what the resource_registry is for 08:29:15 shardy, yes I get the point of the registry 08:29:30 gfidente: it has been discussed before, but it's not possible because of the order of resolution inside heat 08:29:57 shardy, ah I see; to give some context, we hit an issue where some pre-existing resources changed type in between upgrades 08:30:20 shardy, so they're going to be re-created even though the registry points to a definition identical to the old one 08:30:38 gfidente: Yeah, that's the problem with renaming the resource types 08:31:10 Marek Aufart proposed openstack/tripleo-common: [WIP]Add resource capabilities https://review.openstack.org/232534 08:31:17 gfidente: one thing I've considered is whenter we could maintain a nested stack when a resource changes type (but the name of the resource stays the same) 08:31:42 shardy, in this case we didn't just rename it, we changed it so that we could the definition from the registry 08:31:50 *so that we could change 08:31:56 gfidente: in particular I was thinking it's be good to have a way to migrate from ResourceGroup to AutoscalingGroup without destroying your entire cluster, which isn't currently possible 08:32:17 gfidente: can you point to an example? 08:32:29 if it's only the nested stack that's changed, it should get updated in place 08:32:41 shardy, sure https://review.openstack.org/#/c/222497/16/overcloud-without-mergepy.yaml,cm 08:32:48 but if type: foo changes to type: foo2 it will get replaced 08:33:05 shardy, yeah it's "same issue" you mentioned with the resourcegroup 08:33:58 gfidente: one workaround might be to define some additional aliases in the reource_registry 08:34:04 shardy, the the VIPs resource type gets changed so that we can switch from the registry to a yaml that gets the VIPs from parameter 08:34:30 aliases in the registry? 08:34:38 gfidente: e.g OS::TripleO::Network::Ports::ExternalVipPort: OS::TripleO::Controller::Ports::ExternalPort 08:34:55 then OS::TripleO::Controller::Ports::ExternalPort points to the template yaml 08:35:16 gfidente: you can map one type to another, I'm wondering if that would avoid the resource replacement 08:35:28 yeah exaclty, was going to ask about replacement 08:35:34 good to know you're already on it actually :) 08:35:38 thanks! 08:35:59 gfidente: np, let me know if that works, I haven't tested it 08:36:17 If it does then I can write a heat functional test to ensure it stays working ;) 08:36:30 I see 08:36:34 ok this is definitely worth trying 08:36:37 :) 08:36:48 http://docs.openstack.org/developer/heat/template_guide/environment.html#mapping-resources 08:49:07 shardy, can I get your reviews on https://review.openstack.org/#/c/232051/ and https://review.openstack.org/#/c/222602/ when you have time please 08:50:24 Yanis Guenane proposed openstack/tripleo-heat-templates: neutron: enable nova-event-callback by default https://review.openstack.org/225962 08:52:11 Giulio Fidente proposed openstack/tripleo-heat-templates: Allow customization of Ceph client user https://review.openstack.org/212686 08:52:46 Merged openstack/tripleo-heat-templates: Allow a deployer to specify HAProxy syslog server address https://review.openstack.org/222602 08:55:49 What is the current status of the HA job ? Is it supposed to be passing ? 08:56:02 Rico Lin proposed openstack/python-tripleoclient: Improve readme contents https://review.openstack.org/235234 08:56:23 Derek Higgins proposed openstack/diskimage-builder: Define a default for $YUM https://review.openstack.org/235539 08:57:07 spredzy: yeah it should be http://tripleo.org/cistatus.html 08:57:42 jistr, :o looks nice :) 08:58:05 I remember the old page cssless, this one looks nice. 09:00:17 Untriaged bugs so far: 09:00:18 https://bugs.launchpad.net/tripleo/+bug/1505815 09:00:18 Launchpad bug 1505815 in tripleo "Docs refer to ironic-discoverd instead of ironic-inspector" [Undecided,Confirmed] - Assigned to Ruchi Parmar (ruchiparmar143) 09:00:19 https://bugs.launchpad.net/tripleo/+bug/1506651 09:00:19 Launchpad bug 1506651 in tripleo "Definition of /etc/hosts not correct in docs" [Undecided,New] - Assigned to Ignacio Bravo (ibravo) 09:00:22 https://bugs.launchpad.net/tripleo/+bug/1505430 09:00:22 Launchpad bug 1505430 in tripleo "Triggering puppet runs with config_identifier is error prone and complicates templates" [Undecided,New] - Assigned to Steve Baker (steve-stevebaker) 09:00:23 https://bugs.launchpad.net/diskimage-builder/+bug/1506692 09:00:25 https://bugs.launchpad.net/diskimage-builder/+bug/1505900 09:00:25 Launchpad bug 1506692 in diskimage-builder "Broken test in fedora-minimal/install.d/99-ramdisk" [Undecided,In progress] - Assigned to Martin André (mandre) 09:00:26 Launchpad bug 1505900 in diskimage-builder "Failed to create 'wily' release with DIB_RELEASE" [Undecided,New] 09:00:27 https://bugs.launchpad.net/tripleo/+bug/1504263 09:00:28 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 09:00:29 https://bugs.launchpad.net/diskimage-builder/+bug/1506275 09:00:30 Launchpad bug 1506275 in diskimage-builder "$DIB_OFFLINE and $DIB_DEBIAN_USE_DEBOOTSTRAP_CACHE are redundant" [Undecided,New] 09:00:31 https://bugs.launchpad.net/tripleo/+bug/1504363 09:00:31 Launchpad bug 1504363 in tripleo "Updates to hiera configuration not propogating/making a puppet run" [Undecided,Confirmed] 09:00:33 https://bugs.launchpad.net/tripleo/+bug/1506546 09:00:33 Launchpad bug 1506546 in tripleo "Cisco Nexus ML2 MD: Need unique key for server level keys in hieradata" [Undecided,In progress] - Assigned to Timothy Swanson (tiswanso) 09:03:44 hi all, can we get some eyes here https://review.openstack.org/#/c/219562/ ? 09:04:13 this is needed for https://review.openstack.org/#/c/219349/ which will enable testing DIB correctly in the gate (by building it from source) 09:04:19 gate in Ironic* 09:10:15 Derek Higgins proposed openstack-infra/tripleo-ci: Use the new shortened tripleo ci job names https://review.openstack.org/235420 09:10:16 Derek Higgins proposed openstack-infra/tripleo-ci: Add the new periodic jobs https://review.openstack.org/235421 09:10:17 Derek Higgins proposed openstack-infra/tripleo-ci: Removing cellspacing from the tripleo ci report https://review.openstack.org/235419 09:10:18 Derek Higgins proposed openstack-infra/tripleo-ci: Update job names to add "-ci" https://review.openstack.org/235795 09:13:30 Giulio Fidente proposed openstack/tripleo-heat-templates: Ensure access to RabbitMQ is password protected https://review.openstack.org/210402 09:21:58 Yanis Guenane proposed openstack/tripleo-heat-templates: Manage keystone related settings directly in t-h-t manifests https://review.openstack.org/230357 09:21:59 Yanis Guenane proposed openstack/tripleo-heat-templates: Manage keystone related settings directly in t-h-t manifests https://review.openstack.org/230375 09:23:57 Yanis Guenane proposed openstack/tripleo-heat-templates: Manage keystone related settings directly in t-h-t manifests https://review.openstack.org/230357 09:52:38 hi, what is a standard way add some custom parameters to systemctl.conf for my overcloud controller? 09:53:27 I've found the file /usr/.../puppet/hieradata/controller.yaml but am not sure if that's the right one and how to handle the customization 09:55:01 kukacz, https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/hieradata/common.yaml#L18-L24 09:55:20 but we have no parameter from the top level that allows this customization AFAIK 09:55:49 erf ... sorry I misread systemctl and sysctl :/ nevermind 09:56:23 spredzy, ahh, that was my fault. I really meant sysctl.conf 09:56:53 Oh, then what I said above apply 09:57:53 spredzy: so the way how to customize it correctly is to copy the puppet/hieradata/common.yaml to my custom templates directory and edit it there? 09:58:23 Well the only way I can think of atm to do that without parameter at the top level is to use https://github.com/openstack/tripleo-heat-templates/commit/109931bca3e9403abf468f371d9f118e0b5cacf3 09:59:08 so you create a hiera file (name must match the id of the machine) and specify the sysctl config there as above 09:59:12 it will then get applied 09:59:36 meanwhile you can fill a bug/rfe so we can enable a paramter to take care of that 09:59:59 kukacz, ^ 10:00:00 spredzy: Yes we do, you can pass any hieradata via ExtraConfig parameters 10:00:17 https://review.openstack.org/#/c/233012/ 10:00:21 docs patch showing how 10:00:26 Oops then nevermind, follow shardy's comment 10:00:47 kukacz: ^^ you don't have to copy the templates or hack on them, although yes that would work 10:01:12 spredzy: thanks anyway! 10:01:14 shardy, awesome didn't know that feature 10:01:32 spredzy: Yeah I wired those in relatively recently 10:01:43 they take precedence in the hierarchy over the static hieradata files 10:01:52 shardy: I've already created a custom templates directory due to my network addressing modifications 10:02:07 nit: l213 is it controllerExtraConfig or ControllerExtraConfig (or is it case insensitive?) 10:02:10 shardy, is that a right way to follow to avoid editing in /usr/... ? 10:02:27 spredzy: it's controllerExtraConfig not case insensitive 10:02:33 ack 10:02:39 for some reason that's not captialized, all the other parameters are 10:02:53 we should fix that, but it was like that for a long time for the element based templates 10:02:57 erf I didn't read the whole sentence it was specified at the end 10:03:23 kukacz: Yeah, if you follow my docs you don't have to edit the templates at all 10:03:24 top feature :) 10:03:39 kukacz: you just do openstack overcloud deploy --templates -e my_extra_params.yaml 10:03:48 and put your needed overrides into my_extra_params.yaml 10:04:07 kukacz: obviously you *can* cp -r /usr/share/openstack-heat-templates my_templates 10:04:08 shardy: aha, that's very nice 10:04:21 then do openstack overcloud deploy --templates my_templates 10:04:40 but then it's more painful when you need to rebase the forked version 10:05:00 both approaches will be documented when we land those docs patches 10:05:36 shardy: is this a new feature or does it also apply on RH OSP Director 7 ? 10:06:53 https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/controller.yaml#L979 10:07:28 kukacz: I think it's in the OSPd packages, but you can check by comparing /usr/share/openstack-tripleo-heat-templates/puppet/controller-puppet.yaml with that 10:07:57 If you have the "extraconfig" and "controller_extraconfig" items in the hierarchy list then it should work as in the docs 10:08:44 shardy: I do have it there 10:09:02 shardy: thanks! now I'll try to apply it 10:10:11 also, another question in my mind - is there a way how to make software raid-1 on deployed overcloud nodes? 10:11:43 kukacz: probably very easily for the OS disk, as the deployment ramdisk would need to create the raid, but for any additional disks you could do it via a firstboot script 10:11:46 https://github.com/openstack/tripleo-heat-templates/tree/master/firstboot 10:12:12 kukacz: probably *not* very easily, sorry 10:12:36 lucasagomes: ^^ is there any IPA support for creating sw raid before deploying an image? 10:13:14 shardy, hi there, not really... We started thinking about an abstraction layer in Ironic but we don't have SW raid support it 10:13:24 it conflicts with cleaning (which erease the disk) 10:13:40 lucasagomes: Ok, thanks for the clarification :) 10:13:45 so that the sw raid should be recreated every time after cleaning, we still thinking about it 10:14:01 thanks both! 10:14:21 shardy, http://specs.openstack.org/openstack/ironic-specs/specs/approved/inband-raid-configuration.html this spec, which is already approved 10:14:22 http://docs.openstack.org/developer/tripleo-docs/advanced_deployment/extra_config.html#firstboot-extra-configuration 10:14:37 kukacz: docs on firstboot which would enable configuring secondary disks however you want 10:14:41 talks about software raid, we probably will have support for it in the next cycle 10:14:44 mitaka 10:14:56 lucasagomes: nice! Will check it out, thanks! 10:15:01 shardy, no problem 10:15:01 lucasagomes: great, that would by nice to see! 10:15:32 kukacz, yeah, the spec is a bit broad, since it aims to create a generic interface for raid (be it hw or sw raid) 10:15:33 shardy: thanks. I was thinking applying it for the OS 10:43:39 lucasagomes, are you saying we can already clean the disks in ironic? 10:43:51 gfidente, if used with IPA yes 10:44:04 clean, shred and even extend it for whatever else you need 10:44:11 firmware updates etc... 10:44:24 can you append that as comment https://bugzilla.redhat.com/show_bug.cgi?id=1252260 maybe? 10:44:24 bugzilla.redhat.com bug 1252260 in rhel-osp-director "[RFE] Use ironic to partition Ceph OSD disks automatically during installation" [Unspecified,New] - Assigned to calfonso 10:44:37 gfidente, will do 10:44:44 1 sec 10:44:45 lucasagomes, see https://bugzilla.redhat.com/show_bug.cgi?id=1252260#c8 10:45:30 we probably just need to piece together the puzzle 10:46:41 Lucas Alvares Gomes proposed openstack/diskimage-builder: WIP: Install IPA in a venv https://review.openstack.org/235843 10:50:42 Giulio Fidente proposed openstack/tripleo-heat-templates: Wire Neutron ML2 plugin and OVS agent settings as arrays https://review.openstack.org/187506 10:55:46 gfidente, commented :-) 10:57:55 lucasagomes, so the idea is that we should find the logic when this erase is appropriate to do 10:58:08 lucasagomes, ideally only on 'new stack' creation 10:58:39 I think heat can use special handling for the UPDATE vs CREATE case 10:58:56 gfidente, ironic will erase the disk in 2 stages: First when it goes from "manageable" to "available", where "available" is the state which the node gets exposed to the nova scheduler 10:59:36 and Ironic also will erase it every time the instance is deleted from the node, to clean up the previous tentant data 11:00:03 and that pretty much matches the requirement I think 11:00:37 is this enabled by default? 11:06:36 gfidente, yup, it's enabled by default in Ironic 11:06:44 see clean_nodes config option 11:13:33 so you have one more adept for migration to ipa fwiw 11:15:37 :-) w00t! 11:23:39 Merged openstack/os-net-config: Add Linux Bridge capability to os-net-config ifcfg https://review.openstack.org/219927 11:23:50 Merged openstack/os-net-config: Fix kwarg assignment in objects.py https://review.openstack.org/231247 11:43:05 spredzy: so do you think the swiftoperator issue i pointed out is an issue in the puppet module side? shouldn't it not try and create SwiftOperator if swiftoperator exists? 11:49:55 dprince: https://review.openstack.org/#/c/235795/ 12:08:13 bnemec: the part for the ci monitor script https://review.openstack.org/#/c/235795/1 12:08:25 bnemec: thanks for rebasing my stuff 12:21:02 Emilien Macchi proposed openstack/tripleo-heat-templates: neutron: enable nova-event-callback by default https://review.openstack.org/225962 12:28:09 slagle, hey. This review allows us to move forward now. I submitted https://review.openstack.org/#/c/235886/, asking for feedbacks. Having it in would probably take some time, meanwhile we can go ahead with the current PS 12:28:58 slagle, also for your -1 on https://review.openstack.org/#/c/230357/ what you mention is addressed in a dependent review https://review.openstack.org/#/c/230375/ 12:29:50 hi 12:32:42 am trying to deploy a undercloud using the tutorial but I got some errors :( " Error: Could not start Service[nova-api]: Execution of '/bin/systemctl start openstack-nova-api' returned 1: Job for openstack-nova-api.service failed. See 'systemctl status openstack-nova-api.service' and 'journalctl -xn' for details. 12:32:47 2015-10-15 15:32:59,139 INFO: Wrapped exception: 12:32:50 2015-10-15 15:32:59,140 INFO: Execution of '/bin/systemctl start openstack-nova-api' returned 1: Job for openstack-nova-api.service failed. See 'systemctl status openstack-nova-api.service' and 'journalctl -xn' for details. 12:32:54 2015-10-15 15:32:59,141 INFO: Error: /Stage[main]/Nova::Api/Nova::Generic_service[api]/Service[nova-api]/ensure: change from stopped to running failed: Could not start Service[nova-api]: Execution of '/bin/systemctl start openstack-nova-api' returned 1: Job for openstack-nova-api.service failed. See 'systemctl status openstack-nova-api.service' and 'journalctl -xn' for details. 12:33:00 2015-10-15 15:32:59,374 INFO: Notice: /Stage[main]/Nova::Api/Nova::Generic_service[api]/Service[nova-api]: Triggered 'refresh' from 4 events 12:33:03 2015-10-15 15:33:32,166 INFO: /usr/libexec/os-refresh-config/configure.d/50-puppet-stack-config: line 7: 28617 Killed puppet apply --detailed-exitcodes /etc/puppet/manifests/puppet-stack-config.pp 12:33:07 2015-10-15 15:33:32,176 INFO: + rc=137 12:33:09 2015-10-15 15:33:32,176 INFO: + set -e 12:33:12 2015-10-15 15:33:32,177 INFO: + echo 'puppet apply exited with exit code 137' 12:33:13 spredzy: k, thanks will have a look 12:33:14 2015-10-15 15:33:32,178 INFO: puppet apply exited with exit code 137 12:33:17 2015-10-15 15:33:32,178 INFO: + '[' 137 '!=' 2 -a 137 '!=' 0 ']' 12:33:19 2015-10-15 15:33:32,179 INFO: + exit 137 12:33:22 " 12:33:24 any one can help me ? 12:35:00 thanks 12:35:12 jlourdess: hey o/ - fyi might be nicer to use sthing like http://paste.openstack.org/ to share traces like that. where are you seeing this and are you referrring to http://docs.openstack.org/developer/tripleo-docs/ 12:35:23 ok 12:39:17 marios: thanks. here are the lik 12:39:28 links : http://paste.openstack.org/show/476483/ 12:39:46 http://docs.openstack.org/developer/tripleo-docs 12:43:31 jaosorior, ping 12:43:59 - https://review.openstack.org/#/c/230375/ 12:44:11 spredzy: pong 12:44:53 to answer of what you asked yesterday 12:44:54 Their was a first attemps doing what you proposed during PS6 https://review.openstack.org/#/c/230375/6/puppet/controller.yaml,cm, but then we agreed on that it was going to be done in a later patchset as the scope of the change was much broader than just keystone itself 12:45:17 Was it in PS6 the kind of flexibility you were looking for ? 12:45:38 jlourdess: is that during installation of the undercloud? the error the fpaste doesn't say much - is there anything useful about why nova-api failed to start? 12:45:57 spredzy: That looks reasonable 12:48:15 I guess at some point it would be nice to be able to specify an fqdn instead of showing the IPs in the catalog; But that would be a separate patch 12:48:57 Yanis Guenane proposed openstack/python-tripleoclient: Fix chicken and eggs problem with tripleo-heat-templates https://review.openstack.org/235769 12:49:10 d0ugal, ^ sorry I misunderstood your first sentence :/ 12:49:17 s/sentence/comment 12:50:19 jaosorior, +1 the idea 12:52:10 slagle: https://review.openstack.org/#/c/232105/ got a sec to discuss the back-compat options? 12:53:44 thrash: sure 12:54:21 slagle: so, I think you can solve the back-compat by leaving the old param, and just have both point to the same dest var 12:54:33 and just flip-flop the logic so it makes sense. 12:55:08 right, makes sense 12:55:24 slagle: cool. Just trying to get that stuff moved along. :) 13:00:29 thrash: is there something built into argparse to make sure both aren't specified at the same time? 13:00:41 i can rtfm :) 13:00:45 marios: I ill have a look and replay soon for api 13:00:47 slagle: yes. You can use a mutually exclusive group 13:00:54 k, will have a look 13:01:12 slagle: pretty sure we have an example of one in the deploy 13:01:16 already. 13:03:42 Yanis Guenane proposed openstack/tripleo-heat-templates: Manage keystone related settings directly in t-h-t manifests https://review.openstack.org/230375 13:16:07 spredzy: no problem :) 13:21:38 Florian Fuchs proposed openstack/tripleo-common: [WIP]Adds deployment configuration methods https://review.openstack.org/230315 13:25:45 Florian Fuchs proposed openstack/tripleo-common: [WIP]Adds deployment configuration methods https://review.openstack.org/230315 13:33:52 Emilien Macchi proposed openstack/tripleo-heat-templates: Implement Workers parameters https://review.openstack.org/229471 13:35:54 Emilien Macchi proposed openstack/tripleo-heat-templates: controller/ceilometer: use internalURL for os endpoint type https://review.openstack.org/225999 13:40:46 Emilien Macchi proposed openstack/tripleo-heat-templates: controller/ceilometer: use internalURL for os endpoint type https://review.openstack.org/225999 13:55:26 Merged openstack-infra/tripleo-ci: Update job names to add "-ci" https://review.openstack.org/235795 13:56:33 dprince: can you deploy that ^ 13:56:56 derekh: it'll deploy automatically 13:57:09 dprince: fancy 13:57:44 derekh: at least that was the plan, we'll see 13:58:04 derekh: 10 minutes and we'll know I guess 13:59:15 dprince: ack, the suspense is killing me 14:10:51 derekh: yeah, I had to un-pin it. I forgot I initially pinned tripleo-ci to my (unlanded) patches. It is running now... 14:11:45 dprince: cooleo, thanks 14:11:54 dprince: there landed now right ? 14:14:14 derekh: boom, http://tripleo.org/cistatus.html 14:15:12 dprince: happy days, thanks 14:15:37 derekh: anyways, it is unpinned now so future changes will auto update about every 10 minutes 14:15:51 dprince: okie dokie 14:16:03 derekh: also, I'll set you up w/ creds on this box so you can jump on it too if you'd like 14:16:50 dprince: ack, I can add another login to my collection 14:31:33 Merged openstack/tripleo-heat-templates: Puppet / Compute: allow to run Ephemeral only storage with RBD https://review.openstack.org/202206 15:01:06 Untriaged bugs so far: 15:01:08 https://bugs.launchpad.net/tripleo/+bug/1505815 15:01:08 Launchpad bug 1505815 in tripleo "Docs refer to ironic-discoverd instead of ironic-inspector" [Undecided,Confirmed] - Assigned to Ruchi Parmar (ruchiparmar143) 15:01:10 https://bugs.launchpad.net/diskimage-builder/+bug/1506792 15:01:10 Launchpad bug 1506792 in diskimage-builder "ironic-python-agent fails to run on fedora ramdisk" [Undecided,In progress] - Assigned to Lucas Alvares Gomes (lucasagomes) 15:01:11 https://bugs.launchpad.net/tripleo/+bug/1505430 15:01:11 Launchpad bug 1505430 in tripleo "Triggering puppet runs with config_identifier is error prone and complicates templates" [Undecided,New] - Assigned to Steve Baker (steve-stevebaker) 15:01:13 https://bugs.launchpad.net/diskimage-builder/+bug/1506692 15:01:13 Launchpad bug 1506692 in diskimage-builder "Broken test in fedora-minimal/install.d/99-ramdisk" [Undecided,In progress] - Assigned to Martin André (mandre) 15:01:15 https://bugs.launchpad.net/diskimage-builder/+bug/1505900 15:01:15 Launchpad bug 1505900 in diskimage-builder "Failed to create 'wily' release with DIB_RELEASE" [Undecided,New] 15:01:17 https://bugs.launchpad.net/tripleo/+bug/1504263 15:01:17 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 15:01:19 https://bugs.launchpad.net/diskimage-builder/+bug/1506275 15:01:19 Launchpad bug 1506275 in diskimage-builder "$DIB_OFFLINE and $DIB_DEBIAN_USE_DEBOOTSTRAP_CACHE are redundant" [Undecided,New] 15:01:21 https://bugs.launchpad.net/tripleo/+bug/1506651 15:01:21 Launchpad bug 1506651 in tripleo "Definition of /etc/hosts not correct in docs" [Undecided,New] - Assigned to Ignacio Bravo (ibravo) 15:21:03 Dougal Matthews proposed openstack/tripleo-common: Collect the common logic in tripleo-common https://review.openstack.org/228991 15:21:04 Dougal Matthews proposed openstack/tripleo-common: Rename stack_update to _stack_update https://review.openstack.org/235976 15:25:21 Dougal Matthews proposed openstack/tripleo-common: Collect the common logic in tripleo-common https://review.openstack.org/228991 15:25:21 Dougal Matthews proposed openstack/tripleo-common: Rename stack_update to _stack_update https://review.openstack.org/235976 15:25:22 Dougal Matthews proposed openstack/tripleo-common: Collect constants in one file https://review.openstack.org/235977 15:44:14 marios: ping https://review.openstack.org/#/c/213175/ - can you help me to work on that patch? 15:44:49 Dougal Matthews proposed openstack/tripleo-common: Collect constants in one file https://review.openstack.org/235977 15:45:21 Emilien Macchi proposed openstack/tripleo-heat-templates: neutron: enable nova-event-callback by default https://review.openstack.org/225962 15:46:02 Emilien Macchi proposed openstack/tripleo-heat-templates: nova-api: set catalog_info to InternalURL https://review.openstack.org/225983 15:46:07 Emilien Macchi proposed openstack/tripleo-heat-templates: nova-api: set catalog_info to InternalURL https://review.openstack.org/225983 15:46:42 dprince, slagle: can you look https://review.openstack.org/#/c/225983/ and https://review.openstack.org/#/c/225999/ please ? 15:47:05 thrash: tripleoclient unit tests are failing locally for me with: http://fpaste.org/280032/45010420/ 15:47:29 does that look familiar at all, or did i just hose myself somehow (likely)? 15:48:19 slagle: How did you run the tests? 15:48:39 slagle: AFAIK, it wont work outside of tox because it uses tripleo-common master :/ 15:48:50 just the standard tox -e py27 15:48:56 hmm, odd 15:49:02 i blew away my .tox and tried again too, same thing 15:49:18 d0ugal: yes, i get that too 15:49:53 It worked last time I tried :D 15:49:54 Running now. 15:51:06 slagle, pino|work - worked for me, but that might just be due to my local wheel cache 15:51:20 where is it? 15:51:36 pino|work: The wheel cache? ~/.cache/pip 15:51:45 I just wiped it, so let's see what happens now :) 15:51:58 ah, i tested only wiping .tox 15:52:29 I tried that first and the tests passed 15:52:30 ok, so i was using python-tox from f22 (1.8 something). i pip updated that to latest (2.1.1) and it works now 15:52:44 Interesting 15:52:50 * pino|work is still on f21 15:53:22 I am on tox 2.1.1 and pop 7.1.0 :) 15:53:39 pop? 15:54:16 pip and pop and tix and tox 15:54:20 fox and socks 15:54:55 haha 15:55:03 the fox got a block as a big black box? 15:55:09 pip 7.1 15:55:22 still fails here :/ 15:55:48 sec looking 15:55:54 I am still waiting for it to install 15:55:59 EmilienM: looking 15:56:07 clearing my pip cache when I am on a slow connection is probably a terrible idea 15:56:18 EmilienM: sure, i can try running it tomorrow to get the order right if that's what you mean 15:56:31 marios: tomorrow is saturday :) 15:56:33 EmilienM: err... monday 15:56:35 :) 15:56:51 I don't have an HA setup here 15:57:19 slagle: http://fpaste.org/280038/ after deleting ~/.pip and .tox 15:57:56 EmilienM: i have one but is downstream so may have rebase hell to try it out there... my upstream env is being rebuilt (but today, my undercloud consistently out of memory for image builds) 16:21:56 Marius Cornea proposed openstack/tripleo-docs: Add documentation on how to depoy HA overcloud https://review.openstack.org/235597 16:27:37 Marios Andreou proposed openstack/tripleo-docs: Tidy up docs - fixup missing links to TOC where possible https://review.openstack.org/236003 16:36:18 Derek Higgins proposed openstack/diskimage-builder: Preserve env when calling yum with sudo https://review.openstack.org/236006 16:44:55 pino|work: i had to sudo pip install -U tox to get it fixed for me 16:45:25 Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Enable TLS deployment for the Overcloud https://review.openstack.org/231930 16:47:21 Marius Cornea proposed openstack/tripleo-docs: Add documentation on how to depoy HA overcloud https://review.openstack.org/235597 16:54:20 slagle: I started seeing that yesterday, but chalked it up to something transient. 16:54:47 slagle: looks like d0ugal figured it out? 17:04:53 slagle: Can we land those delorean tripleo.sh patches please? 17:05:08 I've tested all the recent revisions made by derekh and it works well 17:05:30 I'd like to finish a docs patch showing how to use it, but we need those landed first :) 17:06:16 shardy: yes. i reviewed them earlier, i think my +2 probably fell off by now though 17:06:26 sorry, way under water today 17:06:51 slagle: cool, I +2'd them since derekh has been doing all the actual work on them lately 17:07:05 so they just need one last pass I think 17:07:22 CI looks good other than the HA job on the second patch, which must be unrelated 17:08:33 shardy: did you happen to test on centos7? just curious, not going to -1 for that 17:08:45 wondering if you could run this on the undercloud itself 17:08:46 slagle: Yeah, all my testing has been on centos7 17:08:52 cool 17:09:06 slagle: I've tested both, either works provided you've got enough ram and disk 17:11:10 slagle: to clarify, I've tested both patches, on both the host box and the undercloud 17:12:10 also exported the host delorean to the undercloud, as it could be useful to document how to have a persistent tree of git dirs and associated delorean repo on the host 17:12:26 so you can rebuild the UC without losing all your work ;) 17:12:33 sgtm 17:12:37 (don't ask me how I know that :D) 17:12:44 i +a'd both. thx, this will be super useful 17:12:45 Merged openstack/tripleo-common: tripleo.sh Add option to create a local Delorean environment https://review.openstack.org/228156 17:13:04 slagle: thanks! :) 17:13:19 the corresponding ci patch to switch to use these is already up right? 17:13:35 slagle: yep although I've not looked at it today 17:13:40 https://review.openstack.org/#/c/229425/ 17:14:34 Merged openstack/tripleo-common: tripleo.sh Add option to locally build delorean package https://review.openstack.org/228157 17:15:22 slagle: CI patch lgtm and is all green 17:16:24 Merged openstack-infra/tripleo-ci: Use tripleo.sh to build packages https://review.openstack.org/229425 17:24:41 Tzu-Mainn Chen proposed openstack/tuskar: [WIP] Tuskar API v3 https://review.openstack.org/235716 17:40:29 James Slagle proposed openstack/python-tripleoclient: Make hypervisor stat check a validation https://review.openstack.org/232041 17:40:29 James Slagle proposed openstack/python-tripleoclient: Add --skip-validation-checks cli arg https://review.openstack.org/232209 17:40:30 James Slagle proposed openstack/python-tripleoclient: Validation cli args improvements https://review.openstack.org/232105 17:40:30 James Slagle proposed openstack/python-tripleoclient: Make node count check a validation https://review.openstack.org/232106 17:41:40 Merged openstack/tripleo-image-elements: Wires up neutron tunnel_id_ranges and vni_ranges for config https://review.openstack.org/199592 17:45:18 Merged openstack/instack-undercloud: Update undercloud.conf.sample with deprecated names. https://review.openstack.org/225259 17:46:04 Merged openstack/instack-undercloud: Add a RedHat.yaml hieradata file https://review.openstack.org/222310 17:48:37 Merged openstack/instack-undercloud: Remove some legacy stuff https://review.openstack.org/224125 17:48:50 Merged openstack/instack-undercloud: Remove unused elements https://review.openstack.org/227607 17:49:31 slagle: haha... that one bothered me too... but I never found myself actually in that section of the code. 'paramaters' 17:49:43 Merged openstack/tripleo-common: Check if regexp is valid when updating packages https://review.openstack.org/213101 17:49:49 Merged openstack/tripleo-docs: Remove GUI documentation https://review.openstack.org/235552 17:51:36 thrash: you know you have to say "paramaters" in a Larry the cable guy voice 17:51:44 slagle: haha 17:51:51 I was thinking that! 17:51:56 para - maters 17:52:33 Merged openstack/tripleo-heat-templates: Force stop a single node pacemaker on yum update https://review.openstack.org/229711 18:07:51 James Slagle proposed openstack/python-tripleoclient: Make hypervisor stat check a validation https://review.openstack.org/232041 18:07:51 James Slagle proposed openstack/python-tripleoclient: Add --skip-validation-checks cli arg https://review.openstack.org/232209 18:07:52 James Slagle proposed openstack/python-tripleoclient: Validation cli args improvements https://review.openstack.org/232105 18:07:52 James Slagle proposed openstack/python-tripleoclient: Make node count check a validation https://review.openstack.org/232106 18:32:08 Emilien Macchi proposed openstack/tripleo-heat-templates: nova-api: set catalog_info to InternalURL https://review.openstack.org/225983 18:32:23 Emilien Macchi proposed openstack/tripleo-heat-templates: controller/ceilometer: use internalURL for os endpoint type https://review.openstack.org/225999 18:46:25 EmilienM: see my latest on https://review.openstack.org/#/c/234704 18:47:56 slagle: looking 18:48:34 oh dear 18:48:52 the puppet run is idempotent in our CI 18:49:06 so puppet has a trouble to detect the endpoint is already created 18:49:22 it's likely because some informations are not the same between without/with puppet resource 18:49:33 let me check something 18:49:37 yea, and i think the modules need to use id's, not names here 18:49:42 if it's going to fail in this way 18:59:48 slagle: ok I got it. Indeed this is 1/ a bug in osclient 2/ our provider is using service_name['id'] to manage services 19:00:02 it's just in the endpoint creation, it uses the service name 19:01:03 I can see how to patch it but it's sounds an hack of trying to fix osclient 19:01:45 yea, i'm not as concerned about the osclient bug 19:01:58 b/c if the puppet modules work, we won't get tripped by that bug 19:06:52 slagle: the fact is we're strong users of osclient 19:07:00 and we hit each bug of it 19:07:04 which is good and bad 19:07:13 good because we give early feedback when something is broken 19:07:19 bad because we're quite often broken 19:07:23 we have a thread about that 19:07:31 some folks want to rewrite a ruby library 19:08:56 slagle: I'm creating a bug 19:14:34 slagle: https://bugs.launchpad.net/puppet-keystone/+bug/1506996 19:14:34 Launchpad bug 1506996 in puppet-keystone "puppet not able to find non-puppet managed services resources" [Undecided,New] 19:14:45 I'll investigate the bug in osclient too, it needs to be fixed there 20:12:14 Merged openstack/tripleo-docs: Add documentation on how to depoy HA overcloud https://review.openstack.org/235597 20:26:51 EmilienM, I'm pretty sure nova-compute uses the cinder keystone url to attach volumes 20:27:19 that's the very reason why we added it there in the first place 20:39:22 Rohit Pagedar proposed openstack/tripleo-heat-templates: Changes for configuring Nuage https://review.openstack.org/234970 20:58:31 Merged openstack/tripleo-image-elements: [Labs] Update links to Cloud Admin Guide https://review.openstack.org/219945 21:00:18 Untriaged bugs so far: 21:00:20 https://bugs.launchpad.net/tripleo/+bug/1505815 21:00:20 Launchpad bug 1505815 in tripleo "Docs refer to ironic-discoverd instead of ironic-inspector" [Undecided,Confirmed] - Assigned to Ruchi Parmar (ruchiparmar143) 21:00:21 https://bugs.launchpad.net/diskimage-builder/+bug/1506792 21:00:21 Launchpad bug 1506792 in diskimage-builder "ironic-python-agent fails to run on fedora ramdisk" [Undecided,In progress] - Assigned to Lucas Alvares Gomes (lucasagomes) 21:00:23 https://bugs.launchpad.net/tripleo/+bug/1505430 21:00:23 Launchpad bug 1505430 in tripleo "Triggering puppet runs with config_identifier is error prone and complicates templates" [Undecided,New] - Assigned to Steve Baker (steve-stevebaker) 21:00:25 https://bugs.launchpad.net/diskimage-builder/+bug/1506692 21:00:25 Launchpad bug 1506692 in diskimage-builder "Broken test in fedora-minimal/install.d/99-ramdisk" [Undecided,In progress] - Assigned to Martin André (mandre) 21:00:27 https://bugs.launchpad.net/diskimage-builder/+bug/1505900 21:00:28 Launchpad bug 1505900 in diskimage-builder "Failed to create 'wily' release with DIB_RELEASE" [Undecided,New] 21:00:29 https://bugs.launchpad.net/tripleo/+bug/1504263 21:00:29 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 21:00:31 https://bugs.launchpad.net/diskimage-builder/+bug/1506275 21:00:31 Launchpad bug 1506275 in diskimage-builder "$DIB_OFFLINE and $DIB_DEBIAN_USE_DEBOOTSTRAP_CACHE are redundant" [Undecided,New] 21:00:33 https://bugs.launchpad.net/tripleo/+bug/1506651 21:00:34 Launchpad bug 1506651 in tripleo "Definition of /etc/hosts not correct in docs" [Undecided,New] - Assigned to Ignacio Bravo (ibravo) 21:47:42 Giulio Fidente proposed openstack/tripleo-docs: Update instructions to deploy with Ceph https://review.openstack.org/236140 21:48:52 Giulio Fidente proposed openstack/tripleo-docs: Update instructions to deploy with Ceph https://review.openstack.org/236140 22:26:23 OpenStack Proposal Bot proposed openstack/tripleo-common: Updated from global requirements https://review.openstack.org/232282 03:00:18 Untriaged bugs so far: 03:00:23 https://bugs.launchpad.net/tripleo/+bug/1505815 03:00:23 Launchpad bug 1505815 in tripleo "Docs refer to ironic-discoverd instead of ironic-inspector" [Undecided,Confirmed] - Assigned to Ruchi Parmar (ruchiparmar143) 03:00:23 https://bugs.launchpad.net/diskimage-builder/+bug/1506792 03:00:24 https://bugs.launchpad.net/tripleo/+bug/1505430 03:00:24 Launchpad bug 1506792 in diskimage-builder "ironic-python-agent fails to run on fedora ramdisk" [Undecided,In progress] - Assigned to Lucas Alvares Gomes (lucasagomes) 03:00:25 https://bugs.launchpad.net/diskimage-builder/+bug/1506692 03:00:26 Launchpad bug 1505430 in tripleo "Triggering puppet runs with config_identifier is error prone and complicates templates" [Undecided,New] - Assigned to Steve Baker (steve-stevebaker) 03:00:26 https://bugs.launchpad.net/diskimage-builder/+bug/1505900 03:00:27 Launchpad bug 1506692 in diskimage-builder "Broken test in fedora-minimal/install.d/99-ramdisk" [Undecided,In progress] - Assigned to Martin André (mandre) 03:00:29 https://bugs.launchpad.net/tripleo/+bug/1504263 03:00:29 Launchpad bug 1505900 in diskimage-builder "Failed to create 'wily' release with DIB_RELEASE" [Undecided,New] 03:00:30 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 03:00:31 https://bugs.launchpad.net/diskimage-builder/+bug/1506275 03:00:34 https://bugs.launchpad.net/tripleo/+bug/1507054 03:00:34 Launchpad bug 1506275 in diskimage-builder "$DIB_OFFLINE and $DIB_DEBIAN_USE_DEBOOTSTRAP_CACHE are redundant" [Undecided,New] 03:00:35 Launchpad bug 1507054 in tripleo "Allow configuration of mysql percona sst through xtrabackup-v2" [Undecided,New] 09:00:16 Untriaged bugs so far: 09:00:17 https://bugs.launchpad.net/tripleo/+bug/1505815 09:00:17 Launchpad bug 1505815 in tripleo "Docs refer to ironic-discoverd instead of ironic-inspector" [Undecided,Confirmed] - Assigned to Ruchi Parmar (ruchiparmar143) 09:00:19 https://bugs.launchpad.net/diskimage-builder/+bug/1506792 09:00:19 Launchpad bug 1506792 in diskimage-builder "ironic-python-agent fails to run on fedora ramdisk" [Undecided,In progress] - Assigned to Lucas Alvares Gomes (lucasagomes) 09:00:21 https://bugs.launchpad.net/tripleo/+bug/1505430 09:00:21 Launchpad bug 1505430 in tripleo "Triggering puppet runs with config_identifier is error prone and complicates templates" [Undecided,New] - Assigned to Steve Baker (steve-stevebaker) 09:00:23 https://bugs.launchpad.net/diskimage-builder/+bug/1506692 09:00:23 Launchpad bug 1506692 in diskimage-builder "Broken test in fedora-minimal/install.d/99-ramdisk" [Undecided,In progress] - Assigned to Martin André (mandre) 09:00:25 https://bugs.launchpad.net/diskimage-builder/+bug/1505900 09:00:25 Launchpad bug 1505900 in diskimage-builder "Failed to create 'wily' release with DIB_RELEASE" [Undecided,New] 09:00:27 https://bugs.launchpad.net/tripleo/+bug/1504263 09:00:28 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 09:00:29 https://bugs.launchpad.net/diskimage-builder/+bug/1506275 09:00:30 Launchpad bug 1506275 in diskimage-builder "$DIB_OFFLINE and $DIB_DEBIAN_USE_DEBOOTSTRAP_CACHE are redundant" [Undecided,New] 09:00:31 https://bugs.launchpad.net/tripleo/+bug/1507054 09:00:32 Launchpad bug 1507054 in tripleo "Allow configuration of mysql percona sst through xtrabackup-v2" [Undecided,New] 15:00:17 Untriaged bugs so far: 15:00:19 https://bugs.launchpad.net/tripleo/+bug/1505815 15:00:19 Launchpad bug 1505815 in tripleo "Docs refer to ironic-discoverd instead of ironic-inspector" [Undecided,Confirmed] - Assigned to Ruchi Parmar (ruchiparmar143) 15:00:20 https://bugs.launchpad.net/diskimage-builder/+bug/1506792 15:00:20 Launchpad bug 1506792 in diskimage-builder "ironic-python-agent fails to run on fedora ramdisk" [Undecided,In progress] - Assigned to Lucas Alvares Gomes (lucasagomes) 15:00:21 https://bugs.launchpad.net/tripleo/+bug/1505430 15:00:21 Launchpad bug 1505430 in tripleo "Triggering puppet runs with config_identifier is error prone and complicates templates" [Undecided,New] - Assigned to Steve Baker (steve-stevebaker) 15:00:23 https://bugs.launchpad.net/diskimage-builder/+bug/1506692 15:00:23 Launchpad bug 1506692 in diskimage-builder "Broken test in fedora-minimal/install.d/99-ramdisk" [Undecided,In progress] - Assigned to Martin André (mandre) 15:00:25 https://bugs.launchpad.net/diskimage-builder/+bug/1505900 15:00:25 Launchpad bug 1505900 in diskimage-builder "Failed to create 'wily' release with DIB_RELEASE" [Undecided,New] 15:00:27 https://bugs.launchpad.net/tripleo/+bug/1504263 15:00:28 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 15:00:29 https://bugs.launchpad.net/diskimage-builder/+bug/1506275 15:00:30 Launchpad bug 1506275 in diskimage-builder "$DIB_OFFLINE and $DIB_DEBIAN_USE_DEBOOTSTRAP_CACHE are redundant" [Undecided,New] 15:00:31 https://bugs.launchpad.net/tripleo/+bug/1507054 15:00:31 Launchpad bug 1507054 in tripleo "Allow configuration of mysql percona sst through xtrabackup-v2" [Undecided,New] 21:00:16 Untriaged bugs so far: 21:00:18 https://bugs.launchpad.net/tripleo/+bug/1505815 21:00:18 Launchpad bug 1505815 in tripleo "Docs refer to ironic-discoverd instead of ironic-inspector" [Undecided,Confirmed] - Assigned to Ruchi Parmar (ruchiparmar143) 21:00:20 https://bugs.launchpad.net/diskimage-builder/+bug/1506792 21:00:20 Launchpad bug 1506792 in diskimage-builder "ironic-python-agent fails to run on fedora ramdisk" [Undecided,In progress] - Assigned to Lucas Alvares Gomes (lucasagomes) 21:00:21 https://bugs.launchpad.net/tripleo/+bug/1505430 21:00:21 Launchpad bug 1505430 in tripleo "Triggering puppet runs with config_identifier is error prone and complicates templates" [Undecided,New] - Assigned to Steve Baker (steve-stevebaker) 21:00:24 https://bugs.launchpad.net/diskimage-builder/+bug/1506692 21:00:24 Launchpad bug 1506692 in diskimage-builder "Broken test in fedora-minimal/install.d/99-ramdisk" [Undecided,In progress] - Assigned to Martin André (mandre) 21:00:25 https://bugs.launchpad.net/diskimage-builder/+bug/1505900 21:00:26 Launchpad bug 1505900 in diskimage-builder "Failed to create 'wily' release with DIB_RELEASE" [Undecided,New] 21:00:27 https://bugs.launchpad.net/tripleo/+bug/1504263 21:00:28 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 21:00:29 https://bugs.launchpad.net/diskimage-builder/+bug/1506275 21:00:29 Launchpad bug 1506275 in diskimage-builder "$DIB_OFFLINE and $DIB_DEBIAN_USE_DEBOOTSTRAP_CACHE are redundant" [Undecided,New] 21:00:31 https://bugs.launchpad.net/tripleo/+bug/1507054 21:00:34 Launchpad bug 1507054 in tripleo "Allow configuration of mysql percona sst through xtrabackup-v2" [Undecided,New] 21:25:14 Dan Prince proposed openstack/tripleo-heat-templates: WIP: Support for composable roles https://review.openstack.org/236243 02:28:17 greghaynes proposed openstack/diskimage-builder: Add --force to grub-install https://review.openstack.org/236661 03:00:16 Untriaged bugs so far: 03:00:18 https://bugs.launchpad.net/tripleo/+bug/1505815 03:00:18 Launchpad bug 1505815 in tripleo "Docs refer to ironic-discoverd instead of ironic-inspector" [Undecided,Confirmed] - Assigned to Ruchi Parmar (ruchiparmar143) 03:00:19 https://bugs.launchpad.net/diskimage-builder/+bug/1506792 03:00:19 Launchpad bug 1506792 in diskimage-builder "ironic-python-agent fails to run on fedora ramdisk" [Undecided,In progress] - Assigned to Lucas Alvares Gomes (lucasagomes) 03:00:21 https://bugs.launchpad.net/tripleo/+bug/1505430 03:00:21 Launchpad bug 1505430 in tripleo "Triggering puppet runs with config_identifier is error prone and complicates templates" [Undecided,New] - Assigned to Steve Baker (steve-stevebaker) 03:00:23 https://bugs.launchpad.net/diskimage-builder/+bug/1506692 03:00:24 Launchpad bug 1506692 in diskimage-builder "Broken test in fedora-minimal/install.d/99-ramdisk" [Undecided,In progress] - Assigned to Martin André (mandre) 03:00:25 https://bugs.launchpad.net/diskimage-builder/+bug/1505900 03:00:26 Launchpad bug 1505900 in diskimage-builder "Failed to create 'wily' release with DIB_RELEASE" [Undecided,New] 03:00:27 https://bugs.launchpad.net/tripleo/+bug/1504263 03:00:27 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 03:00:29 https://bugs.launchpad.net/diskimage-builder/+bug/1506275 03:00:29 Launchpad bug 1506275 in diskimage-builder "$DIB_OFFLINE and $DIB_DEBIAN_USE_DEBOOTSTRAP_CACHE are redundant" [Undecided,New] 03:00:31 https://bugs.launchpad.net/tripleo/+bug/1507054 03:00:31 Launchpad bug 1507054 in tripleo "Allow configuration of mysql percona sst through xtrabackup-v2" [Undecided,New] 09:00:18 Untriaged bugs so far: 09:00:20 https://bugs.launchpad.net/tripleo/+bug/1505815 09:00:20 Launchpad bug 1505815 in tripleo "Docs refer to ironic-discoverd instead of ironic-inspector" [Undecided,Confirmed] - Assigned to Ruchi Parmar (ruchiparmar143) 09:00:20 https://bugs.launchpad.net/diskimage-builder/+bug/1506792 09:00:21 Launchpad bug 1506792 in diskimage-builder "ironic-python-agent fails to run on fedora ramdisk" [Undecided,In progress] - Assigned to Lucas Alvares Gomes (lucasagomes) 09:00:22 https://bugs.launchpad.net/tripleo/+bug/1505430 09:00:22 Launchpad bug 1505430 in tripleo "Triggering puppet runs with config_identifier is error prone and complicates templates" [Undecided,New] - Assigned to Steve Baker (steve-stevebaker) 09:00:24 https://bugs.launchpad.net/diskimage-builder/+bug/1506692 09:00:25 Launchpad bug 1506692 in diskimage-builder "Broken test in fedora-minimal/install.d/99-ramdisk" [Undecided,In progress] - Assigned to Martin André (mandre) 09:00:26 https://bugs.launchpad.net/diskimage-builder/+bug/1505900 09:00:27 Launchpad bug 1505900 in diskimage-builder "Failed to create 'wily' release with DIB_RELEASE" [Undecided,New] 09:00:28 https://bugs.launchpad.net/tripleo/+bug/1504263 09:00:29 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 09:00:30 https://bugs.launchpad.net/diskimage-builder/+bug/1506275 09:00:30 Launchpad bug 1506275 in diskimage-builder "$DIB_OFFLINE and $DIB_DEBIAN_USE_DEBOOTSTRAP_CACHE are redundant" [Undecided,New] 09:00:33 https://bugs.launchpad.net/tripleo/+bug/1507054 09:00:34 Launchpad bug 1507054 in tripleo "Allow configuration of mysql percona sst through xtrabackup-v2" [Undecided,New] 12:38:45 Dan Prince proposed openstack/tripleo-heat-templates: WIP: Support for composable roles https://review.openstack.org/236243 15:00:17 Untriaged bugs so far: 15:00:19 https://bugs.launchpad.net/tripleo/+bug/1505815 15:00:19 Launchpad bug 1505815 in tripleo "Docs refer to ironic-discoverd instead of ironic-inspector" [Undecided,Confirmed] - Assigned to Ruchi Parmar (ruchiparmar143) 15:00:20 https://bugs.launchpad.net/diskimage-builder/+bug/1506792 15:00:21 Launchpad bug 1506792 in diskimage-builder "ironic-python-agent fails to run on fedora ramdisk" [Undecided,In progress] - Assigned to Lucas Alvares Gomes (lucasagomes) 15:00:22 https://bugs.launchpad.net/tripleo/+bug/1505430 15:00:22 Launchpad bug 1505430 in tripleo "Triggering puppet runs with config_identifier is error prone and complicates templates" [Undecided,New] - Assigned to Steve Baker (steve-stevebaker) 15:00:24 https://bugs.launchpad.net/diskimage-builder/+bug/1506692 15:00:24 Launchpad bug 1506692 in diskimage-builder "Broken test in fedora-minimal/install.d/99-ramdisk" [Undecided,In progress] - Assigned to Martin André (mandre) 15:00:26 https://bugs.launchpad.net/diskimage-builder/+bug/1505900 15:00:27 Launchpad bug 1505900 in diskimage-builder "Failed to create 'wily' release with DIB_RELEASE" [Undecided,New] 15:00:28 https://bugs.launchpad.net/tripleo/+bug/1504263 15:00:28 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 15:00:30 https://bugs.launchpad.net/diskimage-builder/+bug/1506275 15:00:31 Launchpad bug 1506275 in diskimage-builder "$DIB_OFFLINE and $DIB_DEBIAN_USE_DEBOOTSTRAP_CACHE are redundant" [Undecided,New] 15:00:32 https://bugs.launchpad.net/tripleo/+bug/1507054 15:00:32 Launchpad bug 1507054 in tripleo "Allow configuration of mysql percona sst through xtrabackup-v2" [Undecided,New] 17:26:08 Dan Prince proposed openstack/tripleo-puppet-elements: Update puppet-ceph git location https://review.openstack.org/236721 17:29:53 Dan Prince proposed openstack/tripleo-heat-templates: WIP: Support for composable roles https://review.openstack.org/236243 20:09:43 Dan Prince proposed openstack-infra/tripleo-ci: Fix so that we build tripleo-puppet-elements pkgs https://review.openstack.org/236740 20:10:25 Dan Prince proposed openstack/tripleo-puppet-elements: Update puppet-ceph git location https://review.openstack.org/236721 20:10:48 Dan Prince proposed openstack/tripleo-heat-templates: WIP: Support for composable roles https://review.openstack.org/236243 21:00:17 Untriaged bugs so far: 21:00:18 https://bugs.launchpad.net/tripleo/+bug/1505815 21:00:18 Launchpad bug 1505815 in tripleo "Docs refer to ironic-discoverd instead of ironic-inspector" [Undecided,Confirmed] - Assigned to Ruchi Parmar (ruchiparmar143) 21:00:20 https://bugs.launchpad.net/diskimage-builder/+bug/1506792 21:00:20 Launchpad bug 1506792 in diskimage-builder "ironic-python-agent fails to run on fedora ramdisk" [Undecided,In progress] - Assigned to Lucas Alvares Gomes (lucasagomes) 21:00:22 https://bugs.launchpad.net/tripleo/+bug/1505430 21:00:22 Launchpad bug 1505430 in tripleo "Triggering puppet runs with config_identifier is error prone and complicates templates" [Undecided,New] - Assigned to Steve Baker (steve-stevebaker) 21:00:24 https://bugs.launchpad.net/diskimage-builder/+bug/1506692 21:00:24 Launchpad bug 1506692 in diskimage-builder "Broken test in fedora-minimal/install.d/99-ramdisk" [Undecided,In progress] - Assigned to Martin André (mandre) 21:00:26 https://bugs.launchpad.net/diskimage-builder/+bug/1505900 21:00:26 Launchpad bug 1505900 in diskimage-builder "Failed to create 'wily' release with DIB_RELEASE" [Undecided,New] 21:00:28 https://bugs.launchpad.net/tripleo/+bug/1504263 21:00:28 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 21:00:30 https://bugs.launchpad.net/diskimage-builder/+bug/1506275 21:00:30 Launchpad bug 1506275 in diskimage-builder "$DIB_OFFLINE and $DIB_DEBIAN_USE_DEBOOTSTRAP_CACHE are redundant" [Undecided,New] 21:00:32 https://bugs.launchpad.net/tripleo/+bug/1507054 21:00:32 Launchpad bug 1507054 in tripleo "Allow configuration of mysql percona sst through xtrabackup-v2" [Undecided,New] 23:22:33 Merged openstack-infra/tripleo-ci: Fix so that we build tripleo-puppet-elements pkgs https://review.openstack.org/236740 23:24:29 Dan Prince proposed openstack/tripleo-heat-templates: WIP: Support for composable roles https://review.openstack.org/236243 23:25:28 Merged openstack/tripleo-puppet-elements: Update puppet-ceph git location https://review.openstack.org/236721 00:49:02 Dan Prince proposed openstack-infra/tripleo-ci: Force build of tripleo-puppet-elements https://review.openstack.org/236760 01:22:11 Dan Prince proposed openstack/tripleo-heat-templates: WIP: Support for composable roles https://review.openstack.org/236243 03:00:17 Untriaged bugs so far: 03:00:19 https://bugs.launchpad.net/tripleo/+bug/1505815 03:00:19 Launchpad bug 1505815 in tripleo "Docs refer to ironic-discoverd instead of ironic-inspector" [Undecided,Confirmed] - Assigned to Ruchi Parmar (ruchiparmar143) 03:00:20 https://bugs.launchpad.net/diskimage-builder/+bug/1506792 03:00:20 Launchpad bug 1506792 in diskimage-builder "ironic-python-agent fails to run on fedora ramdisk" [Undecided,In progress] - Assigned to Lucas Alvares Gomes (lucasagomes) 03:00:22 https://bugs.launchpad.net/tripleo/+bug/1505430 03:00:22 Launchpad bug 1505430 in tripleo "Triggering puppet runs with config_identifier is error prone and complicates templates" [Undecided,New] - Assigned to Steve Baker (steve-stevebaker) 03:00:24 https://bugs.launchpad.net/diskimage-builder/+bug/1506692 03:00:26 https://bugs.launchpad.net/diskimage-builder/+bug/1505900 03:00:26 Launchpad bug 1506692 in diskimage-builder "Broken test in fedora-minimal/install.d/99-ramdisk" [Undecided,In progress] - Assigned to Martin André (mandre) 03:00:27 Launchpad bug 1505900 in diskimage-builder "Failed to create 'wily' release with DIB_RELEASE" [Undecided,New] 03:00:29 https://bugs.launchpad.net/tripleo/+bug/1504263 03:00:29 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 03:00:30 https://bugs.launchpad.net/diskimage-builder/+bug/1506275 03:00:31 Launchpad bug 1506275 in diskimage-builder "$DIB_OFFLINE and $DIB_DEBIAN_USE_DEBOOTSTRAP_CACHE are redundant" [Undecided,New] 03:00:32 https://bugs.launchpad.net/tripleo/+bug/1507054 03:00:32 Launchpad bug 1507054 in tripleo "Allow configuration of mysql percona sst through xtrabackup-v2" [Undecided,New] 05:02:30 bnemec: ping 07:41:41 shardy: Hey, quick question, is the ControlFixedIPs parameter in the overcloud-without-mergepy.yaml template empty if you just do "openstack overcloud create"? Or is that value populated somewhere in the client? 07:44:28 jaosorior: I think it's left empty - you can check by doing heat stack-show overcloud | grep ControlFixed 07:48:17 shardy: Ah, I see. Thanks 07:58:28 shardy: By the way, is there a reason why there is a GlanceProtocol parameter? A similar parameter doesn't seem to exist for any other service 08:02:47 jaosorior, I think that was going to be added with the other changes from spredzy 08:03:14 all of those should probably go into a resource-specific template so we can alter them basing on SSL status 08:03:26 gfidente: Actually I'm currently working on that 08:03:47 I'm adding a map similar to the similar to the ServiceNetMap 08:03:49 It looks like that param is pretty old, but there's a bunch of other intra-service URLs hard-coded to http 08:03:50 unless we pass Protocol as parameter for each an every service from the SSL yaml 08:04:05 I wonder why e.g nova doesn't get the neutron endpoints from keystone? 08:04:22 instead of via the nova::network::neutron::neutron_url config 08:04:40 gfidente: I'm actually thinking of doing that. The ssl environment will define a parameter_defaults with a properly made map 08:04:51 currently the map aims to enable ssl only for the public endpoints 08:04:59 shardy: Good question :/ 08:05:03 jaosorior, yeah I see what you mean, also good I think 08:05:22 jaosorior: it may be worth investigating if the services do default to keystone, and just allow for config file overrides 08:05:24 gfidente: Of course, or do you think doing it another way would be better? 08:05:24 shardy, I don't know which services require the url and which do keystone lookup 08:05:53 gfidente: me neither (other than heat), but it's probably worth investgating before we update all those config values 08:06:46 jaosorior, I was thinking about a different resource definition for the endpoints but it's probably overkilling, shardy might vote on this 08:07:29 jaosorior: If we end up with a service-catalog style mapping in the templates then either TripleO, or a bunch of services, are doing something really wrong 08:08:02 e.g heat talks to all services, but we only need to know about the keystone URL 08:08:04 well, for now replacing those hardcoded 'http://' to using a map would actually end up changing the endpoints from keystone, as far as I can see from spredzy's changes (which I'm doing the changes on top of) 08:08:44 jaosorior: hmm, yeah I forgot puppet was creating the endpoints now, I guess we need the mapping for that 08:09:02 my point was more that mapping shouldn't need to be wired in to all the services, it should be specific to keystone 08:09:23 or, potentially that should be possible, modulo some investigation ;) 08:09:39 jaosorior, don't we need to change only the protocol? 08:10:25 gfidente: That's what I'm changing, yes 08:10:41 jaosorior, so the map is protocol per service? 08:12:21 gfidente, shardy: so the definition of the parameter currently looks like this http://paste.openstack.org/show/476641/ 08:12:49 and it would be overriden if we would enable TLS (via an environment file) 08:13:01 yeah I see it, seems good to me 08:13:51 +1 lgtm, then we can potentially clean up the various hard-coded url configs later 08:14:22 gfidente: I'm also doing something similar to what you have in this CR https://review.openstack.org/#/c/222497 08:14:46 So what I'm trying to do is to enable the deployer to push a different certificate/key per controller 08:15:20 oh you mean passing the certs in an array so that each collects its own from the indes? 08:15:21 *index 08:15:22 So basically I copied the NodeIndex parameter 08:15:26 ok 08:15:29 gfidente: Exactly 08:15:44 that idea was really useful :D 08:15:56 And the solution is much cleaner than what I proposed initially 08:16:03 I don't know if shardy likes it? 08:16:41 do we have that index or something similar if/when we move to autoscalingroup? 08:16:44 gfidente: I can upload a draft in some minutes (still need to test it) so shardy and you can take a look at what I'm talking about 08:17:09 Still need to test the thing though, that's why I say a draft 08:17:20 you're already super-fast I think 08:17:21 no worries 08:26:03 gfidente: Yeah I'm fine with node %index%, but yeah it will prevent us moving to ASG so we'll need to find a way around that if/when that happens 08:34:17 morning all, anybody else around to look at this commit to unblock the ci jobs https://review.openstack.org/#/c/236760/1 08:35:03 derekh: done! 08:35:04 Merged openstack-infra/tripleo-ci: Force build of tripleo-puppet-elements https://review.openstack.org/236760 08:35:37 shardy: thanks, 08:39:49 Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Enable TLS deployment for the Overcloud https://review.openstack.org/231930 08:40:12 gfidente, shardy: ^^ There's a draft of the idea. Still need to test it though. 08:51:49 anyone seen neutron failing to start with 08:51:50 říj 19 08:39:16 overcloud-controller-0 neutron-server[5175]: 2015-10-19 08:39:16.947 5175 ERROR neutron.service ValueError: Empty module name 08:51:52 ? 08:52:29 seems like this somehow didn't make it onto my system or i don't know what happened https://github.com/openstack/neutron/blob/master/setup.cfg#L106 08:52:40 didn't change anything since my last setup 08:52:52 (afaik) 08:53:13 jistr: not seen that one... but last time i ran upstream env was sometime midweek last 08:53:50 jistr: is this clean upstream tht? 08:54:03 marios: yea 08:55:31 jistr: k, well fwiw, will revisit my upstream env this afternoon (memory issues on friday during ./tripleo-common/scripts/tripleo.sh --overcloud-images, did something cause the undercloud vm mem requirements to ^^^ if you know btw?) 08:56:11 EmilienM: ping, hey, left some more notes on the review. i just had lots of fun rebasing so i can test against downstream (trying to get the sequencing right for the apache inclusions... also couple more nits/thoughts grateful when you have a minute) 08:56:27 marios: i don't know, but you can work around that with https://github.com/jistr/inlunch/blob/cce3a07a10a8c025319bff7a4fd2f48922bb7f0a/answers.yml.example#L126 08:57:07 (i've had UC memory issues since long time ago) 08:57:23 jistr: thanks ! :) we should frame that url somewhere (I probably looked there but didn't recognise the relevance of DIB_NO_TMPFS) 09:00:17 Untriaged bugs so far: 09:00:19 https://bugs.launchpad.net/tripleo/+bug/1505815 09:00:19 Launchpad bug 1505815 in tripleo "Docs refer to ironic-discoverd instead of ironic-inspector" [Undecided,Confirmed] - Assigned to Ruchi Parmar (ruchiparmar143) 09:00:20 https://bugs.launchpad.net/diskimage-builder/+bug/1506792 09:00:20 Launchpad bug 1506792 in diskimage-builder "ironic-python-agent fails to run on fedora ramdisk" [Undecided,In progress] - Assigned to Lucas Alvares Gomes (lucasagomes) 09:00:23 https://bugs.launchpad.net/tripleo/+bug/1505430 09:00:23 Launchpad bug 1505430 in tripleo "Triggering puppet runs with config_identifier is error prone and complicates templates" [Undecided,New] - Assigned to Steve Baker (steve-stevebaker) 09:00:24 https://bugs.launchpad.net/diskimage-builder/+bug/1506692 09:00:24 Launchpad bug 1506692 in diskimage-builder "Broken test in fedora-minimal/install.d/99-ramdisk" [Undecided,In progress] - Assigned to Martin André (mandre) 09:00:25 https://bugs.launchpad.net/diskimage-builder/+bug/1505900 09:00:25 Launchpad bug 1505900 in diskimage-builder "Failed to create 'wily' release with DIB_RELEASE" [Undecided,New] 09:00:28 https://bugs.launchpad.net/tripleo/+bug/1504263 09:00:29 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 09:00:29 https://bugs.launchpad.net/diskimage-builder/+bug/1506275 09:00:30 Launchpad bug 1506275 in diskimage-builder "$DIB_OFFLINE and $DIB_DEBIAN_USE_DEBOOTSTRAP_CACHE are redundant" [Undecided,New] 09:00:32 https://bugs.launchpad.net/tripleo/+bug/1507054 09:00:32 Launchpad bug 1507054 in tripleo "Allow configuration of mysql percona sst through xtrabackup-v2" [Undecided,New] 09:01:43 aha! https://bugzilla.redhat.com/show_bug.cgi?id=1271389 09:01:43 bugzilla.redhat.com bug 1271389 in rdo-manager "neutron-server fails to start when using short name (ml2) for core_plugin" [Urgent,New] - Assigned to calfonso 09:04:12 aha! https://github.com/redhat-openstack/tripleo-docs/commit/26d63c1abf991b3a7938ab93af73ccb9fd7e4b03 09:04:15 trown|outttypeww++ 09:04:17 will try that 09:35:06 has anybody seen this? http://logs.openstack.org/30/231930/8/check-tripleo/gate-tripleo-ci-ironic-overcloud-f21puppet-nonha/4bff75e/console.html#_2015-10-19_09_08_49_299 10:11:52 jistr: I put that patch up for the upstream docs too https://review.openstack.org/#/c/235575/ 10:15:28 trown|outttypeww why was the comment about the libvirt-type flag removed in that CR? 10:18:48 jaosorior: not intentional... I see other issues where that patch is based on top of the RDO docs too, I will WIP and fix later, thanks! 10:37:22 jistr, shardy, gfidente: Have you seen some errors of os-refresh-config in the upstream ci? 10:39:35 jaosorior, ping your bug earlier heat/keystone 10:40:06 should be fixed in few minutes - we were missing a review upstream puppet-heat that wasn't merged https://review.openstack.org/#/c/232366 causing this I think 10:40:56 spredzy: Ah! Thanks :D 10:41:13 spredzy: By the way, I uploaded a new patch for the TLS stuff on top of your work. 10:41:43 jaosorior, cool. I ll take a loot at it 10:42:28 spredzy: I still don't get to properly test it since my current dev environment is not playing nice... But if you can check that it makes sense I would appreciate it. 10:43:57 sure thing. Will take a look after lunch 11:17:03 FYI - the tripleo-ci jobs are all failing because of a patch in puppet-keystone that broke backwards compat https://review.openstack.org/#/c/226624/ 11:17:39 They should start passing again, when this patch merges into puppet-heat https://review.openstack.org/#/c/232366/ 11:42:19 shardy, around? 12:02:07 marios: ok will look, thanks 12:02:45 EmilienM: np, on a call, will update comments later 12:19:53 rhallisey: Hey, yup, just been grabbing lunch 12:23:23 shardy, I actually think I figured it out. For the local docker registry we need instack's ip 12:23:33 I think I need to use puppet here 12:24:07 since I need that ip to be used as the namespace it needs to be passed around the heat templtes 12:25:13 Has anybody seen an error such as this: parse error: Invalid string: control characters from U+0000 through U+001F must be escaped at line 32, column 30 12:25:19 when running instack-virt-setup? 12:25:29 jistr: ^ 12:25:35 rhallisey: you can derive it from the heat metadata IP which is passed in to all instances on create I think 12:26:02 rhallisey: sec, dprince posted a patch which does that for os-net-config validation 12:26:12 shardy, wait heat meta data will have instack's ip? 12:26:15 pino|work: thx 12:26:21 jaosorior: https://github.com/jistr/inlunch/blob/cce3a07a10a8c025319bff7a4fd2f48922bb7f0a/answers.yml.example#L104 12:26:31 rhallisey: Yes, because we poll heat, which is running on the instack undercloud VM 12:26:43 jistr|mtg: Thanks man! 12:26:57 so assuming you're happy to consider the local registry and the undercloud heat co-located for now, that should work 12:27:21 I think that's fine 12:27:37 jistr|mtg: aw...that export didn't do the trick :/ 12:27:44 rhallisey: https://review.openstack.org/#/c/197739/7/elements/os-net-config/os-refresh-config/configure.d/20-os-net-config 12:28:10 shardy, ok cool 12:28:19 rhallisey: Note, that only works for the CFN transport, which is the current default, but I'd really prefer we worked out a generic way to get the IP independent of transport 12:28:37 rhallisey: I'll give some more thought to how we do that, but that should get things working OK for now 12:29:22 rhallisey: actually, correction, that does handle multiple transports, a previous revision didn't 12:29:46 so you can just copy the get_metadata_ip function 12:29:49 shardy, what do you mean by multiple transports 12:30:06 rhallisey: there are a few ways for the agents on each node to poll heat 12:30:26 rhallisey: each requires a different os-apply-config query, which is why there's a for loop there 12:32:05 shardy, ok I see. So can I just copy that bash part into a heat script within the template? 12:32:11 is that what you meant? 12:32:17 shardy, I tried the double mapping of the resource type but didn't work, got the resource re-created 12:32:18 rhallisey: yeah, that should work 12:32:19 jprovazn, ^^ 12:32:27 shardy, ok great! 12:32:41 thank you! 12:32:45 rhallisey: np! 12:33:03 gfidente: Hmm, Ok, will have a think about possible alternative approaches then :( 12:35:43 gfidente: probably lame question, but in this double=,apping workaround, shouldn't it map back to 7.0 type? so in this case to OS::TripleO::Controller::Ports::ExternalPort? 12:36:29 Is there a known issue with building images? I seem to be missing ironic-python-agent.initramfs 12:38:14 Brad P. Crochet proposed openstack/python-tripleoclient: Do not throw an error on validation when tunneling is disabled https://review.openstack.org/228446 12:38:33 d0ugal, What did you follow to build them? 12:39:30 therve: Yeah, I was missing that Yum repo thing. Thanks 12:39:37 No problem :) 12:39:50 The error is well hidden in that sea of logs 12:42:08 jprovazn, I deployed with some recent template, I think as long as you make an arbitrary resource map to what it was we're recreating the same scenario 12:42:43 where the resource is different but maps back to what it was at the time of the deployment 12:45:33 gfidente: ack me tries with: OS::TripleO::CustomVIP: OS::TripleO::Network::Ports::ExternalVipPort 12:46:46 jprovazn, yeah that's what I did 12:55:25 therve: d0ugal, DIB_YUM_REPO_CONF will not work if there are any packages with weird NVR, for that you need to use the delorean-repo element since it installs yum-plugin-priorities 12:56:01 https://github.com/redhat-openstack/tripleo-docs/commit/26d63c1abf991b3a7938ab93af73ccb9fd7e4b03 12:56:26 I have a similar patch for openstack/tripleo-docs, but I screwed it up, fixing now 12:57:26 Ah, makes sense 13:02:20 jprovazn, news? 13:02:24 John Trowbridge proposed openstack/tripleo-docs: Modify instructions for using delorean to build images https://review.openstack.org/235575 13:02:35 shardy, maybe I'm not familiar enough here, but is there a way to get the instack IP into an environment file? 13:02:47 d0ugal: therve ^ that is the docs patch 13:03:20 gfidente: it failed, but because of an unrealted reason :/ 13:03:28 *unrelated 13:04:49 rhallisey: No, unless you hard-code it 13:05:16 rhallisey: you could add e.g a UndercloudIp parameter to the template, then pass the parameter value in an environment file 13:05:27 but I assume you want something more automatic than that 13:06:11 jaosorior: do you have some more output regarding the unicode parsing failure? 13:06:27 jaosorior, jistr I think that is the known issue with jq to new 13:06:31 jaosorior: i'm thinking maybe in your case it's something else than a time/date which caused the problem 13:06:34 sudo yum downgrade jq-1.3-2.el7 13:06:41 sudo yum install -y yum-plugin-versionlock 13:06:41 sudo yum versionlock add jq 13:06:43 rhallisey: I've tried automagic things to help obtain the instack IP before. I think a parameter in t-h-t for whatever you need is probably best 13:06:57 shardy, ok I think this could work.. What I think needs to change is we need to specify the namescape in the heat template instead of the env file 13:07:14 rhallisey, shardy: rather than UndercloudIp could we be more specific? Like what is this parameter for?? 13:07:23 dprince, local docker registry 13:07:43 since the namespace is not the same everytime 13:07:44 jistr: This is the end of it http://pastebin.com/jjqi6uMU 13:07:44 rhallisey: My preference would be DockerRegistryIp then 13:07:57 rhallisey: just add a parameter for that and be done w/ it :) 13:07:58 dprince: yup, I was just using that name as an example 13:08:03 jaosorior, yeah the jq thing 13:08:08 dprince, so are you saying do that in the env file? 13:08:12 jaosorior, sudo yum downgrade jq-1.3-2.el7 13:08:16 gfidente: alright, will try that out 13:08:26 jaosorior: try 'export LC_ALL=C' before running the undercloud install 13:08:31 anything <1.5 should do AFAIK 13:08:32 rhallisey: you could provide an example in a docker environment file 13:08:49 rhallisey: the actual parameter definition probably goes into one of the heat templates however 13:08:52 jistr, oh indeed does that work with 1.5 ? 13:09:25 shardy: now quite working yet but I'm really optimistic with this approach https://review.openstack.org/#/c/236243/ 13:09:27 shardy, so you were saying I can specify in a template and pass back into the enf file? 13:09:36 gfidente: i don't know, but overwriting LC_TIME fixed issues for me... jaosorior probably has a different locale so the problematic value could be something else then time/date 13:09:48 I'm confused I guess 13:09:52 so i don't know if it will fix his problem but i think it's worth trying 13:09:57 shardy: question *is*, when can I use my new map_merge function (when it lands) 13:10:04 not sure what prints unicode chars in his case 13:10:13 in my case it was timestamps in log messages 13:11:02 I first went for the jq version lock 13:11:45 jistr: gfidente jaosorior (sorry, just reading back). if this is what you are referring to: jq issue @ https://bugzilla.redhat.com/show_bug.cgi?id=1265403 https://review.openstack.org/#/c/228034 13:11:45 bugzilla.redhat.com bug 1265403 in instack-undercloud "instack-undercloud fails to setup seed vm, parse error while creating ssh key" [High,Closed: notabug] - Assigned to jslagle 13:12:34 marios++ 13:12:49 aah 13:12:51 ok 13:13:04 jaosorior: so worst case you can just apply that to /usr/libexec/openstack-tripleo/devtest_testenv.sh 13:13:23 https://review.openstack.org/#/c/228034/4/scripts/devtest_testenv.sh 13:14:08 dprince: realistically, we can't use new-for-mitaka features until we cut the release/stable branches 13:14:13 so it was in the description of a ssh key probably, interesting... 13:14:41 dprince: I looked at the patch tho and it looks good 13:14:46 ryansb: looks like https://review.openstack.org/#/c/225246/6 needs some love (and the patch it depends on) 13:15:03 shardy: yeah, and I can't do this without that new function. So I'd perhaps like to discuss the merit of carrying that function in Tripleo (assuming it lands) until Mitaka is released 13:15:54 dprince: I think tripleo can just use trunk heat, that part is fine 13:15:59 shardy: waiting to do this till Mitaka Heat is going to kill us, we are already need this to assist w/ composability and 3rd party integration 13:16:05 but we need to have the stable branch first to support e.g rdo-manager 13:16:13 which is currently building from tripleo trunk 13:16:22 shardy: fine by me 13:17:49 shardy: anyways, I'm working how how this affects pacemaker today and will push another set of initial patches soon. 13:17:53 dprince: Yeah, cool, IMO the problem has never been how can we consume latest stuff in TripleO - that's already solved, it's just how do we not break those on stable branches at the same time 13:18:09 dprince: great, look forward to checking it out :) 13:18:26 well, the instack-virt-setup seems to have succeeded with the jq version locking 13:18:26 shardy: fair enough, there has been pushback in the pasted about uses non-released Heat functions I think 13:18:55 dprince: you'll have to bump heat_template_format version to use it anyway, which implies trunk heat 13:19:01 * dprince should have written some of these Heat functions months ago 13:19:18 dprince: the stable branch solves that, because that is what supports released openstack 13:19:31 * shardy has been going on about this for weeks/months for exactly this reason 13:19:57 dprince: if we actually have a stable branch, nobody can argue that trunk can't consume trunk from other projects 13:20:15 shardy: well, they can. but they have less of an argument 13:20:18 Marek Aufart proposed openstack/python-tripleoclient: Disable command run under root user https://review.openstack.org/221720 13:21:10 dprince: well, I the whole reason I'm trying to push on this is so we can define this 13:21:29 if we document stable releases are supported via the stable branch, then those arguments are invalid 13:21:51 honestly I'm kinda baffled by how much friction the proposal has encountered 13:22:08 it's something we desparately need, and is well aligned with what nearly all other projects do 13:22:10 shardy: I think it was just wording 13:23:09 shardy: It will happen. I'd say it is safe to set into motion the non-contentious work items today.... 13:23:30 dprince: Ok, cool, I'll look in to doing that 13:23:42 dprince: a first step could be cutting the t-h-t branch so you can use map_merge ;) 13:23:56 Julia Kreger proposed openstack/diskimage-builder: WIP: iso element compatability for ubuntu EFI https://review.openstack.org/236973 13:23:58 trown: Thanks! sorry, got distracted :) 13:24:09 shardy: FWIW, this is why I'm a slacker on specs... cause they slow me down sometimes ;) 13:24:12 * dprince is so bad 13:24:33 super useful to read them once they are agreed on finally though 13:24:41 Merged openstack/python-tripleoclient: Improve Coverage for tripleoclient/utils.py https://review.openstack.org/228609 13:26:51 thrash: indeed, onit 13:26:57 ryansb: thx 13:28:56 Giulio Fidente proposed openstack/tripleo-docs: Update instructions to deploy with Ceph https://review.openstack.org/236140 13:29:16 jistr, marios, gfidente: well... now I got this error in the undercloud install http://pastebin.com/yb04EdSD any idea on what that might be? 13:29:34 dprince: Yeah, FWIW the Heat spec process isn't normally such hard work as it is (IME) for TripleO :\ 13:29:38 jaosorior: yeah 13:29:44 that's a fresh one 13:30:05 jaosorior: export DIB_REPOLOCATION_puppet_ceph=https://git.openstack.org/openstack/puppet-ceph.git 13:30:17 sigh sounds like we need to fix that 13:30:20 jaosorior: and there's another one 13:30:22 export DIB_REPOLOCATION_puppet_heat=https://review.openstack.org/openstack/puppet-heat 13:30:22 export DIB_REPOREF_puppet_heat=refs/changes/66/232366/4 13:30:53 gfidente: it's already fixed i thought, just a delorean package wasn't (isn't) available? 13:30:55 shardy: i'm a fan of having less details in our specs, and focusing more on "what" and the end result vs. the "how" 13:30:59 * jistr checks 13:31:08 jistr, ack I see in the source it points to /openstack/ 13:31:09 leave the "how" for the reviews on the patches 13:31:12 gfidente: https://github.com/openstack/tripleo-puppet-elements/commit/bd061dbe34349e052d6affc2d52687e40f1ee0ef 13:31:15 yeh 13:31:32 Florian Fuchs proposed openstack/tripleo-common: [WIP]Adds deployment configuration methods https://review.openstack.org/230315 13:31:56 slagle: Yeah, I guess I see it as a place to capture the intent of some work, and discuss any interface changes, not really a place to bike-shed every concievable aspect of the "how" 13:32:11 the heat one is also fixed, just didn't go through gate i think, i rechecked it earlier today 13:32:18 s/rechecked/reverified/ 13:32:23 slagle: I think the branch one has been particularly bad due to history and the scope of the change tho, which I guess is to be expected 13:36:44 jistr: Giving it another try with the exports you mentioned 13:39:01 Florian Fuchs proposed openstack/tripleo-common: [WIP]Adds deployment configuration methods https://review.openstack.org/230315 13:39:04 jaosorior: i see the puppet-heat change went in, so the puppet-heat exports shouldn't be needed, but they won't hurt either 13:40:32 thrash: sigh, yet another rewrite to do... would be nice if you could tell me *in advance* of all of this, instead of let me doing ping-wait-pong with patches 13:43:44 Emilien Macchi proposed openstack/instack-undercloud: neutron: enable nova-event-callback by default https://review.openstack.org/234325 13:43:45 pino|work: it's a recent development. 13:44:03 Emilien Macchi proposed openstack/instack-undercloud: neutron: enable nova-event-callback by default https://review.openstack.org/234325 13:44:09 slagle: ^ 13:44:14 I think it will work 13:44:42 pino|work: it shouldn't be too bad really... The other code simplifies things dramatically. 13:44:51 spredzy: ^ 13:45:04 EmilienM: will have a look 13:45:06 spredzy: I don't think I need keystone resources access anymore to enable nova events 13:45:12 pino|work: here is the patch that I've proposed... https://review.openstack.org/#/c/235569/ 13:45:19 spredzy: we can now use tenant name, so no need to lookup the tenant id :) 13:45:31 spredzy: that means I don't need to manage puppet resources in the catalog 13:48:34 Emilien Macchi proposed openstack/tripleo-heat-templates: neutron: enable nova-event-callback by default https://review.openstack.org/225962 13:48:39 slagle: and for overcloud ^ 13:49:13 slagle: fyi, the 2 patches I just sent are just config files changes by puppet (not like before, where we switch keystone resources management) 13:51:12 thrash: ah, yet to be approved... sigh 13:54:11 Giulio Fidente proposed openstack/tripleo-heat-templates: Ensure memcached resource uses interleave=true https://review.openstack.org/236990 13:55:08 thrash: given that you did that work: why not generalize that builder you did to not rely exclusively on elements? 13:55:48 thrash: this way, we can migrate to better solutions than dib altogether, not even using elements 13:56:05 thrash: shouldn't be hard, once you did the first draft of that, right? 13:58:13 pino|work: the way I see it, it doesn't really even rely on elements. 13:58:39 + def build_image(self, image_path, image_type, node_dist, arch, elements, 13:58:40 + options, packages): 13:58:42 yes, it does 13:58:51 it can be empty. 13:59:05 in favour of what? 13:59:16 how would you suggest that be passed in then? 14:00:05 or better yet, how would you suggest to generalize it? 14:00:16 operations, not elements 14:00:25 but i understand dib does not allow that workflow... 14:01:04 that seems like semantics to me... 14:01:18 does virt-dib do something different? 14:01:29 virt-dib runs the elements 14:01:50 what if we switch away from the elements-based workflow entirely? 14:02:17 is that even being talked about anywhere? 14:02:24 gfidente, marios: feel free to take over https://review.openstack.org/#/c/225983/ 14:02:36 marios: let me know when you got time to talk about keystone/wsgi 14:02:48 thrash: right now? :) 14:02:55 pino|work: yes, right now. :) 14:03:07 Oh. I see what you did there. :P 14:04:03 That really sounds like prefactoring for something that isn't even being considered/worked on. 14:04:32 EmilienM: sure, can talk now if u like 14:04:43 thrash: "isn't considered" does not imply "discarded a priori", right? 14:05:48 Yanis Guenane proposed openstack/instack-undercloud: Manage keystone related configuration via Puppet https://review.openstack.org/234704 14:06:31 marios: /b rhos-dev 14:06:36 oops 14:06:42 wrong weechat command :) 14:08:58 marios: I'm investigating https://review.openstack.org/#/c/225983/6/puppet/hieradata/compute.yaml,cm 14:10:38 pino|work: I just don't see that as a priority atm 14:10:42 EmilienM: sure ok. so once I got over the duplicate include (I have this applied on poodle rhos setup) as on the review puppet-ha job, I now get an error from puppet-horizon, which is also trying to bringup apache 14:10:58 marios: please take over the wsgi stuff, I can't help anymore. 14:11:12 EmilienM: heh "thanks very much!" 14:11:22 marios: :p 14:11:31 thrash: i didn't imply anywhere it needs to be a priority, just it might be a possibility 14:12:00 pino|work: I will concede that. :) 14:12:29 Gael Chamoulaud proposed openstack/tripleo-heat-templates: Add Puppet 4.x lint checks https://review.openstack.org/234150 14:12:30 Gael Chamoulaud proposed openstack/tripleo-heat-templates: Make puppet manifests compliant with Puppet 4.x https://review.openstack.org/225664 14:16:28 thrash: anyway, my point was to not create this new api with dib-isms all around, to not make alternative image building methods more complicated than needed 14:17:07 thrash: for example, in ImageBuildManager things like skip_base and docker_target 14:17:19 pino|work: and my point is that those image building methods don't exist. So I can't create an API for it. 14:17:28 marios, gfidente: I think you're right, catalog_info is needed by nova/volume/cinder.py / cinderclient class, consummed bu nova/compute/manager 14:17:33 pino|work: and those are optional. 14:17:46 marios, gfidente: I'm about to fix puppet-nova 14:18:00 thrash: which is not what i'm asking. i'm asking to not make future implementations of those more complicated and in need of refactor than what actually needed 14:18:09 pino|work: driven by the yaml. 14:19:00 EmilienM: so anyway, sure i will keep poking, i recall an issue (gotcha! https://bugzilla.redhat.com/show_bug.cgi?id=1247547#c5 ) so puppet-horizon expects to manage apache because of horizon::wsgi::apache - so there might be a clash there with '::keystone::wsgi::apache': (perhaps it has a similar expectation) 14:19:00 bugzilla.redhat.com bug 1247547 in openstack-tripleo-heat-templates "overcloud: HA: httpd service should not be started on boot by systemd." [High,On_dev] - Assigned to gfidente 14:19:21 EmilienM: sorry was searching for that comment... reading 14:19:24 marios: it should work 14:19:28 thrash: let's take the skip_base option: ImageBuildManager adds -n for it to the options, so any ImageBuilder subclass would get it among its options 14:19:36 marios: our CI is deploying Horizon + Keystone in WSGI and it works 14:19:42 marios: (puppet CI) 14:20:00 pino|work: true. 14:20:04 thrash: this is what i call dib-isms and make life harder for non-dib methods 14:20:17 *this* is what i'm asking to not do with the new api 14:20:58 pino|work: so that should be pushed down into the actual builder class. 14:21:03 i can see that. 14:21:09 thrash: of course it's your call (since it's your patch), but then i will need to change that part myself later when rebasing on it my virt-dib patch 14:21:23 EmilienM: ok well i say that because of the error i see like http://paste.openstack.org/raw/476693/ 14:21:37 pino|work: I agree. I was being lazy by appending them to the options. 14:22:23 marios: that's because keystone is running eventlet before starting apache 14:22:31 I fixed it in puppet-keystone 14:22:50 marios: https://github.com/openstack/puppet-keystone/commit/7120da9844e96a6898c3a406c2e36f0d38724552 14:24:00 EmilienM: thanks looking 14:24:28 EmilienM: ok great - so my poodle setup is likely lacking that... thanks for the pointer 14:26:31 Marek Aufart proposed openstack/python-tripleoclient: Disable command run under root user https://review.openstack.org/221720 14:30:28 Ryan Brown proposed openstack/python-tripleoclient: Move more functions to utils.py https://review.openstack.org/225246 14:30:28 Ryan Brown proposed openstack/python-tripleoclient: Consolidate module constants. https://review.openstack.org/225245 14:31:44 Brian Demers proposed openstack/os-net-config: Added natural sort for interfaces in utils.ordered_active_nics() https://review.openstack.org/210688 14:35:54 Emilien Macchi proposed openstack/tripleo-heat-templates: nova: set catalog_info to InternalURL https://review.openstack.org/225983 14:36:06 marios, gfidente ^ 14:36:15 Emilien Macchi proposed openstack/tripleo-heat-templates: controller/ceilometer: use internalURL for os endpoint type https://review.openstack.org/225999 14:37:23 EmilienM, I wonder how the other services are behaving as well 14:37:26 looks like they 14:37:43 would benefit same type of change or is it really only nova and ceilometer defaulting to public? 14:38:02 Brad P. Crochet proposed openstack/tripleo-common: Build image files from definitions in yaml https://review.openstack.org/235569 14:38:17 gfidente, marios: I'm done with THT patches: https://review.openstack.org/225983 and https://review.openstack.org/225999 - feel free to push over if you need something else, but I think it's ready to merge now. 14:38:26 Bob Ball proposed openstack/diskimage-builder: Add support for Xen PV disks https://review.openstack.org/226249 14:45:32 shardy, hey still around? 14:46:05 rhallisey: hi, yup 14:46:23 shardy: hey is there a way to get the heat API IP in teh template directly? 14:46:57 the heat meta data script you sent me works I guess, but we would have to put it in 2 places 14:47:09 I see heat_api_node_ips .. 14:47:31 can we get it from puppet, like using ^ 14:47:35 Lokesh Jain proposed openstack/tripleo-heat-templates: Changes for configuring Nuage https://review.openstack.org/230116 14:47:52 I don't think we want to get it from puppet 14:48:18 Jiri Stransky proposed openstack/tripleo-heat-templates: Support NFS backend for Glance (via Pacemaker) https://review.openstack.org/199152 14:48:26 Slower, rhallisey: Yeah that's not what you want, it's for the overcloud heat I think 14:48:41 I'd just put the script in two places, or pass a parameter in 14:48:57 shardy: there's no parameter for the heat API in the template you are deploying? 14:49:22 Slower: No, nothing which will help in this case 14:49:27 huh 14:50:18 shardy: or nova server etc? 14:50:35 Slower: you can create heat stacks via a remote heat using the OS::Heat::Stack resource, but that won't give you the IP of the heat service 14:50:51 Slower: well yeah there's OS::Nova::Server 14:51:03 not sure if that will always be correct though I guess 14:51:09 in case of multinode undercloud 14:51:23 Lokesh Jain proposed openstack/tripleo-heat-templates: Changes for configuring Nuage https://review.openstack.org/230116 14:51:35 Slower: I'm not sure even nova exposes the IP of the nova API service does it? 14:52:02 shard: well we 14:52:13 shardy: well we're just cheating really 14:52:16 we want the instack IP 14:52:55 shardy: this is in order to configure the local docker registry.. to have a sane default 14:53:48 shardy: the other idea is to generate it in an environment from the setup script 14:53:50 Merged openstack/tripleo-docs: Update docs to show using ExtraConfig parameters https://review.openstack.org/233012 14:53:59 Ben Nemec proposed openstack/python-tripleoclient: Don't parse redhat-release for distro info https://review.openstack.org/234885 14:54:08 shardy: which might actually be more correct 14:54:13 Slower: Yeah, I think a parameter is perfectly fine 14:54:22 there are a bunch of ways we can automatically populate it later 14:54:32 for now just documenting how to set it is OK IMO 14:54:42 right 14:57:19 pino|work: i've updated that patch. Is that more to your liking? :) 14:59:37 thrash: will check later, i was reviewing other patches 14:59:44 pino|work: ack 15:00:16 Untriaged bugs so far: 15:00:18 https://bugs.launchpad.net/tripleo/+bug/1505815 15:00:18 Launchpad bug 1505815 in tripleo "Docs refer to ironic-discoverd instead of ironic-inspector" [Undecided,Confirmed] - Assigned to Ruchi Parmar (ruchiparmar143) 15:00:20 https://bugs.launchpad.net/tripleo/+bug/1507556 15:00:20 Launchpad bug 1507556 in tripleo "[tripleoclient] should ignore validation of tunneling params if tunneling is disabled" [Undecided,In progress] - Assigned to Brad P. Crochet (brad-9) 15:00:21 https://bugs.launchpad.net/diskimage-builder/+bug/1507631 15:00:21 Launchpad bug 1507631 in diskimage-builder "Diskimage-builder (DIB) produces a non-bootable RHEL 6.7 QCOW2-image" [Undecided,New] 15:00:24 https://bugs.launchpad.net/diskimage-builder/+bug/1506792 15:00:24 Launchpad bug 1506792 in diskimage-builder "ironic-python-agent fails to run on fedora ramdisk" [Undecided,In progress] - Assigned to Lucas Alvares Gomes (lucasagomes) 15:00:25 https://bugs.launchpad.net/tripleo/+bug/1505430 15:00:25 Launchpad bug 1505430 in tripleo "Triggering puppet runs with config_identifier is error prone and complicates templates" [Undecided,New] - Assigned to Steve Baker (steve-stevebaker) 15:00:27 https://bugs.launchpad.net/diskimage-builder/+bug/1506692 15:00:27 Launchpad bug 1506692 in diskimage-builder "Broken test in fedora-minimal/install.d/99-ramdisk" [Undecided,In progress] - Assigned to Martin André (mandre) 15:00:29 https://bugs.launchpad.net/diskimage-builder/+bug/1505900 15:00:29 Launchpad bug 1505900 in diskimage-builder "Failed to create 'wily' release with DIB_RELEASE" [Undecided,New] 15:00:31 https://bugs.launchpad.net/tripleo/+bug/1504263 15:00:31 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 15:00:33 https://bugs.launchpad.net/diskimage-builder/+bug/1506275 15:00:33 Launchpad bug 1506275 in diskimage-builder "$DIB_OFFLINE and $DIB_DEBIAN_USE_DEBOOTSTRAP_CACHE are redundant" [Undecided,New] 15:00:35 https://bugs.launchpad.net/tripleo/+bug/1507054 15:00:36 Launchpad bug 1507054 in tripleo "Allow configuration of mysql percona sst through xtrabackup-v2" [Undecided,New] 15:05:04 Got "No image with a name or ID of 'bm-deploy-kernel' exists." in the CI, any idea of what that might be? http://logs.openstack.org/30/231930/8/check-tripleo/gate-tripleo-ci-ironic-overcloud-f21puppet-nonha/41b4b4a/console.html#_2015-10-19_13_47_37_686 15:10:54 jaosorior: check a few lines above http://logs.openstack.org/30/231930/8/check-tripleo/gate-tripleo-ci-ironic-overcloud-f21puppet-nonha/41b4b4a/console.html#_2015-10-19_13_47_32_355 15:11:09 jaosorior: i think that's what you saw on your local env, right? 15:11:22 maybe we have another breakage 15:14:54 jaosorior: in the undercloud logs i see Cannot open: http://rdo.fedorapeople.org/openstack-kilo/rdo-release-kilo.rpm. Skipping. 15:15:09 could be intermittent, maybe recheck could do 15:15:18 (and re-run locally) 15:16:03 maybe they did maintenance or something 15:19:32 jistr: Alright, did a recheck 15:20:48 bnemec: ping 15:21:13 jaosorior: pong 15:22:09 bnemec: Hey, you had mentioned that there were some issues with nova that were gonna be problematic for the enabling of TLS, can you elaborate on that? 15:23:13 jaosorior: Yeah, it's basically what I describe in the commit message on https://review.openstack.org/#/c/223330/ 15:23:47 However, after discussing it some with people from the projects involved it sounds like that isn't a complete fix. 15:24:05 Heat, for example, can have nested addresses in its response that will also be wrong. 15:24:34 It sounds like there's a header we can set before passing requests to Heat that will make it do the right thing, but I haven't had a chance to test it. 15:24:48 And I don't know if it works for Nova. Apparently support for this sort of thing in OpenStack is a bit spotty. 15:25:08 bnemec: looking at that patch... wouldn't setting the forwarding mode to http be problematic when trying to terminate TLS in HAProxy? 15:27:20 jaosorior: That's not specifically referring to the protocol type, it just enables header rewriting for HTTP responses. 15:27:52 It's what we're using for Heat with SSL on the undercloud today, so I can say with reasonable certainty that it does what it's supposed to, it's just not enough. 15:28:14 I see 15:28:46 where can I read more details about it? To understand it better 15:29:22 shardy, what's a good substitute for FN::join? 15:29:23 shardy: what's the best way to join two parameters? 15:29:28 haha 15:29:32 :) 15:29:40 shardy: we tried Fn::Join but it says it's unsupported? 15:30:23 Slower: Have you tried list_join? http://docs.openstack.org/developer/heat/template_guide/hot_spec.html#list-join 15:30:32 jaosorior: There's an upstream discussion in http://lists.openstack.org/pipermail/openstack-dev/2015-September/074871.html 15:31:16 bnemec: Thanks, I'll give it a read 15:39:13 jaosorior: oh right! 15:39:19 jaosorior: yeah that's working thx! 15:40:23 Slower: No problem, glad I could help :D 16:20:05 Ryan Hallisey proposed openstack/tripleo-heat-templates: Update docker compute environment to use json config https://review.openstack.org/234313 16:20:05 Ryan Hallisey proposed openstack/tripleo-heat-templates: Add local docker registry support https://review.openstack.org/237071 17:14:48 Ben Nemec proposed openstack/python-tripleoclient: Don't log exceptions redundantly https://review.openstack.org/237097 17:45:37 slagle: so I am in the process of getting all of the tripleo packages into the GA repo for RDO, I am having to use git hash tarballs since there are no stable/liberty branches 17:45:58 slagle: it looks like for openstack-tripleo you have already been doing that. 17:46:26 Ben Nemec proposed openstack/python-tripleoclient: Allow adding new overcloud passwords https://review.openstack.org/237116 17:46:28 slagle: any objection to me updating that hash to the current-passed-ci delorean hash? 17:51:05 Steven Hardy proposed openstack/tripleo-heat-templates: Allow ctlplane network or ID, and default to "ctlplane" https://review.openstack.org/232101 17:51:05 Steven Hardy proposed openstack/tripleo-heat-templates: Update overcloud template description and Count constraints https://review.openstack.org/232084 17:51:05 Steven Hardy proposed openstack/tripleo-heat-templates: Align some defaults with with ControllerCount default https://review.openstack.org/232122 18:17:47 trown: you mean in the tripleo spec? 18:17:53 trown: if so, go4it. no objection 18:18:13 delorean trunk should already be building latest each time anyway 18:19:22 slagle: right... found out this morning that none of the tripleo packages are in the RDO liberty release GA repo :( 18:19:38 so I am working on getting them all in there 18:19:50 since GA is Wed 18:19:58 trown: so what needs to be updated, the fedora distgit specs? 18:20:51 slagle: that would be one way... I am doing an rdo-liberty branch on the openstack-packages repos since I am not the fedora maintainer for any of these 18:21:13 slagle: then I have to ping apevec to actually do the builds in centos 18:21:32 https://github.com/openstack-packages/python-tripleoclient/commit/c5a94c6e0b145658ab264e422b39d82ad4ece2e6 is an example 18:22:36 oh i see, so the specs are from openstack-packages, and the builds are done in cbs 18:22:45 slagle: is HA job red? 18:22:57 EmilienM: RTFT man 18:23:33 last one was red but i see several green before that :) 18:24:28 slagle: tripleo e-mails go in SPAM, that's why... damn it 18:24:37 lol 18:24:52 EmilienM: really though, you can check here: http://tripleo.org/cistatus.html 18:25:05 slagle: sorry for asking, I'll know next time 18:25:08 slagle: so I have a good news 18:25:21 https://review.openstack.org/#/c/234325/ and https://review.openstack.org/#/c/225962/ pass CI 18:25:28 ok cool 18:25:33 slagle: without yanis's work 18:26:47 slagle: I'm testing the HA job now, it's failing upstream 18:27:13 Steven Hardy proposed openstack/tripleo-specs: Add spec for release branch policy https://review.openstack.org/221811 18:27:59 dprince: fyi I added a comment about the master branches not requiring backwards compat wrt openstack features when we have release branches 18:28:13 re our conversation earlier about Heat map_merge 18:28:50 Emilien Macchi proposed openstack/tripleo-heat-templates: neutron: enable nova-event-callback by default https://review.openstack.org/225962 18:29:36 slagle: one change related to an earlier comment you made, is apparently, according to dtantsur|afk, we have to use "stable/foo" instad of "release/foo" due to zuul 18:30:01 I don't have time to hack on zuul, so I've just reverted to the normal branch naming in the spec 18:30:18 ok, wfm 19:07:24 Tzu-Mainn Chen proposed openstack/tuskar: [WIP] Tuskar API v3 https://review.openstack.org/235716 19:46:04 Ryan Hallisey proposed openstack/tripleo-common: Use Tripleo.sh to setup a local Docker Registry https://review.openstack.org/237173 19:50:14 slagle: bnemec is the version of diskimage-builder and dib-utils in fedora good enough for liberty? 19:50:33 trown: i don't know for sure, but I doubt it 19:52:22 hmm.. dib-utils looks good... not sure about diskimage-builder... I think I will do the source tarball for that 20:02:33 Heh, there hasn't been a commit to dib-utils since February. 20:10:11 i wouldnt have guessed the fedora pkg was that new :) 20:24:15 slagle: ha job is now passing for https://review.openstack.org/#/c/225962/ 20:25:25 ok, i shall have a look 20:59:56 Ben Nemec proposed openstack/tripleo-common: Fix cherry-pick comments https://review.openstack.org/237198 21:00:18 Untriaged bugs so far: 21:00:20 https://bugs.launchpad.net/tripleo/+bug/1505815 21:00:21 https://bugs.launchpad.net/tripleo/+bug/1507556 21:00:23 https://bugs.launchpad.net/diskimage-builder/+bug/1507631 21:00:24 Launchpad bug 1505815 in tripleo "Docs refer to ironic-discoverd instead of ironic-inspector" [Undecided,Confirmed] - Assigned to Ruchi Parmar (ruchiparmar143) 21:00:25 https://bugs.launchpad.net/diskimage-builder/+bug/1506792 21:00:25 Launchpad bug 1507556 in tripleo "[tripleoclient] should ignore validation of tunneling params if tunneling is disabled" [Undecided,In progress] - Assigned to Brad P. Crochet (brad-9) 21:00:26 Launchpad bug 1507631 in diskimage-builder "Diskimage-builder (DIB) produces a non-bootable RHEL 6.7 QCOW2-image" [Undecided,New] 21:00:27 https://bugs.launchpad.net/tripleo/+bug/1505430 21:00:28 Launchpad bug 1506792 in diskimage-builder "ironic-python-agent fails to run on fedora ramdisk" [Undecided,In progress] - Assigned to Lucas Alvares Gomes (lucasagomes) 21:00:29 https://bugs.launchpad.net/diskimage-builder/+bug/1506692 21:00:29 Launchpad bug 1505430 in tripleo "Triggering puppet runs with config_identifier is error prone and complicates templates" [Undecided,New] - Assigned to Steve Baker (steve-stevebaker) 21:00:30 Launchpad bug 1506692 in diskimage-builder "Broken test in fedora-minimal/install.d/99-ramdisk" [Undecided,In progress] - Assigned to Martin André (mandre) 21:00:31 https://bugs.launchpad.net/diskimage-builder/+bug/1505900 21:00:32 Launchpad bug 1505900 in diskimage-builder "Failed to create 'wily' release with DIB_RELEASE" [Undecided,New] 21:00:34 https://bugs.launchpad.net/tripleo/+bug/1504263 21:00:34 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 21:00:35 https://bugs.launchpad.net/diskimage-builder/+bug/1506275 21:00:35 Launchpad bug 1506275 in diskimage-builder "$DIB_OFFLINE and $DIB_DEBIAN_USE_DEBOOTSTRAP_CACHE are redundant" [Undecided,New] 21:00:37 https://bugs.launchpad.net/tripleo/+bug/1507054 21:00:38 Launchpad bug 1507054 in tripleo "Allow configuration of mysql percona sst through xtrabackup-v2" [Undecided,New] 21:55:36 anyone here run dib on rhel-7? 00:11:57 Merged openstack/diskimage-builder: Add flake8 to requirements https://review.openstack.org/229780 00:49:03 OpenStack Proposal Bot proposed openstack/diskimage-builder: Updated from global requirements https://review.openstack.org/237351 00:55:12 OpenStack Proposal Bot proposed openstack/tuskar: Updated from global requirements https://review.openstack.org/233887 01:07:43 Martin André proposed openstack/diskimage-builder: Fix uniqueness check of initrd in fedora-minimal https://review.openstack.org/235705 01:41:18 Merged openstack/diskimage-builder: Fix devuser pubkey defaults https://review.openstack.org/234490 02:03:29 Dan Prince proposed openstack/tripleo-heat-templates: WIP: Support for composable roles https://review.openstack.org/236243 02:03:29 Dan Prince proposed openstack/tripleo-heat-templates: Create Glance API and Glance Registry Roles https://review.openstack.org/237370 02:18:25 Augustina Ragwitz proposed openstack/diskimage-builder: Sanitize distro elements for minimal vs cloud https://review.openstack.org/211859 02:37:07 Tzu-Mainn Chen proposed openstack/tripleo-specs: Spec for TripleO overcloud deployment REST API https://review.openstack.org/230432 03:00:17 Untriaged bugs so far: 03:00:18 https://bugs.launchpad.net/tripleo/+bug/1505815 03:00:18 Launchpad bug 1505815 in tripleo "Docs refer to ironic-discoverd instead of ironic-inspector" [Undecided,Confirmed] - Assigned to Ruchi Parmar (ruchiparmar143) 03:00:20 https://bugs.launchpad.net/tripleo/+bug/1507556 03:00:20 Launchpad bug 1507556 in tripleo "[tripleoclient] should ignore validation of tunneling params if tunneling is disabled" [Undecided,In progress] - Assigned to Brad P. Crochet (brad-9) 03:00:22 https://bugs.launchpad.net/diskimage-builder/+bug/1507631 03:00:22 Launchpad bug 1507631 in diskimage-builder "Diskimage-builder (DIB) produces a non-bootable RHEL 6.7 QCOW2-image" [Undecided,New] 03:00:24 https://bugs.launchpad.net/diskimage-builder/+bug/1506792 03:00:25 Launchpad bug 1506792 in diskimage-builder "ironic-python-agent fails to run on fedora ramdisk" [Undecided,In progress] - Assigned to Lucas Alvares Gomes (lucasagomes) 03:00:26 https://bugs.launchpad.net/tripleo/+bug/1505430 03:00:27 Launchpad bug 1505430 in tripleo "Triggering puppet runs with config_identifier is error prone and complicates templates" [Undecided,New] - Assigned to Steve Baker (steve-stevebaker) 03:00:28 https://bugs.launchpad.net/diskimage-builder/+bug/1506692 03:00:28 Launchpad bug 1506692 in diskimage-builder "Broken test in fedora-minimal/install.d/99-ramdisk" [Undecided,In progress] - Assigned to Martin André (mandre) 03:00:30 https://bugs.launchpad.net/diskimage-builder/+bug/1505900 03:00:30 Launchpad bug 1505900 in diskimage-builder "Failed to create 'wily' release with DIB_RELEASE" [Undecided,New] 03:00:32 https://bugs.launchpad.net/tripleo/+bug/1504263 03:00:32 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 03:00:34 https://bugs.launchpad.net/diskimage-builder/+bug/1506275 03:00:35 Launchpad bug 1506275 in diskimage-builder "$DIB_OFFLINE and $DIB_DEBIAN_USE_DEBOOTSTRAP_CACHE are redundant" [Undecided,New] 03:00:36 https://bugs.launchpad.net/tripleo/+bug/1507054 03:00:36 Launchpad bug 1507054 in tripleo "Allow configuration of mysql percona sst through xtrabackup-v2" [Undecided,New] 03:08:07 Vinay Rao proposed openstack/tripleo-heat-templates: This change makes the following parameters user configurable: https://review.openstack.org/230764 06:56:23 Giulio Fidente proposed openstack/tripleo-puppet-elements: Add python-memcached to list of installed pkgs on controllers https://review.openstack.org/237426 08:02:22 trown|outttypeww, when you're here, could you have a look: https://bugzilla.redhat.com/show_bug.cgi?id=1270818 08:02:22 bugzilla.redhat.com bug 1270818 in rdo-manager "Two ironic-inspector processes are running on the undercloud, breaking the introspection" [High,New] - Assigned to calfonso 08:27:20 Hey guys, I'm seeing errors like this when trying to deploy a basic overcloud (no scale parameters) http://pastebin.com/pt7Ad2LR :/ 08:27:27 any idea of what that might be? 08:28:02 aaah, jistr 08:28:12 good morning :) 08:28:18 good morning dude :D 08:28:30 sorry to bother you since you just made it online 08:28:35 ever seen something like this? http://pastebin.com/pt7Ad2LR 08:29:03 (I think I owe you a beer by now for all the help) 08:30:31 jaosorior, Looks like an issue with nova? I'd look in the compute logs 08:30:48 "Exceeded max scheduling attempts 3 for instance ..." 08:31:52 yeah does 'ironic node-list' print something? 08:32:52 more ~cd .. 08:32:59 vaaah 08:33:03 therve: the compute logs seems to complain like this PortBindingFailed: Binding failed for port 9ef74564-0a4f-48de-94ba-9b78674de12e, please check neutron logs for more information. 08:33:23 jistr: Yeah, they show the 6 nodes that it should :/ 08:33:23 jaosorior, Check if neutron is running 08:33:34 Especially the ovs agent 08:35:29 therve neutron-server seems to be running 08:36:23 I just did a restart to the neutron-server to see if that helps 08:36:49 jaosorior, What about neutron-openvswitch-agent? 08:36:55 Or whatever it's caleld 08:37:22 therve I was just wandering what the name was haha 08:37:27 lets see 08:37:35 Just run systemctl otherwise 08:38:20 yeah, doesn't appear to be in the list offered by the openstack-service. Lets see what systemctl says 08:39:22 Cyril Lopez proposed openstack/tripleo-heat-templates: Implement Neutron enable_isolated_metadata parameters https://review.openstack.org/237464 08:45:29 therve: ovs was dead :/ 08:46:10 jaosorior, ovs or the neutron agent? 08:46:33 neutron-openvswitch-agent 08:46:59 jaosorior, https://bugs.launchpad.net/neutron/+bug/1494281 by any chance? :) 08:46:59 Launchpad bug 1494281 in neutron "neutron-openvswitch-agent is crashing with "invalid literal for int() with base 10" error" [Medium,In progress] - Assigned to bharath (bharath-7) 08:48:27 therve: Indeed, that's in the logs 08:49:31 That's an annoying one 08:50:37 The neutron team seems pretty unconcerned too 08:51:17 :( 08:51:56 The proposed fix seems... like its gonna take a bit https://review.openstack.org/#/c/225001/ 08:52:16 Yeah because it's somewhat wrong 08:52:32 It simply works around the issue 08:52:57 Cyril Lopez proposed openstack/tripleo-heat-templates: Implement Neutron enable_isolated_metadata parameters https://review.openstack.org/237464 08:58:18 jaosorior, FWIW you should be able to unblock yourself with some ovs calls 08:58:58 my knowledge about OVS is pretty lacking TBH :/ 09:00:17 Untriaged bugs so far: 09:00:19 https://bugs.launchpad.net/tripleo/+bug/1505815 09:00:19 Launchpad bug 1505815 in tripleo "Docs refer to ironic-discoverd instead of ironic-inspector" [Undecided,Confirmed] - Assigned to Ruchi Parmar (ruchiparmar143) 09:00:21 https://bugs.launchpad.net/tripleo/+bug/1507556 09:00:21 Launchpad bug 1507556 in tripleo "[tripleoclient] should ignore validation of tunneling params if tunneling is disabled" [Undecided,In progress] - Assigned to Brad P. Crochet (brad-9) 09:00:22 https://bugs.launchpad.net/diskimage-builder/+bug/1507631 09:00:22 Launchpad bug 1507631 in diskimage-builder "Diskimage-builder (DIB) produces a non-bootable RHEL 6.7 QCOW2-image" [Undecided,New] 09:00:24 https://bugs.launchpad.net/diskimage-builder/+bug/1506792 09:00:24 Launchpad bug 1506792 in diskimage-builder "ironic-python-agent fails to run on fedora ramdisk" [Undecided,In progress] - Assigned to Lucas Alvares Gomes (lucasagomes) 09:00:26 https://bugs.launchpad.net/diskimage-builder/+bug/1506692 09:00:27 Launchpad bug 1506692 in diskimage-builder "Broken test in fedora-minimal/install.d/99-ramdisk" [Undecided,In progress] - Assigned to Martin André (mandre) 09:00:28 https://bugs.launchpad.net/diskimage-builder/+bug/1505900 09:00:29 Launchpad bug 1505900 in diskimage-builder "Failed to create 'wily' release with DIB_RELEASE" [Undecided,New] 09:00:30 https://bugs.launchpad.net/tripleo/+bug/1504263 09:00:31 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 09:00:32 https://bugs.launchpad.net/diskimage-builder/+bug/1506275 09:00:32 Launchpad bug 1506275 in diskimage-builder "$DIB_OFFLINE and $DIB_DEBIAN_USE_DEBOOTSTRAP_CACHE are redundant" [Undecided,New] 09:00:34 https://bugs.launchpad.net/tripleo/+bug/1507054 09:00:34 Launchpad bug 1507054 in tripleo "Allow configuration of mysql percona sst through xtrabackup-v2" [Undecided,New] 09:10:16 URGENT TRIPLEO TASKS NEED ATTENTION 09:10:16 +-------------------------------------+-------------------------------+-------------------+-------------+ 09:10:16 | Title | URL | Members | Last Active | 09:10:16 +-------------------------------------+-------------------------------+-------------------+-------------+ 09:10:16 | Could not find resource 'Anchor[hea | https://trello.com/c/dWWL3dnT | **NEEDS MEMBERS** | 31 min | 09:10:17 +-------------------------------------+-------------------------------+-------------------+-------------+ 09:15:34 therve: figured it out 09:23:57 derekh, are you working on ^ ? 09:24:27 I might have a patch for it 09:25:00 spredzy: I've submitted a revert to puppet-heat , was going to get back to it in a few minutes, if you have a patch for it fire ahead and give it a go 09:27:30 derekh, ack, will submit it then 09:27:49 is there a way to view the heat logs from the openstack CI? 09:28:42 jaosorior: download and unzip the undercloud tarball 09:30:02 jaosorior: then you can use the systemd journal 09:30:09 JOBLOGS ]$ journalctl --file ./undercloud.tar.xz_/var/log/journal/0af04d3c78a943ae8f3cc26602e374f2/system.journal -u openstack-heat-engine 09:30:17 jaosorior: ^ 09:30:38 dude, nice! 10:10:14 URGENT TRIPLEO TASKS NEED ATTENTION 10:10:14 +-------------------------------------+-------------------------------+-------------------+-------------+ 10:10:14 | Title | URL | Members | Last Active | 10:10:14 +-------------------------------------+-------------------------------+-------------------+-------------+ 10:10:14 | Could not find resource 'Anchor[hea | https://trello.com/c/dWWL3dnT | **NEEDS MEMBERS** | 91 min | 10:10:15 +-------------------------------------+-------------------------------+-------------------+-------------+ 10:12:27 Derek Higgins proposed openstack/tripleo-common: Update the repo whitelist to reflect what we CI https://review.openstack.org/229906 11:10:15 URGENT TRIPLEO TASKS NEED ATTENTION 11:10:15 +-------------------------------------+-------------------------------+-------------------+-------------+ 11:10:15 | Title | URL | Members | Last Active | 11:10:15 +-------------------------------------+-------------------------------+-------------------+-------------+ 11:10:15 | Could not find resource 'Anchor[hea | https://trello.com/c/dWWL3dnT | **NEEDS MEMBERS** | 151 min | 11:10:16 +-------------------------------------+-------------------------------+-------------------+-------------+ 11:29:42 Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Enable TLS deployment for the Overcloud https://review.openstack.org/231930 11:53:35 Derek Higgins proposed openstack/diskimage-builder: Nothing to see here https://review.openstack.org/139627 11:54:06 hahaha I get too curious when I see a commit message like that 11:54:30 * derekh believes he found a bug in grep 12:10:17 URGENT TRIPLEO TASKS NEED ATTENTION 12:10:17 +-------------------------------------+-------------------------------+-------------------+-------------+ 12:10:17 | Title | URL | Members | Last Active | 12:10:17 +-------------------------------------+-------------------------------+-------------------+-------------+ 12:10:17 | Could not find resource 'Anchor[hea | https://trello.com/c/dWWL3dnT | **NEEDS MEMBERS** | 211 min | 12:10:18 +-------------------------------------+-------------------------------+-------------------+-------------+ 12:34:45 Anybody has time to review this CR? https://review.openstack.org/#/c/222189/ it would be reaaaally useful for the TLS part :D 12:41:51 dprince: Hey, when you're around I'm looking for a +A on the release branch spec (https://review.openstack.org/#/c/221811/) 12:42:18 I think we reached sufficient consensus to go ahead and cut the branches, then work to put support in place via tripleo.sh and CI 12:42:40 ideally I'd love to push on that for a few days and announce initial support for stable/liberty around summit 12:43:18 Yanis Guenane proposed openstack/python-tripleoclient: Fix chicken and eggs problem with tripleo-heat-templates https://review.openstack.org/235769 12:44:07 shardy: ack 12:45:02 Martin Mágr proposed openstack/tripleo-heat-templates: Ceph cluster validation https://review.openstack.org/225248 12:45:57 shardy: +A 12:46:19 dprince: thanks! I'll go ahead and get the project-config changes posted so we can create the branches 12:46:43 shardy: sounds good 12:47:58 Merged openstack/tripleo-specs: Add spec for release branch policy https://review.openstack.org/221811 12:58:00 Derek Higgins proposed openstack/tripleo-common: Pin puppet-heat https://review.openstack.org/237568 12:58:04 spredzy: ^ 12:59:27 derekh, thanks, I am working on it. Just that the revert is valid for a given topology, and I would like to avoid commit that are revert of reverts 13:00:39 marios, bnemec shardy ping https://review.openstack.org/#/c/235769 - Should I abandon the parent review ? 13:00:53 And let's be it revisited at the end of next cycle ? 13:01:15 spredzy: I'd just like to avoid another few hours of blocked ci, anyways gotta run and do a school pickup 13:04:02 spredzy: assuming you mean https://review.openstack.org/#/c/231395/ imo yes too soon 13:04:31 marios, yes that is the one I meant. Dependend review I should have said 13:05:06 spredzy: (mostly for others if interested and just to confirm, i understood which you meant, sorry, didn't mean to be pedantic) 13:05:49 marios, didn't take it this way no worries :). I does make more sense. 13:06:00 ;) 13:10:14 URGENT TRIPLEO TASKS NEED ATTENTION 13:10:14 +-------------------------------------+-------------------------------+-------------------+-------------+ 13:10:14 | Title | URL | Members | Last Active | 13:10:14 +-------------------------------------+-------------------------------+-------------------+-------------+ 13:10:14 | Could not find resource 'Anchor[hea | https://trello.com/c/dWWL3dnT | **NEEDS MEMBERS** | 271 min | 13:10:15 +-------------------------------------+-------------------------------+-------------------+-------------+ 13:14:01 dprince: Hey, can you add me to the tripleo-release group please? 13:14:05 https://review.openstack.org/#/admin/groups/200,members 13:14:33 I assume that's going to be the most appropriate group to use for branch-create ACLs, as it's already used for some tripleo related tag/create entries 13:17:55 Does anybody have an idea on why this patch https://review.openstack.org/#/c/231930/ has a comment by jenkins that it depends on a patch that failed to merge? Even if the patch above it doesn't seem to have a problem :/ 13:19:08 spredzy ping 13:19:44 jaosorior, pong 13:20:05 spredzy: Do you mind if I rebase your changes starting here https://review.openstack.org/#/c/230357/14 with the current master? 13:20:25 jaosorior, no go ahead 13:21:24 Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Manage keystone related settings directly in t-h-t manifests https://review.openstack.org/230357 13:21:25 Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Manage keystone related settings directly in t-h-t manifests https://review.openstack.org/230375 13:21:25 Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Enable TLS deployment for the Overcloud https://review.openstack.org/231930 13:22:08 spredzy: Done 13:27:45 shardy: done 13:29:20 dprince: thx 13:31:44 trown: should be, yes: http://fpaste.org/281425/14453455/ 13:31:58 trown: ignore, sorry 13:32:02 :) 13:32:10 too late already clicked 13:32:28 the secret is out 13:39:09 Does anyone know why t-h-t ACLs specify diskimage-builder-core vs tripleo-core? 13:39:12 https://github.com/openstack-infra/project-config/blob/master/gerrit/acls/openstack/tripleo-heat-templates.config 13:39:40 I'm assuming that should be switched to be consitently tripleo-core now the groups working on each have diverged? 13:40:38 i think that's just some legacy 13:40:49 from when the core group was called diskimage-builder-core 13:40:54 it could be switched to tripleo-core 13:41:12 slagle: Ok, thanks for confirming, I'll clean it up with the changes for stable branches 13:43:50 jaosorior, I was looking at the TLS change, I think it's all good 13:44:14 I was wondering, we can change the protocol now but still have the issue with endpoints being created from IPs instead of hostnames right? 13:44:29 that means clients get a warning but can technically work 13:44:39 is this correct? 13:44:47 gfidente: I'm currently testing it and for some reason the trust anchor didn't get properly extracted (maybe I missed an environment flag...gonna check that out after my finnish course) 13:45:12 Hmm, actually looks like we don't use the repo-specific ACLs anyway now, only the general tripleo.config 13:45:31 yeah, would really like to get an option to get hostnames instead of the IPs 13:45:55 jaosorior, ok we need to work on that too then, but not necessarily in that change 13:46:15 gfidente: https://review.openstack.org/#/c/222189/ this change which you co-authored would really be a step forward to ease testing this 13:47:10 gfidente: Then deployers could specify the VIP as subject name in the certificates and testing would be easier 13:47:18 Anyway, I'll be back in a couple of hours 13:47:21 jaosorior, yes I see what you mean 13:47:54 but that is meant to create the IPs too I am not sure if we can reuse it as-is 13:48:32 jaosorior, also, should the certificate contain the VIP it won't get the warning either 13:48:44 just saying, not that we want to do that, but for testing ... 13:54:02 slagle: do we need a branch for openstack/instack as well as instack-undercloud? 13:55:30 shardy: instack-undercloud, yes 13:55:33 instack, i'd say no 13:55:51 slagle: Ok, thanks, they share the common acl atm, so I'll split it 13:58:10 Meeting time in 2 minutes! 14:07:33 Ryan Hallisey proposed openstack/tripleo-specs: Containerize TripleO blueprint https://review.openstack.org/223182 14:10:19 URGENT TRIPLEO TASKS NEED ATTENTION 14:10:19 +-------------------------------------+-------------------------------+-------------------+-------------+ 14:10:19 | Title | URL | Members | Last Active | 14:10:19 +-------------------------------------+-------------------------------+-------------------+-------------+ 14:10:19 | Could not find resource 'Anchor[hea | https://trello.com/c/dWWL3dnT | **NEEDS MEMBERS** | 331 min | 14:10:20 +-------------------------------------+-------------------------------+-------------------+-------------+ 14:26:34 Merged openstack-infra/tripleo-ci: Destroy previous test env VM's earlier https://review.openstack.org/230129 15:00:17 Untriaged bugs so far: 15:00:19 https://bugs.launchpad.net/tripleo/+bug/1505815 15:00:20 https://bugs.launchpad.net/tripleo/+bug/1507556 15:00:22 https://bugs.launchpad.net/diskimage-builder/+bug/1507631 15:00:22 Launchpad bug 1505815 in tripleo "Docs refer to ironic-discoverd instead of ironic-inspector" [Undecided,Confirmed] - Assigned to Ruchi Parmar (ruchiparmar143) 15:00:24 https://bugs.launchpad.net/diskimage-builder/+bug/1506792 15:00:24 Launchpad bug 1507556 in tripleo "[tripleoclient] should ignore validation of tunneling params if tunneling is disabled" [Undecided,In progress] - Assigned to Brad P. Crochet (brad-9) 15:00:25 Launchpad bug 1507631 in diskimage-builder "Diskimage-builder (DIB) produces a non-bootable RHEL 6.7 QCOW2-image" [Undecided,New] 15:00:26 https://bugs.launchpad.net/diskimage-builder/+bug/1506692 15:00:26 Launchpad bug 1506792 in diskimage-builder "ironic-python-agent fails to run on fedora ramdisk" [Undecided,In progress] - Assigned to Lucas Alvares Gomes (lucasagomes) 15:00:27 Launchpad bug 1506692 in diskimage-builder "Broken test in fedora-minimal/install.d/99-ramdisk" [Undecided,In progress] - Assigned to Martin André (mandre) 15:00:28 https://bugs.launchpad.net/diskimage-builder/+bug/1505900 15:00:29 Launchpad bug 1505900 in diskimage-builder "Failed to create 'wily' release with DIB_RELEASE" [Undecided,New] 15:00:30 https://bugs.launchpad.net/tripleo/+bug/1504263 15:00:30 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 15:00:32 https://bugs.launchpad.net/tripleo/+bug/1507930 15:00:32 Launchpad bug 1507930 in tripleo "tripleo-docs instackenv.json content" [Undecided,New] 15:00:34 https://bugs.launchpad.net/diskimage-builder/+bug/1506275 15:00:34 Launchpad bug 1506275 in diskimage-builder "$DIB_OFFLINE and $DIB_DEBIAN_USE_DEBOOTSTRAP_CACHE are redundant" [Undecided,New] 15:00:36 https://bugs.launchpad.net/tripleo/+bug/1507054 15:00:36 Launchpad bug 1507054 in tripleo "Allow configuration of mysql percona sst through xtrabackup-v2" [Undecided,New] 15:02:53 gfidente: bind_host? 15:03:04 dprince, yep here 15:03:37 can't we just pass the bind_host as param to the role? 15:04:20 gfidente: not initially 15:04:43 gfidente: there isn't (yet) a mechanism to smoothly deal w/ the per server settings 15:05:01 gfidente: basically, all the network settings (bind hosts) 15:05:48 gfidente: the best we could do would be to "template" them and do some sort of awkward str_replace to substitue them in for specific servers 15:06:14 gfidente: I think perhaps we could get there, but it might require more Heat functions 15:08:01 dprince, I might miss something 15:08:20 derekh, ok, where should we start. Is this the repo I want? https://github.com/openstack-infra/tripleo-ci 15:08:46 dprince: what's the key to lookup the per-host data? 15:08:58 dprince: we already have a method to lookup based on a host uuid: 15:09:01 https://review.openstack.org/#/c/215013/ 15:09:13 could that be used in this case? 15:09:17 dprince, yeah I think I was going where shardy is already 15:09:27 https://review.openstack.org/#/c/215116/5 15:09:33 dprince, is it just a lookup method to get the data for a specific server missing? 15:09:59 basically pass in a json blob with the per-node stuff, then look it up with a unique identifier in the node 15:10:02 shardy: right, but that would allow you to pass in things from the top level. That isn't what we need here 15:10:15 URGENT TRIPLEO TASKS NEED ATTENTION 15:10:15 +-------------------------------------+-------------------------------+-------------------+-------------+ 15:10:15 | Title | URL | Members | Last Active | 15:10:15 +-------------------------------------+-------------------------------+-------------------+-------------+ 15:10:15 | Could not find resource 'Anchor[hea | https://trello.com/c/dWWL3dnT | **NEEDS MEMBERS** | 391 min | 15:10:16 +-------------------------------------+-------------------------------+-------------------+-------------+ 15:10:23 shardy: we need a mechanism to customize IPs that are assigned via the undercloud Neutron 15:10:48 shardy, gfidente: I' 15:10:52 dprince: Ah, I see , so each node gets the same IP every time 15:11:00 dprince, oh the external-lb spec adds the yaml to assign IPs without creating the ports in neutron FWIW 15:11:05 I then I get those from the node index 15:11:14 shardy, gfidente: I've got ideas for mechanisms to do this. But these initial patches are already quite large so I'd like to table this as an improvement 15:11:27 rhallisey: sorry can we circle back, looks like we have anoth regression in puppet-nova 15:11:41 dprince: ack 15:11:59 shardy, gfidente: even without the network customization feature for each role I think the interface is still quite useful 15:12:11 derekh, sure no problem. I'll check in tomorrow. 15:12:53 Tzu-Mainn Chen proposed openstack/tripleo-specs: Spec for TripleO overcloud deployment REST API https://review.openstack.org/230432 15:14:05 rhallisey: thanks and sorry 15:14:12 no problem 15:15:46 Tzu-Mainn Chen proposed openstack/tuskar: [WIP] Tuskar API v3 https://review.openstack.org/235716 15:16:03 Derek Higgins proposed openstack/tripleo-common: Pin puppet-heat and puppet-nova https://review.openstack.org/237568 15:25:09 URGENT TRIPLEO TASKS NEED ATTENTION 15:25:09 +-------------------------------------+-------------------------------+-------------------+-------------+ 15:25:09 | Title | URL | Members | Last Active | 15:25:09 +-------------------------------------+-------------------------------+-------------------+-------------+ 15:25:09 | Could not find resource 'Anchor[hea | https://trello.com/c/dWWL3dnT | **NEEDS MEMBERS** | 406 min | 15:25:10 | Error: Duplicate declaration: Nova_ | https://trello.com/c/4zjL5tNt | **NEEDS MEMBERS** | 0 min | 15:25:10 +-------------------------------------+-------------------------------+-------------------+-------------+ 15:37:52 derekh: you're not looking into the duplicate declaration right? i think i see the problem and the solution, so i'll tackle that unless someone else has already started 15:38:49 jistr: correct, I've submitted a revert in the puppet module and a pin in tripleo.sh, so either of those will work of a fix if you have one 15:39:31 ah i see the pin above. yeah that won't hurt. For the puppet-nova one we can update t-h-t to follow the new way. I'll do that. 15:40:49 hmm CI failed on the pin patch 15:43:49 no such remote ref http://logs.openstack.org/68/237568/2/check-tripleo/gate-tripleo-ci-ironic-overcloud-f21puppet-ceph/a958398/console.html#_2015-10-20_15_33_33_200 15:44:23 right, there's an export for puppet_heat twice 15:45:24 * jistr pushing an amendment in a few secs 15:48:35 gfidente: I raised https://bugs.launchpad.net/heat/+bug/1508115 15:48:35 Launchpad bug 1508115 in heat "StackResource updates can destroy more than is needed" [Undecided,New] 15:49:00 zaneb: I'd be interested in your thoughts on that re how feasable doing things differently in heat would actually be 15:49:21 I may dig into it later in the week if I have time 15:49:43 shardy, so if the type name is changed 15:49:51 shardy, not just the yaml describing the type 15:50:10 gfidente: Yes, if you change the type name, we always replace the resource 15:50:26 derekh: i'm thinking let's try to fix puppet-nova and pin puppet-heat. If we pin puppet-nova, we'll have a two way dependency on unpinning it later anyway. (One patch to tripleo-common and one to t-h-t.) 15:50:29 I'm suggesting we investigate special-casing that for StackResource objects, e.g anything backed by a nested stack 15:50:32 derekh: i'm on it 15:51:03 gfidente: That would mean we have the same end-state, but any existing resources would remain, if they exist in both the old and new nested stack definition 15:51:08 shardy, I was hoping to get that sorted with the douple mapping, where type name changes but the registry maps the new type name to the new type name? 15:51:21 shardy: I would guess that it will be messy, but feasible, and I agree that it'd be a good thing to do 15:51:21 gfidente: I thought you said that didn't work? 15:51:29 shardy, yes it didn't 15:51:44 gfidente: Ok, well this is my alternative idea ;) 15:52:10 shardy, ok but my scenario is as follows 15:52:15 jistr: ok 15:52:28 1. we have a resource names "A" defined by x.yaml 15:53:19 2. to make it possible to customize the resource, we change it name to "B" so from the registry we can point it to either x.yaml or y.yaml 15:53:30 *we change its TYPE name 15:54:20 3. we do an upgrade where "A" still points at x.yaml and "B" also points at x.yaml 15:54:32 how do we avoid the resource now typed by B from being recreated 15:54:34 Florian Fuchs proposed openstack/tripleo-common: [WIP]Adds deployment configuration methods https://review.openstack.org/230315 15:54:34 ? 15:54:36 gfidente: Yeah, that's exactly what I've described in the bug (or at least tried to) 15:54:49 then it was me missing it 15:54:56 gfidente: current behaviour is we completely delete "A" then create a new stack "B" 15:55:11 gfidente: proposed behavior, special-case inside heat so we update "A" and call it "B" 15:55:33 oh but we still have other resources typed by A in the new stack though 15:55:38 is that acceptable? 15:55:48 e.g the underlying nested stack is updated instead of replaced 15:55:51 gfidente: yes 15:56:22 because in the scenario we mainly attempt at driving some of the pre-existing resources from a different type 15:56:44 which can optionally be pointed to the same yaml describing the resource exactly as it was by the previous type 15:57:21 sort of, decouple two resources previously described both by the same type 15:57:55 which now can optionally be described by the same yaml, even though the actual type name is different 15:58:05 Jiri Stransky proposed openstack/tripleo-heat-templates: Fix puppet-nova duplicate declaration https://review.openstack.org/237655 16:01:01 Jiri Stransky proposed openstack/tripleo-common: Pin puppet-heat and puppet-nova https://review.openstack.org/237568 16:01:02 derekh: we need this for CI to pass? https://review.openstack.org/#/c/237568/2 16:01:21 derekh: submitted a patch which should fix the nova problem and added a depends-on ^ 16:01:27 dprince: ^ 16:01:35 jistr: oh, I see. Okay, thanks 16:02:26 shardy, I hope the comment https://bugs.launchpad.net/heat/+bug/1508115/comments/2 makes sense 16:02:26 Launchpad bug 1508115 in heat "StackResource updates can destroy more than is needed" [Undecided,New] 16:02:26 (the nova fix seems easy (hopefully :) ) and we'd need an amendment of the pin patch anyway) 16:03:28 gfidente: I added a worked example based on your comment, let me know if I have correctly understood 16:04:06 dprince: jistr so it looks like EmilienM has a patch for t-h-t to use the new puppet-nova https://review.openstack.org/#/c/225983/ 16:04:06 jistr, I think EmilienM was trying to fix that too https://review.openstack.org/#/c/225983 16:04:23 i see, yeah let's go with his 16:04:51 I was not fixing anything, I was updating a manifest due to a change in puppet-nova 16:05:13 EmilienM, you don't fix anything indeed! ever! 16:05:16 your fault 16:06:06 but passing it to common I think was better so ideally all nova services switch to internalURL 16:06:09 Merged openstack/tripleo-heat-templates: nova: set catalog_info to InternalURL https://review.openstack.org/225983 16:06:33 ok, I'm going to update the pin again to remove puppet-nova 16:07:09 derekh: i removed the pin already, but if you're updating, please remove the depends-on 16:07:28 shardy, ack, sorry but my english isn't great, the sample is what I was thinking as well! 16:07:59 gfidente: no worries, I should have added a more verbose example in the initial report, thanks! 16:08:18 I'll try to poke at the code later and figure out how it might actually work 16:08:44 jistr: ack, doing it now 16:08:46 Derek Higgins proposed openstack/tripleo-common: Pin puppet-heat and puppet-nova https://review.openstack.org/237568 16:09:50 Derek Higgins proposed openstack/tripleo-common: Pin puppet-heat https://review.openstack.org/237568 16:10:19 URGENT TRIPLEO TASKS NEED ATTENTION 16:10:19 +-------------------------------------+-------------------------------+-------------------+-------------+ 16:10:19 | Title | URL | Members | Last Active | 16:10:19 +-------------------------------------+-------------------------------+-------------------+-------------+ 16:10:19 | Could not find resource 'Anchor[hea | https://trello.com/c/dWWL3dnT | **NEEDS MEMBERS** | 451 min | 16:10:20 | Error: Duplicate declaration: Nova_ | https://trello.com/c/4zjL5tNt | **NEEDS MEMBERS** | 45 min | 16:10:20 +-------------------------------------+-------------------------------+-------------------+-------------+ 16:12:40 this being hardcoded to kilo is killing me :( https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/v1/overcloud_image.py#L420 16:13:07 derekh: thanks! 16:13:54 jistr: thank you too, 16:27:04 Derek Higgins proposed openstack/diskimage-builder: Use DIB_EPEL_MIRROR when finding the epel-release package https://review.openstack.org/237676 16:27:24 Zane Bitter proposed openstack/tripleo-docs: Fix typo https://review.openstack.org/237678 16:40:41 gfidente: Hey dude 16:40:43 you there? 16:40:49 jaosorior, pong so and so 16:43:30 gfidente: hey, so why had you said that we cannot use https://review.openstack.org/#/c/222189/ as is? 16:43:56 because I didn't notice you added the http mapping 16:44:04 I amended that we can't use it as-is 16:44:13 lucasagomes, ^^ ? 16:45:31 gfidente: hi! thanks 16:45:35 yeah we may give that a go tomorrow or so 16:45:56 we are currently on site but we are currently trying out the linux-bridge stuff 16:46:01 TLS will come next :-) 16:46:10 lucasagomes, on-site where? 16:46:15 oh wai 16:46:16 wait 16:46:22 * lucasagomes pms you 16:57:26 gfidente: anyway, if you have time, can you review the patches that the TLS patch depends on? :D 16:57:29 Gael Chamoulaud proposed openstack/tripleo-heat-templates: Make puppet manifests compliant with Puppet 4.x https://review.openstack.org/225664 16:57:59 jaosorior, yeah we should merge those 17:10:14 URGENT TRIPLEO TASKS NEED ATTENTION 17:10:14 +-------------------------------------+-------------------------------+-------------------+-------------+ 17:10:14 | Title | URL | Members | Last Active | 17:10:14 +-------------------------------------+-------------------------------+-------------------+-------------+ 17:10:14 | Could not find resource 'Anchor[hea | https://trello.com/c/dWWL3dnT | **NEEDS MEMBERS** | 511 min | 17:10:15 +-------------------------------------+-------------------------------+-------------------+-------------+ 17:19:26 Anybody with cloud-init knowledge around? 17:31:16 v1af31 17:31:28 that is not a password 17:31:32 in case you wonder 17:33:01 gfidente lol 18:10:16 URGENT TRIPLEO TASKS NEED ATTENTION 18:10:16 +-------------------------------------+-------------------------------+-------------------+-------------+ 18:10:16 | Title | URL | Members | Last Active | 18:10:16 +-------------------------------------+-------------------------------+-------------------+-------------+ 18:10:16 | Could not find resource 'Anchor[hea | https://trello.com/c/dWWL3dnT | **NEEDS MEMBERS** | 571 min | 18:10:17 +-------------------------------------+-------------------------------+-------------------+-------------+ 18:11:55 dprince: since you're core on packaging, can you look https://github.com/openstack-packages/aodh/pull/1 please , 19:02:12 Lokesh Jain proposed openstack/tripleo-heat-templates: Changes for configuring Nuage https://review.openstack.org/230116 19:10:14 URGENT TRIPLEO TASKS NEED ATTENTION 19:10:14 +-------------------------------------+-------------------------------+-------------------+-------------+ 19:10:14 | Title | URL | Members | Last Active | 19:10:14 +-------------------------------------+-------------------------------+-------------------+-------------+ 19:10:14 | Could not find resource 'Anchor[hea | https://trello.com/c/dWWL3dnT | **NEEDS MEMBERS** | 631 min | 19:10:15 +-------------------------------------+-------------------------------+-------------------+-------------+ 19:14:49 Ryan Brady proposed openstack/tripleo-common: [WIP] Initial workflow implementation of tripleo-common https://review.openstack.org/227297 19:21:02 dprince: unping about aodh 19:22:19 EmilienM: sorry, whats up? 19:22:37 dprince: I ping'ed you earlier but nevermind :-) 19:23:04 EmilienM: sorry I missed it. I moved and briefly dropped IRC... 19:23:10 np ! 19:23:39 EmilienM: hey, where do we stand on https://review.openstack.org/#/c/237568/ 19:23:48 EmilienM: is there a roll forward we could use instead? 19:24:29 dprince: let's take it on #puppet-openstack 19:24:37 EmilienM: cool 19:26:22 Has anybody ever seen this kind of output from cloud-init http://pastebin.com/jkrGk1Zy ? 19:56:23 Lokesh Jain proposed openstack/tripleo-heat-templates: Changes for configuring Nuage https://review.openstack.org/230116 20:10:17 URGENT TRIPLEO TASKS NEED ATTENTION 20:10:17 +-------------------------------------+-------------------------------+-------------------+-------------+ 20:10:17 | Title | URL | Members | Last Active | 20:10:17 +-------------------------------------+-------------------------------+-------------------+-------------+ 20:10:17 | Could not find resource 'Anchor[hea | https://trello.com/c/dWWL3dnT | **NEEDS MEMBERS** | 691 min | 20:10:18 +-------------------------------------+-------------------------------+-------------------+-------------+ 20:10:38 Lokesh Jain proposed openstack/tripleo-heat-templates: Changes for configuring Nuage https://review.openstack.org/230116 21:00:16 Untriaged bugs so far: 21:00:17 https://bugs.launchpad.net/tripleo/+bug/1505815 21:00:17 Launchpad bug 1505815 in tripleo "Docs refer to ironic-discoverd instead of ironic-inspector" [Undecided,Confirmed] - Assigned to Ruchi Parmar (ruchiparmar143) 21:00:19 https://bugs.launchpad.net/tripleo/+bug/1507556 21:00:19 Launchpad bug 1507556 in tripleo "[tripleoclient] should ignore validation of tunneling params if tunneling is disabled" [Undecided,In progress] - Assigned to Brad P. Crochet (brad-9) 21:00:20 https://bugs.launchpad.net/diskimage-builder/+bug/1507631 21:00:21 Launchpad bug 1507631 in diskimage-builder "Diskimage-builder (DIB) produces a non-bootable RHEL 6.7 QCOW2-image" [Undecided,New] 21:00:23 https://bugs.launchpad.net/diskimage-builder/+bug/1506792 21:00:23 Launchpad bug 1506792 in diskimage-builder "ironic-python-agent fails to run on fedora ramdisk" [Undecided,In progress] - Assigned to Lucas Alvares Gomes (lucasagomes) 21:00:24 https://bugs.launchpad.net/diskimage-builder/+bug/1506692 21:00:25 Launchpad bug 1506692 in diskimage-builder "Broken test in fedora-minimal/install.d/99-ramdisk" [Undecided,In progress] - Assigned to Martin André (mandre) 21:00:26 https://bugs.launchpad.net/diskimage-builder/+bug/1505900 21:00:27 Launchpad bug 1505900 in diskimage-builder "Failed to create 'wily' release with DIB_RELEASE" [Undecided,New] 21:00:29 https://bugs.launchpad.net/tripleo/+bug/1504263 21:00:29 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 21:00:30 https://bugs.launchpad.net/tripleo/+bug/1507930 21:00:30 Launchpad bug 1507930 in tripleo "tripleo-docs instackenv.json content" [Undecided,New] 21:00:32 https://bugs.launchpad.net/diskimage-builder/+bug/1506275 21:00:32 Launchpad bug 1506275 in diskimage-builder "$DIB_OFFLINE and $DIB_DEBIAN_USE_DEBOOTSTRAP_CACHE are redundant" [Undecided,New] 21:00:35 https://bugs.launchpad.net/tripleo/+bug/1507054 21:00:35 Launchpad bug 1507054 in tripleo "Allow configuration of mysql percona sst through xtrabackup-v2" [Undecided,New] 21:10:14 URGENT TRIPLEO TASKS NEED ATTENTION 21:10:14 +-------------------------------------+-------------------------------+-------------------+-------------+ 21:10:14 | Title | URL | Members | Last Active | 21:10:14 +-------------------------------------+-------------------------------+-------------------+-------------+ 21:10:14 | Could not find resource 'Anchor[hea | https://trello.com/c/dWWL3dnT | **NEEDS MEMBERS** | 751 min | 21:10:14 Ryan Brady proposed openstack/tripleo-common: [WIP] Initial workflow implementation of tripleo-common https://review.openstack.org/227297 21:10:15 +-------------------------------------+-------------------------------+-------------------+-------------+ 21:32:13 Ryan Brady proposed openstack/tripleo-common: [WIP] Initial workflow implementation of tripleo-common https://review.openstack.org/227297 21:32:33 Merged openstack/tripleo-heat-templates: Update overcloud template description and Count constraints https://review.openstack.org/232084 21:38:05 Merged openstack/tripleo-heat-templates: Allow ctlplane network or ID, and default to "ctlplane" https://review.openstack.org/232101 21:43:34 Ben Nemec proposed openstack/python-tripleoclient: Don't log exceptions redundantly https://review.openstack.org/237097 22:03:55 Ben Nemec proposed openstack/python-tripleoclient: Don't raise bare Exceptions https://review.openstack.org/237790 22:10:20 URGENT TRIPLEO TASKS NEED ATTENTION 22:10:21 +-------------------------------------+-------------------------------+-------------------+-------------+ 22:10:21 | Title | URL | Members | Last Active | 22:10:21 +-------------------------------------+-------------------------------+-------------------+-------------+ 22:10:21 | Could not find resource 'Anchor[hea | https://trello.com/c/dWWL3dnT | **NEEDS MEMBERS** | 811 min | 22:10:21 +-------------------------------------+-------------------------------+-------------------+-------------+ 22:13:37 OpenStack Proposal Bot proposed openstack/diskimage-builder: Updated from global requirements https://review.openstack.org/237351 22:15:10 Merged openstack/tripleo-docs: Fix typo https://review.openstack.org/237678 22:15:49 Merged openstack/diskimage-builder: Fix tests/test_functions.bash https://review.openstack.org/229334 22:57:10 Merged openstack/tripleo-common: Pin puppet-heat https://review.openstack.org/237568 23:10:22 URGENT TRIPLEO TASKS NEED ATTENTION 23:10:22 +-------------------------------------+-------------------------------+-------------------+-------------+ 23:10:22 | Title | URL | Members | Last Active | 23:10:22 +-------------------------------------+-------------------------------+-------------------+-------------+ 23:10:22 | Could not find resource 'Anchor[hea | https://trello.com/c/dWWL3dnT | **NEEDS MEMBERS** | 871 min | 23:10:23 +-------------------------------------+-------------------------------+-------------------+-------------+ 23:30:21 Dan Prince proposed openstack/tripleo-heat-templates: Support for composable roles https://review.openstack.org/236243 23:30:21 Dan Prince proposed openstack/tripleo-heat-templates: Create Glance API and Glance Registry Roles https://review.openstack.org/237370 23:59:30 Lucas Alvares Gomes proposed openstack/diskimage-builder: Install IPA in a virtual enviroment https://review.openstack.org/235843 00:10:15 URGENT TRIPLEO TASKS NEED ATTENTION 00:10:15 +-------------------------------------+-------------------------------+-------------------+-------------+ 00:10:15 | Title | URL | Members | Last Active | 00:10:15 +-------------------------------------+-------------------------------+-------------------+-------------+ 00:10:15 | Could not find resource 'Anchor[hea | https://trello.com/c/dWWL3dnT | **NEEDS MEMBERS** | 931 min | 00:10:16 +-------------------------------------+-------------------------------+-------------------+-------------+ 01:10:18 URGENT TRIPLEO TASKS NEED ATTENTION 01:10:18 +-------------------------------------+-------------------------------+-------------------+-------------+ 01:10:18 | Title | URL | Members | Last Active | 01:10:18 +-------------------------------------+-------------------------------+-------------------+-------------+ 01:10:18 | Could not find resource 'Anchor[hea | https://trello.com/c/dWWL3dnT | **NEEDS MEMBERS** | 991 min | 01:10:19 +-------------------------------------+-------------------------------+-------------------+-------------+ 02:10:14 URGENT TRIPLEO TASKS NEED ATTENTION 02:10:14 +-------------------------------------+-------------------------------+-------------------+-------------+ 02:10:14 | Title | URL | Members | Last Active | 02:10:14 +-------------------------------------+-------------------------------+-------------------+-------------+ 02:10:14 | Could not find resource 'Anchor[hea | https://trello.com/c/dWWL3dnT | **NEEDS MEMBERS** | 1051 min | 02:10:15 +-------------------------------------+-------------------------------+-------------------+-------------+ 02:28:22 OpenStack Proposal Bot proposed openstack/diskimage-builder: Updated from global requirements https://review.openstack.org/237351 02:33:50 OpenStack Proposal Bot proposed openstack/tripleo-common: Updated from global requirements https://review.openstack.org/232282 03:00:14 Untriaged bugs so far: 03:00:15 https://bugs.launchpad.net/tripleo/+bug/1505815 03:00:15 Launchpad bug 1505815 in tripleo "Docs refer to ironic-discoverd instead of ironic-inspector" [Undecided,Confirmed] - Assigned to Ruchi Parmar (ruchiparmar143) 03:00:18 https://bugs.launchpad.net/tripleo/+bug/1507556 03:00:18 Launchpad bug 1507556 in tripleo "[tripleoclient] should ignore validation of tunneling params if tunneling is disabled" [Undecided,In progress] - Assigned to Brad P. Crochet (brad-9) 03:00:19 https://bugs.launchpad.net/diskimage-builder/+bug/1507631 03:00:20 Launchpad bug 1507631 in diskimage-builder "Diskimage-builder (DIB) produces a non-bootable RHEL 6.7 QCOW2-image" [Undecided,New] 03:00:21 https://bugs.launchpad.net/diskimage-builder/+bug/1506792 03:00:21 Launchpad bug 1506792 in diskimage-builder "ironic-python-agent fails to run on fedora ramdisk" [Undecided,In progress] - Assigned to Lucas Alvares Gomes (lucasagomes) 03:00:24 https://bugs.launchpad.net/diskimage-builder/+bug/1506692 03:00:24 Launchpad bug 1506692 in diskimage-builder "Broken test in fedora-minimal/install.d/99-ramdisk" [Undecided,In progress] - Assigned to Martin André (mandre) 03:00:25 https://bugs.launchpad.net/diskimage-builder/+bug/1505900 03:00:25 Launchpad bug 1505900 in diskimage-builder "Failed to create 'wily' release with DIB_RELEASE" [Undecided,New] 03:00:27 https://bugs.launchpad.net/tripleo/+bug/1504263 03:00:27 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 03:00:29 https://bugs.launchpad.net/diskimage-builder/+bug/1506275 03:00:29 Launchpad bug 1506275 in diskimage-builder "$DIB_OFFLINE and $DIB_DEBIAN_USE_DEBOOTSTRAP_CACHE are redundant" [Undecided,New] 03:00:31 https://bugs.launchpad.net/tripleo/+bug/1507054 03:00:31 Launchpad bug 1507054 in tripleo "Allow configuration of mysql percona sst through xtrabackup-v2" [Undecided,New] 03:10:16 URGENT TRIPLEO TASKS NEED ATTENTION 03:10:16 +-------------------------------------+-------------------------------+-------------------+-------------+ 03:10:16 | Title | URL | Members | Last Active | 03:10:16 +-------------------------------------+-------------------------------+-------------------+-------------+ 03:10:16 | Could not find resource 'Anchor[hea | https://trello.com/c/dWWL3dnT | **NEEDS MEMBERS** | 1111 min | 03:10:17 +-------------------------------------+-------------------------------+-------------------+-------------+ 04:10:16 URGENT TRIPLEO TASKS NEED ATTENTION 04:10:16 +-------------------------------------+-------------------------------+-------------------+-------------+ 04:10:16 | Title | URL | Members | Last Active | 04:10:16 +-------------------------------------+-------------------------------+-------------------+-------------+ 04:10:16 | Could not find resource 'Anchor[hea | https://trello.com/c/dWWL3dnT | **NEEDS MEMBERS** | 1171 min | 04:10:17 +-------------------------------------+-------------------------------+-------------------+-------------+ 05:10:18 URGENT TRIPLEO TASKS NEED ATTENTION 05:10:18 +-------------------------------------+-------------------------------+-------------------+-------------+ 05:10:18 | Title | URL | Members | Last Active | 05:10:18 +-------------------------------------+-------------------------------+-------------------+-------------+ 05:10:18 | Could not find resource 'Anchor[hea | https://trello.com/c/dWWL3dnT | **NEEDS MEMBERS** | 1231 min | 05:10:19 +-------------------------------------+-------------------------------+-------------------+-------------+ 05:59:52 spredzy: Are you around? 06:00:53 jaosorior, yup 06:10:18 URGENT TRIPLEO TASKS NEED ATTENTION 06:10:18 +-------------------------------------+-------------------------------+-------------------+-------------+ 06:10:18 | Title | URL | Members | Last Active | 06:10:18 +-------------------------------------+-------------------------------+-------------------+-------------+ 06:10:18 | Could not find resource 'Anchor[hea | https://trello.com/c/dWWL3dnT | **NEEDS MEMBERS** | 1291 min | 06:10:19 +-------------------------------------+-------------------------------+-------------------+-------------+ 06:53:55 Merged openstack/python-tripleoclient: Set --min-tmpfs for overcloud-full build https://review.openstack.org/234860 06:54:55 Merged openstack/python-tripleoclient: Consolidate module constants. https://review.openstack.org/225245 06:59:06 Is this the error? http://logs.openstack.org/20/221720/4/check-tripleo/gate-tripleo-ci-ironic-overcloud-f21puppet-nonha/15d10d2/console.html#_2015-10-20_14_49_13_814 07:01:10 Merged openstack/python-tripleoclient: Move more functions to utils.py https://review.openstack.org/225246 07:05:06 d0ugald0ugal: I recently added that library to python-tripleoclient 07:05:12 d0ugal 07:05:25 jaosorior: yup, I remember :) 07:05:31 that dependency should have been gotten from the requirements.txt 07:05:35 or was there somewhere else to update? 07:05:52 jaosorior: As far as I know that is the only place for CI 07:06:43 funky 07:06:46 I have no idea why CI is failing for https://review.openstack.org/#/c/221720 :) 07:07:15 but I do remember we had a similar change before that landed and was reverted as it caused issues 07:07:22 so maybe that was never fully resolved 07:07:29 but I can't find any other error in the log 07:07:39 oh, maybe if I rebase the review... 07:07:42 Dougal Matthews proposed openstack/python-tripleoclient: Disable command run under root user https://review.openstack.org/221720 07:09:20 that's quite strange... 07:09:39 d0ugal: How is the tripleoclient installed in the ci? 07:12:32 jaosorior: I don't really know, but if we need passlib to be packaged in rpms that could be the issue 07:12:34 Merged openstack/python-tripleoclient: Don't download fedora-user as part of --all build https://review.openstack.org/234870 07:15:06 aw :( 07:15:49 d0ugal: Seems to be packaged, at least in CentOS 07:15:58 hmm 07:16:11 I'll wait and ask somebody more familiar with CI errors in a bit :) 07:59:49 Dougal Matthews proposed openstack/python-tripleoclient: [WIP] Update tripleoclient to use the common deploy logic https://review.openstack.org/237920 08:10:01 Anyone aware of HA regressions fixed recently, causing neutron-dhcp-agent to fail to start? 08:10:24 https://bugzilla.redhat.com/show_bug.cgi?id=1273680 has been reported against RDO, trying to figure out if it's something we already fixed upstream 08:10:24 bugzilla.redhat.com bug 1273680 in openstack-tripleo-heat-templates "HA overcloud with network isolation deployment fails" [High,New] - Assigned to tsedovic 08:13:08 shardy: looks like the most recent RDO package. the one from a week ago did not act like that 08:15:16 cmyster: Thanks, that's a useful data-point 08:16:00 shardy: so, there was this https://review.openstack.org/#/c/229466/ which changed the pcmk constraints for neutron like https://bugs.launchpad.net/tripleo/+bug/1501378 BUT it looks like the trace there is an actual issue in neutron dhcp agent/ovs 08:16:00 Launchpad bug 1501378 in tripleo "pacemaker neutron resource constraints and cleanup" [High,In progress] - Assigned to Marios Andreou (marios-b) 08:16:40 marios: Hmm, OK thanks - yeah it looks like either a neutron or RPC issue 08:16:52 it's unfortunate the bug doesn't contain the deployment-show output though :( 08:16:54 shardy: i saw this fly by yesterday, not sure if it is related https://bugs.launchpad.net/neutron/+bug/1494281 (could be by way of, openvswitch-agent doesn't start ---> neither does dhcp/l3) 08:16:54 Launchpad bug 1494281 in neutron liberty "neutron-openvswitch-agent is crashing with "invalid literal for int() with base 10" error" [High,Confirmed] 08:17:36 shardy: personally, on my upstream setup yesterday i had issues with neutron svces (but was chasing sthing else for keystone wsgi) - will cycle back today 08:17:58 shardy: it was strange, heat stack went to create complete, but pcs shows neutron services down 08:19:08 marios: Ok, thanks - I saw a couple of regressions around the end of yesterday but I couldn't recall the exact details as was focussed on other things 08:19:26 yesterday I did stumble upon https://bugs.launchpad.net/neutron/+bug/1494281 08:19:26 Launchpad bug 1494281 in neutron liberty "neutron-openvswitch-agent is crashing with "invalid literal for int() with base 10" error" [High,Confirmed] 08:19:30 marios: I assume that means they started, CREATE_COMPLETE, then crashed? 08:19:34 which marios mentioned 08:20:10 and there is no fix merged upstream yet for that 08:20:17 only workaround 08:20:22 *workarounds 08:20:35 shardy: no, well, after create complete i didn't hang around too long (pulled the stack down to test up the other thing) - could be the services were up then went down will check today 08:20:46 jaosorior: thanks (actually you were probably who i saw it fly by from ;) ) 08:21:02 marios: hahaha yeah, probably it was me :P 08:21:26 but yeah, had to change the ovs database to set the segmentation_id for that port to 0 instead of None 08:22:09 shardy: speaking of which (weak link), is there such a thing as 'map_merge' i can't find it at http://docs.openstack.org/developer/heat/template_guide/hot_spec.html#intrinsic-functions 08:22:29 shardy: i am looking at dprince composable roles https://review.openstack.org/#/c/236243/8/puppet/roles/role_map.yaml 08:24:57 Derek Higgins proposed openstack-infra/tripleo-ci: Use local puppet module in DIB https://review.openstack.org/237929 08:26:04 marios: It's not landed yet, it's a new function dprince wrote for Mitaka heat 08:26:08 * shardy finds link 08:26:45 shardy: thanks 08:26:46 https://review.openstack.org/#/c/236663/ 08:27:30 marios: ^^ We'll have to land that, then ensure tripleo-heat-templates has a stable/liberty branch, then ensure dprince's t-h-t changes land on master with the new mitaka HOT version 08:27:51 shardy: yeah was just thinking this is a dependency then if he uses it 08:28:28 marios: yup, I think it's not an issue when we have the stable branches in place, it just means such a patch would not be allowed as a backport to stable/liberty 08:32:58 happy back to the future day! 08:37:59 Derek Higgins proposed openstack/tripleo-common: Update the repo whitelist to reflect what we CI https://review.openstack.org/229906 08:42:48 Derek Higgins proposed openstack-infra/tripleo-ci: Use local puppet module in DIB https://review.openstack.org/237929 08:43:48 bnemec are you around? 09:00:15 Untriaged bugs so far: 09:00:17 https://bugs.launchpad.net/tripleo/+bug/1505815 09:00:17 Launchpad bug 1505815 in tripleo "Docs refer to ironic-discoverd instead of ironic-inspector" [Undecided,Confirmed] - Assigned to Ruchi Parmar (ruchiparmar143) 09:00:19 https://bugs.launchpad.net/tripleo/+bug/1507556 09:00:19 Launchpad bug 1507556 in tripleo "[tripleoclient] should ignore validation of tunneling params if tunneling is disabled" [Undecided,In progress] - Assigned to Brad P. Crochet (brad-9) 09:00:20 https://bugs.launchpad.net/diskimage-builder/+bug/1507631 09:00:20 Launchpad bug 1507631 in diskimage-builder "Diskimage-builder (DIB) produces a non-bootable RHEL 6.7 QCOW2-image" [Undecided,New] 09:00:22 https://bugs.launchpad.net/diskimage-builder/+bug/1506792 09:00:24 https://bugs.launchpad.net/diskimage-builder/+bug/1506692 09:00:24 Launchpad bug 1506792 in diskimage-builder "ironic-python-agent fails to run on fedora ramdisk" [Undecided,In progress] - Assigned to Lucas Alvares Gomes (lucasagomes) 09:00:25 Launchpad bug 1506692 in diskimage-builder "Broken test in fedora-minimal/install.d/99-ramdisk" [Undecided,In progress] - Assigned to Martin André (mandre) 09:00:26 https://bugs.launchpad.net/diskimage-builder/+bug/1505900 09:00:27 Launchpad bug 1505900 in diskimage-builder "Failed to create 'wily' release with DIB_RELEASE" [Undecided,New] 09:00:28 https://bugs.launchpad.net/tripleo/+bug/1504263 09:00:28 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 09:00:30 https://bugs.launchpad.net/diskimage-builder/+bug/1506275 09:00:31 Launchpad bug 1506275 in diskimage-builder "$DIB_OFFLINE and $DIB_DEBIAN_USE_DEBOOTSTRAP_CACHE are redundant" [Undecided,New] 09:00:32 https://bugs.launchpad.net/tripleo/+bug/1507054 09:00:32 Launchpad bug 1507054 in tripleo "Allow configuration of mysql percona sst through xtrabackup-v2" [Undecided,New] 09:21:32 Anybody got questions about CI, put them here and I'll try and answer them https://etherpad.openstack.org/p/tripleo-ci-faq 09:23:40 Derek Higgins proposed openstack-infra/tripleo-ci: Use local puppet module in DIB https://review.openstack.org/237929 09:24:42 derekh, https://review.openstack.org/#/c/237592/ seems to have worked for Heat issues 09:25:17 CI job fails due to Command: ['ovs-vsctl', '--timeout=10', '--oneline', '--format=json', '--', '--columns=type', 'list', 'Interface', 'int-br-ex'] 09:25:37 but not related to heat anymore, hopefully will be merged later today 09:29:34 spredzy: cool, thanks, CI failed because of "Nova_config[cinder/catalog_info] is already declared in file " https://bugs.launchpad.net/tripleo/+bug/1508101 09:29:34 Launchpad bug 1508101 in tripleo "Error: Duplicate declaration: Nova_config[cinder/catalog_info" [Critical,Fix released] - Assigned to Jiří Stránský (jistr) 09:29:45 spredzy: but thats no longer an issue 09:29:58 Derek Higgins proposed openstack/tripleo-common: Remove puppet-heat pin https://review.openstack.org/237959 09:30:19 spredzy: if that ^^ passes then we're good 09:31:09 Derek Higgins proposed openstack/tripleo-common: Remove puppet-heat pin https://review.openstack.org/237959 09:32:08 derekh, if you remove the pin to puppet-heat it will still fail, as the fix hasn't been merged upstream 09:32:15 The anchor issue remains relevant 09:32:26 spredzy: that why it has a depends-on 09:32:45 derekh, my bad haven't fully read the commit message 09:32:54 ack then if it passes we're indeed good 09:46:31 derekh: Hey, question re CI and tripleo.sh 09:46:35 https://review.openstack.org/#/c/235575/2/doc/source/basic_deployment/basic_deployment_cli.rst 09:46:51 In there, it looks like the exports for REPO/FILE are aligned with tripleo.sh 09:47:02 but I don't see USE_DELOREAN_TRUNK anywhere? 09:47:09 * shardy probably just needs more coffee 09:48:40 tripleoclient seems to default that to 0, so I wanted to ensure the docs would be consistent with what we're CI'ing 09:50:50 shardy: looking, but there probably is differences we are ci'ing with 2 extra repositories, these shouldn't be part of the tripleo deployment docs 10:01:24 So the docs are using this element to install the delorean repo http://git.openstack.org/cgit/openstack/tripleo-image-elements/tree/elements/delorean-repo/pre-install.d/01-delorean-repo 10:02:06 tripleo.sh needs another repository so uses DIB_YUM_REPO_CONF http://git.openstack.org/cgit/openstack/tripleo-common/tree/scripts/tripleo.sh#n308 10:02:48 shardy: and ci needs another repo so adds a ci repository here http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/toci_instack.sh#n94 10:03:49 shardy: I'm goinna summarize it here https://etherpad.openstack.org/p/tripleo-ci-faq 10:04:10 derekh: Ok, thanks! 10:30:39 Juan Antonio Osorio Robles proposed openstack/puppet-tripleo: Resolve repeated ports for ssl frontends https://review.openstack.org/237974 10:40:32 shardy: thats how I understand it at the moment, does it make sense ? https://etherpad.openstack.org/p/tripleo-ci-faq 10:40:36 Marek Aufart proposed openstack/tripleo-common: [WIP]Add resource capabilities https://review.openstack.org/232534 10:42:22 Lucas Alvares Gomes proposed openstack/diskimage-builder: Install IPA in a virtual enviroment https://review.openstack.org/235843 10:43:33 derekh: yup, that's great, thanks for clarifying 10:43:47 that looks like excellent content to refactor into a docs patch at some point 10:44:01 shardy: yup 10:54:27 Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Enable TLS deployment for the Overcloud https://review.openstack.org/231930 11:19:41 Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Enable TLS deployment for the Overcloud https://review.openstack.org/231930 11:20:53 Derek Higgins proposed openstack-infra/tripleo-ci: Add ci logs helper script https://review.openstack.org/237992 11:40:48 Another fix from some intermittent failures https://review.openstack.org/#/c/237676/ 11:53:29 Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: external loadbalancer: Add net_vip_map_external https://review.openstack.org/222189 11:53:29 Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Make load balancer deployment optional via template param https://review.openstack.org/222305 11:53:30 Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Allow for usage of pre-allocated IPs for the Controller nodes https://review.openstack.org/222497 12:10:51 Gael Chamoulaud proposed openstack/tripleo-puppet-elements: Gnocchi Integration https://review.openstack.org/238013 12:18:58 Lennart Regebro proposed openstack/python-tripleoclient: Make root directory configurable https://review.openstack.org/230275 12:18:58 Lennart Regebro proposed openstack/python-tripleoclient: Move the PKI initialization into post-config https://review.openstack.org/238016 12:35:40 uuuh...the CI is throwing a bunch of NOT_REGISTERED :/ 12:36:30 yeah everything in check-tripleo queue in zuul just disappeared 12:49:32 oh NOES 12:51:32 indeed :/ 12:53:33 Hmm the project-config change to adjust acl's for stable branches just landed, I wonder if that resets the zuul queues? 12:55:08 relatedly - trown, do you have the list of git sha's you used for the rdo-manger branches handy? 12:55:44 trown: also I was wondering if you'd hacked on tripleo.sh yet to get that working with the rdo-manager repos 12:56:36 shardy: I can get them shortly but they are all in the openstack-packages repos like https://github.com/openstack-packages/python-tripleoclient/blob/rdo-liberty/python-tripleoclient.spec#L1 12:57:17 shardy: I have not really looked at tripleo.sh much, it is on my list to include a CI job for RDO that uses it 12:57:20 trown: Ok no worries, I can use those, just wondered if you had a cut/pasteable list somewhere ;) 12:57:45 trown: Ok, thanks, I'll create the branches and hack on tripleo.sh to consume them 12:57:55 woot woot, thanks 13:06:15 Lennart Regebro proposed openstack/python-tripleoclient: ove the PKI initialization into post-config https://review.openstack.org/238016 13:12:14 Martin Mágr proposed openstack/tripleo-heat-templates: Ceph cluster validation https://review.openstack.org/225248 13:26:27 Gonéri Le Bouder proposed openstack/instack-undercloud: use full path to call the ip command https://review.openstack.org/238069 13:26:41 Florian Fuchs proposed openstack/tripleo-common: [WIP]Adds deployment configuration methods https://review.openstack.org/230315 13:27:21 Brad P. Crochet proposed openstack/python-tripleoclient: Move the PKI initialization into post-config https://review.openstack.org/238016 13:32:17 Marios Andreou proposed openstack/tripleo-common: Adds the ctlplane subnet update for nameserver to tripleo.sh https://review.openstack.org/238072 13:34:38 slagle: sorry if you already considered and decided not to... ^^^ but imo is useful since tripleo.sh is for quick dev setup right? 13:34:57 slagle: (adds ctlplane dns config setup to tripleo.sh) 13:35:06 Hi geys the ci status script no needs this https://review.openstack.org/#/c/235420 13:37:30 Merged openstack-infra/tripleo-ci: Removing cellspacing from the tripleo ci report https://review.openstack.org/235419 13:37:52 Merged openstack-infra/tripleo-ci: Use the new shortened tripleo ci job names https://review.openstack.org/235420 13:38:28 gfidente: shardy thanks 13:40:24 gfidente: ping wrt https://review.openstack.org/#/c/235484/1 13:41:14 gfidente: lgtm and green ci. so this just sets directoy on horizon::vhost_extra_params 13:41:28 (directly) 13:42:32 Merged openstack/tripleo-heat-templates: Sync httpd vhost settings in between pcmk and non-pcmk scenarios https://review.openstack.org/235484 13:47:00 marios: i like it, it's probably fine for now. i wonder if we need this as a config option directly in the undercloud installer 13:47:28 we already create the ctlplane network with dhcp start/end, seems reasonable to have another config for the dns server 13:49:45 rhallisey: sorry about yesterday, wanna talk CI ? 13:50:22 derekh, ya sure! 13:50:45 slagle: yeah that makes sense actually 13:51:04 slagle: will look into it 13:51:24 derekh, so the pieces that I think need to happen are 1) CI 2) job that builds and tags 'tested' containers and pushes them 13:51:27 rhallisey: so I havn't looked at the docker stuff pretty much at all, what do you need to have ? 13:52:30 Jiri Stransky proposed openstack/tripleo-heat-templates: [WIP] OpenContrail integration https://review.openstack.org/238097 13:52:35 rhallisey: so at the moment I am creating periodic jobs for the other tripleo jobs https://review.openstack.org/#/c/235321/ 13:53:14 rhallisey: my plan is to have a "mirror" server for cloud images / repositories and build artifacts of the periodic jobs 13:53:28 Jiri Stransky proposed openstack/tripleo-heat-templates: [WIP] OpenContrail integration https://review.openstack.org/238097 13:53:35 rhallisey: then subsequent jobs could make use of them 13:54:01 rhallisey: one we have a working docker job we could add a periodic jobs that pushes its images to the mirror server ? 13:54:15 rhallisey: does that sounds liek ti would work ? 13:55:51 derekh, ya that sounds good 13:56:25 derekh, where should I start with this? 13:56:43 * rhallisey doesn't know anything is this area 13:58:43 rhallisey: first thing you want is a working job, if this was me I would start with a WIP patch to tripleo-ci to get it doing what you want, I can collaborate with you on it there and once its ready we can add an additional job to do it 14:00:26 rhallisey: the new job would be added to this list (http://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul/layout.yaml#n602 ) if we want it everywhere we have the other jobs 14:00:42 rhallisey: or we could run it only for specific repositories 14:01:47 rhallisey: this for loop in tripleo-ci is how we currently switch between different job type http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/toci_gate_test.sh#n25 14:02:15 rhallisey: so we could flick the docker switches there 14:03:15 rhallisey: To get started if you have any general tripleo-ci questions add them here and I'll answer them https://etherpad.openstack.org/p/tripleo-ci-faq 14:03:28 * derekh is trying to start a tripleo-ci FAQ 14:03:30 derekh, ok great 14:04:59 thanks! 14:06:39 Marios Andreou proposed openstack/tripleo-common: Adds the ctlplane subnet update for nameserver to tripleo.sh https://review.openstack.org/238072 14:20:28 Dan Prince proposed openstack/tripleo-heat-templates: Support for composable roles https://review.openstack.org/236243 14:20:28 Dan Prince proposed openstack/tripleo-heat-templates: Create Glance API and Glance Registry Roles https://review.openstack.org/237370 14:35:35 slagle: huh. looks like 'setup-neutron' already has this https://github.com/openstack/os-cloud-config/blob/master/os_cloud_config/neutron.py#L125 - we just aren't setting in the undercloud post-config element 14:36:41 well it's nice it's already in setup-neutron :) 14:36:53 not using it probably wasn't intentional 14:36:56 just lack of awareness 14:37:20 slagle: yeah i went looking with a view to add it so simple fix anyway 14:38:54 slagle: oh man, we already have it 14:39:22 UNDERCLOUD_NAMESERVER 14:39:28 oh we do? 14:39:48 wait, this is sounding familiar now 14:39:53 sec 14:40:10 https://github.com/openstack/instack-undercloud/blob/master/elements/undercloud-post-config/os-refresh-config/post-configure.d/98-undercloud-setup#L115 14:40:12 oh, hah, the support is there in the call to setup-neutron 14:40:20 just not as a top level config in undercloud.conf :) 14:40:43 yeah. :) and am guessing not in docs either ... checking 14:40:48 Merged openstack/os-cloud-config: Add pxe_irmc driver support into os-cloud-config https://review.openstack.org/188717 14:42:22 probably not, there's not really a way to set that env variable 14:42:22 slagle: so in theory, you can just export UNDERCLOUD_NAMESERVER before "openstack undercloud install" 14:42:33 yea i'm not sure that'd work 14:42:37 slagle: or is that oh yeah in the image sorry 14:42:58 we should just make it be set in undercloud.conf. i can fix this if you'd like 14:43:40 slagle: sure whichever. are you going to change the post-config element? 14:43:51 slagle: nm, i will see the review :) 14:45:09 Gael Chamoulaud proposed openstack/tripleo-heat-templates: Add Puppet 4.x lint checks https://review.openstack.org/234150 14:45:09 Gael Chamoulaud proposed openstack/tripleo-heat-templates: Make puppet manifests compliant with Puppet 4.x https://review.openstack.org/225664 14:46:58 hey folks can I have some eyes on https://review.openstack.org/#/c/235843/ ? Creating an IPA ramdisk with fedora fails without that patch 14:50:41 slagle: so does this belong in undercloud.py then as an option there (and undercloud.config.sample)... anyway, lemme know if you get tied up can poke some more 14:51:11 lucasagomes: looking 14:51:27 marios: yea i think that's where it belongs 14:52:14 Gael Chamoulaud proposed openstack/tripleo-heat-templates: Make puppet manifests compliant with Puppet 4.x https://review.openstack.org/225664 14:52:59 marios: thanks, I'm fixing the things dmitry pointed out too 15:00:15 Untriaged bugs so far: 15:00:17 https://bugs.launchpad.net/tripleo/+bug/1505815 15:00:17 Launchpad bug 1505815 in tripleo "Docs refer to ironic-discoverd instead of ironic-inspector" [Undecided,Confirmed] - Assigned to Ruchi Parmar (ruchiparmar143) 15:00:20 https://bugs.launchpad.net/tripleo/+bug/1507556 15:00:21 Launchpad bug 1507556 in tripleo "[tripleoclient] should ignore validation of tunneling params if tunneling is disabled" [Undecided,In progress] - Assigned to Brad P. Crochet (brad-9) 15:00:23 https://bugs.launchpad.net/diskimage-builder/+bug/1507631 15:00:24 https://bugs.launchpad.net/diskimage-builder/+bug/1506792 15:00:24 Launchpad bug 1507631 in diskimage-builder "Diskimage-builder (DIB) produces a non-bootable RHEL 6.7 QCOW2-image" [Undecided,New] 15:00:26 https://bugs.launchpad.net/diskimage-builder/+bug/1506692 15:00:26 Launchpad bug 1506792 in diskimage-builder "ironic-python-agent fails to run on fedora ramdisk" [Undecided,In progress] - Assigned to Lucas Alvares Gomes (lucasagomes) 15:00:26 https://bugs.launchpad.net/diskimage-builder/+bug/1505900 15:00:27 Launchpad bug 1506692 in diskimage-builder "Broken test in fedora-minimal/install.d/99-ramdisk" [Undecided,In progress] - Assigned to Martin André (mandre) 15:00:28 https://bugs.launchpad.net/tripleo/+bug/1504263 15:00:29 Launchpad bug 1505900 in diskimage-builder "Failed to create 'wily' release with DIB_RELEASE" [Undecided,New] 15:00:30 https://bugs.launchpad.net/diskimage-builder/+bug/1506275 15:00:31 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 15:00:32 https://bugs.launchpad.net/tripleo/+bug/1507054 15:00:32 Launchpad bug 1506275 in diskimage-builder "$DIB_OFFLINE and $DIB_DEBIAN_USE_DEBOOTSTRAP_CACHE are redundant" [Undecided,New] 15:00:33 Launchpad bug 1507054 in tripleo "Allow configuration of mysql percona sst through xtrabackup-v2" [Undecided,New] 15:00:54 lucasagomes: one comment 15:01:04 marios: thanks much 15:01:11 heh read it first ;) 15:08:18 Merged openstack/diskimage-builder: Use DIB_EPEL_MIRROR when finding the epel-release package https://review.openstack.org/237676 15:08:54 lucasagomes: just realised source-install/ 15:09:00 o_O 15:14:24 slagle: i'm now looking into putting docker-registry in the undercloud, but i heard that you thought using puppet might be best? 15:15:34 * jpeeler thought a triple-image-element would be where to start? 15:22:11 marios: yeah it's a bit odd to have source/packages there... And even in the source installation there are packages being added/removed in the creation of the image 15:22:27 which was causing it to fail, so the idea of the patch is just install it in a virtual env to avoid those collision 15:23:49 jpeeler: as much as possible should be done in puppet and not bash 15:25:37 slagle: can you point out where you're talking for the puppet? i would have thought that a new package install would simply be a new image element 15:26:33 dtantsur: hello there 15:26:39 o/ 15:27:43 Giulio Fidente proposed openstack/tripleo-heat-templates: Ensure ping returns after 5mins timeout https://review.openstack.org/238137 15:28:27 jistr, bnemec, marios ^^ I am not sure if 5mins is a good interval there, but the goal seems valid to 15:28:49 dtantsur: so, is that expected to have 2 mac addresses in ironic node-port-list for nodes 15:28:50 I think ukalifon saw the deployment remain stuck because of ping timeout 15:29:21 sasha21, not really, that's why I want to see logs relevant to the processing 15:29:31 gfidente: well maybe 5 mins is even too long imo 15:29:46 gfidente: i mean, where are we deploying? south side of the moon? 15:30:30 marios, I think the script is trying to ping all the IPs the controllers get on all the networks 15:31:01 gfidente: but this is per ping 15:31:10 ah right 15:31:22 but as soon as one fails we exit 15:31:39 gfidente: i mean whatever, sure, 5 mins much better than nothing, just saying maybe 5 mins even too long 15:41:47 grateful for reviews https://review.openstack.org/#/c/235278/ (few days old, easy addition, useful imo. at least is for me) 15:42:09 related docs @ https://review.openstack.org/#/c/235332/ 15:43:04 jistr: i think inlunch does/sorts this for you right? (at least when i used that, i got symlinks for /var/lib/libvirt/images 15:45:21 marios: yeah it does 15:45:28 * jistr checks the patches 15:46:28 jistr: ty sir 15:46:57 marios: i think i'll need to add them to queue for tomorrow though :) 15:47:42 jistr: yeah np :) seriously not urgent 15:47:48 jistr: in fact me too 15:47:52 marios, i thought the "default" pool existed by default? 15:48:06 gfidente: yeah does, just points to /var/lib/libvirt/images 15:48:28 marios, so if you point target to another location, given pool=default, what is going to happen? 15:48:40 gfidente: at that point, isn't defined afairecall 15:48:40 Ryan Hallisey proposed openstack/tripleo-heat-templates: Update docker compute environment to use json config https://review.openstack.org/234313 15:48:40 Ryan Hallisey proposed openstack/tripleo-heat-templates: Add local docker registry support https://review.openstack.org/237071 15:49:04 gfidente: leave that comment and i will confirm/check tomorrow thans 15:49:25 gfidente: actually going running now (well, after i packup office) but yeah. 15:49:26 bai 15:59:13 Ryan Hallisey proposed openstack/tripleo-common: Deploy Docker compute role with tripleo.sh https://review.openstack.org/232853 16:25:14 jpeeler: https://github.com/openstack/instack-undercloud/blob/master/elements/puppet-stack-config/puppet-stack-config.pp 16:30:24 Gael Chamoulaud proposed openstack/tripleo-heat-templates: Make puppet manifests compliant with Puppet 4.x https://review.openstack.org/225664 16:33:19 Merged openstack/instack-undercloud: puppet: restart mysql if config changes https://review.openstack.org/235584 16:36:38 Merged openstack/tripleo-docs: Update instructions to deploy with Ceph https://review.openstack.org/236140 16:42:10 Lucas Alvares Gomes proposed openstack/diskimage-builder: Install IPA in a virtual enviroment https://review.openstack.org/235843 16:52:02 Dan Prince proposed openstack/tripleo-heat-templates: Create Glance API and Glance Registry Roles https://review.openstack.org/237370 17:01:22 Yanis Guenane proposed openstack/tripleo-heat-templates: Manage keystone related settings directly in t-h-t manifests https://review.openstack.org/230357 17:01:23 Yanis Guenane proposed openstack/tripleo-heat-templates: Manage keystone related settings directly in t-h-t manifests https://review.openstack.org/230375 17:02:33 Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Enable TLS deployment for the Overcloud https://review.openstack.org/231930 17:03:24 spredzy: Were the changes you did to the keystone stuff only rebases? 17:04:28 jaosorior, nop, they were a slight addition on the _pacemaker.pp https://review.openstack.org/#/c/230375/25..26/puppet/manifests/overcloud_controller_pacemaker.pp,cm (just look at the -> - arrows - not the rest) 17:05:35 this was done in order to fix HA failture in CI 17:07:21 jaosorior, thanks for the reviews :) 17:08:20 Lucas Alvares Gomes proposed openstack/diskimage-builder: Install IPA in a virtual enviroment https://review.openstack.org/235843 17:08:29 spredzy: No prob Mr. 17:27:24 trown|lunch: FYI I cut the stable/liberty branches for all repos except docs 17:27:28 https://etherpad.openstack.org/p/tripleo-stable-branches 17:28:00 I started making some notes there about the next steps - I'll start on the tripleo.sh stuff tomorrow and chat with derekh about the CI integration 17:28:57 trown|lunch: I'm not familiar with (3), e.g how we update RDO delorean to point to the correct branches 17:29:06 any pointers there would be great :) 17:29:18 shardy: I was just about to say I can handle that part :) 17:29:26 trown: great, thanks! :) 17:29:48 shardy: I think it may be automatic, with fall back to master if stable/liberty does not exist, but I will confirm 17:30:07 dprince: ^^ In there I link to the current tripleo-release and tripleo-core groups, looks like both could use a cleanup 17:30:43 trown: Ok, great, feel free to add notes to the etherpad if as needed 17:33:51 shardy: agree 17:34:14 shardy: perhaps lets discuss this next week and follow up w/ a list thread soon thereafter 17:34:36 dprince: Yup, sounds good 17:34:53 the group cleanup seemed like a PTL-ish activity so I'll leave that to you ;) 17:35:12 I'll be posting to the ML about the stable branches soon (probably tomorrow) after I get things initially working 17:36:39 shardy: I confirmed on #rdo that the 3 should happen automagically 17:36:54 shardy: hi, so besides the HA job (which I think I just fixed)... CI is passing on the new composability stuff 17:37:15 shardy: so it works!, but still want to get feedback because this is quite a drastic change 17:38:09 shardy: a spec would be good too, now that we got a general idea for how this could work 17:39:58 slagle: I'm going to bail on this one I think https://review.openstack.org/#/c/204806/ 17:41:16 slagle: well, I'm mixed on it now. Perhaps still worth doing 17:43:41 dprince: great! I'll check it out - yeah a spec is probably a good plan, if nothing else it'll help communicate the direction 17:43:52 we also need to look into the upgrade implications 17:44:23 shardy: yeah, I'd like to solve that once and for all 17:44:41 although I've been looking into fixes for https://bugs.launchpad.net/heat/+bug/1508115 which may make major nested stack rework less destructive than it currently is :) 17:44:41 Launchpad bug 1508115 in heat "StackResource updates can destroy more than is needed" [High,Triaged] - Assigned to Steven Hardy (shardy) 17:45:53 Yeah, heat should always make a best effort to not destroy things, and when resources move between stacks, we should be able to handle that via either abandon/adopt migration or external_resource references 17:46:04 So, a few things to think about on the heat side 17:46:11 shardy: cool, yeah I'd just have to try it 17:46:22 dprince: ok, wfm. my only gripe there is the unhandled failure which leads to really bad confusion 17:46:45 trown: thanks for the confirmation! :) 17:47:27 shardy: wtr to abandon/adopt I suppose the 2 stacks idea is a similar mechanism 17:47:43 shardy: stack 1 has all the "Openstack" resources 17:48:06 shardy: things like servers, etc. Which honestly I don't see us needing to change the structure of as much 17:48:25 shardy: stack 2... well it could just go away and come back in an entirely different form 17:48:53 shardy: because stack 2 uses external resources from stack 1 it'd be much les of an upgrade concern I think 17:49:26 shardy: I still like the single 'heat stack-update' however 17:49:32 dprince: Yeah, it's mostly not having a OS::Nova::Server move between stacks, as obviously that's destructive atm 17:49:50 Ben Nemec proposed openstack/tripleo-specs: Simplify the TripleO Workflow https://review.openstack.org/238192 17:50:40 shardy: so perhaps we could consider a "break" (nested stack) for all the configuration related stuff, that we could force heat to drop on occasion 17:51:28 dprince: Yeah, so you create a pool of nodes, grouped by network topology, and apply the config via separate nested stacks? 17:51:36 shardy: then swap in the new version and I think problem solved. I understand this is an odd use case for Heat, but all of our stuff under the covers is idempotent 17:51:39 shardy: exactly 17:51:41 e.g everything becomes a *PostDeploy config 17:51:55 (although maybe we rename it for clarity) 17:58:11 shardy: Yeah, right now we do a bunch of config in the nested stack that creates the server. That is probably okay if we make it all "generic". Ideally the nested stack for the server would do just that, plus network isolation stuff 17:58:35 Ben Nemec proposed openstack/tripleo-specs: Simplify the TripleO Workflow https://review.openstack.org/238192 17:58:56 Simplify?!?!? +2! 17:59:05 shardy: but I'd like all of those service specific parameters to go away in those stacks (puppet/controller.yaml) 17:59:22 slagle: yeah, it'd be simpler too :) 17:59:40 slagle: gotta decompose the massive controller first though 18:01:21 oh i was referencing bnemec's spec :) 18:23:05 Dan Prince proposed openstack/diskimage-builder: Use ironic-agent for source-repositories https://review.openstack.org/233088 19:04:19 Dan Prince proposed openstack/tripleo-heat-templates: Create Glance API and Glance Registry Roles https://review.openstack.org/237370 19:05:14 Ryan Hallisey proposed openstack-infra/tripleo-ci: Add Docker job to tripleo CI https://review.openstack.org/238212 19:37:04 Merged openstack/os-net-config: Support multiple addresses assignment with eni https://review.openstack.org/213903 20:04:58 Steve Baker proposed openstack/python-tripleoclient: Include registry on update when other envs are specified https://review.openstack.org/238231 20:19:15 Jeff Peeler proposed openstack/instack-undercloud: Install local docker registry with puppet https://review.openstack.org/238238 20:22:04 Merged openstack/tripleo-heat-templates: Ensure ping returns after 5mins timeout https://review.openstack.org/238137 20:34:54 Dan Prince proposed openstack/instack-undercloud: Correct ironic auth settings https://review.openstack.org/223126 20:48:23 Dan Prince proposed openstack/tripleo-heat-templates: Add validation to ping the default gateway(s) https://review.openstack.org/204806 20:56:27 Dan Prince proposed openstack/os-collect-config: shorter sleeps if metadata changes are detected https://review.openstack.org/189026 21:00:16 Untriaged bugs so far: 21:00:17 https://bugs.launchpad.net/tripleo/+bug/1507556 21:00:17 Launchpad bug 1507556 in tripleo "[tripleoclient] should ignore validation of tunneling params if tunneling is disabled" [Undecided,In progress] - Assigned to Brad P. Crochet (brad-9) 21:00:18 https://bugs.launchpad.net/diskimage-builder/+bug/1507631 21:00:18 Launchpad bug 1507631 in diskimage-builder "Diskimage-builder (DIB) produces a non-bootable RHEL 6.7 QCOW2-image" [Undecided,New] 21:00:21 https://bugs.launchpad.net/diskimage-builder/+bug/1506792 21:00:21 Launchpad bug 1506792 in diskimage-builder "ironic-python-agent fails to run on fedora ramdisk" [Undecided,In progress] - Assigned to Lucas Alvares Gomes (lucasagomes) 21:00:23 https://bugs.launchpad.net/diskimage-builder/+bug/1506692 21:00:24 Launchpad bug 1506692 in diskimage-builder "Broken test in fedora-minimal/install.d/99-ramdisk" [Undecided,In progress] - Assigned to Martin André (mandre) 21:00:25 https://bugs.launchpad.net/diskimage-builder/+bug/1505900 21:00:25 Launchpad bug 1505900 in diskimage-builder "Failed to create 'wily' release with DIB_RELEASE" [Undecided,New] 21:00:26 https://bugs.launchpad.net/tripleo/+bug/1504263 21:00:26 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 21:00:28 https://bugs.launchpad.net/diskimage-builder/+bug/1506275 21:00:29 Launchpad bug 1506275 in diskimage-builder "$DIB_OFFLINE and $DIB_DEBIAN_USE_DEBOOTSTRAP_CACHE are redundant" [Undecided,New] 21:00:30 https://bugs.launchpad.net/tripleo/+bug/1507054 21:00:31 Launchpad bug 1507054 in tripleo "Allow configuration of mysql percona sst through xtrabackup-v2" [Undecided,New] 21:06:33 Dan Sneddon proposed openstack/tripleo-heat-templates: Add IPv6 Support to Isolated Networks https://review.openstack.org/235423 21:14:28 Ben Nemec proposed openstack/instack-undercloud: Support relative certificate paths https://review.openstack.org/221886 21:14:29 Ben Nemec proposed openstack/instack-undercloud: Enable loadbalancer with ssl on undercloud https://review.openstack.org/221885 21:36:56 Emilien Macchi proposed openstack/instack-undercloud: puppet: drop some warnings https://review.openstack.org/234446 21:40:30 Merged openstack/tripleo-heat-templates: controller/ceilometer: use internalURL for os endpoint type https://review.openstack.org/225999 23:12:32 Tzu-Mainn Chen proposed openstack/tuskar: [WIP] Tuskar API v3 https://review.openstack.org/235716 03:00:16 Untriaged bugs so far: 03:00:17 https://bugs.launchpad.net/tripleo/+bug/1507556 03:00:17 Launchpad bug 1507556 in tripleo "[tripleoclient] should ignore validation of tunneling params if tunneling is disabled" [Undecided,In progress] - Assigned to Brad P. Crochet (brad-9) 03:00:18 https://bugs.launchpad.net/diskimage-builder/+bug/1507631 03:00:19 Launchpad bug 1507631 in diskimage-builder "Diskimage-builder (DIB) produces a non-bootable RHEL 6.7 QCOW2-image" [Undecided,New] 03:00:21 https://bugs.launchpad.net/diskimage-builder/+bug/1506792 03:00:21 Launchpad bug 1506792 in diskimage-builder "ironic-python-agent fails to run on fedora ramdisk" [Undecided,In progress] - Assigned to Lucas Alvares Gomes (lucasagomes) 03:00:22 https://bugs.launchpad.net/diskimage-builder/+bug/1506692 03:00:22 Launchpad bug 1506692 in diskimage-builder "Broken test in fedora-minimal/install.d/99-ramdisk" [Undecided,In progress] - Assigned to Martin André (mandre) 03:00:25 https://bugs.launchpad.net/diskimage-builder/+bug/1505900 03:00:25 Launchpad bug 1505900 in diskimage-builder "Failed to create 'wily' release with DIB_RELEASE" [Undecided,New] 03:00:26 https://bugs.launchpad.net/tripleo/+bug/1504263 03:00:26 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 03:00:29 https://bugs.launchpad.net/diskimage-builder/+bug/1506275 03:00:30 Launchpad bug 1506275 in diskimage-builder "$DIB_OFFLINE and $DIB_DEBIAN_USE_DEBOOTSTRAP_CACHE are redundant" [Undecided,New] 03:00:30 https://bugs.launchpad.net/tripleo/+bug/1507054 03:00:31 Launchpad bug 1507054 in tripleo "Allow configuration of mysql percona sst through xtrabackup-v2" [Undecided,New] 05:36:15 Graeme Gillies proposed openstack/os-cloud-config: Add wol ironic driver support https://review.openstack.org/213956 07:53:31 Morning all 07:53:42 Need reviews on a couple of docs patches if anyone has a moment: 07:54:48 https://review.openstack.org/#/q/status:open+project:openstack/tripleo-docs+branch:master+topic:extraconfig2,n,zhttps://review.openstack.org/#/q/status:open+project:openstack/tripleo-docs+branch:master+topic:extraconfig2,n,z 07:54:59 oops, sorry double paste 08:03:44 shardy: done 08:05:05 jaosorior: thanks, will fix the spelling nit 08:11:53 shardy: if you have time can you review the TLS patch? https://review.openstack.org/#/c/231930/ 08:34:03 shardy: will have a look at those in a bit 08:35:05 marios: kk, thanks 08:36:34 jaosorior: looking, re "So heat no longer takes strings separated by commas? Only actual lists?" 08:36:44 comma_delimited_list will take either format 08:38:49 shardy: Seems to be giving errors if I use "string1,string2" 08:38:56 it only worked by giving an actual string 08:39:37 jaosorior: why is the protocol "http" in the ServiceEndpointProtocolMap in enable-tls.yaml? 08:39:57 Oh wait, the internal endpoints don't use SSL 08:40:02 * shardy needs more coffee 08:40:42 jaosorior: That should work fine, can you create a small reproducer with a simple heat template to demonstrate the issue? 08:41:11 Or show me which interface in the patch doesn't work, with examples of good and bad input parameters 08:43:48 shardy: yeah, so admin endpoints and internal endpoints are set as http, while public are set with https 08:44:45 Steven Hardy proposed openstack/tripleo-docs: Add docs for per-node (pre-deploy) config interfaces https://review.openstack.org/233169 08:44:45 Steven Hardy proposed openstack/tripleo-docs: Split node configuration and customization docs https://review.openstack.org/233168 08:47:01 shardy: and where I was getting an error was here in line 5 https://review.openstack.org/#/c/231930/13/environments/inject-trust-anchor-red-hat.yaml 08:47:59 if I would use "update-ca-trust,extract" instead of [update-ca-trust, extract] it would complain about the format 08:48:08 I think jistr saw the same behaviour there 08:50:14 yeah i did 08:59:54 http://paste.openstack.org/show/477107/ 09:00:12 jaosorior, jistr: I can't reproduce that direct via heat, could have been a tripleoclient issue? 09:00:15 Untriaged bugs so far: 09:00:17 https://bugs.launchpad.net/tripleo/+bug/1507556 09:00:17 Launchpad bug 1507556 in tripleo "[tripleoclient] should ignore validation of tunneling params if tunneling is disabled" [Undecided,In progress] - Assigned to Brad P. Crochet (brad-9) 09:00:18 https://bugs.launchpad.net/diskimage-builder/+bug/1507631 09:00:18 Launchpad bug 1507631 in diskimage-builder "Diskimage-builder (DIB) produces a non-bootable RHEL 6.7 QCOW2-image" [Undecided,New] 09:00:21 https://bugs.launchpad.net/diskimage-builder/+bug/1506792 09:00:21 Launchpad bug 1506792 in diskimage-builder "ironic-python-agent fails to run on fedora ramdisk" [Undecided,In progress] - Assigned to Lucas Alvares Gomes (lucasagomes) 09:00:22 https://bugs.launchpad.net/diskimage-builder/+bug/1506692 09:00:22 Launchpad bug 1506692 in diskimage-builder "Broken test in fedora-minimal/install.d/99-ramdisk" [Undecided,In progress] - Assigned to Martin André (mandre) 09:00:24 https://bugs.launchpad.net/diskimage-builder/+bug/1505900 09:00:25 Launchpad bug 1505900 in diskimage-builder "Failed to create 'wily' release with DIB_RELEASE" [Undecided,New] 09:00:26 https://bugs.launchpad.net/tripleo/+bug/1504263 09:00:27 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 09:00:28 https://bugs.launchpad.net/diskimage-builder/+bug/1506275 09:00:29 Launchpad bug 1506275 in diskimage-builder "$DIB_OFFLINE and $DIB_DEBIAN_USE_DEBOOTSTRAP_CACHE are redundant" [Undecided,New] 09:00:30 https://bugs.launchpad.net/tripleo/+bug/1507054 09:00:30 Launchpad bug 1507054 in tripleo "Allow configuration of mysql percona sst through xtrabackup-v2" [Undecided,New] 09:04:42 shardy: http://paste.openstack.org/show/477108/ 09:05:38 jaosorior: can you pls try the minimal test I pasted too? 09:05:55 will do 09:07:27 shardy: funky, it worked http://paste.openstack.org/show/477109/ 09:08:12 jaosorior: hmm 09:09:42 Derek Higgins proposed openstack-infra/tripleo-ci: [WIP] Add mirror server https://review.openstack.org/238414 09:11:50 shardy: Yeah... I think the error comes from somewhere else, as passing that parameter from an environment file seems to work too http://paste.openstack.org/show/477110/ 09:17:28 Jan Provaznik proposed openstack/tripleo-common: Fix node group name when deleting a node https://review.openstack.org/238417 09:21:45 jaosorior: could you please report the bug to launchpad? 09:22:20 jaosorior: haven't we just confirmed that it's likely an issue with jaosorior's patch? 09:22:23 it's still interesting though, i wonder what can tripleoclient pass as a value to Heat which would be unprocessable by comma_delimited_list 09:22:40 hmm 09:22:43 * jistr opens the patch 09:22:57 or, yeah, I suppose it could be tripleoclient mangling things, but IMO there's not enough information here to raise a useful bug 09:23:00 (yet) 09:23:16 jaosorior: I'll try to pull and investigate it later if you've not figured it out by then 09:23:33 shardy: However, the patch works if I use a list 09:24:04 So I wouldn't say it's an issue with the patch itself 09:24:12 Is the parameter defined in more than one template, e.g as a comma_delimited_list in one and "json" in the other? 09:24:26 * shardy greps 09:25:05 hmm 09:27:55 looking at the comment i left on review: Stack failed with status: Resource CREATE failed: resources.Compute: resources[0]: "u'update-ca-trust,extract'" is not a list 09:28:06 i wonder if tripleoclient is actually passing 09:28:26 I don't see the parameter in the debug output in http://paste.openstack.org/show/477108/ 09:28:27 Param: "u'update-ca-trust,extract'" 09:28:59 UpdateTrustAnchorsCommand isn't there tho 09:29:59 I'm planning to document how to create an overcloud direct via heat stack-create, that would help when we need to bisect templates vs tripleoclient issues 09:32:17 Derek Higgins proposed openstack-infra/tripleo-ci: [WIP] Add mirror server https://review.openstack.org/238414 09:37:22 derekh, morning 09:37:33 derekh, I came up with this yesterday https://review.openstack.org/#/c/238212/ 09:38:19 not sure if I did that right... 09:45:12 jaosorior: http://paste.openstack.org/show/477112/ 09:45:22 That shows how to test deploying without tripleoclient 09:46:03 The stack validates OK for me, although I didn't put real certs in the environments so I guess deployment will fail 09:51:32 hmm i just checked out the latest version of the patch, updated the trust anchor RH file to have 09:51:34 UpdateTrustAnchorsCommand: "update-ca-trust,extract" 09:51:48 ran openstack overcloud deploy --templates -e workstationsync/tripleo-heat-templates/environments/inject-trust-anchor-red-hat.yaml 09:51:52 and it's deploying 09:54:39 rhallisey: looking 10:05:20 rhallisey: see comments 10:06:00 derekh, ok cool 10:06:07 shardy: I also missed a change that I had in the machine I was deploying on that I forgot to put on the patch 10:06:18 jistr: ^^ 10:10:55 Ryan Hallisey proposed openstack-infra/tripleo-ci: Add Docker job to tripleo CI https://review.openstack.org/238212 10:12:09 xiexs proposed openstack/os-cloud-config: Refactor _extract_driver_info() to reduce complexity. https://review.openstack.org/214128 10:12:28 Ryan Hallisey proposed openstack/tripleo-common: Deploy Docker compute role with tripleo.sh https://review.openstack.org/232853 10:13:08 Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Enable TLS deployment for the Overcloud https://review.openstack.org/231930 10:13:22 tremble: Will remove the GlancePort parameter in the next patch 10:13:37 derekh, ok that simplifies this 10:13:42 gfidente: Can we re-take the conversation we were having about how to set static IPs now? 10:14:05 Ryan Hallisey proposed openstack-infra/tripleo-ci: [WIP] Add Docker job to tripleo CI https://review.openstack.org/238212 10:14:18 jaosorior, ack 10:15:14 rhallisey: yup, looks like it, once we're ready we'll need to add a job for it 10:15:26 rhallisey: so does it use the same overcloud images ? 10:15:57 derekh, oh I forgot that.. Need fedora-atomic 10:16:21 rhallisey: ya, I reckoned something looked mkissing 10:16:26 *missing 10:18:20 jaosorior: /me takes a look 10:23:12 Lucas Alvares Gomes proposed openstack/os-cloud-config: Allow ipmi_priv_level to be configured for the ipmitool driver https://review.openstack.org/238439 10:24:29 jaosorior, to deploy in network isolation, you pass two more -e 10:24:51 environments/network-isolation.yaml 10:24:58 environments/net-single-nic-with-vlans.yaml 10:25:06 and you don't really need to customize those 10:25:30 but you need the add the following in the custom env.yaml 10:25:34 parameter_defaults: 10:25:34 ControlPlaneSubnetCidr: "24" 10:25:35 ControlPlaneDefaultRoute: "192.0.2.1" 10:25:35 EC2MetadataIp: "192.0.2.1" 10:26:03 when that completes, we add a little more changes into the custom env.yaml to deploy with predictable IPs 10:26:09 and redeploy 10:26:23 but we want to make sure the environment works as expected with the default net isolation templates first 10:26:26 sounds valid? 10:26:30 gfidente: So you do it as multiple runs? 10:26:36 tremble, nono 10:26:55 I thought was just good to make sure the environment works well before adding more salsa 10:27:02 Ah ok 10:28:01 Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Enable TLS deployment for the Overcloud https://review.openstack.org/231930 10:35:21 tremble: that gets rid of the GlancePort parameter ^^ 10:35:30 :) 10:37:03 I'm deploying it now 10:37:18 and after this I'll try network isolation 10:38:16 jaosorior++ 10:38:26 gfidente: Thanks for the tip, by the way :)\ 10:38:39 oh you're all into it already now 10:38:44 can't escape it anymore 10:39:00 Ryan Hallisey proposed openstack/tripleo-common: Deploy Docker compute role with tripleo.sh https://review.openstack.org/232853 10:39:22 derekh, ok all set 10:39:55 jaosorior, so network isolation is going to create a bunch of vlans on top of the single NIC the VMs ahave 10:40:05 gfidente: thanks very much will update the libvirt_vol_pool 10:40:12 marios, don't hate me! 10:41:22 gfidente: terms and conditions apply 10:41:25 gfidente: I definitely owe you a beer for all the help so far 10:41:27 gfidente: thanks man, makes sense 10:42:10 marios, jistr on a side not, not sure if slagle is around 10:42:22 I was thinking to reprise the instack-test-overcloud thing for CI ? 10:42:31 derekh, ^^ ? 10:42:58 gfidente: remind us. did it launch instance? poke at interfaces (ping?) 10:43:03 yeah 10:43:16 from volume and I think it even tried ssh 10:43:20 gfidente: cant/dont we tempest? 10:43:25 I'm sure we can 10:43:44 but I don't know where we stand there 10:44:02 using some of the tempest scenarios would sure be nicer 10:44:16 the script seemed quicker (dirtier!) 10:44:26 well for sure all of these simple cases will be covered there 10:44:53 if we can use it, much better than reinventing. especial bonus it is external tooling to verify the tripleo deployment 10:45:24 ok 10:47:24 gfidente: marios I havn't seen the instack-test-overcloud script 10:47:34 gfidente: marios this is what the old ci did http://paste.openstack.org/show/477125/ 10:47:55 gfidente: marios we should be able to do that now (I've been meaning to) with little effort 10:48:19 gfidente: marios then work on using a subset of tempest tests 10:49:07 derekh: gfidente so since we got nothing right now, simple script to get started makes sense. but with a view to not making it perfect but lookign to use tempest in the longer term 10:49:25 yeah I think marios is right there are many benefits in using tempest, yet if we can get any simple script up quickly, with minimum effort, that I think would be useful 10:49:33 marios, +1 10:50:00 apparently derekh had planned everything already :P 10:50:25 yay, it deployed. now gotta check with network isolation 10:50:27 gfidente: yeah i mean, what derekh showed should really be the simple script, just simple launch ping test 10:50:44 to start with anyway 10:51:04 gfidente: marios yup, anybody want to take it on ? https://trello.com/c/OQlIbGoK/19-toci-start-a-user-instance-and-ping-it 10:51:12 if not I'll get to it 10:51:32 derekh: sure i can have a look in a bit , or can ping you to see if i can help etc? 10:52:23 derekh: meaning, if you're looking forward to getting stuck into that please go ahead, if you got other stuff, i can look in a bit 10:52:51 marios: fire ahead, it might be worth adding it to tripleo.sh --sanitiytest , I'm lost in setting up a mirror server at the moment 10:53:09 derekh: ack 10:53:25 marios: it looks like it might take another few minutes off the CI run time, so you can add them back again with a test ;-) 10:54:02 the universe is balanced once again 10:54:29 derekh, yeah unfortunately it will slow down things 10:54:48 but I feel we have too little sense of things working in the overcloud without it 10:54:52 gfidente: yup, 10:55:07 honestly though launching an instance shouldn't be that much drama 10:55:15 though we also have to ah... the image :/ 10:55:23 yeah at least a couple mins per run 10:58:43 gfidente: derekh we can at least reuse the fedora-user image no? 10:58:50 no need to grab the cirros one 10:59:10 i mean we end up with that on the undercloud node 10:59:33 marios: you could try it, the cirros one might boot faster 10:59:45 marios: I would try both and see 10:59:51 yeah ok 11:01:03 marios: which ever one you use, we can add it here https://review.openstack.org/#/c/238414/2/scripts/mirror-server.pp,cm so it will appear in the local mirror I started yesterday http://66.187.229.84/ 11:02:11 derekh: ah great. well that was the main thought about cirros so cool 11:03:59 marios: in my very scientific sample set of 1, the centos image downloaded in less then 2 seconds http://logs.openstack.org/14/238414/2/check-tripleo/gate-tripleo-ci-f21-nonha/492d408/console.html#_2015-10-22_09_41_32_740 11:11:29 Marek Aufart proposed openstack/tripleo-common: [WIP]Add resource capabilities https://review.openstack.org/232534 11:13:56 Any reviews for this one? https://review.openstack.org/#/c/237974/ should be a quick one 11:15:27 jaosorior, hey thanks 11:17:06 shardy: Thanks for the review dude 11:17:26 jaosorior: np, thanks for the fix :) 11:18:51 jaosorior, nova vnc 11:18:58 and swift proxy have same problem 11:19:13 I wonder what is the policy we put in place there for non-ssl ports? " 11:19:21 gfidente: Let me check 11:21:19 damn 11:21:21 you're right 11:24:49 Merged openstack/puppet-tripleo: Resolve repeated ports for ssl frontends https://review.openstack.org/237974 11:28:24 Ryan Hallisey proposed openstack/tripleo-common: Deploy Docker compute role with tripleo.sh https://review.openstack.org/232853 11:29:05 derekh, ok I think everything is set to add a job for this 11:31:13 Juan Antonio Osorio Robles proposed openstack/puppet-tripleo: Resolve repeated ports for ssl frontends (nova vnc and swift proxy) https://review.openstack.org/238461 11:32:27 gfidente: tried the network isolation with the environments you mentioned...and it seems to get stuck http://pastebin.com/6PXdiWxd 11:33:07 tremble: Responded to your comments on the TLS patch 11:33:24 jaosorior, yeah the client is trying to run the post-deployment config steps against the public vip 11:33:30 jaosorior, which the undercloud does not reach 11:34:37 jaosorior, which version of tht are you using? 11:35:31 gfidente: almost the latest, pretty much pulled the one that my TLS CR is based on 11:35:54 jaosorior, and the undercloud is from rdo? 11:36:09 the client should be using the ip from KeystoneAdminVip heat outut 11:36:21 try 'heat output-show overcloud KeystoneAdminVip' 11:36:32 so... do I cancel that deployment? 11:37:40 no the heat command is supposed to show us which value KeystoneAdminVip has 11:38:06 I'm trying to understand why the post-deployment is launched against 10.0.0.4 which is the public network 11:38:07 $ heat output-show overcloud KeystoneAdminVip 11:38:09 "192.0.2.23" 11:38:23 yeah that is where it should have connected 11:39:07 uhm 11:42:14 rhallisey: great, what repositories does it need to run against? These are the repositories we currently cover along with how many CI jobs were triggered over 5 days 2 weeks ago 11:46:10 derekh, tripleo-heat-templates. What else does normal tripleo CI run against puppet? 11:46:43 derekh, actually, I might change this a little. I want this to build the images so it grabs latest pacakges 11:47:35 Marios Andreou proposed openstack/tripleo-incubator: Adds LIBVIRT_VOL_POOL_TARGET to tripleo create-nodes https://review.openstack.org/235278 11:47:36 rhallisey: the normal tripleo CI runs against all of those, I suppose what I'm wondering is are any of those not relevant to you ? 11:47:52 rhallisey: no prob, we can do this in parallel anyways 11:48:02 derekh, it should run against the same repos 11:48:03 gfidente: https://review.openstack.org/#/c/235278/1..2/scripts/create-nodes thanks 11:48:49 derekh, well ok, we can have a job building and pushing and another job that pull down and uses those images 11:48:56 derekh, that might be best 11:51:03 rhallisey: ok, you'r patch should just be adding the line "gate-tripleo-ci-f21-containers" to this list http://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul/layout.yaml#n610 11:51:47 rhallisey: then for the puppet module it needs to be added to each one seperatly, e.g http://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul/layout.yaml#n7032 11:52:23 rhallisey: I have a patch to add a period job for the other tests, I'll rebase on top of yours when its up 11:52:56 rhallisey: finally, you should make the new job non-voting like this http://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul/layout.yaml#n885 11:53:07 rhallisey: untill we know its reliable 11:54:38 derekh, cool. I'm on it 11:55:33 jaosorior, can you actually check if the ssl ports we assign are anywhere useful more in general>? 11:55:54 jaosorior, not sure if there is any agreement on which ssl port each service should use 11:56:17 jaosorior, having a pattern in the .pp would be nice too 11:56:30 jaosorior, to avoid seeing same error again in the future 11:56:36 gfidente, well, they were not actually being used before. And it seems to me like it's the only place where they are even set. 11:56:53 But after the TLS lands, I actually want to start passing the ports from the templates on to puppet 11:57:05 so those ports would end up as parameters 11:57:35 uhm but still for the defaults 11:57:45 having a pattern would be nice 11:59:05 jaosorior, btw, regarding the deployment being stuck, can you check if tripleo client on the undercloud is carrying this: https://github.com/openstack/python-tripleoclient/blob/dfa236730ae2f13f4c5e322f0d833b64bce45bde/tripleoclient/v1/overcloud_deploy.py#L347 12:02:57 gfidente: not there 12:03:31 jaosorior, ok that is the reason why it points at 10.0.0.4 12:03:37 so looks like you have some old undercloud? 12:04:11 gfidente: might be. Though I deployed it not too long ago 12:04:15 last week or so I think 12:04:43 and running a yum -y upgrade doesn't do anything :/ 12:05:20 jaosorior, the change seems dated 24th of july 12:05:25 how come you don't have it though> 12:05:50 weird :/ 12:06:51 gfidente 12:06:53 wait a minute... 12:06:57 I messed up 12:06:59 it is there 12:07:01 just in a different line 12:08:14 gfidente: Why do you feel that the ports need to change just because they're SSL? 12:09:07 tremble, nah I don't indeed, I trust on doing whatever is best was just thinking how we could avoid them being duplicated again 12:09:27 actually, s/trust/encourage/ :) 12:09:32 Yeah 12:09:37 fix it! 12:09:41 sigh :( 12:11:50 derekh, https://review.openstack.org/#/c/238475/ 12:12:37 derekh, do I need to alter the existing job since I think it looks at tripleo-ci-* 12:13:10 maybe list them out instead 12:13:59 I think it's good to not have the SSL-only version on the 'known non-SSL' port 12:14:13 but if we can do TLS, then I wouldn't use a different port 12:15:17 jaosorior, so regarding the deployment being stuck 12:18:59 gfidente: ? 12:19:16 here we moved from overcloud_ip to keystone_ip https://github.com/openstack/python-tripleoclient/commit/f4fbce88450a3da2cf34ef279bb06b8601c2d608 12:19:31 keystone initialize is here https://github.com/openstack/os-cloud-config/blob/9a16509d0d11c3142b19a90d112e8fe942f154a8/os_cloud_config/keystone.py#L129 12:19:52 I don't see how the client can be using 10.0.0.4 when KeystoneAdminVip returns 192.x 12:19:57 but so it seems from the logs 12:21:03 not sure if you can find something? 12:21:16 I think the version of the tripleoclient is old and is passing the public ip 12:21:28 or something like that 12:24:30 dprince: well I see why ifdown isn't working.. 12:24:34 running ifdown on interface: eth0 cmd: /sbin/ifdown, args: ('eth0',), kwargs: {'check_exit_code': False} 12:24:37 usage: ifdown 12:25:01 that should read /sbin/ifdown, args: eth0, kwargs... 12:25:26 dprince: so we're picking up a (' and a ',) somewhere 12:25:56 Slower: hmmm, so this is just an os-net-config bug? got a patch? 12:26:45 Marios Andreou proposed openstack/tripleo-docs: Adds note about the LIBVIRT_VOL_POOL_TARGET used by create-nodes https://review.openstack.org/235332 12:26:59 dprince: no, I was wondering if you have any ideas.. I have to restart the stack every time to make it show up 12:27:07 dprince: after first run it doesn't do it 12:27:18 dprince: so I'll add a smathering of debug and see if I can figure it out 12:27:48 it's just a time consuming pita 12:33:52 dprince: I actually think we should log stdout/err from execute if -d or -v is set on os-net-config 12:34:31 dprince: as it is all output from execute is ignored including return value 12:35:04 Marios Andreou proposed openstack/tripleo-docs: Fix indendation bug in virtual environment setup https://review.openstack.org/238480 12:36:04 shardy: easy one ^^^ docs nit 12:36:13 dprince: anywho I'll get a few patches up soon hopefully 12:36:56 Slower: FYI we've been talking about switching to having os-net-config run via a script e.g via SoftwareConfig/SoftwareDeployment 12:37:13 that would also solve the stderr thing and make it visible outside of the node via heat 12:37:23 (and provide a much better failure path than just timing out) 12:37:42 shardy: in this case it wouldn't actually solve that I don't think 12:37:52 shardy: os-net-config itself is actually ignoring the output on exec 12:38:33 shardy: there's no way to see it except to modify os-net-config 12:39:15 Slower: aha, I see, I thought os-net-config was just failing 12:39:17 Slower: the output of os-net-config should go to the process that runs it, os-collect-config 12:39:24 even if you can see os-net-config output 12:39:42 I'm running it by hand.. 12:39:55 marios: lgtm! 12:40:07 Slower: http://git.openstack.org/cgit/openstack/tripleo-image-elements/tree/elements/os-net-config/os-refresh-config/configure.d/20-os-net-config#n54 12:40:13 Slower: we run w/ -v 12:40:33 it uses processutils.execute() which reutrns (stdout, stderr) 12:40:35 Slower: so basically you'd like me to build a custom agent image that also deploys w/ -d? 12:40:47 no, os-net-config needs to be fixed :) 12:41:01 Slower: right, that is what I thought initially too 12:41:05 it uses the above call and ignores the return value 12:41:07 Slower: but you mentioned output was a problem 12:41:09 which is stdout/err 12:41:17 for all exec's 12:41:39 in __init__.py 12:41:48 shardy: thanks 12:42:48 https://github.com/openstack/os-net-config/blob/master/os_net_config/__init__.py 12:42:51 line 130 12:42:52 calls 12:43:27 https://github.com/openstack/oslo.concurrency/blob/master/oslo_concurrency/processutils.py 12:43:31 line 188 12:43:44 and then you see the issue 12:44:02 Marek Aufart proposed openstack/tripleo-common: [WIP]Add resource capabilities https://review.openstack.org/232534 12:44:13 I can put up a patch in a bit (or tomorrow, I'll see how it goes) 12:44:32 Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Enable TLS deployment for the Overcloud https://review.openstack.org/231930 12:44:39 tremble: ^^ 12:47:31 jaosorior: ack 12:48:10 tremble: currently deploying that, lets see 12:59:55 imain proposed openstack/os-net-config: Add logging of stdout/err and cmd used for execute. https://review.openstack.org/238486 13:00:54 Lucas Alvares Gomes proposed openstack/diskimage-builder: Install IPA in a virtual enviroment https://review.openstack.org/235843 13:06:25 Florian Fuchs proposed openstack/tripleo-common: [WIP]Adds deployment configuration methods https://review.openstack.org/230315 13:10:25 Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Enable TLS deployment for the Overcloud https://review.openstack.org/231930 13:10:54 tremble: deleted the hidden parameter from the certificates ^^ 13:11:05 ok 13:11:36 tremble: Thanks a lot for the reviews 13:11:59 now you just need to find a core reviewer ;) 13:25:55 trown: Hey, doing some initial tests with stable/liberty - all going OK until image build/upload 13:26:12 trown: It seems we don't build the ipa ramdisk, but image upload expects it 13:26:22 is that something you've already encountered w RDO? 13:26:52 * shardy looks at tripleoclient patches 13:28:00 shardy: you need to `export RDO_RELEASE='liberty'` before the image build 13:28:22 it still defaults to kilo, and there was no IPA in kilo 13:28:35 aha, we should fix that :) 13:28:37 thanks! 13:29:15 shardy: I plan to submit a fix for https://bugs.launchpad.net/tripleo/+bug/1506067 and will include that in it 13:29:15 Launchpad bug 1506067 in tripleo "tripleoclient: defaults for using delorean are based on kilo" [Medium,Triaged] - Assigned to John Trowbridge (trown) 13:29:41 trown: great, sounds good 13:29:59 other than that, everything is working great so far :) 13:30:09 :) 13:30:43 shardy: I should have asked... are you doing delorean or the RDO GA bits? 13:30:58 trown: I'm using centos7-liberty trunk delorean 13:31:27 shardy: i had a question about tripleoclient while it's on your mind 13:31:28 trown, hey man! I have a bad feeling that we need to revive ahc-tools 13:31:38 shardy: ah ok, the image instructions are slightly different, but the delta between those two should not be too much 13:32:01 dtantsur: ya, it has been in the back of my mind as well :( 13:32:12 I assumed folks would go to RDO manager for the RDO GA bits, e.g then upstream TripleO is always trunk (master or stable branch), RDO is always a bit behind that etc 13:32:44 shardy: i saw it was in the list for a stable branch. but given the need for a liberty undercloud to be able to deploy liberty and kilo, and same pattern going forward, shouldnt tripleoclient actually need to be backwards compatible? 13:32:54 trown, we don't have time to move it to tripleo... but we definitely should move it to using new inspector features, wdyt? 13:33:02 slagle: Yeah, probably it does 13:33:18 shardy: ya makes sense https://review.openstack.org/#/c/235575/ is the patch for tripleo docs for the images 13:33:28 slagle: all other python-*clients make backwards compat promises, so perhaps we should do the same 13:33:36 we can always remove the branch if needed :) 13:33:37 shardy: that would be delorean, although you would point to a different repo for liberty obviously 13:34:01 shardy: ok. it's either that or the 2 versions need to be installable side by side...like we plan to do for tht 13:34:12 trown: thanks, will check it out, can't find openstack-ironic-python-agent so I guess I'm still missing a repo 13:35:31 dtantsur: ya, I dont forsee having much time to look at it until during/after summit 13:35:50 slagle: Yeah, just having it backwards compat is probably better, but all the *clients now do actually have stable branches despite the historical backwards compat 13:36:10 I think that is mostly driven by wanting more stable branch CI stability than user requirements tho 13:36:21 dtantsur: it should not be a huge effort to make it work with new inspector features though.. maybe even an afternoon of hacking on it 13:36:30 yeah, maybe 13:36:38 I'm on pto after summit 13:36:39 (I actually opposed that move and discussed it with those involved at the time, FWIW) 13:36:42 dtantsur: the reports will still suck, because that is a giant hack :p 13:37:03 yeah... well, we have what we have 13:37:30 it is my giant hack, so I can say that :) 13:37:37 slagle: actually, wouldn't the N & N-1 release thing work OK regardless of the stable branch? 13:38:04 e.g stable/liberty has to support liberty and kilo, but what's currently master will support mitaka and liberty (and possibly kilo) 13:38:47 So we could backport a patch which enables tripleoclient stable/liberty to deploy liberty and kilo 13:38:58 we just wouldn't backport one related to mitaka 13:39:14 Anyway, +1 on removing the branch if we decide we can do without it 13:39:20 less backports == better ;) 13:40:11 let's discuss next week and decide then (stevebaker's input would be good as he's maintaining the heatclient stable branch) 13:41:14 dprince, you around? 13:41:27 Ryan Brady proposed openstack/tripleo-common: [WIP] Initial workflow implementation of tripleo-common https://review.openstack.org/227297 13:41:53 gfidente: hi, yes 13:42:30 dprince, I think the issue jaosorior is seeing is due to https://github.com/openstack/python-tripleoclient/blob/dfa236730ae2f13f4c5e322f0d833b64bce45bde/tripleoclient/v1/overcloud_deploy.py#L356 13:43:12 do you remember if we added that line there for a reason or if we're safe enough with https://github.com/openstack/python-tripleoclient/blob/dfa236730ae2f13f4c5e322f0d833b64bce45bde/tripleoclient/v1/overcloud_deploy.py#L352 ? 13:43:52 looks like in occ public preveals on host so we end up connecting to public regardless :( 13:43:53 trown: So there's no current-tripleo in http://trunk.rdoproject.org/centos7-liberty/, I guess we'll need to set that up when we've got liberty tripleo CI 13:44:35 current has what I'm missing anyway :) 13:45:11 yep, shardy, or you could use current-passed-ci as that actually does get updated regularly based on RDO CI runs 13:45:37 but we can definitely have a current-tripleo there if it seems useful 13:45:59 shardy: ok, yea, we can discuss more. i guess i don't see how the n-1 support is implied 13:46:15 tripleoclient could rip out postconfig via os-cloud-config tomorrow if the puppet side lands in tht 13:46:26 that would break deploying liberty on master 13:47:05 but that's what we want to prevent anyway 13:47:24 slagle: Yeah, I guess that's an advantage of the branches, e.g insulation from those sorts of changes 13:47:37 Hmm, yeah 13:48:04 if that happened, when we branched mitaka, it couldn't deploy liberty 13:48:21 Eventually we'll need CI coverage of that, e.g stand up a mitaka/master undercloud, then deploy mitaka and liberty underclouds (potentially in different jobs) 13:48:32 yea 13:48:50 I guess the ideal would be an upgrade job, which deploys mitaka undercloud, liberty overcloud, then upgrades it 13:49:05 I know we're a way off that now, but that would give the needed coverage 13:50:32 Giulio Fidente proposed openstack/python-tripleoclient: Do not pass public to keystone.initialize https://review.openstack.org/238514 13:50:34 agreed, that would be awesome 13:53:39 * shardy deploys first stable/liberty overcloud :) 13:53:50 Marios Andreou proposed openstack/tripleo-heat-templates: puppet: run keystone in wsgi https://review.openstack.org/213175 13:54:02 EmilienM: not there yet, i want to see what ci makes of it, will add not in a sec 13:54:05 ^^^^ 13:56:06 If someone can +2A https://review.openstack.org/#/c/235575/ that would be good 13:56:36 it's needed for the stable/liberty docs, just tested and it works 13:56:44 shardy: looking 13:58:20 too late 13:58:40 :) 13:58:56 thief! 13:59:10 Merged openstack/tripleo-docs: Modify instructions for using delorean to build images https://review.openstack.org/235575 14:01:30 Gael Chamoulaud proposed openstack/tripleo-heat-templates: Add Puppet 4.x lint checks https://review.openstack.org/234150 14:01:30 Gael Chamoulaud proposed openstack/tripleo-heat-templates: Make puppet manifests compliant with Puppet 4.x https://review.openstack.org/225664 14:13:46 hi folks, I've set-up a PoC deployment following the steps from https://repos.fedorapeople.org/repos/openstack-m/rdo-manager-docs/liberty/basic_deployment/basic_deployment_cli.html all seems ok besides I can't ssh into an instance booted in overcloud (http://ur1.ca/o5bl3). It seems the post-deploy network steps may miss some entries --- actually I can't even ping the gateway 172.16.23.251 (from instack/undercloud machine) Any hints ap 14:13:46 preciated. 14:14:31 mkovacik: I think the example network in the docs does not match the network created by instack-virt-setup 14:14:49 slagle: okay, I think this should work better https://review.openstack.org/#/c/204806/6 14:16:31 trown, that might indeed be the case ;) are there some docs that match the PoC deployment? 14:19:15 mkovacik: I think if you just delete the 172.16.13.0/24 network you created and replace all those with 192.0.2.0/24 addresses you should be good to go 14:19:51 * mkovacik shall try that 14:20:04 might be worth submitting a patch to openstack/tripleo-docs for that, I think examples should all be based on the virt defaults 14:20:13 since those are the only sane defaults 14:22:27 trown, +1 if this works for me, I'm definitely submitting 14:22:47 mkovacik: btw you are using the RDO-Manager forked docs, I am regularly rebasing those on openstack/tripleo-docs, but they use a more conservative repo setup than what is used in tripleoci 14:30:38 trown, it worked, thanks a lot! 14:31:44 #success First overcloud deployed using new TripleO stable/liberty repos :) 14:31:46 shardy: Added success to Success page 14:32:01 nice one shardy 14:32:09 SHIP IT 14:32:23 trown: Thanks to all the hard work in RDO it's been very easy, but still a good step forward :) 14:34:01 #success RDO Liberty released http://rdoproject.org/repos/openstack-liberty/rdo-release-liberty.rpm 14:34:03 trown: Added success to Success page 14:34:09 trown: I know we said fork tripleo-docs, but you be open to an experiment trying a Stable admonition first? 14:34:35 * derekh wonders what this success page is 14:34:38 all that differs atm is the repo setup, so we could defer branching until that changes 14:35:07 http://superuser.openstack.org/articles/success-bot-helps-share-your-happy-moments-in-the-openstack-community 14:35:10 derekh: ^^ 14:35:27 shardy: I am ok with defering branching until there is some feature documented that will not work on liberty 14:35:55 shardy: at that point I would start having to do reverts in my downstream fork which is kind of gross 14:36:09 but also not impossible 14:36:11 trown: Ok, let's delay until it's obvious it's needed, thanks 14:36:25 shardy: thanks, never heard of it before ;-) 14:36:53 derekh: ya it is new thing 14:40:04 * slagle is tempted to #success teaching derekh about success bot 14:40:17 lol 14:40:57 * derekh was tempted to "#success learned about success bot" but restrained himself 14:53:01 Merged openstack/puppet-tripleo: Resolve repeated ports for ssl frontends (nova vnc and swift proxy) https://review.openstack.org/238461 15:00:15 Untriaged bugs so far: 15:00:17 https://bugs.launchpad.net/tripleo/+bug/1507556 15:00:17 Launchpad bug 1507556 in tripleo "[tripleoclient] should ignore validation of tunneling params if tunneling is disabled" [Undecided,In progress] - Assigned to Brad P. Crochet (brad-9) 15:00:18 https://bugs.launchpad.net/diskimage-builder/+bug/1507631 15:00:20 https://bugs.launchpad.net/diskimage-builder/+bug/1506792 15:00:20 Launchpad bug 1507631 in diskimage-builder "Diskimage-builder (DIB) produces a non-bootable RHEL 6.7 QCOW2-image" [Undecided,New] 15:00:22 https://bugs.launchpad.net/diskimage-builder/+bug/1506692 15:00:22 Launchpad bug 1506792 in diskimage-builder "ironic-python-agent fails to run on fedora ramdisk" [Undecided,In progress] - Assigned to Lucas Alvares Gomes (lucasagomes) 15:00:25 https://bugs.launchpad.net/diskimage-builder/+bug/1505900 15:00:25 Launchpad bug 1506692 in diskimage-builder "Broken test in fedora-minimal/install.d/99-ramdisk" [Undecided,In progress] - Assigned to Martin André (mandre) 15:00:26 https://bugs.launchpad.net/tripleo/+bug/1504263 15:00:27 Launchpad bug 1505900 in diskimage-builder "Failed to create 'wily' release with DIB_RELEASE" [Undecided,New] 15:00:28 https://bugs.launchpad.net/diskimage-builder/+bug/1506275 15:00:29 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 15:00:30 https://bugs.launchpad.net/tripleo/+bug/1507054 15:00:30 Launchpad bug 1506275 in diskimage-builder "$DIB_OFFLINE and $DIB_DEBIAN_USE_DEBOOTSTRAP_CACHE are redundant" [Undecided,New] 15:00:31 Launchpad bug 1507054 in tripleo "Allow configuration of mysql percona sst through xtrabackup-v2" [Undecided,New] 15:01:04 dprince: can you point me to where we build the agent container with dib? 15:01:26 dprince: I figured out I'm dumb.. I also figured out why ifdown isn't working 15:06:04 Dan Prince proposed openstack/tripleo-incubator: Add docker_images_redhat.yaml build-images file https://review.openstack.org/199831 15:06:13 Slower: that is what I use ^^ 15:19:30 dprince: ok well the issue is we don't have ifcfg-eth0 in the agent container when we go to ifdown the interface 15:19:43 dprince: but we can't mount it in with docker because we don't know what it will be 15:20:16 dprince so I think we have to add cp /host/etc/sysconfig/network-scripts/ifcfg-* /etc/sysconfig/network-scripts to our run-os-net-config.sh before we run os-net-config 15:20:39 cp -f I guess 15:20:47 cp -fp maybe 15:22:41 dprince: when I do that locally before os-net-config runs it works 15:23:07 dprince: so I don't know if I can make a patch for that or if it's all outstanding.. you might have to modify a patch for it 15:24:35 Slower: sounds promising to me 15:26:43 dprince: actually what about putting all of run-os-net-config.sh in the template? 15:26:55 dprince: then all the magic is in the template instead of mixed between that and the agent image 15:37:53 dprince: there is any reason for using $build_ring = 'True' and testing it with str2bool(downcase(... in ringbuilder.pp manifests https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/manifests/ringbuilder.pp#L58-L64 15:38:22 dprince: why not using a puppet boolean for build_ring? :/ 15:39:48 Hey can we land marios's https://review.openstack.org/#/c/238480/ docs patch please? 15:40:08 I'm about to update that file and would be good to land that first 15:41:27 +A'd 15:41:35 thanks :) 15:42:01 Merged openstack/tripleo-docs: Fix indendation bug in virtual environment setup https://review.openstack.org/238480 15:46:47 Gael Chamoulaud proposed openstack/tripleo-heat-templates: Make puppet manifests compliant with Puppet 4.x https://review.openstack.org/225664 15:53:03 gfidente still around? 15:55:32 dprince: i did a first high level pass of the composable roles, i think i'll let it sink in and look more closely later :) in general i like it and it also solves our pacemaker hiera issue i think 15:56:20 jaosorior, yes and no 15:57:46 gfidente: was there a response for the stuff we were asking about? 15:57:53 dprince: btw reading the patches i wondered if we could, looking forward, make it even more awesome and support custom roles (node types) and traits (the bits which compose together to form a node type) << just an arbitrary naming i picked to differ between those two 15:58:59 jaosorior, the post config reaching out to public vip? 15:59:54 dprince: having custom roles would allow us to do things like separate out network nodes and scale them independently etc., having custom traits would allow us to do 3rd party extensions nicely i think 16:00:14 for that we'd need to stop directly referencing those from the top-level template though 16:00:24 and drive everything through resource registry in some manner 16:00:27 gfidente: Yeah 16:00:35 which i guess would be tricky in many ways 16:00:40 well, the public parameter that's being passed to the keystone.initialize 16:00:52 jaosorior, I think dprince is not seeing that happening 16:01:23 gfidente, huh? 16:01:25 jaosorior, I am not sure how that can ben 16:02:11 well, the public parameter is gotten from the KeystoneURL output of the stack 16:02:32 or do you mean that he ran it and doesn't get the same thing? 16:02:33 dprince: e.g. just ensuring that the traits get all the info they need from the top-level template would probably not be an easy thing to solve, if one can't make assumption what parameters a trait can take until one looks at the resource registry. I think it could be done by feeding all params into a ParamMap (similarly like we have VipMaps for Vips, for example) 16:03:14 but that maybe feels a bit hacky to begin with 16:03:30 jaosorior, yes I think he got the client connecting to the ctlplane 16:03:39 jaosorior, I am trying myself as well to see what happens 16:05:32 alright 16:06:11 Marios Andreou proposed openstack/tripleo-docs: Adds note about the LIBVIRT_VOL_POOL_TARGET used by create-nodes https://review.openstack.org/235332 16:06:14 dprince: i'm not even sure how feasible those fully custom roles and traits would be with current Heat capabilities, was just wondering if/how to make things even more generic and configurable in the future 16:07:54 Jaume Devesa proposed openstack/tripleo-puppet-elements: Adding networking MidoNet element https://review.openstack.org/232551 16:12:07 jistr: I think having custom roles is the first step in composing things 16:12:36 jistr: I was thinking of doing the 'database' role next 16:12:43 Jaume Devesa proposed openstack/tripleo-puppet-elements: Adding networking MidoNet element https://review.openstack.org/232551 16:12:49 jistr: then perhaps 'network' 16:13:10 jistr: everything becomes a role, and we compose it how we want 16:14:05 gfidente, jaosorior: is there a ticket on the keystone endpoint bug you are seeing? 16:14:55 Steven Hardy proposed openstack/tripleo-docs: Initial instructrions for stable/liberty repos https://review.openstack.org/238600 16:19:35 trown, derekh: ^^ initial docs based on my notes from testing 16:19:49 let me know if anything looks wrong else I'll base a tripleo.sh update on the docs 16:23:08 dprince, no I don't think so, I'll open something in LP if I see it 16:25:45 gfidente: cool, thanks 16:30:24 dprince, no ticket. yet. 17:52:19 Joe Talerico proposed openstack/tripleo-heat-templates: The default balance-tcp is causing issues with deployments. Defaulting to active-backup. https://review.openstack.org/238638 17:54:39 Ben Nemec proposed openstack/instack-undercloud: stackrc improvements for SSL undercloud https://review.openstack.org/238639 18:35:23 Ben Nemec proposed openstack/tripleo-docs: Document installing the undercloud with SSL endpoints https://review.openstack.org/221891 19:54:46 Ben Nemec proposed openstack/tripleo-docs: Document installing the undercloud with SSL endpoints https://review.openstack.org/221891 21:00:16 Untriaged bugs so far: 21:00:19 https://bugs.launchpad.net/tripleo/+bug/1507556 21:00:19 Launchpad bug 1507556 in tripleo "[tripleoclient] should ignore validation of tunneling params if tunneling is disabled" [Undecided,In progress] - Assigned to Brad P. Crochet (brad-9) 21:00:20 https://bugs.launchpad.net/diskimage-builder/+bug/1507631 21:00:20 Launchpad bug 1507631 in diskimage-builder "Diskimage-builder (DIB) produces a non-bootable RHEL 6.7 QCOW2-image" [Undecided,New] 21:00:22 https://bugs.launchpad.net/diskimage-builder/+bug/1506792 21:00:22 Launchpad bug 1506792 in diskimage-builder "ironic-python-agent fails to run on fedora ramdisk" [Undecided,In progress] - Assigned to Lucas Alvares Gomes (lucasagomes) 21:00:23 https://bugs.launchpad.net/diskimage-builder/+bug/1506692 21:00:23 Launchpad bug 1506692 in diskimage-builder "Broken test in fedora-minimal/install.d/99-ramdisk" [Undecided,In progress] - Assigned to Martin André (mandre) 21:00:25 https://bugs.launchpad.net/diskimage-builder/+bug/1505900 21:00:25 Launchpad bug 1505900 in diskimage-builder "Failed to create 'wily' release with DIB_RELEASE" [Undecided,New] 21:00:27 https://bugs.launchpad.net/tripleo/+bug/1504263 21:00:27 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 21:00:29 https://bugs.launchpad.net/diskimage-builder/+bug/1506275 21:00:29 Launchpad bug 1506275 in diskimage-builder "$DIB_OFFLINE and $DIB_DEBIAN_USE_DEBOOTSTRAP_CACHE are redundant" [Undecided,New] 21:00:31 https://bugs.launchpad.net/tripleo/+bug/1507054 21:00:32 Launchpad bug 1507054 in tripleo "Allow configuration of mysql percona sst through xtrabackup-v2" [Undecided,New] 21:42:25 Steve Baker proposed openstack/python-tripleoclient: Log all resource events during overcloud deploy https://review.openstack.org/235032 00:23:14 Steve Baker proposed openstack/python-tripleoclient: Log all resource events during overcloud deploy https://review.openstack.org/235032 02:38:59 OpenStack Proposal Bot proposed openstack/diskimage-builder: Updated from global requirements https://review.openstack.org/237351 02:42:44 OpenStack Proposal Bot proposed openstack/os-cloud-config: Updated from global requirements https://review.openstack.org/231788 03:00:15 Untriaged bugs so far: 03:00:17 https://bugs.launchpad.net/tripleo/+bug/1507556 03:00:17 Launchpad bug 1507556 in tripleo "[tripleoclient] should ignore validation of tunneling params if tunneling is disabled" [Undecided,In progress] - Assigned to Brad P. Crochet (brad-9) 03:00:18 https://bugs.launchpad.net/diskimage-builder/+bug/1507631 03:00:19 Launchpad bug 1507631 in diskimage-builder "Diskimage-builder (DIB) produces a non-bootable RHEL 6.7 QCOW2-image" [Undecided,New] 03:00:21 https://bugs.launchpad.net/diskimage-builder/+bug/1506792 03:00:21 Launchpad bug 1506792 in diskimage-builder "ironic-python-agent fails to run on fedora ramdisk" [Undecided,In progress] - Assigned to Lucas Alvares Gomes (lucasagomes) 03:00:22 https://bugs.launchpad.net/diskimage-builder/+bug/1506692 03:00:22 Launchpad bug 1506692 in diskimage-builder "Broken test in fedora-minimal/install.d/99-ramdisk" [Undecided,In progress] - Assigned to Martin André (mandre) 03:00:24 https://bugs.launchpad.net/tripleo/+bug/1509148 03:00:24 Launchpad bug 1509148 in tripleo "keystone initialize fails if undercloud can't reach overcloud public vip" [Undecided,New] 03:00:26 https://bugs.launchpad.net/diskimage-builder/+bug/1505900 03:00:26 Launchpad bug 1505900 in diskimage-builder "Failed to create 'wily' release with DIB_RELEASE" [Undecided,New] 03:00:28 https://bugs.launchpad.net/tripleo/+bug/1504263 03:00:28 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 03:00:30 https://bugs.launchpad.net/diskimage-builder/+bug/1506275 03:00:30 Launchpad bug 1506275 in diskimage-builder "$DIB_OFFLINE and $DIB_DEBIAN_USE_DEBOOTSTRAP_CACHE are redundant" [Undecided,New] 03:00:33 https://bugs.launchpad.net/tripleo/+bug/1507054 03:00:33 Launchpad bug 1507054 in tripleo "Allow configuration of mysql percona sst through xtrabackup-v2" [Undecided,New] 03:28:15 Merged openstack/python-tripleoclient: Install device-mapper-multipath on overcloud-full https://review.openstack.org/231690 03:52:07 xiexs proposed openstack/os-cloud-config: Refactor _extract_driver_info() to reduce complexity. https://review.openstack.org/214128 05:09:27 Ian Wienand proposed openstack/diskimage-builder: Create YUM_CACHE_DIR in yum-minimal https://review.openstack.org/238772 05:09:28 Ian Wienand proposed openstack/diskimage-builder: Fixup RPM db path when building Fedora on Ubuntu https://review.openstack.org/238773 06:15:49 hi, all 06:16:27 could anybody help me to review a patch https://review.openstack.org/#/c/214128/, thanks in advance. 06:16:46 this patch attempts to refactor a function of os-cloud-config, which includes a lot of if-else statments. 06:18:57 and i will add more drivers into this function such as iscsi drivers, it had to add more if-else statements if no refactoring. 06:25:49 are all travelling to the summit? 07:46:11 Merged openstack/python-tripleoclient: Fix chicken and eggs problem with tripleo-heat-templates https://review.openstack.org/235769 08:28:36 Steven Hardy proposed openstack/tripleo-common: tripleo.sh initial support for stable branches https://review.openstack.org/238839 09:01:06 Marios Andreou proposed openstack/tripleo-docs: Tidy up docs - fixup missing links to TOC where possible https://review.openstack.org/236003 09:18:24 Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Enable TLS deployment for the Overcloud https://review.openstack.org/231930 09:18:43 tremble: Change the paramter's name ^^ 09:24:12 Does anybody know what would usually cause a db_sync connection timeout like this? http://paste.openstack.org/show/477247/ 09:27:34 jaosorior: Is the DB running OK? 09:27:50 IIRC there were some issues w/mongo in HA deployments, but I'm not sure on the details 09:31:34 derekh: Hey, when you get a moment, could you help me add detail to the CI TODO list here? 09:31:37 https://etherpad.openstack.org/p/tripleo-stable-branches 09:31:54 derekh: assume I know nothing about how to wire in the jobs, other than updating toci-instack ;) 09:33:31 shardy: It is indeed mongo 09:57:38 shardy: yup, will do 10:15:49 Is deploying an HA cloud with mysql as a backend for ceilometer supported? 10:15:57 Seem to be getting some weird errors there 10:25:39 jaosorior: It probably should be supported, but I don't think we generally deploy it that way 10:27:27 hewbrocca: I think this might be an issue: https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/controller.yaml#L888 10:52:13 Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Fix password issue with mysql address for ceilometer https://review.openstack.org/238879 10:58:02 that should fix it 11:31:14 Mark Chappell proposed openstack/tripleo-heat-templates: Enable static hostnames for endpoints https://review.openstack.org/238887 11:35:28 jaosorior: cool! 11:35:50 so I was thinking about this shirt "I deploy Overclouds" 11:35:56 but it's late! 11:36:53 gfidente: Going to Japan? 11:37:12 jaosorior, yes! 11:37:31 you should get prepared for the next one 11:38:36 gfidente: yeah, hopefully I get to go to the next one. 12:02:58 Mike Burns proposed openstack/python-tripleoclient: Install bigswitch networking plugin by default https://review.openstack.org/222312 12:07:49 Merged openstack/tripleo-heat-templates: Add validation to ping the default gateway(s) https://review.openstack.org/204806 12:09:30 Mike Burns proposed openstack/python-tripleoclient: remove default Neutron Network and Tunnel types https://review.openstack.org/224870 12:59:20 Merged openstack/tripleo-common: Remove puppet-heat pin https://review.openstack.org/237959 13:31:57 Mark Chappell proposed openstack/tripleo-heat-templates: Refacter ServiceEndpointMap (add two new types) https://review.openstack.org/238887 14:07:38 too many clouds, I can't find the VM i was working on 14:09:45 Found it, ran a traceroute to the IP, feels like walking around the car park pressing the unlock button 14:15:31 rofl 14:58:44 Mark Chappell proposed openstack/tripleo-heat-templates: Manage keystone related settings directly in t-h-t manifests https://review.openstack.org/230357 14:58:45 Mark Chappell proposed openstack/tripleo-heat-templates: Manage keystone related settings directly in t-h-t manifests https://review.openstack.org/230375 14:58:45 Mark Chappell proposed openstack/tripleo-heat-templates: Refacter ServiceEndpointMap and add support for explicit hostnames for endpoints https://review.openstack.org/238887 14:58:46 Mark Chappell proposed openstack/tripleo-heat-templates: Enable TLS deployment for the Overcloud https://review.openstack.org/231930 15:00:17 Untriaged bugs so far: 15:00:18 https://bugs.launchpad.net/tripleo/+bug/1507556 15:00:18 Launchpad bug 1507556 in tripleo "[tripleoclient] should ignore validation of tunneling params if tunneling is disabled" [Undecided,In progress] - Assigned to Brad P. Crochet (brad-9) 15:00:19 https://bugs.launchpad.net/diskimage-builder/+bug/1507631 15:00:20 Launchpad bug 1507631 in diskimage-builder "Diskimage-builder (DIB) produces a non-bootable RHEL 6.7 QCOW2-image" [Undecided,New] 15:00:21 https://bugs.launchpad.net/diskimage-builder/+bug/1506792 15:00:21 Launchpad bug 1506792 in diskimage-builder "ironic-python-agent fails to run on fedora ramdisk" [Undecided,In progress] - Assigned to Lucas Alvares Gomes (lucasagomes) 15:00:24 https://bugs.launchpad.net/diskimage-builder/+bug/1506692 15:00:24 Launchpad bug 1506692 in diskimage-builder "Broken test in fedora-minimal/install.d/99-ramdisk" [Undecided,In progress] - Assigned to Martin André (mandre) 15:00:25 https://bugs.launchpad.net/tripleo/+bug/1509148 15:00:26 Launchpad bug 1509148 in tripleo "keystone initialize fails if undercloud can't reach overcloud public vip" [Undecided,New] 15:00:27 https://bugs.launchpad.net/diskimage-builder/+bug/1505900 15:00:28 Launchpad bug 1505900 in diskimage-builder "Failed to create 'wily' release with DIB_RELEASE" [Undecided,New] 15:00:29 https://bugs.launchpad.net/tripleo/+bug/1504263 15:00:30 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 15:00:31 https://bugs.launchpad.net/diskimage-builder/+bug/1506275 15:00:32 Launchpad bug 1506275 in diskimage-builder "$DIB_OFFLINE and $DIB_DEBIAN_USE_DEBOOTSTRAP_CACHE are redundant" [Undecided,New] 15:00:34 https://bugs.launchpad.net/tripleo/+bug/1507054 15:00:34 Launchpad bug 1507054 in tripleo "Allow configuration of mysql percona sst through xtrabackup-v2" [Undecided,New] 16:05:02 OpenStack Proposal Bot proposed openstack/tripleo-common: Updated from global requirements https://review.openstack.org/232282 16:13:00 Derek Higgins proposed openstack-infra/tripleo-ci: [WIP] Add mirror server https://review.openstack.org/238414 16:36:52 Jaume Devesa proposed openstack/tripleo-puppet-elements: Adding networking MidoNet element https://review.openstack.org/238995 16:38:53 Jaume Devesa proposed openstack/tripleo-puppet-elements: Adding networking MidoNet element https://review.openstack.org/232551 16:43:21 Jaume Devesa proposed openstack/tripleo-heat-templates: [WIP] MidoNet heat templates https://review.openstack.org/238998 18:18:01 Ana Krivokapic proposed openstack/tripleo-specs: Add spec for TripleO UI https://review.openstack.org/239056 18:29:05 OpenStack Proposal Bot proposed openstack/tuskar-ui: Updated from global requirements https://review.openstack.org/239058 19:12:34 Jeff Peeler proposed openstack/python-tripleoclient: Add ability to add custom overcloud image https://review.openstack.org/239090 19:39:06 Joe Talerico proposed openstack/tripleo-heat-templates: The default balance-tcp is causing issues with deployments. Defaulting to active-backup. https://review.openstack.org/238638 19:39:42 Joe Talerico proposed openstack/tripleo-heat-templates: The default balance-tcp is causing issues with deployments. Defaulting to active-backup. https://review.openstack.org/238638 19:40:01 Joe Talerico proposed openstack/tripleo-heat-templates: The default balance-tcp is causing issues with deployments. Defaulting to active-backup. https://review.openstack.org/238638 19:40:16 Joe Talerico proposed openstack/tripleo-heat-templates: The default balance-tcp is causing issues with deployments. Defaulting to active-backup. https://review.openstack.org/238638 19:54:48 anyone around that would know how I could get the value of KeystonePublicApiVirtualIP from within overcloud_controller.pp? 19:55:16 https://bugzilla.redhat.com/show_bug.cgi?id=1272572 seems like an easy fix with that 19:55:16 bugzilla.redhat.com bug 1272572 in rdo-manager "Error: Unable to retrieve volume limit information when accessing System Defaults in Horizon" [High,New] - Assigned to calfonso 21:00:17 Untriaged bugs so far: 21:00:19 https://bugs.launchpad.net/tripleo/+bug/1507556 21:00:19 Launchpad bug 1507556 in tripleo "[tripleoclient] should ignore validation of tunneling params if tunneling is disabled" [Undecided,In progress] - Assigned to Brad P. Crochet (brad-9) 21:00:20 https://bugs.launchpad.net/diskimage-builder/+bug/1507631 21:00:20 Launchpad bug 1507631 in diskimage-builder "Diskimage-builder (DIB) produces a non-bootable RHEL 6.7 QCOW2-image" [Undecided,New] 21:00:22 https://bugs.launchpad.net/diskimage-builder/+bug/1506792 21:00:22 Launchpad bug 1506792 in diskimage-builder "ironic-python-agent fails to run on fedora ramdisk" [Undecided,In progress] - Assigned to Lucas Alvares Gomes (lucasagomes) 21:00:24 https://bugs.launchpad.net/diskimage-builder/+bug/1506692 21:00:24 Launchpad bug 1506692 in diskimage-builder "Broken test in fedora-minimal/install.d/99-ramdisk" [Undecided,In progress] - Assigned to Martin André (mandre) 21:00:26 https://bugs.launchpad.net/tripleo/+bug/1509148 21:00:26 Launchpad bug 1509148 in tripleo "keystone initialize fails if undercloud can't reach overcloud public vip" [Undecided,New] 21:00:28 https://bugs.launchpad.net/diskimage-builder/+bug/1505900 21:00:28 Launchpad bug 1505900 in diskimage-builder "Failed to create 'wily' release with DIB_RELEASE" [Undecided,New] 21:00:30 https://bugs.launchpad.net/tripleo/+bug/1504263 21:00:31 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 21:00:32 https://bugs.launchpad.net/diskimage-builder/+bug/1506275 21:00:32 Launchpad bug 1506275 in diskimage-builder "$DIB_OFFLINE and $DIB_DEBIAN_USE_DEBOOTSTRAP_CACHE are redundant" [Undecided,New] 21:00:34 https://bugs.launchpad.net/tripleo/+bug/1507054 21:00:35 Launchpad bug 1507054 in tripleo "Allow configuration of mysql percona sst through xtrabackup-v2" [Undecided,New] 21:02:57 John Trowbridge proposed openstack/tripleo-heat-templates: Configure cinder [keymgr]/encryption_auth_url https://review.openstack.org/239109 21:15:08 Dan Sneddon proposed openstack/tripleo-heat-templates: Add Management Network For System Administration. https://review.openstack.org/226057 21:17:10 Dan Sneddon proposed openstack/tripleo-heat-templates: Add Management Network For System Administration. https://review.openstack.org/226057 00:21:27 John Trowbridge proposed openstack/tripleo-heat-templates: Configure cinder [keymgr]/encryption_auth_url https://review.openstack.org/239109 00:28:36 OpenStack Proposal Bot proposed openstack/tuskar: Updated from global requirements https://review.openstack.org/233887 00:57:52 Merged openstack/diskimage-builder: Create YUM_CACHE_DIR in yum-minimal https://review.openstack.org/238772 02:06:51 OpenStack Proposal Bot proposed openstack/diskimage-builder: Updated from global requirements https://review.openstack.org/237351 03:00:16 Untriaged bugs so far: 03:00:18 https://bugs.launchpad.net/tripleo/+bug/1507556 03:00:18 Launchpad bug 1507556 in tripleo "[tripleoclient] should ignore validation of tunneling params if tunneling is disabled" [Undecided,In progress] - Assigned to Brad P. Crochet (brad-9) 03:00:19 https://bugs.launchpad.net/diskimage-builder/+bug/1507631 03:00:19 Launchpad bug 1507631 in diskimage-builder "Diskimage-builder (DIB) produces a non-bootable RHEL 6.7 QCOW2-image" [Undecided,New] 03:00:21 https://bugs.launchpad.net/diskimage-builder/+bug/1506792 03:00:21 Launchpad bug 1506792 in diskimage-builder "ironic-python-agent fails to run on fedora ramdisk" [Undecided,In progress] - Assigned to Lucas Alvares Gomes (lucasagomes) 03:00:23 https://bugs.launchpad.net/diskimage-builder/+bug/1506692 03:00:23 Launchpad bug 1506692 in diskimage-builder "Broken test in fedora-minimal/install.d/99-ramdisk" [Undecided,In progress] - Assigned to Martin André (mandre) 03:00:25 https://bugs.launchpad.net/tripleo/+bug/1509148 03:00:26 Launchpad bug 1509148 in tripleo "keystone initialize fails if undercloud can't reach overcloud public vip" [Undecided,New] 03:00:27 https://bugs.launchpad.net/diskimage-builder/+bug/1505900 03:00:28 Launchpad bug 1505900 in diskimage-builder "Failed to create 'wily' release with DIB_RELEASE" [Undecided,New] 03:00:29 https://bugs.launchpad.net/tripleo/+bug/1504263 03:00:29 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 03:00:31 https://bugs.launchpad.net/diskimage-builder/+bug/1506275 03:00:32 Launchpad bug 1506275 in diskimage-builder "$DIB_OFFLINE and $DIB_DEBIAN_USE_DEBOOTSTRAP_CACHE are redundant" [Undecided,New] 03:00:33 https://bugs.launchpad.net/tripleo/+bug/1507054 03:00:33 Launchpad bug 1507054 in tripleo "Allow configuration of mysql percona sst through xtrabackup-v2" [Undecided,New] 07:20:24 James Slagle proposed openstack/instack-undercloud: Make 98-undercloud-setup rerunnable https://review.openstack.org/222227 09:00:16 Untriaged bugs so far: 09:00:18 https://bugs.launchpad.net/tripleo/+bug/1507556 09:00:18 Launchpad bug 1507556 in tripleo "[tripleoclient] should ignore validation of tunneling params if tunneling is disabled" [Undecided,In progress] - Assigned to Brad P. Crochet (brad-9) 09:00:19 https://bugs.launchpad.net/diskimage-builder/+bug/1507631 09:00:19 Launchpad bug 1507631 in diskimage-builder "Diskimage-builder (DIB) produces a non-bootable RHEL 6.7 QCOW2-image" [Undecided,New] 09:00:21 https://bugs.launchpad.net/diskimage-builder/+bug/1506792 09:00:21 Launchpad bug 1506792 in diskimage-builder "ironic-python-agent fails to run on fedora ramdisk" [Undecided,In progress] - Assigned to Lucas Alvares Gomes (lucasagomes) 09:00:23 https://bugs.launchpad.net/diskimage-builder/+bug/1506692 09:00:23 Launchpad bug 1506692 in diskimage-builder "Broken test in fedora-minimal/install.d/99-ramdisk" [Undecided,In progress] - Assigned to Martin André (mandre) 09:00:25 https://bugs.launchpad.net/tripleo/+bug/1509148 09:00:25 Launchpad bug 1509148 in tripleo "keystone initialize fails if undercloud can't reach overcloud public vip" [Undecided,New] 09:00:27 https://bugs.launchpad.net/diskimage-builder/+bug/1505900 09:00:28 Launchpad bug 1505900 in diskimage-builder "Failed to create 'wily' release with DIB_RELEASE" [Undecided,New] 09:00:29 https://bugs.launchpad.net/tripleo/+bug/1504263 09:00:29 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 09:00:31 https://bugs.launchpad.net/diskimage-builder/+bug/1506275 09:00:31 Launchpad bug 1506275 in diskimage-builder "$DIB_OFFLINE and $DIB_DEBIAN_USE_DEBOOTSTRAP_CACHE are redundant" [Undecided,New] 09:00:32 https://bugs.launchpad.net/tripleo/+bug/1507054 09:00:32 Launchpad bug 1507054 in tripleo "Allow configuration of mysql percona sst through xtrabackup-v2" [Undecided,New] 15:49:50 Julia Kreger proposed openstack/diskimage-builder: WIP: iso element compatability for ubuntu EFI https://review.openstack.org/236973 16:33:52 Dmitry Tantsur proposed openstack/diskimage-builder: Add a tox target to run functional tests locally https://review.openstack.org/229335 20:29:30 Derek Higgins proposed openstack-infra/tripleo-ci: [WIP] Add mirror server https://review.openstack.org/238414 21:00:17 Untriaged bugs so far: 21:00:18 https://bugs.launchpad.net/tripleo/+bug/1507556 21:00:18 Launchpad bug 1507556 in tripleo "[tripleoclient] should ignore validation of tunneling params if tunneling is disabled" [Undecided,In progress] - Assigned to Brad P. Crochet (brad-9) 21:00:19 https://bugs.launchpad.net/diskimage-builder/+bug/1507631 21:00:19 Launchpad bug 1507631 in diskimage-builder "Diskimage-builder (DIB) produces a non-bootable RHEL 6.7 QCOW2-image" [Undecided,New] 21:00:22 https://bugs.launchpad.net/diskimage-builder/+bug/1506792 21:00:22 Launchpad bug 1506792 in diskimage-builder "ironic-python-agent fails to run on fedora ramdisk" [Undecided,In progress] - Assigned to Lucas Alvares Gomes (lucasagomes) 21:00:24 https://bugs.launchpad.net/diskimage-builder/+bug/1506692 21:00:24 Launchpad bug 1506692 in diskimage-builder "Broken test in fedora-minimal/install.d/99-ramdisk" [Undecided,In progress] - Assigned to Martin André (mandre) 21:00:25 https://bugs.launchpad.net/tripleo/+bug/1509148 21:00:25 Launchpad bug 1509148 in tripleo "keystone initialize fails if undercloud can't reach overcloud public vip" [Undecided,New] 21:00:27 https://bugs.launchpad.net/diskimage-builder/+bug/1505900 21:00:27 Launchpad bug 1505900 in diskimage-builder "Failed to create 'wily' release with DIB_RELEASE" [Undecided,New] 21:00:30 https://bugs.launchpad.net/tripleo/+bug/1504263 21:00:30 Launchpad bug 1504263 in tripleo "os-net-config mapping by MAC fails if trying to map in alternate order" [Undecided,Triaged] 21:00:31 https://bugs.launchpad.net/diskimage-builder/+bug/1506275 21:00:32 Launchpad bug 1506275 in diskimage-builder "$DIB_OFFLINE and $DIB_DEBIAN_USE_DEBOOTSTRAP_CACHE are redundant" [Undecided,New] 21:00:34 https://bugs.launchpad.net/tripleo/+bug/1507054 21:00:34 Launchpad bug 1507054 in tripleo "Allow configuration of mysql percona sst through xtrabackup-v2" [Undecided,New] 22:26:22 Derek Higgins proposed openstack-infra/tripleo-ci: [WIP] Add mirror server https://review.openstack.org/238414