| *** jcoufal_ has joined #tripleo | 00:13 | |
| *** jcoufal has quit IRC | 00:15 | |
| *** lazy_prince has joined #tripleo | 00:18 | |
| *** lazy_prince has quit IRC | 00:22 | |
| *** bana_k has quit IRC | 00:23 | |
| *** onovy has quit IRC | 00:23 | |
| *** onovy has joined #tripleo | 00:23 | |
| *** saneax is now known as saneax_AFK | 00:31 | |
| *** limao has joined #tripleo | 00:31 | |
| *** dmacpher has joined #tripleo | 00:40 | |
| *** myoung|biab is now known as myoung | 00:49 | |
| *** lblanchard has joined #tripleo | 00:56 | |
| *** rhallisey has quit IRC | 01:05 | |
| *** rbrady has quit IRC | 01:13 | |
| *** chem has quit IRC | 01:19 | |
| *** chem has joined #tripleo | 01:19 | |
| *** fultonj has quit IRC | 01:23 | |
| *** jcoufal_ has quit IRC | 01:28 | |
| *** tbonds has joined #tripleo | 01:34 | |
| *** jcoufal has joined #tripleo | 01:41 | |
| *** bana_k has joined #tripleo | 01:46 | |
| *** tzumainn has quit IRC | 01:48 | |
| *** chem has quit IRC | 01:52 | |
| *** chem has joined #tripleo | 01:53 | |
| *** thrash is now known as thrash|g0ne | 01:53 | |
| *** jcoufal has quit IRC | 02:04 | |
| *** ramishra has quit IRC | 02:04 | |
| *** jcoufal has joined #tripleo | 02:05 | |
| *** milan has quit IRC | 02:06 | |
| *** ramishra has joined #tripleo | 02:07 | |
| *** maeca1 has joined #tripleo | 02:07 | |
| *** milan has joined #tripleo | 02:11 | |
| *** bana_k has quit IRC | 02:29 | |
| *** tbonds has quit IRC | 02:29 | |
| *** milan has quit IRC | 02:36 | |
| *** milan has joined #tripleo | 02:37 | |
| *** yamahata has quit IRC | 02:52 | |
| *** bana_k has joined #tripleo | 02:55 | |
| *** lazy_prince has joined #tripleo | 03:00 | |
| *** dtrainor has quit IRC | 03:03 | |
| *** lazy_prince has quit IRC | 03:05 | |
| *** lazy_prince has joined #tripleo | 03:05 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: WIP - Implement undercloud upgrade job - Mitaka -> Newton https://review.openstack.org/346995 | 03:08 |
|---|---|---|
| *** lblanchard has quit IRC | 03:17 | |
| *** r-mibu has quit IRC | 03:23 | |
| *** r-mibu has joined #tripleo | 03:24 | |
| *** coolsvap has joined #tripleo | 03:28 | |
| *** mgarciam has quit IRC | 03:42 | |
| *** jcoufal has quit IRC | 03:50 | |
| *** masco has joined #tripleo | 04:08 | |
| *** julim has quit IRC | 04:19 | |
| *** saneax_AFK is now known as saneax | 04:28 | |
| *** crinkle has quit IRC | 04:35 | |
| *** crinkle has joined #tripleo | 04:35 | |
| *** links has joined #tripleo | 04:36 | |
| bandini | morning | 04:43 |
| *** maeca1 has quit IRC | 04:45 | |
| bandini | seems CI is failing on ceph stuff now /me looks | 04:55 |
| *** oshvartz has quit IRC | 04:55 | |
| *** rlandy has quit IRC | 04:59 | |
| *** milan has quit IRC | 05:07 | |
| *** milan has joined #tripleo | 05:07 | |
| *** radek_ has joined #tripleo | 05:11 | |
| *** jaosorior has joined #tripleo | 05:19 | |
| bandini | I filed https://bugs.launchpad.net/tripleo/+bug/1608785 for the CI failures, I added the alert tag, but could not set the importance | 05:24 |
| openstack | Launchpad bug 1608785 in tripleo "gate-tripleo-ci-centos-7-ovb-ha CI jobs fail with ceph-mon errors" [Undecided,New] | 05:24 |
| *** limao_ has joined #tripleo | 05:28 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/instack-undercloud: Enable SSL for ironic inspector https://review.openstack.org/332641 | 05:28 |
| jaosorior | bandini: Seems that the same issue appears on the nonha jobs | 05:29 |
| *** limao has quit IRC | 05:30 | |
| *** ramishra has quit IRC | 05:33 | |
| bandini | jaosorior: do you have a link handy? | 05:35 |
| *** ramishra has joined #tripleo | 05:35 | |
| bandini | I did not spot any but I admittedly took a very quick look | 05:35 |
| bandini | jaosorior: could you set the critical tag on the bug, seems I am not allowed to | 05:35 |
| *** chem has quit IRC | 05:35 | |
| jaosorior | bandini: How do you know that If627a50d8aa0cba0f1b4e00dfd43103c7eaff6f6 introduced the problem? It did seem to pass the nonha job there at least | 05:36 |
| jaosorior | bandini: this is the link https://review.openstack.org/#/c/349606/ | 05:36 |
| jaosorior | and the logs I see for the failed deployment are these: http://paste.openstack.org/show/545515/ | 05:37 |
| bandini | jaosorior: I assumed that ceph was not enabled when that job passed, because it got enabled last night | 05:38 |
| jaosorior | I see | 05:39 |
| jaosorior | bandini: Which commit enabled it? | 05:40 |
| bandini | maybe the root cause is something else, I would not exclude it yet. The commit enabling it seems to have passed which might give us more hints: https://review.openstack.org/#/c/346897/ | 05:41 |
| *** derekjhyang has quit IRC | 05:42 | |
| jaosorior | bandini: might be something else. Because it's also failing in the nonha job. Which had no change | 05:42 |
| jaosorior | it does enable the ceph-devel environment though | 05:42 |
| *** bana_k has quit IRC | 05:42 | |
| bandini | jaosorior: do you have a link of a failed nonha job? | 05:43 |
| bandini | I am clearly blind ;) | 05:43 |
| jaosorior | bandini: I sent it already | 05:43 |
| jaosorior | bandini: this is the link https://review.openstack.org/#/c/349606/ to the failed commit | 05:44 |
| bandini | jaosorior: http://paste.openstack.org/show/545515/ this one? I mean a link from the CI not the messages | 05:44 |
| bandini | ah sorry | 05:44 |
| bandini | can I blame lack of coffee? | 05:44 |
| jaosorior | bandini: You sure can haha. I can relate entirely | 05:44 |
| jaosorior | marios: the error you saw in your testing of manila was due to an error in doing a git clone. So nothing related to your commit. Just need to recheck | 05:47 |
| jaosorior | bandini: I have no idea what could have changed yesterday related to ceph that's now breaking :( | 05:50 |
| bandini | jaosorior: it is weird that the review enabling ceph passed all the CI jobs that are now failing, yes | 05:54 |
| jaosorior | bandini: try reverting that commit and lets see if it passes | 05:54 |
| bandini | jaosorior: am downloading the logs of one ci job that passed with the ceph-enabling review and will poke at it. will try a revert to | 05:55 |
| *** trozet has quit IRC | 05:55 | |
| jaosorior | bandini: I can do the revert while you look at those logs. No worries | 05:56 |
| jaosorior | bandini: I'll just propose it and see what CI throws out | 05:56 |
| bandini | jaosorior: ack, thanks | 05:57 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/tripleo-ci: Revert "Enable Ceph in OVB/HA2" https://review.openstack.org/349810 | 05:58 |
| *** florianf has joined #tripleo | 06:03 | |
| *** numans has joined #tripleo | 06:03 | |
| marios | jaosorior: ack thanks i did | 06:04 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/puppet-tripleo: Remove unnecessary certmonger manifest https://review.openstack.org/349812 | 06:06 |
| jaosorior | marios: Hey dude, by the way, all the gates seem to be broken due to a ceph issue. How acquainted are you with it? | 06:06 |
| jaosorior | I mean, with ceph | 06:06 |
| *** ooolpbot has joined #tripleo | 06:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1608785 | 06:10 |
| *** ooolpbot has quit IRC | 06:10 | |
| openstack | Launchpad bug 1608785 in tripleo "CI jobs fail with ceph-mon errors" [Critical,New] | 06:10 |
| *** oshvartz has joined #tripleo | 06:15 | |
| marios | jaosorior: not so much, but depends what the issue is i mean is it packaging or a breaking change or what?for actual ceph/tripleo stuff usually gfidente is your man | 06:18 |
| marios | jaosorior: looking at the bug | 06:19 |
| marios | jaosorior: so looks like bandini has already dub into that a bit? | 06:21 |
| marios | dug even | 06:21 |
| *** bana_k has joined #tripleo | 06:27 | |
| *** dculik|gone is now known as dculik | 06:32 | |
| jaosorior | marios: Hey dude, could you give a read to https://review.openstack.org/#/c/332774/ ? I'm testing it with this test commit https://review.openstack.org/#/c/349606/ and it is setting up TLS using certmonger there :D only thing is that the commit is failing due to the ceph issues. But FWIW the commit affects the undercloud, and the ceph issue is for the overcloud. | 06:34 |
| marios | jaosorior: ack | 06:35 |
| *** pgadiya has joined #tripleo | 06:37 | |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates: Enable Manila integration - as a composable controller service https://review.openstack.org/188137 | 06:38 |
| bandini | marios: I am still looking, but have no idea how the enabling of ceph passed CI. I am probably missing something obvious | 06:42 |
| *** tesseract- has joined #tripleo | 06:43 | |
| *** liverpooler has joined #tripleo | 06:45 | |
| *** bana_k has quit IRC | 06:46 | |
| *** leanderthal|afk is now known as leanderthal | 06:47 | |
| *** bootsha has joined #tripleo | 06:48 | |
| *** mcornea has joined #tripleo | 06:56 | |
| *** pgadiya has quit IRC | 07:01 | |
| *** aufi has joined #tripleo | 07:06 | |
| *** pgadiya has joined #tripleo | 07:07 | |
| *** jpena|off is now known as jpena | 07:07 | |
| ccamacho | morning guys! | 07:08 |
| *** ooolpbot has joined #tripleo | 07:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1608785 | 07:10 |
| *** ooolpbot has quit IRC | 07:10 | |
| openstack | Launchpad bug 1608785 in tripleo "CI jobs fail with ceph-mon errors" [Critical,New] | 07:10 |
| jaosorior | ccamacho sup dude | 07:11 |
| ccamacho | jaosorior, mind to send some fresh air from Finland to Spain!?? :P | 07:11 |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-docs: Deploy and Scale Swift https://review.openstack.org/293311 | 07:13 |
| jaosorior | ccamacho hahaha would if I could. Wish it was a bit warmer here | 07:14 |
| *** devvesa has joined #tripleo | 07:15 | |
| openstackgerrit | Carlos Camacho proposed openstack/puppet-tripleo: Fix mistral profile steps https://review.openstack.org/343346 | 07:16 |
| *** hewbrocca-afk is now known as hewbrocca | 07:19 | |
| openstackgerrit | Carlos Camacho proposed openstack/tripleo-heat-templates: Composable Mistral services https://review.openstack.org/323436 | 07:19 |
| *** cschwede has joined #tripleo | 07:19 | |
| *** zoli_gone-proxy is now known as zoliXXL | 07:19 | |
| *** jpich has joined #tripleo | 07:20 | |
| *** sshnaidm|afk has quit IRC | 07:24 | |
| jaosorior | marios: hey dude, at what step do we add the ramdisk, kernel and image_source to the ironic nodes? | 07:24 |
| *** sshnaidm|afk has joined #tripleo | 07:25 | |
| *** ccamacho is now known as ccamacho|afk | 07:26 | |
| marios | jaosorior: so used to be with openstack baremetal configure boot | 07:27 |
| marios | jaosorior: but i think this is now changed? see the tripleo docs... that what you mean? | 07:27 |
| *** abehl has joined #tripleo | 07:28 | |
| jaosorior | marios: I did try configure boot, but I can't do an overcloud deploy, and doing an ironic node-validate I get this: http://paste.openstack.org/show/545545/ | 07:28 |
| marios | This does not require docs as it was an issue found during the test upgrades | 07:29 |
| marios | haha sorry paste err | 07:29 |
| marios | jaosorior: hmm that is stgrange..but are you deploying mitaka? or master? | 07:29 |
| *** jpich has quit IRC | 07:29 | |
| jaosorior | marios: master | 07:30 |
| *** shardy has joined #tripleo | 07:30 | |
| marios | jaosorior: i think for master you shouldn't have to do configure boot anymore dtantsur|afk or lucasagomes are probably the people to ask though | 07:30 |
| *** dsariel has joined #tripleo | 07:30 | |
| jaosorior | shardy: We're not deploying swift-proxy anymore, right? | 07:30 |
| jaosorior | shardy: Seems that we're only configuring the file store for glance. So if swift is around it fails cause it doesn't accept that file store. So just wanted to make sure if that's correct. | 07:33 |
| *** ebarrera has joined #tripleo | 07:35 | |
| *** mbound has joined #tripleo | 07:35 | |
| shardy | jaosorior: we should still be deploying swift-proxy | 07:37 |
| shardy | https://github.com/openstack/tripleo-heat-templates/blob/master/overcloud-resource-registry-puppet.yaml#L180 | 07:37 |
| shardy | https://github.com/openstack/tripleo-heat-templates/blob/master/overcloud.yaml#L391 | 07:38 |
| jaosorior | shardy: what about the undercloud? | 07:38 |
| shardy | jaosorior: we're still deploying swift on the undercloud, we just removed some rax specific auth middleware AFAIK | 07:38 |
| jaosorior | shardy: cause it seems to me that we only configure the file store https://github.com/openstack/instack-undercloud/blob/master/elements/puppet-stack-config/puppet-stack-config.yaml.template#L75 | 07:39 |
| jaosorior | if that's the case I could submit a fix | 07:39 |
| shardy | we did switch out the glance backend | 07:39 |
| jaosorior | I see | 07:39 |
| shardy | but that was supposed to be temporary, I'm not sure if it's been reverted yet | 07:39 |
| shardy | we were hitting issues with timeouts uploading images IIRC, so we switched to a file backend just to get CI working | 07:39 |
| *** dmacpher has quit IRC | 07:40 | |
| shardy | jaosorior: If you look at git blame, you'll see the bug reference where we switched that to file | 07:40 |
| shardy | you can propose to revert that, but we should recheck a few times to ensure the problems are now fixed | 07:40 |
| shardy | I think the adjustments made to the proxy pipeline should have fixed it, but not confirmed yet | 07:41 |
| *** jpich has joined #tripleo | 07:41 | |
| *** zoliXXL is now known as zoli|brb | 07:42 | |
| shardy | https://github.com/openstack/instack-undercloud/commit/b8c5ac736733e28315364a0c9e70465b6f41166d | 07:42 |
| shardy | jaosorior: ^^ | 07:42 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/instack-undercloud: Revert "glance: disable swift backend" https://review.openstack.org/349843 | 07:42 |
| jaosorior | shardy ^^ | 07:43 |
| jaosorior | lets see how that goes | 07:43 |
| shardy | https://github.com/openstack/instack-undercloud/commit/0b42b6df23f64a7b13d7f809ecd4c1642640e3a7 | 07:43 |
| shardy | that's the commit which was supposed to fix the OOM problems | 07:43 |
| shardy | so you might want to reference that in your commit message | 07:44 |
| jaosorior | will do | 07:44 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/instack-undercloud: Revert "glance: disable swift backend" https://review.openstack.org/349843 | 07:47 |
| openstackgerrit | Christopher Brown proposed openstack/tripleo-heat-templates: Enable Neutron LBaaS Integration https://review.openstack.org/313933 | 07:54 |
| openstackgerrit | Gabriele Cerami proposed openstack/tripleo-quickstart: WIP: Reproducing specific CI run https://review.openstack.org/340156 | 07:54 |
| *** yamahata has joined #tripleo | 07:56 | |
| *** ohamada has joined #tripleo | 07:58 | |
| jaosorior | bandini: Doing the revert https://review.openstack.org/#/c/349810/ seems to make the ha gate pass | 08:02 |
| bandini | jaosorior: okay, that is expected. before you mentioned something about the ceph-devel stuff, could you expand a bit? is that a special repo? | 08:04 |
| jaosorior | bandini: No, just a heat environment that is added to the nonha jobb | 08:04 |
| bandini | ah ok | 08:04 |
| jaosorior | that pretty much enables ceph there | 08:04 |
| *** Goneri has joined #tripleo | 08:07 | |
| openstackgerrit | Carlos Camacho proposed openstack/tripleo-heat-templates: Disable mistral services by default. https://review.openstack.org/349408 | 08:09 |
| *** ooolpbot has joined #tripleo | 08:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1608785 | 08:10 |
| *** ooolpbot has quit IRC | 08:10 | |
| openstack | Launchpad bug 1608785 in tripleo "CI jobs fail with ceph-mon errors" [Critical,New] | 08:10 |
| *** jaosorior has quit IRC | 08:11 | |
| *** jaosorior has joined #tripleo | 08:12 | |
| shardy | bandini: Hey, thanks for the ceph related CI bug report - we'll have to chat with gfidente when he gets in | 08:12 |
| shardy | we can revert ceph coverage, but we need a plan to reinstate it, ideally including a switch to ceph jewel | 08:13 |
| *** zoli|brb is now known as zoli | 08:15 | |
| *** zoli is now known as zoliXXL | 08:15 | |
| *** TSCHAK_ has joined #tripleo | 08:15 | |
| *** jmiu has quit IRC | 08:16 | |
| *** jmiu_ is now known as jmiu | 08:16 | |
| bandini | shardy: I am truly puzzled as to how the enabling of ceph passed CI, there were no errors there. | 08:16 |
| *** TSCHAK_ has quit IRC | 08:16 | |
| bandini | Maybe the issue is somewhere else | 08:16 |
| *** jmiu_ has joined #tripleo | 08:16 | |
| *** jeckersb has quit IRC | 08:16 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/tripleo-ci: Revert "Enable Ceph in OVB/HA2" https://review.openstack.org/349810 | 08:17 |
| *** TSCHAK has quit IRC | 08:17 | |
| *** jaosorior_ has joined #tripleo | 08:18 | |
| *** TSCHAK has joined #tripleo | 08:18 | |
| *** yamahata has quit IRC | 08:18 | |
| shardy | bandini: Yeah I'm not sure yet either, will dig into it if gfidente isn't around soon | 08:18 |
| *** chem has joined #tripleo | 08:18 | |
| *** karthiks has quit IRC | 08:19 | |
| *** devvesa has quit IRC | 08:20 | |
| *** karthiks has joined #tripleo | 08:20 | |
| bandini | I'd wait a bit, gfidente should be online shortly | 08:20 |
| *** jeckersb has joined #tripleo | 08:21 | |
| *** jaosorior has quit IRC | 08:21 | |
| assassin | bandini++ | 08:21 |
| openstackgerrit | Sanjay Upadhyay proposed openstack/tripleo-heat-templates: SR-IOV nic agent changes in line with composable roles. https://review.openstack.org/332190 | 08:22 |
| *** Tristitia has quit IRC | 08:22 | |
| *** fungi has quit IRC | 08:22 | |
| bandini | assassin++ hello, sir! long time no see ;) | 08:22 |
| *** dtantsur|afk is now known as dtantsur | 08:22 | |
| assassin | bandini: same to you :) I'm guessing you might hear more from me now when I'm getting in to EPM and the whole RHOSP/NFV game | 08:23 |
| bandini | assassin: aye, glad to have you around ;) | 08:23 |
| assassin | he says that now.. | 08:24 |
| dtantsur | jaosorior_, hi! configure boot is not required on master, and the validation failure you see is normal, when nova is usecd | 08:24 |
| assassin | ;) | 08:24 |
| *** sshnaidm|afk is now known as sshnaidm | 08:24 | |
| bandini | assassin: true that, I am sure I will live in fear from now on :D | 08:24 |
| *** Tristitia has joined #tripleo | 08:24 | |
| assassin | *grin* | 08:24 |
| *** fungi has joined #tripleo | 08:24 | |
| assassin | bandini: nah, I will play nice. still learning a lot | 08:24 |
| *** links has quit IRC | 08:26 | |
| *** greghaynes has quit IRC | 08:29 | |
| *** pgadiya has quit IRC | 08:34 | |
| *** pgadiya has joined #tripleo | 08:36 | |
| *** greghaynes has joined #tripleo | 08:40 | |
| *** isq_ has quit IRC | 08:40 | |
| *** isq_ has joined #tripleo | 08:41 | |
| *** links has joined #tripleo | 08:41 | |
| *** devvesa has joined #tripleo | 08:47 | |
| *** mbound has quit IRC | 08:47 | |
| *** bootsha has quit IRC | 08:53 | |
| *** gfidente has joined #tripleo | 08:53 | |
| *** gfidente has quit IRC | 08:53 | |
| *** gfidente has joined #tripleo | 08:53 | |
| *** bootsha has joined #tripleo | 08:53 | |
| *** stendulker has joined #tripleo | 08:54 | |
| *** bootsha has quit IRC | 08:54 | |
| *** snecklifter has quit IRC | 08:55 | |
| *** electrofelix has joined #tripleo | 08:56 | |
| *** mbound has joined #tripleo | 08:56 | |
| *** bootsha has joined #tripleo | 08:58 | |
| *** bootsha has quit IRC | 08:59 | |
| *** fzdarsky has joined #tripleo | 09:00 | |
| *** bootsha has joined #tripleo | 09:00 | |
| *** bootsha has quit IRC | 09:01 | |
| *** bootsha has joined #tripleo | 09:01 | |
| openstackgerrit | Michele Baldessari proposed openstack/tripleo-heat-templates: Next generation HA architecture work https://review.openstack.org/314208 | 09:05 |
| shardy | d0ugal, jpich: Hey, trying out some of the latest tripleoclient + mistral stuff | 09:08 |
| d0ugal | Hey :) | 09:08 |
| shardy | is there a patch for "plan create"? | 09:08 |
| shardy | d0ugal: Hi! | 09:08 |
| d0ugal | shardy: For the CLI? Not that I am aware of. | 09:09 |
| shardy | d0ugal: so I tried the overcloud deploy patches, and as expected it failed, but the first thing I noticed, is there's no overcloud container in swift | 09:09 |
| shardy | so, I assume I manually run the workflow | 09:09 |
| shardy | but it seems like by default we need to create the plan based on the --templates input | 09:09 |
| d0ugal | shardy: Right, I had been triggering the workflow for that manually. I think the idea is for the default plan to be created at install time. | 09:09 |
| shardy | but probably also allow operators to pre-create plans, then select then on deploy time | 09:09 |
| jpich | Hey | 09:10 |
| shardy | d0ugal: Ok, can you paste the commands you're using to save me some time please? | 09:10 |
| *** ooolpbot has joined #tripleo | 09:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1608785 | 09:10 |
| *** ooolpbot has quit IRC | 09:10 | |
| openstack | Launchpad bug 1608785 in tripleo "CI jobs fail with ceph-mon errors" [Critical,New] | 09:10 |
| * shardy still getting familiar with mistralclient | 09:10 | |
| shardy | jpich: Hi! | 09:10 |
| jpich | d0ugal, shardy: I've been wondering about plan management from the CLI as well (I'm looking at it from the context of https://blueprints.launchpad.net/tripleo-ui/+spec/tripleo-ui-mistral-refactoring ) | 09:10 |
| jpich | That is something we're going to want as well, right? | 09:11 |
| shardy | It seems like something that will fit with both the CLI and UI flow to me | 09:11 |
| d0ugal | shardy: you just trigger the workflow with mistral, something like: mistral execution-create tripleo.plan_management.v1.create_default_deployment_plan '{"container":"overcloud"}' | 09:11 |
| bandini | gfidente: morning sir. can you hit me with a cluebat about https://bugs.launchpad.net/tripleo/+bug/1608785 ? | 09:11 |
| openstack | Launchpad bug 1608785 in tripleo "CI jobs fail with ceph-mon errors" [Critical,New] | 09:11 |
| *** dculik has quit IRC | 09:11 | |
| d0ugal | shardy: I need to check that is correct, I am just fixing some dev env issues :( | 09:11 |
| shardy | d0ugal: np, I figured that was what I'd need, but there's also a create_deployment_plan | 09:12 |
| shardy | and I wasn't sure how to pass my local t-h-t checkout to it | 09:12 |
| d0ugal | shardy: create_deployment_plan accepts a JSON "dict" of templates IIRC | 09:12 |
| shardy | we need that in the tripleoclient patch really | 09:12 |
| d0ugal | shardy: the default one uses the packaged templates | 09:12 |
| shardy | d0ugal: ack, my local templates are about 30 commits diverged from master at this point, but I can revert to the trunk ones or build a tht package ;) | 09:13 |
| d0ugal | shardy: Yeah, we do. I think for the deploy command we will need to create plans on the fly | 09:13 |
| jpich | d0ugal: Now I wonder if I reimplemented something that already exists... Where is this? (Create plan from JSON dict) | 09:13 |
| *** akrivoka has joined #tripleo | 09:13 | |
| shardy | d0ugal: Ok, if you'd be able to hack that into your tripleoclient path that would be good | 09:13 |
| shardy | I'm happy to help with test/reviews today | 09:13 |
| d0ugal | jpich: I may be miss-remembering, let me check. | 09:13 |
| d0ugal | shardy: Great, thanks. | 09:13 |
| d0ugal | shardy: My plan today is to update the CLI patch and test again with rbrady's updates. Then I'll add the plan creation | 09:14 |
| shardy | https://github.com/openstack/tripleo-common/blob/master/workbooks/plan_management.yaml#L8 | 09:14 |
| d0ugal | I need to totally rework that patch, it leaves a ton of dead code which needs untangled too. fun | 09:14 |
| shardy | jpich: that's the workflow I was looking at | 09:14 |
| shardy | but it appears to take a container input | 09:14 |
| shardy | so I assume you push all-the-things to swift | 09:15 |
| shardy | then pass in the container ID | 09:15 |
| shardy | (not tested) | 09:15 |
| d0ugal | shardy, jpich: Oh yeah, I think maybe the UI manually adds the files to the container. | 09:15 |
| jpich | shardy: Thanks - okay! This one assumes the container already exists | 09:15 |
| shardy | Where I became confused is create_default_deployment_plan also takes a "container" input | 09:15 |
| shardy | shouldn't that be "container_name", if we create a container in that workflow? | 09:15 |
| d0ugal | shardy: We really need some documentation strings in these workflows :) | 09:15 |
| jpich | d0ugal: Right, that's where https://review.openstack.org/#/c/347830/ will come in I think then | 09:16 |
| shardy | d0ugal: +1000 :) | 09:16 |
| openstackgerrit | Christopher Brown proposed openstack/tripleo-heat-templates: Enable Neutron LBaaS Integration https://review.openstack.org/313933 | 09:16 |
| d0ugal | shardy: documentation in the workflows combined with this should be quite nice: https://review.openstack.org/#/c/341572/ | 09:16 |
| jaosorior_ | dtantsur: thanks for the clarification | 09:16 |
| *** jaosorior_ is now known as jaosorior | 09:16 | |
| d0ugal | jpich: oh cool, I shall check that one out! | 09:16 |
| *** snecklifter has joined #tripleo | 09:17 | |
| jpich | d0ugal: If I can give a hand with some of the client patches around plan management let me know, I don't want to duplicate work | 09:17 |
| shardy | d0ugal: Yeah, that looks good, just need the content now :) | 09:17 |
| gfidente | bandini that's the newer puppet-ceph module which can't cope with the old version of ceph installed | 09:17 |
| gfidente | let me comment on the bug | 09:18 |
| d0ugal | Also, I said "comment" but really we should use the description property: https://github.com/openstack/tripleo-common/blob/master/workbooks/baremetal.yaml#L32 | 09:18 |
| gfidente | bandini is this happening for all jobs? | 09:18 |
| shardy | jpich: we can do some testing, but I'm a bit worried about the field size limit | 09:18 |
| bandini | gfidente: it seems so yes | 09:18 |
| d0ugal | jpich: Thanks, if you have started any of it I can always use that! At the moment I have a broken env, I hit some weird issues last week. | 09:18 |
| shardy | the blob of data is huge (and increasing) | 09:18 |
| shardy | so we might find it's more reliable to just push the data to swift | 09:18 |
| shardy | seen quite a few weird timeouts with mistral, this seeems like a good way to DoS it :D | 09:19 |
| shardy | I guess we can test it and find out tho | 09:19 |
| *** bootsha has quit IRC | 09:19 | |
| jaosorior | gfidente: yep, both nonha and ha | 09:19 |
| openstackgerrit | Saravanan KR proposed openstack/tripleo-heat-templates: Added a new composable service for enabling DPDK on compute nodes. https://review.openstack.org/344084 | 09:21 |
| panda | pabelanger: I see that periodic jobs succeded in the last few days. In the logs I see too that the overcloud images are uploaded correclty, so why images in http://8.43.87.241/builds/current-tripleo/ are only updated to the 21 jul ? | 09:21 |
| jpich | shardy: I don't necessarily disagree :-) I think there's a lot of concerns on the UI team about doing too much outside of workflows and ending up diverging with the CLI, but in general Swift does seem better equipped to directly deal with large data input | 09:21 |
| jpich | florianf ^ - about continuing to push the JSON to Swift manually rather than use a workflow | 09:21 |
| shardy | jpich: ack, understood - lets give it a try and see how it goes | 09:22 |
| *** gchamoul has joined #tripleo | 09:22 | |
| jpich | shardy: At this point with the couple hundreds default templates we were just a bit over the limit (~1070K) for reference | 09:22 |
| *** dculik has joined #tripleo | 09:26 | |
| d0ugal | jpich: What limit is this? | 09:26 |
| *** ccamacho|afk is now known as ccamacho | 09:27 | |
| d0ugal | jpich: Oh, is this the limit for workflow input? | 09:27 |
| jpich | d0ugal: Task input size, yeah ( execution_field_size_limit_kb ) | 09:27 |
| d0ugal | heh, that amuses me for some reason :) | 09:27 |
| d0ugal | 1070K should be enough for anyone! | 09:28 |
| jpich | d0ugal: Not when you have a couple hundreds heat templates contents included in there :) | 09:28 |
| *** links has quit IRC | 09:28 | |
| shardy | We'll have to figure out the relationship between that and the heat request limit: | 09:29 |
| shardy | https://github.com/openstack/heat/blob/master/heat/common/wsgi.py#L192 | 09:29 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-puppet-elements: Revert "bump puppet-ceph to stable/hammer" https://review.openstack.org/343667 | 09:29 |
| shardy | really it would be better if we just passed the swift container ID around | 09:29 |
| shardy | currently heat only supports that on the client side | 09:29 |
| shardy | we could fix that in future tho | 09:29 |
| flaper87 | shardy: is it possible to set none as the default for a type number ? | 09:29 |
| flaper87 | shardy: in a heat template, I mena | 09:29 |
| flaper87 | mean* | 09:29 |
| florianf | jpich, shardy: Making the UI upload the plans to swift weakens the separation of the API and the UI considerably IMHO. For instance tripleo-common uses swift metadata to distinguish plan containers from other swift containers. If that metadata changes on the tripleo-common side, plans uploaded by the UI will not be recognized by -common any more. Also, one could argue that the storage method is just an implementation detail and | 09:30 |
| florianf | we're limited in our choice of storage options, if it has to be supported by both -common and the ui. | 09:30 |
| shardy | flaper87: No, a null default will look like a missing defaut | 09:31 |
| shardy | default | 09:31 |
| flaper87 | shardy: Asking because of this: https://bugzilla.redhat.com/show_bug.cgi?id=1361285#c5 | 09:31 |
| openstack | bugzilla.redhat.com bug 1361285 in rhel-osp-director "Glance deployed with single worker" [Unspecified,New] - Assigned to athomas | 09:31 |
| flaper87 | shardy: any recommendations on how to fix it? | 09:31 |
| flaper87 | I thought to remove the default value | 09:31 |
| flaper87 | but apparently that's not a good idea based on what you just said | 09:32 |
| shardy | flaper87: Don't most services interpret 0 the same as unset? | 09:33 |
| florianf | jpich, shardy: Also, why make the UI implement its own workflows if we have mistral for that? Especially since we're hitting the upload ceiling only by a little bit right now. | 09:33 |
| shardy | if glance doesn't we might need to handle it in puppet-glance | 09:33 |
| *** paramite has joined #tripleo | 09:33 | |
| flaper87 | shardy: Glance doesn't and it's been like that since Glance was created :/ | 09:34 |
| *** bootsha has joined #tripleo | 09:34 | |
| flaper87 | Other projects changed on the way | 09:34 |
| d0ugal | florianf: I guess changing this is a discussion that we need to have with Mistral folk | 09:34 |
| d0ugal | florianf, jpich: is there a bug for the input limit? | 09:34 |
| jpich | d0ugal: It's a configuration setting | 09:35 |
| d0ugal | oh okay, so we can just change it | 09:35 |
| jpich | d0ugal: https://review.openstack.org/#/c/348216/ | 09:35 |
| *** dculik has quit IRC | 09:35 | |
| florianf | d0ugal: jpich has found a mistral.conf option to increase the limit. But, sure, we should talk to them about how they chose the default and what they think about increasing it. Is that what you mean? | 09:35 |
| d0ugal | florianf: That is what I meant, but I didn't know we could change it easily :) | 09:36 |
| shardy | flaper87: fair enough - I can't see any way to fix it in the template, so the only options are either make glance treat 0 the same as null, or special-case this in puppet-glance to do the same (e.g translate the 0 in to None) | 09:38 |
| *** links has joined #tripleo | 09:39 | |
| *** mgould|afk is now known as mgould | 09:47 | |
| *** hewbrocca is now known as hewbrocca-afk | 09:49 | |
| *** tosky has joined #tripleo | 09:59 | |
| sshnaidm | shardy, hi, do you know which periodic jobs should pass for promoting image? I see upgrades job never passed there | 09:59 |
| shardy | sshnaidm: All of them I believe, so the failing upgrades job probably is the reason we've not promoted in a week | 10:02 |
| shardy | sshnaidm: can you please raise a bug with details of the failure? | 10:02 |
| paramite | Guys, can anybody please check the sensu spec? https://review.openstack.org/#/c/347399/ | 10:02 |
| shardy | (or even better send a patch which fixes it ;) | 10:03 |
| sshnaidm | shardy, I didn't see upgrades job succeeds at all - *never* :) not during the week, so I'm curios if it's counted at all | 10:03 |
| sshnaidm | shardy, and how can I see what's counted..? | 10:04 |
| shardy | sshnaidm: it certainly used to work | 10:04 |
| shardy | sshnaidm: if you grep -R "promote" in the tripleo-ci repo, you'll see a puppet manifest which runs a promote.sh script | 10:06 |
| shardy | it shows the list of jobs | 10:06 |
| shardy | ./scripts/mirror-server/mirror-server.pp: command => "timeout 10m /opt/stack/tripleo-ci/scripts/mirror-server/promote.sh current-tripleo periodic-tripleo-ci-centos-7-ovb-ha periodic-tripleo-ci-centos-7-ovb-nonha" | 10:06 |
| shardy | grep ftw ;) | 10:06 |
| sshnaidm | shardy, yep, no upgrades job there, thanks :) | 10:07 |
| shardy | so we're actually not currently including the upgrades job by the looks of it | 10:07 |
| openstackgerrit | Giulio Fidente proposed openstack-infra/tripleo-ci: Install Ceph Hammer and/or Jewel from CentOS Storage SIG repos https://review.openstack.org/340504 | 10:07 |
| shardy | it's probably been removed since rh1 was moved | 10:07 |
| sshnaidm | shardy, no, it runs every day | 10:07 |
| sshnaidm | shardy, http://status-tripleoci.rhcloud.com/ | 10:07 |
| shardy | sshnaidm: No, I mean I12251459f5a86f7e7e1c322279b10e62201129a9 removed it from that list before we moved rh1 | 10:08 |
| shardy | https://review.openstack.org/#/c/338735/ | 10:08 |
| sshnaidm | shardy, I think it was an issue with this, and now I recall something about it'll be fixed when using rh1 again.. maybe | 10:08 |
| shardy | we need that job working again, but first we need rh1 fully operational | 10:09 |
| gfidente | bandini jaosorior so I think we need to wait for those two patches to go through CI and see | 10:09 |
| shardy | sshnaidm: I highly doubt it'll just be fixed, we'll have regressions due to the temporary lack of coverage | 10:09 |
| sshnaidm | shardy, acc. to this, the images should have been promoted yesterday | 10:09 |
| *** ooolpbot has joined #tripleo | 10:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1608785 | 10:10 |
| *** ooolpbot has quit IRC | 10:10 | |
| openstack | Launchpad bug 1608785 in tripleo "CI jobs fail with ceph-mon errors" [Critical,In progress] - Assigned to Giulio Fidente (gfidente) | 10:10 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-puppet-elements: Revert "bump puppet-ceph to stable/hammer" https://review.openstack.org/343667 | 10:11 |
| shardy | sshnaidm: Hmm, yes it seems so - we need to figure out why the promote didn't work then | 10:12 |
| jaosorior | gfidente: Which patch? | 10:12 |
| gfidente | jaosorior https://review.openstack.org/#/c/343667/ and its depends-on | 10:13 |
| shardy | sshnaidm: lets chat with slagle when he comes online as he should have access to look at the logs from that cron job | 10:13 |
| jaosorior | I see | 10:13 |
| jaosorior | shardy: who can I talk to regarding mistral? | 10:13 |
| d0ugal | jaosorior: What do you need? | 10:13 |
| shardy | jaosorior: what about mistral? | 10:13 |
| jaosorior | d0ugal: I was finally able to reproduce the error that we were seeing in CI when enabling SSL for ironic-inspector | 10:14 |
| sshnaidm | shardy, ok | 10:14 |
| jaosorior | and I see the same issue without SSL actually | 10:14 |
| *** zoliXXL is now known as zoli|lunch | 10:14 | |
| jaosorior | It seems that the wait_for_finish action for the baremetal introspection workflow gets defined incorrectly | 10:14 |
| jaosorior | and it gets defined without an input | 10:15 |
| jaosorior | for some reason | 10:15 |
| jaosorior | d0ugal: Here's the relevant log http://logs.openstack.org/39/345539/1/check-tripleo/gate-tripleo-ci-centos-7-ovb-nonha/2704811/console.html#_2016-08-02_06_46_29_572036 | 10:15 |
| jaosorior | Adding some extra output to the logs | 10:16 |
| jaosorior | the definition looks like this: | 10:16 |
| jaosorior | ActionDefinition {\'definition\': None, \'name\': u\'baremetal_introspection.wait_for_finish\', \'tags\': None, \'attributes\': {u\'client_method_name\': u\'wait_for_finish\'}, \'created_at\': \'2016-08-02 10:04:47\', \'action_class\': u\'mistral.actions.openstack.actions.BaremetalIntrospectionAction\', \'updated_at\': None, \'spec\': None, \'is_system\': True, \'input\': u\'\', \'scope\': u\'public\', | 10:17 |
| jaosorior | \'project_id\': u\'<default-project>\', \'id\': u\'7e595b7c-2c1a-4c13-ae37-1a12f8bb712c\', \'description\': None} | 10:17 |
| jaosorior | d0ugal | 10:17 |
| d0ugal | jaosorior: This is where/how the action was defined if that helps. https://review.openstack.org/#/c/319582/ | 10:17 |
| *** milan has quit IRC | 10:18 | |
| jaosorior | d0ugal: And what actually populates that workflow? | 10:18 |
| jaosorior | for some reason it seems to me that it's getting defined incorrectly | 10:19 |
| jaosorior | as far as I can see from those logs | 10:19 |
| shardy | https://github.com/openstack/tripleo-common/blob/master/workbooks/baremetal.yaml#L194 | 10:19 |
| d0ugal | jaosorior: https://github.com/openstack/tripleo-common/blob/master/workbooks/baremetal.yaml#L192-L200 | 10:19 |
| * d0ugal is too slow | 10:19 | |
| *** limao_ has quit IRC | 10:19 | |
| jaosorior | d0ugal, shardy: Any idea why it would populate the "inputs" for that action as empty? instead of adding the uuids as is clearly defined in that yaml? | 10:20 |
| d0ugal | jaosorior: The input should be passed into the workflow. https://github.com/openstack/tripleo-common/blob/master/workbooks/baremetal.yaml#L174 | 10:20 |
| *** chem has quit IRC | 10:20 | |
| jaosorior | d0ugal: Right, so the input is passed. Apparently | 10:21 |
| shardy | Yeah, so I'd check back to where start_introspection ran, and ensure that input is OK | 10:21 |
| jaosorior | but from what I see, it seems that the definition of the action didn't expect any input | 10:21 |
| shardy | assuming it is, I'm not sure | 10:21 |
| d0ugal | jaosorior: but that workflow is called by the introspect_manageable_nodes woorkflow. | 10:21 |
| d0ugal | jaosorior: so, I guess for some reason it isn't finding any nodes in a manageable state to introspect | 10:21 |
| d0ugal | jaosorior: https://github.com/openstack/tripleo-common/blob/master/workbooks/baremetal.yaml#L230 | 10:21 |
| d0ugal | jaosorior: The result of that is passed to the introspect workflow: https://github.com/openstack/tripleo-common/blob/master/workbooks/baremetal.yaml#L241-L243 | 10:22 |
| jaosorior | d0ugal: Are you sure? From the logs I passed you this is what it reports: InputException: Invalid input [name=baremetal_introspection.wait_for_finish, class=NoneType, unexpected=[u\'uuids\']]\n' | 10:23 |
| jaosorior | which, looking by the code, it seems that it got the "uuids" input, but it didn't expect it | 10:23 |
| d0ugal | jaosorior: oh, that is weird. I missed that. | 10:23 |
| d0ugal | jaosorior: We always pass an input to wait_for_finish. | 10:24 |
| openstackgerrit | Babu Shanmugam proposed openstack/tripleo-heat-templates: OVN heat templates https://review.openstack.org/307734 | 10:24 |
| jaosorior | d0ugal: right | 10:24 |
| jaosorior | so here's the issue | 10:24 |
| jaosorior | wait_for_finish should be defined with the "uuids" input | 10:24 |
| jaosorior | which gets passed by the workflow, as you and shardy pointed out to me | 10:24 |
| jaosorior | BUT | 10:24 |
| jaosorior | in the issue that I'm seeing | 10:24 |
| jaosorior | and that is shown in the logs I passed from CI | 10:25 |
| d0ugal | Yeah, for some reason it isn't expecting an input now. That is weird. | 10:25 |
| jaosorior | yeah | 10:25 |
| jaosorior | I don't understand X_x | 10:25 |
| d0ugal | jaosorior: I want to dig around more, but my dev env is currently not working - so I need to get that up first. | 10:26 |
| openstackgerrit | Karthik S proposed openstack/tripleo-heat-templates: Add max_reserved_vfs in THT https://review.openstack.org/345352 | 10:26 |
| d0ugal | unrelated: Is CI using quickstart now? | 10:26 |
| jaosorior | AFAIK no | 10:26 |
| jaosorior | d0ugal: I ended up setting it up with quickstart, and then running tripleo.sh like in CI | 10:27 |
| d0ugal | aha | 10:27 |
| jaosorior | so I could reproduce what's seen in CI | 10:27 |
| d0ugal | jaosorior: I want to do that too :) | 10:27 |
| d0ugal | jaosorior: How do you setup with quickstart? | 10:27 |
| *** bootsha has quit IRC | 10:28 | |
| *** dtantsur is now known as dtantsur|lunch | 10:28 | |
| jaosorior | d0ugal: ./quickstart.sh --config config/general_config/ha.yml -R master --no-clone $HOST | 10:28 |
| d0ugal | jaosorior: Thanks | 10:28 |
| jaosorior | d0ugal: pretty much documented a bit of it here http://jaormx.github.io/2016/deploying-master-tripleo-quickstart/ | 10:29 |
| d0ugal | jaosorior: oh, useful. Thanks! | 10:31 |
| *** bootsha has joined #tripleo | 10:31 | |
| d0ugal | I find it weird having quickstart on my local machine. Maybe that is just me | 10:31 |
| jaosorior | d0ugal: What do you mean? | 10:33 |
| d0ugal | jaosorior: nevermind, I just miss-read something in the README :) | 10:33 |
| * d0ugal needs more coffee | 10:33 | |
| jaosorior | lol | 10:34 |
| jaosorior | actually that's a good idea | 10:34 |
| jaosorior | I should brew some too :P | 10:34 |
| *** rodrigods has quit IRC | 10:36 | |
| *** rodrigods has joined #tripleo | 10:36 | |
| jaosorior | d0ugal: Are you acquainted with the mistral-db-sync command? | 10:40 |
| jaosorior | I think stumbled upon the issue | 10:40 |
| d0ugal | jaosorior: Not really. I only know the command because I have called it a few times :) | 10:41 |
| d0ugal | jaosorior: and I got a weird error last week which caused me to look at the source briefly. | 10:41 |
| jaosorior | d0ugal: so, when I ran that, I found this stuff: http://paste.openstack.org/show/545599/ | 10:41 |
| jaosorior | d0ugal: which is weird, cause the keystone endpoints point to 192.0.2.1 | 10:42 |
| d0ugal | oh, interesting. | 10:42 |
| d0ugal | jaosorior: because it isn't https? | 10:42 |
| jaosorior | d0ugal: So it seems it used to work, cause before the TLS commit, inspector was listening on 0.0.0.0 | 10:42 |
| jaosorior | and the TLS commit made it listen in a specific address | 10:42 |
| d0ugal | right | 10:42 |
| jaosorior | so the issue is not TLS itself, is the fact that inspector is no longer in 0.0.0.0 | 10:42 |
| jaosorior | which was messed up in the first place | 10:43 |
| d0ugal | Yeah, that is weird. | 10:43 |
| jaosorior | d0ugal: Any idea where the db-sync source code is for mistral? | 10:45 |
| jaosorior | I find db-manage but not sync | 10:45 |
| d0ugal | jaosorior: Yup... | 10:45 |
| d0ugal | jaosorior: huh, hold on. What was the exact command you ran for that? | 10:47 |
| jaosorior | d0ugal" mistral-db-sync --verbose | 10:48 |
| d0ugal | oh | 10:48 |
| d0ugal | okay, I can't find that. | 10:48 |
| jaosorior | d0ugal: I couldn't find it either, but I went to the code of that in /usr/bin/mistral-db-sync (which I have no idea where it came from... but sure...) and this is what I found: http://paste.openstack.org/show/545601/ | 10:50 |
| d0ugal | jaosorior: https://github.com/openstack/mistral/blob/master/tools/sync_db.py | 10:51 |
| d0ugal | jaosorior: so I guess it is the rpm packaging that makes it a command | 10:53 |
| jaosorior | right | 10:53 |
| jaosorior | that makes more sense | 10:53 |
| d0ugal | jaosorior: http://paste.openstack.org/show/545602/ I asked the Mistral PTL | 10:54 |
| d0ugal | jaosorior: so I think we should be using mistral-db-manage, not that command. | 10:54 |
| jaosorior | great! | 10:55 |
| jaosorior | let me try with db-manage then | 10:55 |
| shardy | d0ugal: I think I've found the reason for the 401: Unauthorized issue with the deployment workflow | 10:55 |
| shardy | we're passing unresolved references to swift in the environment | 10:55 |
| shardy | looking into fixing it now | 10:55 |
| jaosorior | d0ugal: Are you sure? It seems to me that the db-manage doesn't really do the workflow re-population | 10:57 |
| jaosorior | which is what sync_db does | 10:58 |
| d0ugal | shardy: Great! Thanks | 10:58 |
| *** mbound has quit IRC | 10:58 | |
| d0ugal | jaosorior: I've not looked, just passing on the reply I got in #openstack-mistral :) | 10:58 |
| shardy | jaosorior: Neither does workflow re population | 11:01 |
| shardy | you have to delete them then create them again | 11:01 |
| shardy | db-manage populate only repopulates the actions | 11:02 |
| openstackgerrit | Julie Pichon proposed openstack/tripleo-common: Mistral workflow for creating a deployment plan from JSON input https://review.openstack.org/347830 | 11:02 |
| shardy | https://github.com/openstack/instack-undercloud/blob/master/elements/undercloud-install/os-refresh-config/post-configure.d/98-undercloud-setup#L101 | 11:02 |
| shardy | jaosorior: that's why we do that ^^ | 11:02 |
| shardy | https://github.com/openstack/tripleo-common/blob/master/README.rst | 11:03 |
| shardy | see that for how to repopulate the actions | 11:03 |
| jaosorior | I see | 11:03 |
| shardy | notw however, if there's errors in the action plugins, it's possible for the entrypoints to get registered but not be in a loadable state | 11:04 |
| jaosorior | alright, well, the workflows contain actions... so I had misunderstood. But right. it does repopulate actions | 11:04 |
| shardy | db-manage populate seems to silently ignore that :( | 11:04 |
| jaosorior | shardy: right, and that seems to be happening | 11:04 |
| jaosorior | now that ironic-inspector is listening on something else than 0.0.0.0 | 11:04 |
| jaosorior | not sure where it gets the address it tries to poke from | 11:04 |
| shardy | pip install entry_point_inspector && epi group show mistral.actions | 11:05 |
| shardy | that's how I debugged this yesterday | 11:05 |
| shardy | There was some stuff in "Error" column, which db-manage just ignored | 11:05 |
| *** ramishra has quit IRC | 11:06 | |
| jaosorior | no errors marked there :/ | 11:07 |
| shardy | jaosorior: Ok, so the action is probably getting loaded and it's a runtime error? | 11:08 |
| openstackgerrit | Ana Krivokapic proposed openstack/instack-undercloud: Add default plan creation https://review.openstack.org/349532 | 11:08 |
| jaosorior | shardy: It seems to me that's the case | 11:08 |
| jaosorior | there's an error, but it loads the action anyway | 11:08 |
| jaosorior | except that it doesn't get the values it needs from the action | 11:08 |
| *** ramishra has joined #tripleo | 11:08 | |
| shardy | You may need to add some debug to the actions and trace the flow through via the mistral log (or a debugger I guess, I'm using logging) | 11:09 |
| *** dtantsur|lunch is now known as dtantsur | 11:10 | |
| *** ooolpbot has joined #tripleo | 11:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1608785 | 11:10 |
| openstack | Launchpad bug 1608785 in tripleo "CI jobs fail with ceph-mon errors" [Critical,In progress] - Assigned to Giulio Fidente (gfidente) | 11:10 |
| *** ooolpbot has quit IRC | 11:10 | |
| snecklifter | ah, i was just about to ask if ci was broken... | 11:10 |
| *** sshnaidm is now known as sshnaidm|afk | 11:10 | |
| *** weshay has quit IRC | 11:11 | |
| *** shardy is now known as shardy_lunch | 11:14 | |
| *** tbonds has joined #tripleo | 11:18 | |
| *** hrybacki|appt is now known as hrybacki | 11:18 | |
| *** tbonds has quit IRC | 11:20 | |
| *** stendulker has quit IRC | 11:24 | |
| *** rhallisey has joined #tripleo | 11:30 | |
| mandre | can I get a good soul to review https://review.openstack.org/#/c/322178/? it's blocking work on tripleo-validations | 11:31 |
| *** dculik has joined #tripleo | 11:35 | |
| marios | mandre: i'm just a soul who's intentions are good | 11:36 |
| marios | mandre: i'll have a look later/tomorrow morning if it is still around added to review queue | 11:36 |
| openstackgerrit | Michele Baldessari proposed openstack/tripleo-heat-templates: Add Aodh composable roles https://review.openstack.org/333556 | 11:40 |
| openstackgerrit | Michele Baldessari proposed openstack/tripleo-heat-templates: Next generation HA architecture work https://review.openstack.org/314208 | 11:40 |
| *** paramite is now known as paramite|afk | 11:40 | |
| mandre | marios: don't let me be misunderstood, I'll take any soul :) | 11:40 |
| marios | :D | 11:41 |
| *** hewbrocca-afk is now known as hewbrocca | 11:44 | |
| *** bfournie has quit IRC | 11:46 | |
| remix_tj | hello, does tripleo-image-elements are used during disk-image build or only copied into the overcloud-full image and used when deploying? because i'm experiencing issues with corosync.conf file not being created | 11:46 |
| remix_tj | (and so corosync is not starting and deployment fails) | 11:47 |
| *** thrash|g0ne is now known as thrash | 11:48 | |
| EmilienM | hello | 11:48 |
| *** zoli|lunch is now known as zoli | 11:48 | |
| *** zoli is now known as zoliXXL | 11:48 | |
| *** lucasagomes is now known as lucas-hungry | 11:51 | |
| *** sshnaidm|afk is now known as sshnaidm | 11:52 | |
| jaosorior | d0ugal: So... I found the issue | 11:53 |
| jaosorior | I'm not entirely sure if it's in mistral or python-ironic-client though | 11:53 |
| d0ugal | hah | 11:53 |
| *** dprince has joined #tripleo | 11:55 | |
| d0ugal | jaosorior: What is happening? | 11:55 |
| jaosorior | d0ugal: so it seems that the python-ironic-client for some reason tries to do a request... for something... (still don't know what) in the __init__ method | 11:55 |
| d0ugal | :-/ | 11:55 |
| jaosorior | and since mistral doesn't pass anything to the client | 11:55 |
| jaosorior | it fails | 11:56 |
| jaosorior | so | 11:56 |
| jaosorior | mistral could pass a keystone session | 11:56 |
| jaosorior | and that would work for a lot of clients (not sure if all) | 11:56 |
| jaosorior | and in the end it will get a correct endpoint | 11:56 |
| jaosorior | instead of falling into the default that python-ironic-inspector-client is using | 11:56 |
| jaosorior | sorry, it was python-ironic-inspector-client, not ironic-client | 11:56 |
| *** weshay has joined #tripleo | 11:57 | |
| jaosorior | ooor, I could try to figure out what ironic-inspector is trying to poke | 11:57 |
| dtantsur | jaosorior, if keystone session is not provided in __init__, it will always default to localhost | 11:57 |
| dtantsur | (legacy compatibility, sigh) | 11:58 |
| jaosorior | yeah, I see that | 11:58 |
| jaosorior | which is causing issues with mistral | 11:58 |
| jaosorior | cause mistral doesn't give any keystone session when it's trying to populate the actions | 11:58 |
| jaosorior | so it tries to create a client without arguments | 11:58 |
| jaosorior | and something in python-ironic-inspector-client is trying to access that endpoint... which isn't listening | 11:58 |
| jaosorior | yeah | 12:00 |
| jaosorior | it's the check_api_version | 12:00 |
| jaosorior | dtantsur: http://paste.openstack.org/show/545612/ | 12:00 |
| dtantsur | jaosorior, correct. it essentially detects the current IP and adds a port to it. we need to pass session for real (not passing it is pending deprecation) | 12:01 |
| *** ramishra has quit IRC | 12:01 | |
| jaosorior | d0ugal: so yeah... I don't know if this could be fixed in python-ironic-inspector-client or mistral | 12:02 |
| *** fultonj has joined #tripleo | 12:02 | |
| d0ugal | jaosorior: Right, good question. I suspect Mistral. | 12:02 |
| jaosorior | dtantsur: the issue with fixing it in mistral is that it needs to work for ALL the openstack clients it supports | 12:03 |
| d0ugal | jaosorior: I mean, it's weird behaviour in inspector client, but it has done that forever I think. | 12:03 |
| d0ugal | True | 12:03 |
| d0ugal | :/ | 12:03 |
| jaosorior | d0ugal: an easy fix would be to have that version to be lazily initialized | 12:03 |
| d0ugal | Yeah | 12:03 |
| jaosorior | not sure what dtantsur would think of that | 12:03 |
| d0ugal | hrm I am not sure. | 12:03 |
| d0ugal | Maybe worth asking in mistral about it or discussing in a bug | 12:04 |
| jaosorior | have that _check_api_version() call done only when the version is needed | 12:04 |
| jaosorior | dtantsur: Would that sound reasonable? | 12:04 |
| *** jpena is now known as jpena|lunch | 12:04 | |
| dtantsur | jaosorior, how would it help you though? | 12:04 |
| jaosorior | dtantsur: That call wouldn't be done in the client initialization, and then we could just continue the action creation | 12:05 |
| jaosorior | not much is really done with the instance really | 12:05 |
| *** shardy_lunch is now known as shardy | 12:05 | |
| dtantsur | jaosorior, yeah, but it will fail later, right? | 12:05 |
| jaosorior | dtantsur: not really, it's pretty much discarded after getting the method signature it needs | 12:05 |
| dtantsur | but what happens when we actually use inspection? | 12:06 |
| jaosorior | dtantsur: It's a big for loop that goes around a lot of openstack clients, and creates some "actions" to finally put them in a database | 12:06 |
| jaosorior | dtantsur: When inspection is done from mistral, it creates a valid client, AFAIK it does so with an auth_token and not a session though | 12:07 |
| dtantsur | correct, imagine we're past this point. now we're trying to do introspection. will it work? | 12:07 |
| dtantsur | how can it create a valid client? without a session it will default to the same URL... | 12:07 |
| jaosorior | dtantsur: it gets the service catalog from the token | 12:07 |
| jaosorior | mistral is heavily tied to keystone v2 (I don't like it... but changing it is out of scope from my side) | 12:07 |
| dtantsur | jaosorior, who gets the catalog and how is going to use it? | 12:08 |
| *** Guest52823 is now known as honza | 12:08 | |
| dtantsur | jaosorior, I think we have to do something like https://github.com/openstack/mistral/blob/master/mistral/actions/openstack/actions.py#L437-L449 | 12:08 |
| jaosorior | dtantsur: the mistral process gets the keystone catalog from the auth_token that the user provided in the request. It will use it to get the endpoints for the clients | 12:08 |
| *** milan has joined #tripleo | 12:09 | |
| *** honza is now known as Guest8644 | 12:09 | |
| dtantsur | jaosorior, could you point me to the place where mistral passes the inspector_url to ironic-inspector-client? I can't find it | 12:09 |
| dtantsur | ah, nm, I see | 12:09 |
| jaosorior | dtantsur: https://github.com/openstack/mistral/blob/master/mistral/actions/openstack/actions.py#L394 | 12:09 |
| dtantsur | then this endpoint is wrong, right? | 12:10 |
| jaosorior | dtantsur: yeah, that's the code that actually does actions. The code that I was talking about that populates the actions is different though | 12:10 |
| *** ooolpbot has joined #tripleo | 12:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1608785 | 12:10 |
| *** ooolpbot has quit IRC | 12:10 | |
| openstack | Launchpad bug 1608785 in tripleo "CI jobs fail with ceph-mon errors" [Critical,In progress] - Assigned to Giulio Fidente (gfidente) | 12:10 |
| jaosorior | dtantsur: no, this endpoint is correct. | 12:10 |
| dtantsur | jaosorior, I think I start to understand now | 12:10 |
| dtantsur | lemme check what we do | 12:10 |
| jaosorior | dtantsur: here's the problematic code https://github.com/openstack/mistral/blob/master/mistral/actions/openstack/action_generator/base.py#L86 | 12:11 |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-common: Remove the unused service_host arg from node registration https://review.openstack.org/326036 | 12:11 |
| jaosorior | dtantsur: it goes here https://github.com/openstack/mistral/blob/master/mistral/actions/openstack/base.py#L75 and eventually tries to create the client class here https://github.com/openstack/mistral/blob/master/mistral/actions/openstack/base.py#L71 | 12:12 |
| jaosorior | but, as you can see, that aim to create a "fake client" meaning is just an instance that they use to fetch the method signature, and then they discard that object. | 12:12 |
| *** numans has quit IRC | 12:12 | |
| dtantsur | jaosorior, this is a bad assumption really... also I don't get why they need to call __init__ to get methods | 12:12 |
| dtantsur | well, maybe because of sub-objects | 12:12 |
| *** paramite|afk is now known as paramite | 12:13 | |
| jaosorior | dtantsur: I don't fully understand either. I just started snooping around this code today because I have some issues with getting ironic-inspector to have TLS enabled | 12:14 |
| jaosorior | dtantsur: So a workaround that doesn't really affect anybody would be to have the _check_api_version call in the client to be lazy. So it would only be called when the version is needed | 12:16 |
| dtantsur | the version is needed there, that's the problem | 12:16 |
| dtantsur | the client checks that the requested API version is supported by the server | 12:16 |
| jaosorior | dtantsur: where is that done? | 12:16 |
| dtantsur | jaosorior, in _check_api_version | 12:17 |
| *** maeca1 has joined #tripleo | 12:18 | |
| *** bfournie has joined #tripleo | 12:19 | |
| jaosorior | dtantsur: right. it seems we can fix this in mistral | 12:21 |
| dtantsur | I hope so | 12:21 |
| *** tzumainn has joined #tripleo | 12:24 | |
| *** ramishra has joined #tripleo | 12:30 | |
| jaosorior | EmilienM: Hey dude, can you take another look at https://review.openstack.org/#/c/345183/ ? | 12:31 |
| EmilienM | jaosorior: sure | 12:31 |
| EmilienM | jaosorior: is it already done in master? | 12:32 |
| jaosorior | EmilienM: It is done. But the code had to be different since in master we have the endpoint abstractions. Which don't exist in stable/mitaka | 12:33 |
| EmilienM | jaosorior: +2 | 12:33 |
| jaosorior | EmilienM: thanks | 12:33 |
| jaosorior | EmilienM: anything up for review? | 12:34 |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-quickstart: Devmode: rename delorean current so any existing repo is overwritten https://review.openstack.org/335953 | 12:34 |
| EmilienM | gfidente: do you need me to revert a puppet-ceph patch? | 12:34 |
| jaosorior | EmilienM: I already submitted a revert | 12:34 |
| jaosorior | EmilienM: https://review.openstack.org/#/c/349810/ | 12:35 |
| EmilienM | jaosorior: ah this one, ok | 12:35 |
| EmilienM | jaosorior: for now, no need to review my work, thanks | 12:36 |
| openstackgerrit | Martin Mágr proposed openstack/tripleo-heat-templates: Availability monitoring agents support https://review.openstack.org/254788 | 12:36 |
| EmilienM | jaosorior: I continue my work to implement the undercloud upgrade job mitaka -> newton | 12:36 |
| *** jdob has joined #tripleo | 12:36 | |
| EmilienM | do we have anyone looking at ironic/api/wsgi issue? | 12:36 |
| EmilienM | dtantsur: ^ | 12:36 |
| jaosorior | shardy: Will it work if on an instack-undercloud patch I do a Depends-On that points to a mistral patch? | 12:38 |
| dtantsur | EmilienM, not me, sorry :( ETOOMANYTHINGSTODO | 12:38 |
| EmilienM | :) | 12:39 |
| EmilienM | jaosorior: no | 12:39 |
| EmilienM | or not afik | 12:39 |
| jaosorior | :( | 12:40 |
| gfidente | EmilienM nope | 12:41 |
| gfidente | the two submission I linked from the LP bug | 12:42 |
| gfidente | can deploy master successfully | 12:42 |
| gfidente | I am trying to figure why the stable branches are failing | 12:42 |
| gfidente | EmilienM shardy bandini jaosorior https://review.openstack.org/#/c/343667/ | 12:42 |
| *** numans has joined #tripleo | 12:42 | |
| EmilienM | gfidente: +2 | 12:44 |
| gfidente | EmilienM I can't spot why the stable branches are failing | 12:44 |
| gfidente | there is a trap here | 12:44 |
| gfidente | http://logs.openstack.org/04/340504/8/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha-liberty/adaf221/console.html#_2016-08-02_10_30_59_278161 | 12:44 |
| gfidente | but I can't see what failed before that? | 12:44 |
| gfidente | do you guys see anything wrong? | 12:45 |
| *** rlandy has joined #tripleo | 12:45 | |
| *** jayg|g0n3 is now known as jayg | 12:46 | |
| bandini | gfidente: http://logs.openstack.org/04/340504/8/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha-liberty/adaf221/logs/postci.txt.gz | 12:46 |
| bandini | oh my link might be a red herring | 12:47 |
| *** lblanchard has joined #tripleo | 12:47 | |
| bandini | gfidente: ignore me. indeed, seems odd | 12:47 |
| *** links has quit IRC | 12:48 | |
| EmilienM | gfidente: looking | 12:51 |
| pabelanger | panda: I am not sure what http://8.43.87.241/ is or if I have access. I think this is a private server in that tripleo uses to host images from. First step should be moving the contents to tarballs.openstack.org, replacing private infrastructure. | 12:53 |
| EmilienM | gfidente: http://logs.openstack.org/04/340504/8/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha-mitaka/f53cef9/console.html#_2016-08-02_10_43_32_607330 | 12:53 |
| EmilienM | Public key for ceph-common-0.94.5-1.el7.x86_64.rpm is not installed | 12:53 |
| EmilienM | GPG key retrieval failed: [Errno 14] curl#37 - "Couldn't open file /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-Storage" | 12:53 |
| *** trozet has joined #tripleo | 12:53 | |
| gfidente | hmm | 12:53 |
| gfidente | it's copying the .repo file | 12:54 |
| gfidente | but it doesn't have the gpgkey in the image | 12:54 |
| gfidente | thanks EmilienM :) | 12:54 |
| *** jcoufal has joined #tripleo | 12:54 | |
| *** dmsimard|afk is now known as dmsimard | 12:55 | |
| gfidente | the standard keys are bundled with the image, but not the sig key | 12:55 |
| *** egafford has joined #tripleo | 12:55 | |
| gfidente | dah sounds like the easy win here is to disable gpgcheck | 12:55 |
| gfidente | unless maybe slagle has some ideas? :) | 12:56 |
| gfidente | slagle there is a .repo file I am copying in the image via https://review.openstack.org/#/c/340504/8/scripts/tripleo.sh | 12:56 |
| gfidente | it needs a matching key to verify the rpm signature | 12:56 |
| gfidente | I don't think we have any DIB element which can install the gpgkey? | 12:56 |
| *** morazi has joined #tripleo | 12:58 | |
| *** jpena|lunch is now known as jpena | 13:00 | |
| *** zoliXXL is now known as zoli|brb | 13:00 | |
| *** rbrady has joined #tripleo | 13:01 | |
| *** bvandenh has quit IRC | 13:03 | |
| *** lucas-hungry is now known as lucasagomes | 13:03 | |
| openstackgerrit | Giulio Fidente proposed openstack-infra/tripleo-ci: Install Ceph Hammer and/or Jewel from CentOS Storage SIG repos https://review.openstack.org/340504 | 13:07 |
| gfidente | ^^ that should fix it | 13:07 |
| gfidente | (for liberty/mitaka too) | 13:07 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci: Remove pinning of delorean https://review.openstack.org/349995 | 13:08 |
| *** saneax is now known as saneax_AFK | 13:09 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci: Remove pinning of delorean https://review.openstack.org/349995 | 13:10 |
| EmilienM | gfidente: no gpgcheck? :P | 13:10 |
| *** ooolpbot has joined #tripleo | 13:11 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 13:11 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1608785 | 13:11 |
| *** ooolpbot has quit IRC | 13:11 | |
| openstack | Launchpad bug 1608785 in tripleo "CI jobs fail with ceph-mon errors" [Critical,In progress] - Assigned to Giulio Fidente (gfidente) | 13:11 |
| openstackgerrit | Julie Pichon proposed openstack/tripleo-common: Add boot option argument to the Node Registration Mistral workflow https://review.openstack.org/348875 | 13:11 |
| openstackgerrit | yolanda.robla proposed openstack-infra/tripleo-ci: Add --yes flags to heat stack-delete for liberty https://review.openstack.org/347202 | 13:12 |
| openstackgerrit | Honza Pokorny proposed openstack/tripleo-ui: Add packaging spec and config files https://review.openstack.org/344932 | 13:12 |
| *** Guest8644 is now known as honza | 13:12 | |
| gfidente | EmilienM yeah I think it's the easiest way for now | 13:15 |
| gfidente | alternatively we should create an element which installs the .repo and the matching gpgkey | 13:16 |
| EmilienM | gfidente: +2 | 13:16 |
| gfidente | EmilienM the problem is that DIB will copy those .repo files into the target image | 13:16 |
| gfidente | and it doesn't copy the key | 13:17 |
| *** links has joined #tripleo | 13:20 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci: Don't use AFS mirrors if working locally https://review.openstack.org/350003 | 13:21 |
| openstackgerrit | afazekas proposed openstack/os-cloud-config: Force tty usage on remote ssh https://review.openstack.org/350004 | 13:21 |
| *** bootsha has quit IRC | 13:21 | |
| openstackgerrit | Giulio Fidente proposed openstack-infra/tripleo-ci: Install Ceph Hammer and/or Jewel from CentOS Storage SIG repos https://review.openstack.org/340504 | 13:23 |
| *** bootsha has joined #tripleo | 13:24 | |
| *** zoli|brb is now known as zoli | 13:25 | |
| *** zoli is now known as zoliXXL | 13:25 | |
| *** julim has joined #tripleo | 13:26 | |
| *** julim has quit IRC | 13:26 | |
| *** bootsha has quit IRC | 13:30 | |
| *** morazi has quit IRC | 13:31 | |
| openstackgerrit | afazekas proposed openstack/os-cloud-config: Force tty usage on remote ssh https://review.openstack.org/350004 | 13:31 |
| *** pgadiya has quit IRC | 13:32 | |
| EmilienM | bnemec: hey, I'm wondering something about https://review.openstack.org/#/c/346995/19/scripts/deploy.sh L67 | 13:32 |
| EmilienM | shouldn't I run yum update -y and not specify the instack-undercloud package? | 13:33 |
| *** morazi has joined #tripleo | 13:33 | |
| *** julim has joined #tripleo | 13:33 | |
| Jokke_ | marios: ping | 13:35 |
| *** mbound has joined #tripleo | 13:35 | |
| *** akshai has joined #tripleo | 13:36 | |
| jpich | florianf: Do you see any issue with jtomasek's comment that we use Mistral actions directly from the UI when it makes sense, rather than create workflows that only have one action + zaqar notification? It seems like a good idea to me, that'd limit workflow proliferation and generally keep things readable | 13:38 |
| marios | Jokke_: o/ | 13:38 |
| jpich | florianf: (That's a reference to the plan listing at https://review.openstack.org/#/c/347932/) | 13:38 |
| Jokke_ | marios: I was looking the Manila patches again | 13:39 |
| Jokke_ | marios: https://vhbin.net/pjwn6wbgjr4z/ | 13:39 |
| d0ugal | jpich: My only issue with that approach is that it makes documentation a bit more confusing. | 13:39 |
| *** mgarciam has joined #tripleo | 13:40 | |
| d0ugal | jpich: We can't just tell people to use workflows, they also need to trigger actions and the behaviour is then a little different | 13:40 |
| Jokke_ | "Error: Cannot reassign variable manila_generic_enable at /etc/puppet/modules/tripleo/manifests/profile/pacemaker/manila.pp:125 on node overcloud-controller-0.localdomain" | 13:40 |
| *** mbound has quit IRC | 13:40 | |
| marios | Jokke_: looking | 13:41 |
| florianf | jpich: It's a valid point. I'll update the patch. | 13:41 |
| *** rbrady has quit IRC | 13:42 | |
| openstackgerrit | Honza Pokorny proposed openstack/tripleo-ui: Add packaging spec and config files https://review.openstack.org/344932 | 13:42 |
| jpich | d0ugal: Isn't a workflow with only one action somewhat counter-intuitive though? Especially from simple CRUD tasks like create/list/delete a thing | 13:42 |
| marios | Jokke_: great, i mean not great but seems like legit /easy fixup... is that local test? i am waiting for ci but latest run on http://logs.openstack.org/61/342961/8/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/6267a97/logs/overcloud-controller-0/var/log/messages is failing for the ceph-mon issue https://bugs.launchpad.net/tripleo/+bug/1608785 | 13:43 |
| openstack | Launchpad bug 1608785 in tripleo "CI jobs fail with ceph-mon errors" [Critical,In progress] - Assigned to Giulio Fidente (gfidente) | 13:43 |
| Jokke_ | marios: that's https://review.openstack.org/#/c/188137/ and https://review.openstack.org/#/c/342961 cherrypicked on top of thi morning's master | 13:43 |
| Jokke_ | marios: yup It's on my own system | 13:43 |
| Jokke_ | marios: let me know if you want to poke it and I'll get you access | 13:44 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: WIP - Implement undercloud upgrade job - Mitaka -> Newton https://review.openstack.org/346995 | 13:44 |
| marios | Jokke_: can you add a comment on the review and i will check it momentarily and hopefuly we get a green run when ci is sorted for that ceph mon issue | 13:44 |
| tzumainn | d0ugal, I agree with jpich, having unneeded workflows runs counter to what my original hopes were for the tripleo-api | 13:44 |
| Jokke_ | marios: absolutely! | 13:44 |
| marios | Jokke_: i expect is just a nit in the way we are passing from tht and the way the puppet-tripleo is written, so no need to poke at the env for now thanks though | 13:45 |
| Jokke_ | marios: will add the comment to the don't merge" | 13:45 |
| d0ugal | jpich: Yeah, sure. I guess it is fine. | 13:45 |
| marios | Jokke_: yeah sure man -1 at will :) | 13:45 |
| d0ugal | jpich: The other limitation is that if we ever need to add more steps to it then we need to do it all in an action | 13:45 |
| marios | Jokke_: or woman sorry we have never met | 13:45 |
| marios | ha | 13:45 |
| openstackgerrit | Honza Pokorny proposed openstack/tripleo-ui: Add prepublish script to bundle dependencies https://review.openstack.org/350019 | 13:46 |
| d0ugal | jpich: Workflows keep a consistent interface and give us flexibility. I'm happy for it to be called directly, but I think there are downsides. | 13:46 |
| tzumainn | d0ugal, sure, but for fairly limited actions like a role list or plan list, I think a workflow is a pretty big overhead | 13:46 |
| d0ugal | tzumainn: indeed, mistral is a big overhead in general there ;) | 13:47 |
| tzumainn | :P | 13:47 |
| *** fultonj has quit IRC | 13:48 | |
| jtomasek | d0ugal: we should document the actions in tripleo docs I guess | 13:48 |
| d0ugal | tzumainn, jpich: IIRC, we also don't version action names yet. Maybe we need to do that? | 13:48 |
| jpich | d0ugal, tzumainn: I guess we can see about workflow-ising things that make sense once they start unduly growing too | 13:48 |
| tzumainn | d0ugal, in terms of API access to tripleo logic, I kinda see workflows as a necessary evil - as an outside consumer, I'd far prefer something more direct when needed | 13:48 |
| d0ugal | jtomasek: We need to document the workflows too /o\ | 13:48 |
| jpich | d0ugal: That makes a lot of sense | 13:48 |
| jpich | Yay to moar documentation in general | 13:48 |
| florianf | d0ugal, jpich, jtomasek: +1 to documentation. I, for one, didn't even consider using actions directly. | 13:48 |
| jtomasek | florianf: using actions directly has great benefit of working just like an ordinary API call request > response | 13:49 |
| d0ugal | jpich: Yeah, it is just a difficult transition if we say to people "use this action directly" then later we tell them to all transition to a workflow. There is no good way to do that. | 13:49 |
| *** fultonj has joined #tripleo | 13:49 | |
| d0ugal | tzumainn: Sure, I see your point. How do you propose we version actions? | 13:50 |
| *** fultonj has quit IRC | 13:50 | |
| openstackgerrit | Martin André proposed openstack/instack-undercloud: Introduce 'enable_ui' option https://review.openstack.org/344140 | 13:50 |
| d0ugal | florianf: Yeah, it is weird. I get the feeling it is in Mistral more for testing purposes. | 13:50 |
| tzumainn | d0ugal, haha, honestly, I don't know - I haven't gone far into mistral | 13:50 |
| EmilienM | tripleo meeting in 10 min https://etherpad.openstack.org/p/tripleo-meeting-items | 13:50 |
| tzumainn | d0ugal, honestly, I wanted an API : ) | 13:50 |
| d0ugal | tzumainn: Really? You should have said! :-P | 13:50 |
| tzumainn | d0ugal, I KNOW RIGHT | 13:50 |
| florianf | d0ugal, jpich, jtomasek: I agree it's not really consistent and having 2 different approaches can be problematic. Still, from a user view, it's incredibly more intuitive, since apparently actions behave much more like the usual rest api. | 13:51 |
| d0ugal | florianf: Sure | 13:51 |
| marios | Jokke_: thanks very much | 13:51 |
| tzumainn | florianf, d0ugal, jpich, jtomasek, I agree - if I'm an outside consumer and I'm querying, say, the list of plans, and then learn I have to monitor notifications to get the result - I'm going to be kinda confused as to why I have to do all of that | 13:51 |
| hewbrocca | tzumainn: seriously you need to learn to speak up for yourself better | 13:52 |
| florianf | d0ugal: but then again, you're totally right about api changes to include more complex things. | 13:52 |
| d0ugal | I wonder if we need to define which actions are "public" and which are private? I don't think we plan on supporting the interface for all of them. | 13:52 |
| tzumainn | hewbrocca, :( | 13:52 |
| Jokke_ | marios: np! | 13:52 |
| tzumainn | d0ugal, out of curiosity, what are some actions that wouldn't be 'public'? | 13:52 |
| pabelanger | EmilienM: thanks, plan on attending | 13:53 |
| *** rbrady has joined #tripleo | 13:53 | |
| d0ugal | tzumainn: You are going to have to monitor the result for all the workflows anyway - so one consistent way to consume tripleo seems easier to me. I have yet to figure out how to call actions in a way that works well from the CLI | 13:53 |
| *** fultonj has joined #tripleo | 13:54 | |
| d0ugal | tzumainn: well, we expect people to call list plans directly for example, but are we going to support calling the introspection actions directly? | 13:54 |
| d0ugal | tzumainn: we don't version them like we do with the workflows, so how are we going to manage breaking changes etc. | 13:54 |
| jtomasek | d0ugal: well yes, but calling an action is much less code on the client side compared to calling and listening to workflow | 13:54 |
| jtomasek | (at least in GUI) | 13:55 |
| d0ugal | jtomasek: except you can use the same generic code for one way - with actions you need two. | 13:55 |
| *** rhallisey has quit IRC | 13:55 | |
| tzumainn | d0ugal, same generic code != simpler, I'd argue | 13:55 |
| d0ugal | Anyway, I don't have a problem with calling actions directly. | 13:55 |
| *** mgarciam has quit IRC | 13:55 | |
| d0ugal | tzumainn: I disagree :) | 13:55 |
| d0ugal | Anyway - we need to solve documenting this and how to version it IMO | 13:55 |
| *** fultonj has quit IRC | 13:56 | |
| d0ugal | Otherwise it is fine. | 13:56 |
| jtomasek | d0ugal: +1 | 13:56 |
| *** stendulker has joined #tripleo | 13:56 | |
| *** fultonj has joined #tripleo | 13:56 | |
| *** fultonj has quit IRC | 13:56 | |
| *** rbrady has quit IRC | 13:56 | |
| d0ugal | rbrady: Have you thought about versioning the action names at all? | 13:56 |
| shardy | d0ugal: We'd have to register each version under a different entry point I think | 13:56 |
| *** fultonj has joined #tripleo | 13:56 | |
| d0ugal | shardy: Yeah, I think so. | 13:56 |
| shardy | e.g tripleo.v1.create_plan | 13:57 |
| shardy | probably wouldn't be so bad | 13:57 |
| d0ugal | shardy: Just like we do with the workflows. | 13:57 |
| jtomasek | +1 | 13:57 |
| d0ugal | shardy: Should I just go ahead and do that? Before we land more | 13:57 |
| *** ebalduf has joined #tripleo | 13:58 | |
| shardy | d0ugal: possibly, maybe we need to discuss when we rev the version first | 13:58 |
| shardy | e.g can we consider master unstable then rev it once per cycle? | 13:58 |
| *** dtantsur is now known as dtantsur|mtg | 13:58 | |
| shardy | Meeting in 1min | 13:59 |
| shardy | https://wiki.openstack.org/wiki/Meetings/TripleO | 13:59 |
| *** saneax_AFK is now known as saneax | 13:59 | |
| *** Goneri has quit IRC | 14:00 | |
| pabelanger | EmilienM: gfidente: centos storage now mirrored to AFS: http://mirror.dfw.rax.openstack.org/centos/7/storage/x86_64/ | 14:02 |
| EmilienM | cool | 14:02 |
| EmilienM | I'll patch puppet CI to use it later | 14:03 |
| paramite | guys, please can somebody find few minutes to review avail. monitoring spec, so we are not blocked by this? https://review.openstack.org/#/c/347399/ | 14:04 |
| *** Goneri has joined #tripleo | 14:04 | |
| paramite | http://m.memegen.com/8l18e1.jpg | 14:05 |
| *** rhallisey has joined #tripleo | 14:09 | |
| hewbrocca | EmilienM slagle ^^^ important, if you have any time at all | 14:10 |
| *** ooolpbot has joined #tripleo | 14:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1608785 | 14:10 |
| *** ooolpbot has quit IRC | 14:10 | |
| openstack | Launchpad bug 1608785 in tripleo "CI jobs fail with ceph-mon errors" [Critical,In progress] - Assigned to Giulio Fidente (gfidente) | 14:10 |
| *** links has quit IRC | 14:12 | |
| EmilienM | paramite, hewbrocca: currently in tripleo meeting | 14:13 |
| *** masco has quit IRC | 14:14 | |
| openstackgerrit | Marios Andreou proposed openstack/puppet-tripleo: Fixup nit in manila pacemaker profile, duplicate variable https://review.openstack.org/350046 | 14:17 |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates: Enable Manila integration - as a composable controller service https://review.openstack.org/188137 | 14:18 |
| *** rbrady has joined #tripleo | 14:19 | |
| *** tosky has quit IRC | 14:25 | |
| weshay | panda, can you update https://bugs.launchpad.net/tripleo/+bug/1572976 please | 14:27 |
| openstack | Launchpad bug 1572976 in tripleo "Periodically Apache httpd restart fails" [Medium,Triaged] - Assigned to Gabriele Cerami (gcerami) | 14:27 |
| *** stendulker has quit IRC | 14:28 | |
| *** Hazelesque has quit IRC | 14:28 | |
| *** tosky has joined #tripleo | 14:28 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Add --yes flags to heat stack-delete for liberty https://review.openstack.org/347202 | 14:29 |
| *** pkovar has joined #tripleo | 14:29 | |
| sshnaidm | gfidente, hi | 14:29 |
| gfidente | sshnaidm hey | 14:29 |
| *** oshvartz has quit IRC | 14:30 | |
| *** oshvartz has joined #tripleo | 14:31 | |
| *** saneax is now known as saneax_AFK | 14:31 | |
| *** Hazelesque has joined #tripleo | 14:32 | |
| *** pkovar has quit IRC | 14:36 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/tripleo-ci: Move image uploads to tarballs.openstack.org https://review.openstack.org/350061 | 14:36 |
| sshnaidm | gfidente, can I help you with mitaka and tripleoclient investigation? weshay told me there is a problem with version there..? | 14:37 |
| *** aufi has quit IRC | 14:44 | |
| *** rlandy has quit IRC | 14:46 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/tripleo-ci: Move image uploads to tarballs.openstack.org https://review.openstack.org/350061 | 14:48 |
| *** saneax_AFK is now known as saneax | 14:50 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/tripleo-ci: Move image uploads to tarballs.openstack.org https://review.openstack.org/350061 | 14:52 |
| *** jmiu has quit IRC | 14:53 | |
| *** jmiu has joined #tripleo | 14:53 | |
| *** hjensas has quit IRC | 14:54 | |
| openstackgerrit | Martin Mágr proposed openstack/tripleo-specs: Availability monitoring support https://review.openstack.org/347399 | 15:00 |
| *** rlandy has joined #tripleo | 15:00 | |
| *** dtrainor has joined #tripleo | 15:02 | |
| slagle | bnemec: shardy : since we were discussin specs...i actually think that ^^ is a good example and is about right in terms of detail | 15:02 |
| slagle | if you wanted to review :) | 15:03 |
| *** dtrainor has quit IRC | 15:07 | |
| pabelanger | Regarding 350061, what is the idea about using a cached version of ipa_images.tar and overcloud-full.tar in tripleo-ci? Which problems are they trying to solve for? | 15:07 |
| *** dtantsur|mtg is now known as dtantsur | 15:08 | |
| beagles | slagle: will the spec jenkins job balk at extra WS at the EOLN? | 15:08 |
| *** mbound has joined #tripleo | 15:10 | |
| *** ooolpbot has joined #tripleo | 15:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1608785 | 15:10 |
| *** ooolpbot has quit IRC | 15:10 | |
| openstack | Launchpad bug 1608785 in tripleo "CI jobs fail with ceph-mon errors" [Critical,In progress] - Assigned to Giulio Fidente (gfidente) | 15:10 |
| *** abehl has quit IRC | 15:10 | |
| gfidente | ^^ good news abot the above, multinode job passed | 15:11 |
| beagles | slagle, ne'er mind doesn't seem to | 15:12 |
| shardy | paramite, slagle: agreed, looks good, just a couple of minor comments but +2 | 15:12 |
| *** bana_k has joined #tripleo | 15:13 | |
| sshnaidm | slagle, hi, is this mirror-server with promote script on rh1 or rh2? | 15:14 |
| beagles | EmilienM, shardy, if you get a sec -> https://review.openstack.org/#/c/332147/. Maybe get a blueprint knocked off of the list :) | 15:14 |
| *** ebalduf has quit IRC | 15:15 | |
| *** dtrainor has joined #tripleo | 15:15 | |
| * paramite checking | 15:16 | |
| gfidente | more good news, liberty job passed as well | 15:17 |
| *** ramishra has quit IRC | 15:17 | |
| *** ebarrera has quit IRC | 15:18 | |
| *** ebalduf has joined #tripleo | 15:20 | |
| hewbrocca | wut | 15:21 |
| *** milan has quit IRC | 15:22 | |
| openstackgerrit | Keith Schincke proposed openstack/tripleo-heat-templates: Add support for deploying RGW on controllers. (Composable) https://review.openstack.org/289027 | 15:24 |
| EmilienM | beagles, dprince: https://review.openstack.org/#/c/332147/ | 15:24 |
| EmilienM | I'm afraid we duplicate a service, and just trick some parameters... | 15:25 |
| EmilienM | I have the feeling we could do better here | 15:25 |
| *** yamahata has joined #tripleo | 15:25 | |
| *** ramishra has joined #tripleo | 15:26 | |
| EmilienM | beagles: I -1'ed the patch, I think we don't have to duplicate code here | 15:26 |
| dprince | EmilienM: some of the neutron features, DVR, and NFV type deployments may well be implemented better as separate stacks | 15:26 |
| *** oshvartz has quit IRC | 15:27 | |
| gfidente | mitaka passed as well | 15:27 |
| dprince | EmilienM: to be clear we may be able to "extend" another class | 15:27 |
| EmilienM | dprince: and? beagles is using the same puppet profile | 15:27 |
| dprince | EmilienM: but I absolutely think we still need neutron-l3-compute-dvr.yaml | 15:27 |
| EmilienM | it's just a matter of hieradata here | 15:27 |
| dprince | EmilienM: totally happy to de-dup hiera so long as it works the same | 15:28 |
| EmilienM | what beagles should do is to add parameters in neutron-l3 service with defaults values for a regular l3 agent, and override the values when loading the dvr environment | 15:28 |
| EmilienM | and not duplicate the neutron l3 service | 15:29 |
| EmilienM | this approach doesn't scale and will add lot of duplicated code if we continue that way... | 15:29 |
| EmilienM | that is why we use hiera + puppet here, because you can reuse the same class but with different data | 15:29 |
| EmilienM | so let's do it that way and don't duplicate yaml files here. | 15:29 |
| openstackgerrit | Martin Mágr proposed openstack/tripleo-specs: Availability monitoring support https://review.openstack.org/347399 | 15:29 |
| EmilienM | I have a strong opinion about this thing | 15:29 |
| beagles | what's the proper way to configure the agent_mode differently depending on whether it is a compute or controller? | 15:30 |
| beagles | EmilienM, ^ | 15:30 |
| *** devvesa has quit IRC | 15:30 | |
| beagles | you would have to modify both controller.yaml and compute.yaml atm I think | 15:31 |
| beagles | whereas I was seeking a simple on/off mechanism | 15:31 |
| EmilienM | let me think | 15:32 |
| EmilienM | yeah you found the simpliest way to do it, but it's duplicating things | 15:32 |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates: DO NOT MERGE testing manila https://review.openstack.org/342961 | 15:32 |
| EmilienM | beagles: a simple off/on mechanism could be done in https://review.openstack.org/#/c/332147/16/environments/neutron-ovs-dvr.yaml | 15:33 |
| EmilienM | by setting the right parameters | 15:33 |
| beagles | EmilienM, still not sure how we'd get NeutronL3AgentMode to be dvr_snat on the controller and dvr on the compute nodes | 15:35 |
| EmilienM | beagles: let me look | 15:36 |
| *** lucasagomes has quit IRC | 15:37 | |
| d0ugal | rbrady: Do you mind if I rebase https://review.openstack.org/#/c/298732/ on https://review.openstack.org/#/c/337615/ ? | 15:38 |
| rbrady | d0ugal: please do | 15:38 |
| jaosorior | bnemec: Hey dude, I was finally able to reproduce the ironic-inspector issue | 15:40 |
| jaosorior | bnemec: It turned out to be an issue in Mistral. Submitted a fix for it | 15:40 |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-common: Adds Deployment Action & Workflow https://review.openstack.org/298732 | 15:40 |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-common: Adds action for template processing https://review.openstack.org/337615 | 15:40 |
| d0ugal | rbrady: ^ done | 15:40 |
| openstackgerrit | Merged openstack/puppet-pacemaker: Removing ignoring the action parameter in the agent_generator.rb so that the manifests for the fencing scripts will have the parameter available to them. https://review.openstack.org/332237 | 15:40 |
| *** dtrainor has quit IRC | 15:41 | |
| shardy | rbrady, d0ugal: Hey FYI I debugged the mistral deployment patch, and commented on https://review.openstack.org/#/c/337615/11 | 15:41 |
| bnemec | jaosorior: Glad to hear it. :-) | 15:42 |
| shardy | I think the problem is we're passing a swift container path into heatclient, then giving a function which asserts it's not a swift object | 15:42 |
| shardy | so heat doesn't pass the credentials to swift and downloading it fails | 15:42 |
| shardy | hit another problem now, but the stack is create in progress :) | 15:42 |
| d0ugal | shardy: oh, nice! | 15:42 |
| shardy | can we get that fix added, and rebase those tripleo-common patches? | 15:43 |
| shardy | e.g so they're in a series | 15:43 |
| shardy | pretty soon we can probably prove this in CI | 15:43 |
| d0ugal | shardy: I just did a rebase based on your comment | 15:43 |
| *** karthiks is now known as karthiks|afk | 15:43 | |
| d0ugal | shardy: but yeah, I can add that fix too. I just got a working undercloud finally | 15:43 |
| *** lucasagomes has joined #tripleo | 15:44 | |
| shardy | Oh yeah, so you did, thanks | 15:44 |
| shardy | I'm debugging the next issue noew | 15:44 |
| shardy | now | 15:44 |
| gfidente | master passed as well, results incoming | 15:45 |
| *** dtantsur is now known as dtantsur|afk | 15:45 | |
| d0ugal | rbrady: Do we not need an entry point in setup.cfg for this action? https://review.openstack.org/#/c/337615 | 15:46 |
| shardy | d0ugal: I think the next error is coming from tripleoclient (I'm running your wip patch) | 15:46 |
| shardy | 'NoneType' object has no attribute 'find' | 15:46 |
| shardy | I'll need to re-run with --debug to see exactly where | 15:46 |
| shardy | but AFAICS the mistral part worked | 15:46 |
| shardy | d0ugal: we also need to refactor the endpoint stuff from the client, or finally move that into puppet | 15:47 |
| d0ugal | shardy: ah, probably. That patch is a mess, I basically stopped to investigate mistral. | 15:47 |
| d0ugal | shardy: +1 | 15:47 |
| shardy | Ok, well hopfully things are unblocked now | 15:48 |
| EmilienM | shardy: could you also look https://review.openstack.org/#/c/332147/ and give feedback please? | 15:49 |
| shardy | EmilienM: this was actually why I asked for your feeback previously, when you said "almost good" | 15:51 |
| EmilienM | beagles: why don't you override ComputeServices in your env? | 15:51 |
| shardy | I'm open to refactoring it, but perhaps you can help beagles with it given the previous feedback | 15:51 |
| EmilienM | shardy: I missed this duplicated thing, my bad | 15:51 |
| shardy | EmilienM: we can't append to the *Services lists yet | 15:52 |
| *** jaosorior has quit IRC | 15:52 | |
| *** abehl has joined #tripleo | 15:52 | |
| shardy | So the environment would need an entirely duplicated list | 15:52 |
| EmilienM | mhh not good | 15:52 |
| EmilienM | I guess we have no choice to accept this design | 15:52 |
| shardy | ramishra is working on a new heat feature which will fix that | 15:52 |
| beagles | right... was going to say | 15:52 |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Install Ceph Hammer and/or Jewel from CentOS Storage SIG repos https://review.openstack.org/340504 | 15:53 |
| *** leanderthal is now known as leanderthal|afk | 15:53 | |
| gfidente | fiiiuuuu | 15:53 |
| shardy | https://review.openstack.org/#/c/346850/ | 15:53 |
| shardy | EmilienM: that's the heat patch | 15:53 |
| shardy | my preference would be to accept the duplication for now, then refactor when that lands | 15:54 |
| shardy | hopefully in time for newton | 15:54 |
| *** jaosorior has joined #tripleo | 15:54 | |
| EmilienM | beagles: I can +2 it, but I'm worried about this path | 15:54 |
| *** weshay is now known as weshay_lunch | 15:54 | |
| EmilienM | ok I'll +A | 15:54 |
| beagles | EmilienM, fwiw, I concur | 15:54 |
| beagles | EmilienM, neutron poses a few tangly bits that I would like to sort out, this is one of them | 15:54 |
| EmilienM | beagles: done. | 15:54 |
| beagles | EmilienM, thanks man | 15:54 |
| *** dtrainor has joined #tripleo | 15:55 | |
| * beagles adds himself to the heat patch | 15:55 | |
| *** liverpooler has quit IRC | 15:55 | |
| beagles | which seems pretty sweet | 15:55 |
| *** zoliXXL is now known as zoli|gone | 15:57 | |
| *** liverpooler has joined #tripleo | 15:58 | |
| *** noshankus has joined #tripleo | 15:58 | |
| *** zoli|gone is now known as zoli_gone-proxy | 15:58 | |
| vkmc | hey guys, quick question... in step 5. here http://tripleo.org/installation/installation.html it mentions that you can git checkout puppet modules that are not yet merged, is it possible to do the same with heat templates? | 15:58 |
| *** bootsha has joined #tripleo | 15:59 | |
| openstackgerrit | Merged openstack/tripleo-puppet-elements: Revert "bump puppet-ceph to stable/hammer" https://review.openstack.org/343667 | 15:59 |
| shardy | vkmc: Yes, just do openstack overcloud deploy --templates /path/to/the/templates | 16:01 |
| shardy | by default we use /user/share/openstack-tripleo-heat-templates, but you can point it at a local git checkout | 16:01 |
| vkmc | shardy, awesome, thanks | 16:02 |
| EmilienM | shardy, bnemec, slagle: please review https://review.openstack.org/350135 | 16:03 |
| *** weshay_lunch is now known as weshay | 16:04 | |
| rbrady | d0ugal: we only need setupy.cfg entry points for actions that need to be called directly | 16:04 |
| rbrady | d0ugal: the ProcessTemplatesAction is meant to be used with inheritence, but not called directly atm | 16:04 |
| *** jcoufal_ has joined #tripleo | 16:04 | |
| rbrady | shardy: thanks for debugging. I noticed this morning there is a large difference in the environment | 16:06 |
| *** jaosorior has quit IRC | 16:06 | |
| *** jcoufal has quit IRC | 16:07 | |
| sshnaidm | bnemec, hi, do you know maybe how is created the pip2-freeze log file here? http://logs.openstack.org/00/348900/1/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/d7c1c6f/logs | 16:08 |
| EmilienM | bnemec: did you see my question earlier about yum update on undercloud? | 16:08 |
| bnemec | EmilienM: Yeah, I think the answer depends on the outcome of the spec I linked. | 16:08 |
| EmilienM | mhh ok | 16:09 |
| *** numans has quit IRC | 16:09 | |
| bnemec | Personally I'm in favor of updating all the things, but there's disagreement on that. | 16:09 |
| EmilienM | puppet needs to know what was updated | 16:09 |
| EmilienM | oh in fact no, since we run db sync all the time | 16:10 |
| EmilienM | though we need to make sure services are restarted | 16:10 |
| bnemec | EmilienM: That's why we should stop all the services before updating. | 16:10 |
| bnemec | For other reasons too, which are discussed in the spec. | 16:10 |
| bnemec | Package updates don't work right if the service is running. | 16:11 |
| EmilienM | ok | 16:11 |
| EmilienM | I'll read the spec first :) | 16:11 |
| EmilienM | bnemec: yeah, upgrade is hanging for almost 1h on the upgrade | 16:11 |
| bnemec | Exactly :-) | 16:12 |
| hewbrocca | https://review.openstack.org/#/c/347399/ looks about ready to go folks | 16:12 |
| hewbrocca | lobbying for paramite :) | 16:12 |
| * bnemec is reviewing | 16:12 | |
| sshnaidm | does anybody know how is created pip2-freeze file in tripleo-ci? http://logs.openstack.org/00/348900/1/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/d7c1c6f/logs | 16:12 |
| EmilienM | hewbrocca: the spec looks good now, +2 | 16:12 |
| bnemec | sshnaidm: It's just the output of "pip freeze" on the node. | 16:12 |
| bnemec | ETOOMANYPINGS | 16:13 |
| sshnaidm | bnemec, on undercloud node? | 16:13 |
| sshnaidm | bnemec, I don't see where is it executed | 16:13 |
| bnemec | sshnaidm: Yes | 16:13 |
| sshnaidm | oh, it's in devstack repo.. | 16:15 |
| dmsimard | I suppose you guys have seen "[openstack-dev] [tc] persistently single-vendor projects" ? | 16:15 |
| EmilienM | no, we have actual work to do | 16:15 |
| EmilienM | #sorry | 16:16 |
| bnemec | #notsorry | 16:16 |
| bnemec | :-) | 16:16 |
| *** bootsha has quit IRC | 16:19 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/tripleo-ci: Move image uploads to tarballs.openstack.org https://review.openstack.org/350061 | 16:19 |
| *** mbound has quit IRC | 16:21 | |
| ccamacho | hey shardy, mind to read http://paste.openstack.org/show/545682/ ?? I'm not sure if this is worth enough for a blueprint | 16:22 |
| *** bootsha has joined #tripleo | 16:22 | |
| *** maeca1 has quit IRC | 16:23 | |
| serverascode | where do end users discuss tripleo issues? it seems this is a dev channel? | 16:25 |
| openstackgerrit | Carlos Camacho proposed openstack/tripleo-heat-templates: Next generation HA architecture work https://review.openstack.org/314208 | 16:26 |
| *** krotscheck is now known as krot_sickleave | 16:27 | |
| pabelanger | EmilienM: bnemec slagle: I'd like to get some feedback on https://review.openstack.org/#/c/350153/ It adds a new publisher to tripleo-ci for the purpose of uploading images to tarballs.o.o. | 16:28 |
| pabelanger | the purpose is to remove the private infrastructure tripleo-ci is using | 16:28 |
| openstackgerrit | Pradeep Kilambi proposed openstack/puppet-tripleo: Implement aodh profiles https://review.openstack.org/332854 | 16:28 |
| EmilienM | bnemec: what would you update before running puppet in the upgrade process? should I update instack* openstack-tripleo-* puppet-* openstack-puppet-* os-* diskimage-builder and then let puppet update the rest? | 16:28 |
| EmilienM | pabelanger: ok looking now | 16:29 |
| EmilienM | pabelanger: why || true? isn't -p enough? | 16:29 |
| pabelanger | EmilienM: I am removing it now | 16:30 |
| EmilienM | ok | 16:30 |
| pabelanger | EmilienM: there is so much code path in triplo-ci, it is possible it doesn't exist | 16:30 |
| slagle | pabelanger: i think that might break the repo promotion | 16:31 |
| slagle | pabelanger: promote.sh promotes based on the dirs created by upload.cgi (i think) | 16:31 |
| slagle | pabelanger: would probably have to also update promote.sh to look at t.o.o instead, as long as we get the repo that was used from there somehow | 16:32 |
| bnemec | EmilienM: I would update everything so we don't even have to deal with that question. | 16:32 |
| EmilienM | bnemec: ok | 16:33 |
| pabelanger | slagle: ya, I want to land 350153 first, which adds the publisher to JJB. Then we can recheck 350061 a few times making sure it doesn't break things | 16:33 |
| bnemec | It's a big part of the reason I'm in favor of the stop, update, puppet flow. | 16:33 |
| *** tosky has quit IRC | 16:34 | |
| pabelanger | slagle: in fact, we should likely talk about the promote.sh script and how it works. I think we'll need to redesign it | 16:36 |
| pabelanger | but not right now | 16:36 |
| *** akrivoka has quit IRC | 16:36 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: WIP - Implement undercloud upgrade job - Mitaka -> Newton https://review.openstack.org/346995 | 16:37 |
| *** Goneri has quit IRC | 16:37 | |
| *** hewbrocca is now known as hewbrocca-afk | 16:37 | |
| *** jmiu_ is now known as jmiu|lunch | 16:38 | |
| EmilienM | bnemec: your spec is excellent | 16:39 |
| bnemec | EmilienM: Thanks | 16:40 |
| *** Goneri has joined #tripleo | 16:40 | |
| bnemec | I realize it's a bear to review, but it turns out there's a lot of stuff in undercloud upgrades that we haven't previously given a lot of thought to. | 16:40 |
| *** mcornea has quit IRC | 16:42 | |
| openstackgerrit | Paul Belanger proposed openstack/tripleo-quickstart: Update directory layout per playbook best practices https://review.openstack.org/349076 | 16:44 |
| *** lucasagomes is now known as lucas-dinner | 16:44 | |
| slagle | jtomasek: did you want to comment on https://review.openstack.org/#/c/347399 as it relates to the gui? | 16:46 |
| * jtomasek looks | 16:47 | |
| *** jpich has quit IRC | 16:47 | |
| jtomasek | slagle: ah, yeah, I didn't get to it today:/ I'll do that first thing tomorrow, ok? | 16:48 |
| slagle | sure, np | 16:48 |
| *** saneax is now known as saneax_AFK | 16:49 | |
| *** tesseract- has quit IRC | 16:51 | |
| *** yamahata has quit IRC | 16:55 | |
| *** noshankus has quit IRC | 16:57 | |
| *** chlong has quit IRC | 16:58 | |
| *** dsariel has quit IRC | 16:59 | |
| *** jpena is now known as jpena|off | 16:59 | |
| *** ohamada has quit IRC | 17:02 | |
| *** julim has quit IRC | 17:02 | |
| bnemec | slagle: Any reason I shouldn't uncomment both of the commented cron jobs on the mirror server: http://paste.openstack.org/show/545695/ ? | 17:03 |
| slagle | bnemec: not that I can think of | 17:04 |
| bnemec | slagle: Okay, I'll go ahead and do that then. | 17:04 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci: WIP: DONT MERGE TESTING https://review.openstack.org/316436 | 17:06 |
| *** julim has joined #tripleo | 17:06 | |
| *** paramite has quit IRC | 17:08 | |
| openstackgerrit | Paul Belanger proposed openstack/tripleo-quickstart: Update directory layout per playbook best practices https://review.openstack.org/349076 | 17:08 |
| *** chlong has joined #tripleo | 17:11 | |
| pabelanger | weshay: I'd like to have a talk with you and your team about changing your usage of data_files with ansible roles. For example: https://github.com/redhat-openstack/ansible-role-tripleo-inventory/blob/master/setup.cfg#L24 | 17:12 |
| pabelanger | weshay: I don't think any python project should be installing things into the file system. We've been trying to stop this practice for a while in openstack. | 17:13 |
| pabelanger | weshay: additionally, they are getting installed into a no default path that ansible doesn't search, which means you then need to modify ansible.cfg | 17:13 |
| *** Goneri has quit IRC | 17:13 | |
| pabelanger | weshay: I propose we use stop having pip install the files and use either ansible-galaxy or zuul-cloner to put them into the properly location. | 17:14 |
| *** mbound has joined #tripleo | 17:16 | |
| *** ebarrera has joined #tripleo | 17:17 | |
| openstackgerrit | Paul Belanger proposed openstack/tripleo-quickstart: Update directory layout per playbook best practices https://review.openstack.org/349076 | 17:17 |
| *** crinkle has quit IRC | 17:17 | |
| *** bana_k has quit IRC | 17:18 | |
| *** gfidente has quit IRC | 17:21 | |
| *** maeca1 has joined #tripleo | 17:24 | |
| openstackgerrit | Paul Belanger proposed openstack/tripleo-quickstart: Update directory layout per playbook best practices https://review.openstack.org/349076 | 17:24 |
| *** liverpooler has quit IRC | 17:25 | |
| openstackgerrit | Paul Belanger proposed openstack/tripleo-quickstart: Update directory layout per playbook best practices https://review.openstack.org/349076 | 17:27 |
| *** Goneri has joined #tripleo | 17:28 | |
| weshay | pabelanger, ya.. we can discuss it, what are some of the alternatives? | 17:32 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: WIP - Implement undercloud upgrade job - Mitaka -> Newton https://review.openstack.org/346995 | 17:32 |
| weshay | ansible-galaxy does install config | 17:32 |
| weshay | or playbooks | 17:32 |
| *** yamahata has joined #tripleo | 17:32 | |
| * weshay looks at zuul-cloner | 17:32 | |
| openstackgerrit | Paul Belanger proposed openstack/tripleo-quickstart: Update directory layout per playbook best practices https://review.openstack.org/349076 | 17:32 |
| pabelanger | weshay: we can simply clone the roles into the playbook/roles folder, that should work out of box. I am testing that now. we usually use git, ansible-galaxy or zuul-cloner to do that | 17:34 |
| EmilienM | shardy, bnemec, slagle: could you look https://review.openstack.org/#/c/350135/ please? before pabelanger approves it | 17:36 |
| bnemec | EmilienM: So the goal with that is just quicker feedback on whether a patch breaks the undercloud install? | 17:38 |
| *** numans has joined #tripleo | 17:38 | |
| dsneddon | bnemec, slagle, shardy: If any of you have a moment, I think this BZ might be hitting a Heat-related bug, even though the error is networking-related: https://bugzilla.redhat.com/show_bug.cgi?id=1361877 | 17:39 |
| openstack | bugzilla.redhat.com bug 1361877 in rhel-osp-director "upgrade failed: Failed to detach interface" [Urgent,New] - Assigned to dsneddon | 17:39 |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci: Delete ping test environment in periodic jobs https://review.openstack.org/346134 | 17:39 |
| openstackgerrit | Paul Belanger proposed openstack/tripleo-quickstart: Update directory layout per playbook best practices https://review.openstack.org/349076 | 17:40 |
| openstackgerrit | Brent Eagles proposed openstack/tripleo-heat-templates: Add environment file to enable DVR https://review.openstack.org/332147 | 17:43 |
| beagles | EmilienM, shardy dprince: so close.. had to update service name to the proper format ^ | 17:44 |
| EmilienM | bnemec: yeah for now | 17:45 |
| bnemec | EmilienM: Okay, I'm fine with that. | 17:46 |
| EmilienM | bnemec: we'll probably remove this job once we have upgrade :) | 17:47 |
| EmilienM | but in the meantime, it's still useful feedback | 17:47 |
| EmilienM | since we're using the bits | 17:47 |
| bnemec | EmilienM: Sure, and it'll be good to exercise the job a bunch and see how stable it is. | 17:48 |
| *** abehl has quit IRC | 17:49 | |
| EmilienM | good | 17:51 |
| openstackgerrit | Paul Belanger proposed openstack/tripleo-quickstart: Update directory layout per playbook best practices https://review.openstack.org/349076 | 17:53 |
| *** electrofelix has quit IRC | 17:54 | |
| openstackgerrit | Merged openstack/os-net-config: Add adapter teaming support using teamd for ifcfg-systems https://review.openstack.org/339854 | 17:55 |
| *** numans has quit IRC | 17:56 | |
| *** crinkle has joined #tripleo | 18:00 | |
| *** Goneri has quit IRC | 18:00 | |
| *** mgould is now known as mgould|afk | 18:02 | |
| *** mbound has quit IRC | 18:02 | |
| openstackgerrit | Paul Belanger proposed openstack/tripleo-quickstart: Update directory layout per playbook best practices https://review.openstack.org/349076 | 18:07 |
| *** bana_k has joined #tripleo | 18:07 | |
| *** bootsha has quit IRC | 18:09 | |
| pabelanger | weshay: do you know who to ping about fixing the artifacts on failing jobs? https://ci.centos.org/artifacts/rdo/jenkins-tripleo-quickstart-gate-master-delorean-quick-basic-288/ is empty | 18:09 |
| *** shardy has quit IRC | 18:10 | |
| weshay | pabelanger, ya.. we need to change how we invoke logging... to be completely independent of quickstart I think | 18:11 |
| weshay | pabelanger, https://ci.centos.org/view/rdo/view/tripleo-gate/job/tripleo-quickstart-gate-master-delorean-quick-basic/ | 18:12 |
| *** bootsha has joined #tripleo | 18:12 | |
| openstackgerrit | Paul Belanger proposed openstack/tripleo-quickstart: Update directory layout per playbook best practices https://review.openstack.org/349076 | 18:14 |
| openstackgerrit | Paul Belanger proposed openstack/tripleo-quickstart: Update directory layout per playbook best practices https://review.openstack.org/349076 | 18:19 |
| openstackgerrit | Paul Belanger proposed openstack/tripleo-quickstart: Update directory layout per playbook best practices https://review.openstack.org/349076 | 18:20 |
| openstackgerrit | Paul Belanger proposed openstack/tripleo-quickstart: Update directory layout per playbook best practices https://review.openstack.org/349076 | 18:24 |
| ayoung | EmilienM, getting frustrated by Federation support in Puppet modules. It seems to be 1/2 done. Is there some way we could just stop trying to be smart in the puppet modules and just autogenerate the puppet code from the oslo-config code in Keystone? | 18:25 |
| ayoung | I have a feeling that the puppet modules are going to hard code things that are just not correct for all distros | 18:25 |
| ayoung | like | 18:25 |
| *** jrist has quit IRC | 18:27 | |
| ayoung | if $admin_port { | 18:27 |
| ayoung | concat::fragment { 'configure_mellon_on_port_35357': | 18:27 |
| ayoung | target => "${keystone::wsgi::apache::priority}-keystone_wsgi_admin.conf", | 18:27 |
| EmilienM | ayoung: can you talk about it on #puppet-openstack ? | 18:28 |
| ayoung | Its a little scary that someone wrote that code, and I have no idea what they tested it against | 18:28 |
| ayoung | EmilienM, the issue is that Tripleo uses it, and I am trying to make this work with Tripleo | 18:28 |
| EmilienM | ayoung: and ping richm and iury about that | 18:28 |
| EmilienM | I haven't writen any code about that | 18:28 |
| ayoung | the fact that it is two different projects is not helping. | 18:28 |
| *** jrist has joined #tripleo | 18:28 | |
| ayoung | EmilienM, I know...I was about to git blame it... | 18:29 |
| *** julim has quit IRC | 18:29 | |
| *** jrist has quit IRC | 18:30 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: WIP - Implement undercloud upgrade job - Mitaka -> Newton https://review.openstack.org/346995 | 18:30 |
| openstackgerrit | Paul Belanger proposed openstack/tripleo-quickstart: Update directory layout per playbook best practices https://review.openstack.org/349076 | 18:31 |
| ayoung | EmilienM, its all iury | 18:31 |
| *** julim has joined #tripleo | 18:31 | |
| EmilienM | ayoung: I agree with you | 18:32 |
| EmilienM | let me look code | 18:32 |
| ayoung | EmilienM, so, looks like no support for WebSSO, which would be tricky to do | 18:32 |
| ayoung | it has to configure both Keystone and Horizon | 18:32 |
| EmilienM | why is it tricky? | 18:33 |
| ayoung | EmilienM, YOU NEED THE fqdn OF THE Horizon server in Keystone | 18:33 |
| ayoung | trusted_dashboard value | 18:33 |
| ayoung | also, looks like it only supports one IdP IIUC | 18:33 |
| openstackgerrit | Joe Talerico proposed openstack/tripleo-heat-templates: Glance worker fix https://review.openstack.org/350219 | 18:33 |
| ayoung | <Location /v3/OS-FEDERATION/identity_providers/<%= scope['keystone::federation::mellon::idp_name']-%>/protocols/<%= scope['keystone::federation::mellon::protocol_name']-%>/auth> | 18:34 |
| EmilienM | ayoung: it's just a param, right? | 18:34 |
| EmilienM | we can use puppet-horizon and give the parameter | 18:34 |
| rook | EmilienM ^^^ Glance fix for the # of workers for THT | 18:34 |
| EmilienM | rook: ok, looking now | 18:34 |
| ayoung | EmilienM, so, in Tripleo, I would want this: | 18:34 |
| ayoung | 1. Uswe FQDNs EV-REE_WHERE! | 18:34 |
| ayoung | and thsu the openstack.<domain> value would be the hostname for the dashboard | 18:35 |
| EmilienM | rook: we'll see how it works in CI and if the worker param is empty | 18:35 |
| EmilienM | ayoung: ok cool, I see no problem at this | 18:35 |
| ayoung | and the same value stuck into hiera is used for the dashboard and generates the propert trusted dashboad value | 18:35 |
| EmilienM | I don't understand what is the prob with puppet here | 18:35 |
| ayoung | EmilienM, but Federation is necessarily multi value | 18:35 |
| ayoung | so on a deploy, the trusted dashboard value should also be able to add addtionnal trusted dashaboards...lets ignore that for now, though | 18:36 |
| ayoung | as that is 3rd party integration, and we are not there yet | 18:36 |
| ayoung | we need to ber able to create multiple IdPs | 18:36 |
| ayoung | and each Idp needs to be able to have multipe protocols | 18:36 |
| ayoung | for protocols, at a minimum we need saml2, oidc and Kerberos | 18:37 |
| ayoung | probably x509 too | 18:37 |
| EmilienM | good, and? | 18:37 |
| ayoung | and multiple mappings | 18:37 |
| EmilienM | you can do all of this with tripleo+puppet | 18:37 |
| EmilienM | we already handle complex configurations for neutron / cinder | 18:37 |
| EmilienM | we didn't wait for Keystone to have some :) | 18:37 |
| ayoung | EmilienM, then what is 'keystone::federation::mellon::idp_name' | 18:37 |
| EmilienM | yeah, we can change it | 18:37 |
| EmilienM | what did iury was a PoC | 18:38 |
| EmilienM | he was a first iteration | 18:38 |
| EmilienM | I'm sure nobody uses it right now | 18:38 |
| ayoung | EmilienM, those POCs need to not step on values. It would be better if we stopped making the puppet modules have any business logic in them | 18:38 |
| EmilienM | I'm sure we can continue this work and make it better | 18:38 |
| EmilienM | ayoung: ok, so where do you want to do it? | 18:38 |
| ayoung | EmilienM, I would be perfectly happy with things like this | 18:39 |
| ayoung | keystone::config::keystone_config: | 18:39 |
| ayoung | federation/sso_callback_template: | 18:39 |
| ayoung | value: /etc/keystone/sso_callback_template.html | 18:39 |
| ayoung | in tripleo heat templates, with a rule that says " match all of the hiera values like this with the expected values in keystone.conf" | 18:39 |
| ayoung | we could autogenerate the puppet to do that from http://git.openstack.org/cgit/openstack/keystone/tree/keystone/conf | 18:40 |
| EmilienM | please do not use keystone_config directly, and add the param to puppet-keystone | 18:40 |
| ayoung | EmilienM, I don't have that luxury | 18:40 |
| ayoung | I need to show that something works with Mitaka | 18:40 |
| pabelanger | weshay: who can I ping to help land https://review.openstack.org/#/c/348999 and https://review.openstack.org/#/c/349000 | 18:40 |
| EmilienM | I don't understand the autogenerate thing. | 18:40 |
| EmilienM | RDO already build packages using autogenerated config | 18:41 |
| EmilienM | and Puppet only overrides the parameters | 18:41 |
| ayoung | EmilienM, If I add a config option to Keystone, that should be the end of the discussion. Puppet should be a thin layer generated from that. One True source of truth | 18:41 |
| EmilienM | http://logs.openstack.org/69/232369/21/check/gate-puppet-openstack-integration-3-scenario003-tempest-centos-7/03ad6e4/logs/etc/keystone/keystone.conf.txt.gz | 18:41 |
| EmilienM | I am really confused by what you actually want. | 18:42 |
| EmilienM | "we could autogenerate" etc... | 18:42 |
| openstackgerrit | Paul Belanger proposed openstack/tripleo-quickstart: Update directory layout per playbook best practices https://review.openstack.org/349076 | 18:42 |
| ayoung | EmilienM, You know how the oslo-config code is used to generate the default config files for all the projects? | 18:43 |
| ayoung | each rule is definied like this | 18:43 |
| ayoung | http://git.openstack.org/cgit/openstack/keystone/tree/keystone/conf/identity.py#n32 | 18:43 |
| ayoung | for domain specific drivers | 18:43 |
| ayoung | cfg.BoolOpt | 18:44 |
| *** jbadiapa has quit IRC | 18:44 | |
| EmilienM | yes | 18:44 |
| ayoung | give me a single puppet module generated from the oslo-config definitions generated from Keystone | 18:44 |
| ayoung | call it puppet-keystone-conf | 18:44 |
| ayoung | split that out from any puppet work done to initialize the server, which is based on calls to keystone-manage bootstrap | 18:45 |
| ayoung | and a thrid puppet module for making API calls | 18:45 |
| ayoung | all three of those puppet modules should be autogenerated, not hand coded, based on the code in Keystone | 18:45 |
| openstackgerrit | Sarath Kumar Sankaran Kutty proposed openstack/os-net-config: Enable os-net-config to support and configure NFVSwitch https://review.openstack.org/345599 | 18:46 |
| EmilienM | in the Summit, we discussed many times of tools to generate configurations based on config parameters in the project | 18:46 |
| EmilienM | but nobody worked on it | 18:46 |
| *** ebarrera has quit IRC | 18:46 | |
| ayoung | Then, a separate module for dealing with SAML/mellon, nothing knowing about Keystone. | 18:46 |
| EmilienM | so we continue that way | 18:46 |
| EmilienM | all projects do the same also (ansible, chef, etc) | 18:46 |
| *** maeca1 has quit IRC | 18:46 | |
| openstackgerrit | Paul Belanger proposed openstack/tripleo-quickstart: Update directory layout per playbook best practices https://review.openstack.org/349076 | 18:47 |
| ayoung | EmilienM, anyway, I don't know if I can do what I need to do with the current code setup | 18:48 |
| ayoung | I am kindof coding by trial and error | 18:48 |
| openstackgerrit | Paul Belanger proposed openstack/tripleo-quickstart: Update directory layout per playbook best practices https://review.openstack.org/349076 | 18:49 |
| openstackgerrit | Paul Belanger proposed openstack/tripleo-quickstart: Update directory layout per playbook best practices https://review.openstack.org/349076 | 18:51 |
| openstackgerrit | Paul Belanger proposed openstack/tripleo-quickstart: Don't hide output from get-node.sh https://review.openstack.org/350229 | 18:54 |
| *** serverascode has left #tripleo | 18:57 | |
| *** bootsha has quit IRC | 19:03 | |
| *** mbound has joined #tripleo | 19:03 | |
| *** julim has quit IRC | 19:04 | |
| openstackgerrit | Paul Belanger proposed openstack/tripleo-quickstart: Update directory layout per playbook best practices https://review.openstack.org/349076 | 19:05 |
| *** julim has joined #tripleo | 19:06 | |
| openstackgerrit | Merged openstack/puppet-tripleo: New composable service: FluentD https://review.openstack.org/343546 | 19:07 |
| *** mbound has quit IRC | 19:08 | |
| openstackgerrit | Paul Belanger proposed openstack/tripleo-quickstart: Update directory layout per playbook best practices https://review.openstack.org/349076 | 19:09 |
| openstackgerrit | Justin Kilpatrick proposed openstack/tripleo-quickstart: Add documentation for add-on roles https://review.openstack.org/346733 | 19:09 |
| openstackgerrit | Paul Belanger proposed openstack/tripleo-quickstart: Update directory layout per playbook best practices https://review.openstack.org/349076 | 19:14 |
| openstackgerrit | Paul Belanger proposed openstack/tripleo-quickstart: Update directory layout per playbook best practices https://review.openstack.org/349076 | 19:26 |
| *** jdob has quit IRC | 19:28 | |
| *** jdob has joined #tripleo | 19:28 | |
| *** weshay has quit IRC | 19:30 | |
| openstackgerrit | Jeff Peeler proposed openstack/tripleo-common: Change input for UpdateCapabilitiesAction https://review.openstack.org/349732 | 19:33 |
| *** jdob has quit IRC | 19:39 | |
| ayoung | slagle, I set a handful of options in a post config, and not all of them got applied to the keystone.conf. How do I track what happened? I see nothing in the journal. | 19:39 |
| ayoung | the values are here: | 19:39 |
| *** jdob has joined #tripleo | 19:39 | |
| ayoung | https://paste.fedoraproject.org/400045/14701648/ | 19:39 |
| ayoung | and most of them were set, but not | 19:39 |
| ayoung | federation/sso_callback_template: or federation/trusted_dashboard: | 19:39 |
| openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates: Add Aodh composable roles https://review.openstack.org/333556 | 19:39 |
| ayoung | what logging is applicable on the controller node? | 19:40 |
| slagle | ayoung: do you see those options anywhere under /etc/puppet/hieradata on the overcloud nodes you are expecting to have them set? | 19:40 |
| ayoung | slagle, lets see... | 19:40 |
| ayoung | sudo grep -rn trusted_dashboard /etc/puppet/hieradata/ | 19:41 |
| ayoung | returns nothing | 19:41 |
| *** coolsvap has quit IRC | 19:41 | |
| slagle | did you pass this file during the deployment? | 19:41 |
| slagle | with -e? | 19:41 |
| ayoung | slagle, yep | 19:42 |
| ayoung | slagle, and there was another value in there that did get set | 19:42 |
| ayoung | sudo grep -rn remote_id_attribute /etc/puppet/hieradata/ | 19:42 |
| ayoung | /etc/puppet/hieradata/controller_extraconfig.yaml:10: "federation/remote_id_attribute": { | 19:42 |
| ayoung | slagle, and then I see both of those in the journal, too | 19:43 |
| ayoung | Aug 02 17:27:59 overcloud-controller-2 os-collect-config[2436]: Notice: /Stage[main]/Keystone::Config/Keystone_config[federation/remote_id_attribute]/ensure: created | 19:43 |
| slagle | ayoung: yea that's the file it should be in | 19:44 |
| slagle | ayoung: did you pass any other -e's on the deployment command line? | 19:44 |
| ayoung | slagle, quite a few. It is a quickstart | 19:44 |
| ayoung | slagle, but none were keystone specific that I know of. I can confirm, though | 19:45 |
| slagle | ayoung: yea i'm not familiar with what quickstart passes | 19:45 |
| ayoung | -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/net-single-nic-with-vlans.yaml -e $HOME/network-environment.yaml | 19:45 |
| ayoung | mostly networky things | 19:45 |
| ayoung | -e /usr/share/openstack-tripleo-heat-templates/environments/puppet-pacemaker.yaml | 19:45 |
| ayoung | so it enables pacemaker | 19:46 |
| ayoung | and then -e /home/stack/deploy_env.yaml is my stuff from the paste above | 19:46 |
| slagle | you could try a stack-show on the overcloud and Controller nested stack to see what the parameter value of controllerExtraConfig is | 19:46 |
| slagle | ayoung: something else to try would be to nest all the key/value pairs under a single keystone::config::keystone_config | 19:49 |
| slagle | i wonder if the last is winning | 19:49 |
| ayoung | slagle, ah.... | 19:49 |
| ayoung | that would make sense...so it would be like | 19:49 |
| slagle | i'm putting my money on this being the issue | 19:49 |
| ayoung | slagle, thanks, I'll try that | 19:50 |
| ayoung | slagle, Highlander syndrome strikes again | 19:51 |
| *** dprince has quit IRC | 19:51 | |
| ayoung | https://www.youtube.com/watch?v=sqcLjcSloXs | 19:51 |
| *** bootsha has joined #tripleo | 19:57 | |
| * bnemec watched Highlander recently | 19:57 | |
| bnemec | It was...underwhelming | 19:57 |
| *** jrist has joined #tripleo | 19:58 | |
| *** florianf has quit IRC | 20:00 | |
| *** julim has quit IRC | 20:04 | |
| slagle | maybe you could review https://review.openstack.org/#/c/347008/ to help you forget | 20:06 |
| *** weshay has joined #tripleo | 20:18 | |
| *** sarath has joined #tripleo | 20:28 | |
| ayoung | bnemec, but admit it, EmilienM could pass for Christopher Lambert. | 20:30 |
| ayoung | slagle, that worked. | 20:30 |
| ayoung | thanks | 20:30 |
| slagle | cool | 20:31 |
| *** openstackgerrit_ has joined #tripleo | 20:35 | |
| *** openstackgerrit_ has quit IRC | 20:36 | |
| openstackgerrit | Matthew Thode proposed openstack/diskimage-builder: Optionally remove portage files https://review.openstack.org/348536 | 20:38 |
| ayoung | slagle, http://adam.younglogic.com/2016/08/customizing-a-tripleo-quickstart-deploy/ | 20:41 |
| ayoung | slagle, bnemec is it possible to deploy such that the authUrl and endpoints use domain names instead of ip addresses? | 20:48 |
| *** bootsha has quit IRC | 20:48 | |
| *** fragatina has joined #tripleo | 20:50 | |
| *** fragatina has quit IRC | 20:50 | |
| *** fragatina has joined #tripleo | 20:51 | |
| openstackgerrit | Paul Belanger proposed openstack/tripleo-quickstart: [WIP] POC using ansible-galaxy https://review.openstack.org/350263 | 20:52 |
| *** snecklifter has quit IRC | 20:52 | |
| slagle | ayoung: i don't think so. not until we land the patch to switch endpoint creation to ansible | 20:53 |
| slagle | to puppet rather | 20:53 |
| ayoung | Freudian slip there slagle | 20:54 |
| EmilienM | slagle: what do you think of making gate-tripleo-ci-centos-7-nonha-multinode-nv voting? | 20:55 |
| EmilienM | I currently see 0 blocker | 20:55 |
| slagle | EmilienM: the last thing i wanted to double check was if the bug where the depends-on puppet modules was not used is truly fixed | 20:56 |
| slagle | so we could push up some test patches with a depends-on on some puppet change | 20:57 |
| slagle | and make sure that it's used on the oc node | 20:57 |
| EmilienM | I tested this week end | 20:58 |
| EmilienM | but I am going to try again now | 20:58 |
| *** kbyrne has quit IRC | 20:58 | |
| slagle | yea lets double check since we swithced to packaged puppet modules as well | 20:58 |
| EmilienM | oki doc | 20:58 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: TripleO Overcloud CI test - do not merge https://review.openstack.org/350266 | 21:00 |
| EmilienM | slagle: done https://review.openstack.org/350265 and https://review.openstack.org/350266 | 21:00 |
| EmilienM | so we test again depends-on and single patches in puppet modules themselves | 21:01 |
| EmilienM | I took gnocchi because we deploy it by default on overcloud and we don't deploy it at all on undercloud | 21:01 |
| *** akshai has quit IRC | 21:02 | |
| slagle | merci vu cu | 21:02 |
| EmilienM | vu cu? or beaucoup? :P | 21:02 |
| slagle | yea that's the one | 21:04 |
| jmiu|lunch | haha | 21:06 |
| *** jmiu has quit IRC | 21:06 | |
| openstackgerrit | Paul Belanger proposed openstack/tripleo-quickstart: [WIP] Stop using data_files https://review.openstack.org/350271 | 21:06 |
| *** jmiu has joined #tripleo | 21:06 | |
| *** jmiu has quit IRC | 21:06 | |
| *** jmiu|lunch is now known as jmiu | 21:06 | |
| *** fultonj has quit IRC | 21:07 | |
| *** jmiu_ has joined #tripleo | 21:07 | |
| *** jcoufal_ has quit IRC | 21:08 | |
| *** akshai has joined #tripleo | 21:09 | |
| *** akshai has quit IRC | 21:10 | |
| *** mbound has joined #tripleo | 21:11 | |
| *** panda is now known as panda|Zz | 21:14 | |
| *** kbyrne has joined #tripleo | 21:19 | |
| *** rhallisey has quit IRC | 21:22 | |
| *** jayg is now known as jayg|g0n3 | 21:26 | |
| EmilienM | bnemec: when you upgraded from mitaka to newton, what did hang? mysql? | 21:26 |
| *** rlandy has quit IRC | 21:27 | |
| bnemec | EmilienM: Yeah, I had issues with the database because I didn't have anything to handle the major version bump. | 21:29 |
| bnemec | Or at least I assume that was the reason. | 21:29 |
| EmilienM | bnemec: how did you upgrade it? mysqldump/restore? | 21:29 |
| bnemec | ayoung: You can deploy with domain name endpoints. You have to alter the endpoint map though. | 21:29 |
| ayoung | bnemec, what about the auth URL and the HOrizon URL? | 21:30 |
| *** rlandy has joined #tripleo | 21:30 | |
| EmilienM | oh there is a procedure | 21:30 |
| EmilienM | https://mariadb.com/kb/en/mariadb/upgrading-from-mariadb-55-to-mariadb-100/ | 21:30 |
| ayoung | bnemec, basically, I care about the HTTPD stuff | 21:30 |
| bnemec | ayoung: The tls-enabled version is here: https://github.com/openstack/tripleo-heat-templates/blob/master/environments/enable-tls.yaml | 21:30 |
| bnemec | EmilienM: Yeah, I think that's in the proposed upgrade doc. Let me find it. | 21:30 |
| *** lblanchard has quit IRC | 21:31 | |
| ayoung | bnemec, I think it is "cloudname" that I want as a FQDN | 21:31 |
| bnemec | ayoung: Right, you set that via the CloudName parameter. | 21:31 |
| bnemec | EmilienM: See line 70 in https://review.openstack.org/#/c/308985/8/doc/source/post_deployment/upgrade.rst | 21:32 |
| EmilienM | ok | 21:33 |
| EmilienM | bnemec: why do we need that in mitaka -> newton | 21:33 |
| EmilienM | is it 5.5 -> 10 ? | 21:33 |
| bnemec | EmilienM: I'm not sure, but it looked like I was getting the version upgrade there. It's possible those docs are based on a downstream process where the package version may have changed at a different time. | 21:33 |
| bnemec | This is another good reason to automate it somehow so we just look at the version of the package, and don't try to guess based on the releases involved. | 21:34 |
| EmilienM | mhh in mitaka we already have MariaDB 10.1 | 21:35 |
| ayoung | bnemec, is that something I can set at the top level, so just append CloudName=openstack.ayoung-dell-t1700.test to an env file and pass it to openstack overcloud deploy with a -e option? | 21:36 |
| bnemec | ayoung: Yeah, there's an example here: http://docs.openstack.org/developer/tripleo-docs/advanced_deployment/ssl.html#dns-endpoint-configuration | 21:36 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: WIP - Implement undercloud upgrade job - Mitaka -> Newton https://review.openstack.org/346995 | 21:37 |
| *** fragatina has quit IRC | 21:38 | |
| openstackgerrit | Matthew Thode proposed openstack/diskimage-builder: Optionally remove portage files https://review.openstack.org/348536 | 21:43 |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Break out bootstrapping of subnodes setup https://review.openstack.org/347008 | 21:44 |
| *** bfournie has quit IRC | 22:13 | |
| *** weshay has quit IRC | 22:19 | |
| *** rlandy has quit IRC | 22:29 | |
| *** mbound has quit IRC | 22:41 | |
| *** rlandy has joined #tripleo | 22:43 | |
| *** mbound has joined #tripleo | 22:49 | |
| *** david-lyle has quit IRC | 22:50 | |
| *** david-lyle has joined #tripleo | 22:51 | |
| *** yamahata has quit IRC | 22:52 | |
| *** mbound has quit IRC | 22:54 | |
| *** lucas-dinner has quit IRC | 22:54 | |
| pabelanger | tripleo-quickstart-core, I've added the first experimental job for the project: //review.openstack.org/350301 | 22:58 |
| pabelanger | While we won't be testing a full deploy, we should be able to run something. I picked bare to start | 22:58 |
| *** lucasagomes has joined #tripleo | 22:59 | |
| pabelanger | actually, bare isn't a thing | 22:59 |
| ayoung | pabelanger, don't you get annoyed by how cut and pastr drops things beforethe : | 23:00 |
| ayoung | http://review.openstack.org/350301 | 23:00 |
| *** saneax_AFK is now known as saneax | 23:00 | |
| pabelanger | ya, failbot | 23:01 |
| pabelanger | boat* | 23:01 |
| *** gfidente has joined #tripleo | 23:01 | |
| ayoung | I like failbot | 23:01 |
| *** julim has joined #tripleo | 23:03 | |
| *** Ryjedo has quit IRC | 23:12 | |
| *** yamahata has joined #tripleo | 23:13 | |
| *** bfournie has joined #tripleo | 23:29 | |
| *** radek_ has quit IRC | 23:34 | |
| *** radek_ has joined #tripleo | 23:46 | |
| *** morazi has quit IRC | 23:49 | |
| *** dprince has joined #tripleo | 23:56 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
