| *** openstack has joined #tripleo | 00:03 | |
| *** chem`` has quit IRC | 00:07 | |
| *** rbrady has quit IRC | 00:07 | |
| *** r-mibu has quit IRC | 00:26 | |
| *** r-mibu has joined #tripleo | 00:26 | |
| *** tzumainn has quit IRC | 00:35 | |
| *** penick has quit IRC | 00:42 | |
| *** chlong has quit IRC | 00:46 | |
| *** openstackstatus has quit IRC | 00:46 | |
| *** panda has quit IRC | 00:48 | |
| *** thrash is now known as thrash|g0ne | 00:48 | |
| *** panda has joined #tripleo | 00:49 | |
| *** openstack has joined #tripleo | 00:54 | |
| *** dkelson has joined #tripleo | 01:10 | |
| dkelson | I'm trying to do some customization of my overcloud controller using NodeExtraConfigPost with puppet, but when I run "openstack service list" (after setting the appropriate env variables using hieradata for the password, and URL) I get "ERROR: openstack The service catalog is empty.". However logging into the controller, setting the same env variables and running the same command results in the expected output. | 01:15 |
|---|---|---|
| *** fzdarsky_ has joined #tripleo | 01:18 | |
| openstackgerrit | Ben Swartzlander proposed openstack/tripleo-heat-templates: Add integration with NetApp Manila driver https://review.openstack.org/188138 | 01:19 |
| *** fzdarsky|afk has quit IRC | 01:22 | |
| *** TSCHAK has joined #tripleo | 01:27 | |
| openstackgerrit | Liu Qing proposed openstack/diskimage-builder: Fix variable unbound error while REPOREF="*" https://review.openstack.org/320859 | 01:28 |
| *** TSCHAK has quit IRC | 01:44 | |
| *** openstackgerrit has quit IRC | 01:49 | |
| *** TSCHAK has joined #tripleo | 01:52 | |
| *** rhallisey has quit IRC | 01:53 | |
| *** TSCHAK has quit IRC | 01:55 | |
| *** dmacpher-afk has joined #tripleo | 01:57 | |
| *** openstackgerrit has joined #tripleo | 01:57 | |
| *** chlong has joined #tripleo | 01:59 | |
| *** TSCHAK has joined #tripleo | 02:08 | |
| *** saneax_AFK is now known as saneax | 02:24 | |
| *** myoung has joined #tripleo | 02:24 | |
| *** openstack has joined #tripleo | 02:38 | |
| *** openstackstatus has joined #tripleo | 02:40 | |
| *** ChanServ sets mode: +v openstackstatus | 02:40 | |
| *** myoung has quit IRC | 02:40 | |
| *** dmacpher-afk is now known as dmacpher | 03:11 | |
| *** saneax is now known as saneax_AFK | 03:14 | |
| *** julim has joined #tripleo | 03:24 | |
| *** julim has quit IRC | 03:30 | |
| *** saneax_AFK is now known as saneax | 03:30 | |
| *** links has joined #tripleo | 03:34 | |
| *** karthiks has quit IRC | 03:45 | |
| *** ramishra has quit IRC | 03:59 | |
| *** rlandy has quit IRC | 04:10 | |
| *** saneax is now known as saneax_AFK | 04:14 | |
| *** chlong has quit IRC | 04:15 | |
| *** yamahata has joined #tripleo | 04:17 | |
| *** masco has joined #tripleo | 04:21 | |
| *** chlong has joined #tripleo | 04:34 | |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-quickstart: Tempest support for quickstart https://review.openstack.org/318751 | 04:38 |
| *** panda has quit IRC | 04:48 | |
| *** panda has joined #tripleo | 04:49 | |
| *** gumbyblake has joined #tripleo | 04:59 | |
| *** jaosorior has joined #tripleo | 05:05 | |
| *** stendulker has joined #tripleo | 05:12 | |
| *** dsariel has joined #tripleo | 05:15 | |
| *** yolanda_ has quit IRC | 05:23 | |
| coolsvap | morning folks | 05:29 |
| *** bnemec has quit IRC | 05:30 | |
| *** bnemec has joined #tripleo | 05:31 | |
| jaosorior | coolsvap: Hey man | 05:33 |
| *** mikelk has joined #tripleo | 05:33 | |
| coolsvap | jaosorior, hello | 05:34 |
| *** apetrich has joined #tripleo | 05:37 | |
| *** apetrich has quit IRC | 05:39 | |
| *** apetrich has joined #tripleo | 05:40 | |
| openstackgerrit | Merged openstack/tripleo-image-elements: Remove tuskar element https://review.openstack.org/320908 | 05:40 |
| *** bnemec has quit IRC | 05:44 | |
| *** ramishra has joined #tripleo | 05:47 | |
| *** chlong has quit IRC | 05:47 | |
| *** dsariel has quit IRC | 05:47 | |
| *** bnemec has joined #tripleo | 05:52 | |
| *** yolanda_ has joined #tripleo | 05:52 | |
| *** rcernin has joined #tripleo | 05:54 | |
| *** florianf has joined #tripleo | 05:56 | |
| *** karthiks has joined #tripleo | 05:57 | |
| *** chlong has joined #tripleo | 05:59 | |
| *** bvandenh has quit IRC | 06:00 | |
| *** itamarl has joined #tripleo | 06:03 | |
| *** bnemec has quit IRC | 06:12 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/tripleo-ci: Add MysqlInternal endpoint to enable-tls https://review.openstack.org/321363 | 06:12 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Pass MysqlVirtualIP via EndpointMap https://review.openstack.org/305751 | 06:13 |
| *** bnemec has joined #tripleo | 06:14 | |
| *** bvandenh has joined #tripleo | 06:15 | |
| *** fragatina has quit IRC | 06:18 | |
| *** bnemec has quit IRC | 06:23 | |
| *** yolanda_ has quit IRC | 06:29 | |
| *** numans has joined #tripleo | 06:31 | |
| *** yolanda has joined #tripleo | 06:31 | |
| *** pkovar has joined #tripleo | 06:36 | |
| *** bnemec has joined #tripleo | 06:36 | |
| *** jprovazn has joined #tripleo | 06:39 | |
| *** dsariel has joined #tripleo | 06:51 | |
| *** openstack has joined #tripleo | 06:54 | |
| *** ccamacho|hungry is now known as ccamacho | 06:59 | |
| *** mcornea has joined #tripleo | 07:01 | |
| *** oshvartz has joined #tripleo | 07:01 | |
| *** paramite has joined #tripleo | 07:06 | |
| *** greghaynes has quit IRC | 07:08 | |
| *** mikelk has quit IRC | 07:14 | |
| *** ebarrera has joined #tripleo | 07:20 | |
| *** greghaynes has joined #tripleo | 07:21 | |
| *** athomas has joined #tripleo | 07:22 | |
| *** olap has joined #tripleo | 07:26 | |
| *** liverpooler has joined #tripleo | 07:27 | |
| *** gfidente has joined #tripleo | 07:30 | |
| jaosorior | marios: +1 on landing heat | 07:30 |
| *** dmacpher has quit IRC | 07:30 | |
| gfidente | jaosorior, so I had to skip tests in 4.0 | 07:31 |
| gfidente | I think it's best we can do for now | 07:31 |
| jaosorior | gfidente: Yeah man, I couldn't think of anything else myself | 07:32 |
| jaosorior | those backward incompatible changes in puppet are pretty harsh to deal with :/ | 07:32 |
| *** olap has quit IRC | 07:32 | |
| gfidente | well it works with puppet > 4.1 | 07:32 |
| marios | jaosorior: yeah, my hesitation is cos i missed that cfn endpoitn stuff on the tht side - but I think they are good to go now | 07:32 |
| gfidente | because they fixed the visibility issue | 07:32 |
| gfidente | so you *can* use call_functions(hiera from a 3x puppet and it will see the 3x version of hiera() | 07:33 |
| gfidente | so the tests pass on -latest- and on 3x | 07:33 |
| marios | jaosorior: +Ad the puppet-tripleo | 07:33 |
| gfidente | they just fail in 4.0 | 07:33 |
| marios | gfidente: that review you mentioned only has +1 still | 07:33 |
| gfidente | marios, yeah I know :( | 07:34 |
| marios | gfidente: oh isee | 07:34 |
| marios | gfidente: yeah ok, just saw what it was | 07:34 |
| *** gumbyblake has quit IRC | 07:34 | |
| *** olap has joined #tripleo | 07:34 | |
| gfidente | marios, thanks! :) | 07:35 |
| *** dtantsur|afk is now known as dtantsur | 07:36 | |
| *** aufi has joined #tripleo | 07:36 | |
| openstackgerrit | Merged openstack/puppet-tripleo: Add Heat profiles https://review.openstack.org/313019 | 07:39 |
| *** chem`` has joined #tripleo | 07:41 | |
| *** ifarkas has joined #tripleo | 07:42 | |
| jaosorior | gfidente | 07:43 |
| jaosorior | by the way | 07:43 |
| jaosorior | I found the issue with the database uris in the MysqlInternal endpoint patch | 07:43 |
| *** hjensas has quit IRC | 07:43 | |
| jaosorior | gfidente: It needed the addition of that endpoint to tripleo-ci's enable-tls https://review.openstack.org/#/c/321363/ | 07:43 |
| openstackgerrit | Martin André proposed openstack/tripleo-validations: Initialize README https://review.openstack.org/321396 | 07:43 |
| jaosorior | that environment is a sneaky bastard X_x which makes me want to push more for this https://review.openstack.org/#/c/308182/ we would avoid all those issues if that merged | 07:44 |
| *** bvandenh has quit IRC | 07:45 | |
| gfidente | jaosorior, oh right | 07:45 |
| gfidente | in tripleo-ci | 07:45 |
| gfidente | I faced this a week ago | 07:45 |
| gfidente | https://review.openstack.org/#/c/318050/ | 07:46 |
| bandini | is the ci completely fubared atm? | 07:46 |
| gfidente | but it didn't come up to my ming | 07:46 |
| gfidente | jaosorior, great so let's merge that | 07:46 |
| *** zoli_gone-proxy is now known as zoliXXL | 07:46 | |
| gfidente | but yeah agreed ideally we need to get rid of that custom map in CI or find a different way to customize things | 07:46 |
| bandini | https://review.openstack.org/#/c/319432 done a third recheck but it never seems to find any vms | 07:46 |
| gfidente | jaosorior++ :) | 07:46 |
| jaosorior | bandini: yeah, CI is pretty flaky :/ | 07:47 |
| bandini | jaosorior: any bug open about or do I need to take out my voodoo dolls? | 07:47 |
| bandini | :) | 07:47 |
| jaosorior | there should be a bug, I think, shardy and derekh were working on it the other day | 07:48 |
| jaosorior | bandini: But the voodoo dalls and a rabbit foot would sure help | 07:48 |
| bandini | ack, thanks will look for it and cc myself | 07:48 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/tripleo-ci: Add MysqlInternal endpoint to enable-tls https://review.openstack.org/321363 | 07:50 |
| ccamacho | hey! quick question, for stable/liberty all gates are also broken? https://review.openstack.org/#/c/321060/1 Or is a temporary issue? | 07:51 |
| jaosorior | what? | 07:51 |
| jaosorior | only upgrades was broken | 07:51 |
| jaosorior | the rest of them being broken is an actual problem | 07:51 |
| bandini | ccamacho: I see the *same* issue | 07:52 |
| ccamacho | i have pushed a patch to tht in stable/liberty without changes, and upgrades, ha and nonha are getting issues.. | 07:52 |
| *** dbecker has joined #tripleo | 07:52 | |
| ccamacho | :( | 07:52 |
| marios | ccamacho: nice patch :) | 07:52 |
| bandini | eheh | 07:52 |
| jaosorior | ccamacho: the error seems to be in the undercloud deployment | 07:52 |
| ccamacho | yeahp, I thought it was a temporary issue | 07:54 |
| jaosorior | not sure if this is related | 07:54 |
| jaosorior | May 25 15:46:46 instack kdumpctl: No memory reserved for crash kernel. | 07:54 |
| bandini | I saw the issue yesterday as well | 07:54 |
| jaosorior | May 25 15:46:46 instack kdumpctl: Starting kdump: [FAILED] | 07:54 |
| jaosorior | May 25 15:46:46 instack systemd: kdump.service: main process exited, code=exited, status=1/FAILURE | 07:54 |
| jaosorior | May 25 15:46:46 instack systemd: Failed to start Crash recovery kernel arming. | 07:54 |
| bandini | it is okay if kdump fails, we can ignore that | 07:54 |
| jaosorior | guys, any idea where we can get the puppet logs from the undercloud? | 07:55 |
| *** athomas has quit IRC | 07:55 | |
| jaosorior | found them | 07:55 |
| jaosorior | nevermind | 07:55 |
| *** bvandenh has joined #tripleo | 07:55 | |
| jaosorior | ceilometer is broken | 07:55 |
| jaosorior | http://paste.openstack.org/show/505580/ | 07:56 |
| jaosorior | ccamacho: tried posting a message in #rdo about it, seems it's an issue with the ceilometer-alarm package | 07:57 |
| ccamacho | jaosorior I will follow it, thanks! | 07:58 |
| jaosorior | well, maybe it's the package, or maybe it's the repo setup for stable/liberty | 07:58 |
| *** zoliXXL is now known as zoli|wfh | 07:59 | |
| *** paramite is now known as paramite|afk | 08:00 | |
| *** athomas has joined #tripleo | 08:00 | |
| zoli|wfh | good morning | 08:01 |
| *** chlong has quit IRC | 08:04 | |
| *** cmyster has quit IRC | 08:07 | |
| *** dtantsur is now known as dtantsur|bbl | 08:07 | |
| *** paramite|afk is now known as paramite | 08:08 | |
| *** cmyster has joined #tripleo | 08:12 | |
| *** cmyster has quit IRC | 08:12 | |
| *** cmyster has joined #tripleo | 08:12 | |
| *** afazekas is now known as afazekas|sick | 08:12 | |
| bandini | jaosorior: I will file a BZ against RDO, ok? (at least we track this) | 08:14 |
| *** paramite is now known as paramite|afk | 08:14 | |
| jaosorior | bandini: You mean for the ceilometer issue? | 08:14 |
| jaosorior | yeah, only thing is that I'm not sure if it's an RDO issue, or is it that we misconfigured the repos. shardy and derekh would be more capable of debugging that | 08:15 |
| *** paramite|afk is now known as paramite | 08:16 | |
| *** electrofelix has joined #tripleo | 08:18 | |
| bandini | jaosorior: ack, I'll start with an LP bug and then we can create an rdo bz if needed | 08:21 |
| jaosorior | bandini: Sounds good to me | 08:21 |
| bandini | although I havent seen any changes related to liberty in tripleo-ci recently | 08:22 |
| jaosorior | bandini: I think it's either an error in packaging, or an error in our repo setup | 08:22 |
| *** lucas-dinner is now known as lucasagmes | 08:22 | |
| *** lucasagmes is now known as lucasagomes | 08:22 | |
| *** dmk0202 has joined #tripleo | 08:23 | |
| bandini | jaosorior: https://bugs.launchpad.net/tripleo/+bug/1585937 I could not find the repolist in the logs though | 08:31 |
| openstack | Launchpad bug 1585937 in tripleo "liberty ci is broken (undercloud install fails)" [Undecided,New] | 08:31 |
| jaosorior | bandini: thanks | 08:32 |
| bandini | I need to install a liberty system anyway, so I guess I'll get to the same error | 08:33 |
| *** sshnaidm has quit IRC | 08:34 | |
| jaosorior | bandin | 08:36 |
| jaosorior | *bandini, let me know when you have a liberty deployment | 08:36 |
| jaosorior | have an unrelated question about that | 08:36 |
| bandini | jaosorior: claro ;) | 08:37 |
| * bandini flexes his spanish brain muscles | 08:38 | |
| jaosorior | jajaja va pues. Gracias | 08:38 |
| *** derekh has joined #tripleo | 08:39 | |
| openstackgerrit | Jiri Stransky proposed openstack/tripleo-heat-templates: [EARLY WIP] Composable Upgrades https://review.openstack.org/321416 | 08:39 |
| jistr | marios, social: if you have a moment at some point, it would be great if you could check out ^^^ | 08:40 |
| *** tosky has joined #tripleo | 08:40 | |
| jistr | i was able to "no-op upgrade" keystone and glance with that | 08:41 |
| marios | jistr: ack thx | 08:41 |
| jistr | marios: i also thought about the increased flexibility of the whole solution we talked about, and i think it could be doable at least to some extent | 08:41 |
| *** paramite is now known as paramite|afk | 08:42 | |
| jistr | also i got back to the per-service rather than per-component structuring on the lowest level, to fully map 1-1 to the composable services | 08:42 |
| jaosorior | anybody has a stable/mitaka environment around? | 08:43 |
| jistr | it's not the best at some spots (e.g. as we discussed, one cannot update RPMs for glance-api and glance-registry separately, as they're one RPM), but it shouldn't break, as those two services *must* be upgraded at the same time always anyway | 08:43 |
| jistr | to keep AMQP messaging formats in sync | 08:44 |
| *** jaosorior is now known as jaosorior_lunch | 08:45 | |
| openstackgerrit | Carlos Camacho proposed openstack/tripleo-docs: Composable roles within services Tutorial https://review.openstack.org/311512 | 08:47 |
| *** panda has quit IRC | 08:48 | |
| marios | jistr: i see ok thanks for the info, will have a look and we can catchup later/tomorrow again | 08:49 |
| *** panda has joined #tripleo | 08:49 | |
| jistr | marios: sure thing, thanks | 08:49 |
| *** dsariel has quit IRC | 08:52 | |
| ansiwen | EmilienM: ping nova composable roles | 08:55 |
| *** saneax_AFK is now known as saneax | 08:55 | |
| jistr | marios, social: one possible complication wrt Mistral->Ansible connection is that Ansible is very filesystem focused, we won't be able to feed it URLs for playbooks/inventories etc., we'd have to generate these things to some tmp directory on the undercloud (possibly not under /tmp itself b/c security, but something similar) | 08:59 |
| *** devvesa has joined #tripleo | 09:00 | |
| *** jtomasek_ has joined #tripleo | 09:00 | |
| *** jtomasek_ has quit IRC | 09:02 | |
| *** tesseract has joined #tripleo | 09:02 | |
| *** jtomasek_ has joined #tripleo | 09:02 | |
| *** zoli|wfh is now known as zoli|mtg | 09:03 | |
| *** paramite|afk is now known as paramite | 09:03 | |
| jistr | marios, social: as a next step i'll start looking into Mistral a bit to see how we could generate these things, and it will give me a chance to think about "could we use *directly* Mistral without Ansible for upgrades". On high level Ansible seems better suited for system management operations, but if the Mistral->Ansible connection turns out to be too costly, we might investigate Mistral-only approach too. | 09:04 |
| marios | jistr: i like the first pass... you *have* decomposed into per service-component upgrade definitions, but can orchestrate with a higher template | 09:04 |
| jistr | marios: yea we'll have to approach the component's one way or another | 09:05 |
| jistr | *components | 09:05 |
| jistr | lemme fpaste sth | 09:05 |
| *** nijaba has quit IRC | 09:06 | |
| marios | jistr: did you get a chance to see https://review.openstack.org/#/c/321027/ - going to go work on that some more in a bit and tidy up | 09:06 |
| jistr | marios: ah not yet, will look at that today | 09:06 |
| *** dsariel has joined #tripleo | 09:07 | |
| marios | jistr: k thanks (just reminder ;) ) | 09:07 |
| *** nijaba has joined #tripleo | 09:07 | |
| *** jtomasek_ has quit IRC | 09:08 | |
| social | jistr: here, sorry for being late | 09:10 |
| jistr | marios: if we have a bit of per-service info and a list of upgrade phases, we could autogenerate what a component means essentially: https://paste.fedoraproject.org/371002/64253768/ | 09:10 |
| jistr | ^^ this alone can be used to autogenerate the glance.yaml and keystone.yaml ansible playbooks | 09:10 |
| social | jistr: definetly not /tmp but /var/tmp ,) | 09:10 |
| jistr | so e.g. if there's a vendor plugin that needs to update at the same time as neutron | 09:11 |
| jistr | it just says `upgrade_phase: neutron` | 09:11 |
| jistr | and its upgrade runs together with neutron | 09:11 |
| marios | jistr: phase for grouping | 09:11 |
| jistr | yea | 09:11 |
| *** paramite is now known as paramite|afk | 09:12 | |
| jistr | the list of phases is still static in what i pasted above | 09:12 |
| jistr | but it might be a good start | 09:12 |
| marios | jistr: service manifest it is :) | 09:12 |
| marios | jistr: right? I mean to say, the 'upgrade_phase' could be added/carried by the service into the service manifest | 09:13 |
| ansiwen | shardy somewhere explained the difference of composable services vs. composable role. somebody remembers where that was? | 09:13 |
| jistr | marios: well... in theory yea but i'm not sure if we should put this info into the service's Heat templates | 09:13 |
| jistr | marios: the main issue is that to get any sort of sane info from the files we have in t-h-t, they have to be crunched through Heat, which is bleargh | 09:14 |
| *** anshul has joined #tripleo | 09:14 | |
| *** paramite|afk is now known as paramite | 09:14 | |
| *** anshul is now known as Guest45481 | 09:14 | |
| jistr | marios: i.e. if we put service manifests where i put them until now, to read that manifest after updating something, you have to do a stack-update... yuck | 09:14 |
| marios | jistr: but the service manifest would be generated by the deployment itself | 09:15 |
| marios | jistr: ok so i guess you mean, at time of upgrade, we'd need a 'fresh' service manfiest or sthing | 09:15 |
| marios | jistr: and to get that we'd have to 'redeploy' (stack update) | 09:15 |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui: Initial zaqar websockets https://review.openstack.org/319070 | 09:16 |
| jistr | marios: yea an ability to have fresh info for what was in the fpaste would be better i think. We still have to read something from the Heat stack (at least the list of roles+services deployed, and a list of machines and their roles) but the smaller the interface, the better the chance you don't need to stack-update to get the info you need. | 09:17 |
| jistr | marios: yes exactly as you said, "redeploy" | 09:17 |
| *** devvesa has quit IRC | 09:17 | |
| *** devvesa has joined #tripleo | 09:18 | |
| *** sambetts|afk is now known as sambetts | 09:18 | |
| openstackgerrit | Gabriele Cerami proposed openstack/tripleo-quickstart: Add options to download and handle overcloud images https://review.openstack.org/321057 | 09:19 |
| *** larstobi has quit IRC | 09:19 | |
| *** larstobi has joined #tripleo | 09:24 | |
| bandini | jaosorior_lunch: so with these steps liberty install actually worked https://paste.fedoraproject.org/371010/46425476/ | 09:26 |
| matbu | hi, is someone know if there is some prebuilt OC images somewhere ? | 09:26 |
| panda | matbu: http://artifacts.ci.centos.org/artifacts/rdo/images/mitaka/delorean/stable/overcloud-full.tar | 09:27 |
| matbu | panda: cool thx | 09:28 |
| *** dtantsur|bbl is now known as dtantsur | 09:28 | |
| social | jistr: what about related packages for example you usually have some deps, those will be updated when? | 09:29 |
| *** sshnaidm has joined #tripleo | 09:30 | |
| jistr | social: whenever something pulls them in i'm afraid. The basic guideline is "whenever you start a per-service upgrade, don't restart any unupgraded services" as upgraded deps can break them on restart. I mentioned it in the spec too. Unfortunately i don't think there's a way around this with "traditional packaging". | 09:31 |
| jistr | this would be a non-issue after we switch to containers | 09:31 |
| social | jistr: and that's the issue depending on config you have soft deps in openstack packages | 09:32 |
| jistr | oof soft deps | 09:32 |
| social | jistr: this is solved in puppet by installing if some class is set up :/ | 09:32 |
| social | yet the service might not start if we don't update that | 09:32 |
| social | even db-sync failed on neutron if you didn't have fwaas | 09:33 |
| jistr | we can always list more packages to update, provided that we *know* what those are | 09:34 |
| jistr | social: thanks for bringing this up | 09:34 |
| social | jistr: which will mean we'll reproduce the logic from puppet ;.; | 09:34 |
| *** jaosorior_lunch is now known as jaosorior | 09:35 | |
| jaosorior | bandini funky... | 09:35 |
| jaosorior | shardy, derekh are you around? | 09:35 |
| social | jistr: i would still go with your approach for now but we should have this possibility in mind | 09:35 |
| jistr | heh i guess. "a list of packages" is not a huge amount of logic though | 09:36 |
| jistr | social: you mean upgrading via Puppet? | 09:36 |
| social | jistr: no, even though I'm for using puppet if we are using it to deploy and just overriding package from installed to latest, but just stick with ansible and if issue with soft deps shows up add it to ansible | 09:37 |
| *** devvesa has quit IRC | 09:38 | |
| derekh | jaosorior: whats up? | 09:39 |
| jaosorior | derekh: Hey dude, we've been seeing failures in the stable/liberty branch | 09:40 |
| jaosorior | derekh: https://bugs.launchpad.net/tripleo/+bug/1585937 | 09:40 |
| openstack | Launchpad bug 1585937 in tripleo "liberty ci is broken (undercloud install fails)" [Undecided,New] | 09:40 |
| jistr | social: well just overriding from installed to latest won't cut it, there's also making sure that things are only running at the same time if they actually can talk the same AMQP to each other, and they don't run when DB sync is being performed underneath, and the PM requirement of doing a per-service (per-component) upgrade. Puppet won't do much good there, unless we do some serious hacks. | 09:40 |
| jaosorior | and what's strange enough is taht bandini actually was able to deploy liberty with pretty similar steps as we do in the CI | 09:40 |
| jaosorior | any idea what's up? O_o | 09:40 |
| jaosorior | derekh: This is what bandini did https://paste.fedoraproject.org/371010/46425476/ | 09:41 |
| derekh | jaosorior: looking | 09:41 |
| jistr | social: though to be fair, per-service upgrade is a concern regardless of the approach we pick. I'm still kinda untrusting towards OpenStack itself that it will flawlessly run with some components upgraded and some not. | 09:41 |
| bandini | jaosorior, derekh: looking at the ci, I see I did not do "tripleo.sh --delorean-setup", not sure how relevant that is | 09:43 |
| derekh | bandini: which log file are you looking at? | 09:44 |
| bandini | jaosorior: I see though that my undercloud has instack-undercloud-2.2.8-0.20160523200751.121f79f.el7.centos.noarch as a verision. how do I check if I really did get a liberty undercloud? | 09:45 |
| bandini | derekh: http://logs.openstack.org/60/321060/1/check-tripleo/gate-tripleo-ci-f22-ha/2be66f9/console.html | 09:45 |
| bandini | derekh: logs from the following review https://review.openstack.org/#/c/321060 | 09:45 |
| bandini | jaosorior: these are the repos I end up with https://paste.fedoraproject.org/371016/42562281/ . Seems sane for liberty I guess | 09:50 |
| *** devvesa has joined #tripleo | 09:54 | |
| derekh | bandini: jaosorior I suspect maybe the squid has cached an old repository, going to restart it now | 09:54 |
| jaosorior | alrighty | 09:56 |
| derekh | bandini: jaosorior we used to have a rule in the squid server to prevent repomd.xml from being cached in yum repositories but its gone, os-apply-config probably stomped over it | 09:56 |
| derekh | jaosorior: bandini putting it back now | 09:56 |
| bandini | derekh: ack, let me know when you want me to recheck a liberty job of mine | 09:56 |
| bandini | thanks! | 09:56 |
| derekh | bandini: go for it, let me know if it helps | 09:58 |
| bandini | derekh: ack, thanks | 09:58 |
| derekh | bandini: restarted it again, noticed another config difference, your just should be ok unless the timing was unlucky | 10:02 |
| derekh | jaosorior: bandini I've updated the os-apply-config template on the squid server so that a restart shouldn't cause this to happen again | 10:03 |
| derekh | hopefuly it solves the problem | 10:03 |
| *** mikelk has joined #tripleo | 10:04 | |
| bandini | ack | 10:07 |
| *** rhallisey has joined #tripleo | 10:16 | |
| hoobaman | guys? | 10:17 |
| hoobaman | can i combine external and floating into one subnet/vlan? | 10:17 |
| hoobaman | kilo | 10:17 |
| hoobaman | the manual states | 10:18 |
| hoobaman | Network 6 - External, Floating IP (mapped after Overcloud creation) | 10:18 |
| social | marios: about dbsync triggered from puppet, you'd have to notify it and if we update the package only thing that can notify it is config change | 10:18 |
| hoobaman | however i only see a working possibility when i am using 2 vlans/subnets? | 10:18 |
| hoobaman | the manual staes 6 vlans are sufficiant, however how do i see , i need 7 | 10:22 |
| jaosorior | gfidente | 10:25 |
| jaosorior | Hey dude | 10:25 |
| jaosorior | there seems to be some issue now that we added the MySQLInternal | 10:25 |
| jaosorior | upgrades is failing since it complaigns about a hiera value | 10:25 |
| jaosorior | and the value it complaigns about it this one | 10:25 |
| jaosorior | mysql_virtual_ip: [fd00:fd00:fd00:2000::11] | 10:25 |
| hoobaman | i just dont understand how i can have external/floatin ip on the same vlan (virtual vlan tagged interface) | 10:25 |
| *** jprovazn has quit IRC | 10:37 | |
| bandini | derekh: same error http://logs.openstack.org/32/319432/1/check-tripleo/gate-tripleo-ci-f22-ha/600f583/logs/undercloud/var/log/undercloud_install.txt.gz | 10:37 |
| bandini | jaosorior: ^ | 10:37 |
| sshnaidm | does anybody know why we have two delorean repos in CI - delroean-current.repo with special packages and delorean.repo with different hashes? | 10:40 |
| *** zoli|mtg is now known as zoli|wfh | 10:47 | |
| *** rhallisey has quit IRC | 10:49 | |
| derekh | sshnaidm: the one with special packages is because we use current master packages for all of the tripleo packages | 10:50 |
| derekh | sshnaidm: the other one is the non tripleo packages | 10:50 |
| ccamacho | sshnaidm, Might be http://tripleo.org/environments/environments.html delorean.repo from current-tripleo and delorean-current.repo from RDO (for core openstack packages)? | 10:50 |
| derekh | sshnaidm: for these we have a pinned repository that is updated periodically if the periodic job passes with them | 10:50 |
| derekh | bandini: ok, looking | 10:51 |
| sshnaidm | thanks, now it's clear.. | 10:53 |
| *** yolanda has quit IRC | 10:57 | |
| *** jprovazn has joined #tripleo | 10:57 | |
| openstackgerrit | Marios Andreou proposed openstack/puppet-tripleo: Adds the puppet-tripleo manifests for manila https://review.openstack.org/313527 | 10:58 |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates: Composable manila tripleo-heat-templates side https://review.openstack.org/315658 | 10:58 |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates: Enable Manila integration https://review.openstack.org/188137 | 10:58 |
| jaosorior | gfidente: so the issue is that most of the usages of that IP are with the uri map, which puts IPv6 in brackets. While only one value (mysql_virtual_ip) takes the bracketless | 11:00 |
| hewbrocca | this stuff makes me insane | 11:01 |
| jaosorior | hewbrocca: ?? | 11:01 |
| hewbrocca | the fact that we have AFAICT 17 different ways to write an IPv6 address, and they're all incompatible | 11:02 |
| jaosorior | ah that | 11:02 |
| jaosorior | yep, sucks | 11:02 |
| hewbrocca | could we not have picked *one way* for OpenStack and stuck with it? | 11:02 |
| jaosorior | well, nah | 11:03 |
| jaosorior | that would be too easy :P | 11:03 |
| jaosorior | it's openstack, remember | 11:03 |
| *** yolanda has joined #tripleo | 11:04 | |
| *** pkovar has quit IRC | 11:07 | |
| *** devvesa has quit IRC | 11:08 | |
| *** links has quit IRC | 11:08 | |
| hoobaman | what ports should be open on the pubapi/external side? | 11:10 |
| hoobaman | only keystone or more? | 11:10 |
| jaosorior | hoobaman: well, depends if you expect users to be able to use nova and such from the outside network or if you expect them to only use horizon | 11:12 |
| jaosorior | it also depends if you're using the default SSL settings, because then the ports change | 11:13 |
| derekh | bandini: jaosorior I *think* the problem is that openstack-ceilometer-common is being installed from the delorean-deps repository http://buildlogs.centos.org/centos/7/cloud/x86_64/openstack-liberty/ | 11:13 |
| derekh | version 5.0.3-1 | 11:13 |
| derekh | the undercloud install tries to install | 11:13 |
| derekh | openstack-ceilometer-alarm and is trying to pull it from the delorean-liberty repository | 11:13 |
| derekh | http://trunk.rdoproject.org/centos7-liberty/current/ | 11:13 |
| derekh | the version in there is less then what is in delorean-deps 5.0.3-0.20160506162756 | 11:13 |
| derekh | jaosorior: bandini ^ | 11:13 |
| *** stendulker_ has joined #tripleo | 11:13 | |
| derekh | I didn't know the delorean deps repo contained openstack packages | 11:14 |
| jaosorior | neither did I | 11:14 |
| bandini | surprising yes, I thought it was for deps only | 11:15 |
| *** stendulker has quit IRC | 11:17 | |
| *** ccamacho is now known as ccamacho|lunch | 11:18 | |
| *** stendulker_ has quit IRC | 11:21 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Pass MysqlVirtualIP via EndpointMap https://review.openstack.org/305751 | 11:23 |
| *** qasims has joined #tripleo | 11:25 | |
| *** wendar has quit IRC | 11:31 | |
| *** wendar has joined #tripleo | 11:31 | |
| *** rhallisey has joined #tripleo | 11:32 | |
| *** julim has joined #tripleo | 11:32 | |
| bandini | derekh: is it an rdo issue in your view? | 11:32 |
| *** remix_tj has joined #tripleo | 11:33 | |
| *** rbrady has joined #tripleo | 11:34 | |
| derekh | bandini: possibly, although I can't reproduce it outside of our CI | 11:35 |
| remix_tj | Hello, i'm installing an overcloud composed by 3 controller nodes. Do i need to specify something additional to --control-scale 3 | 11:35 |
| remix_tj | ? | 11:36 |
| remix_tj | i've succesfully deployed with only one controller, but with 3 fails due to mysql errors (tables already existing, columns already existing and similar) | 11:36 |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Nothing to see here https://review.openstack.org/321113 | 11:37 |
| remix_tj | i'm using latest rdo-release (not delorean stable) | 11:38 |
| bandini | remix_tj: likely you need to add -e /usr/share/openstack-tripleo-heat-templates/environments/puppet-pacemaker.yaml to your deploy command | 11:38 |
| bandini | that will give you an ha control plane on the three controllers | 11:38 |
| derekh | bandini: jaosorior ok, found out exactly whats going on now | 11:42 |
| jaosorior | ??? | 11:42 |
| jaosorior | O_O\ | 11:42 |
| derekh | when we build the undercloud image we pre install a load of packages on it, here http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/toci_instack.sh#n222 | 11:42 |
| derekh | see the package list in that line | 11:42 |
| derekh | the key here is that one of the packages that is getting installed is yum-plugin-priorities | 11:43 |
| mandre | hello, can I get a couple of cores to reviews patches in https://review.openstack.org/#/q/project:openstack/tripleo-validations so that we can initiate work on tripleo-validations? | 11:43 |
| derekh | so the priorties we have set on the repositories don't have any effect during this package install | 11:43 |
| *** dsariel has quit IRC | 11:44 | |
| derekh | and openstack-ceilometer-common ends up coming from the delorean-deps repo | 11:44 |
| derekh | becausde its release is greater | 11:44 |
| *** pkovar has joined #tripleo | 11:44 | |
| derekh | then after this any package installs try to respect the priorities set on the db (as now yum-plugin-priorities is installed) | 11:45 |
| dkelson | I'm trying to do some customization of my overcloud controller using NodeExtraConfigPost with puppet, but when I run "openstack service list" (after setting the appropriate env variables using hieradata for the password, and URL) I get "ERROR: openstack The service catalog is empty.". However logging into the controller, setting the same env variables and running the same command results in the expected output. | 11:46 |
| derekh | so while the undercloud is being installed it trys to install openstack-ceilometer-alarm from the delorean/liberty repo as it has a higher priority | 11:46 |
| derekh | and fails becasue the release is lower | 11:46 |
| derekh | bandini: jaosorior ^ | 11:46 |
| remix_tj | bandini: ok. This may solve the problems i'm experiencing with the deploy? http://pastebin.com/tibmZv60 (these are the errors that has been found on a failing controller node) | 11:46 |
| jaosorior | uhm... | 11:47 |
| jaosorior | alright | 11:47 |
| jaosorior | derekh: So, can we pre-install packages in two steps there? | 11:47 |
| derekh | so I suggest you talk to rdo to see if they expect that the delorean/liberty repo should always have a NVR hight then whats in the delorean-deps repo and if so they might need to fix somthing | 11:48 |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates: Initial rework of pacemaker_common_functions for M..N upgrades https://review.openstack.org/321027 | 11:48 |
| hewbrocca | derekh: that'đ entirely excellent | 11:48 |
| derekh | if not we can change CI to may not preinstall packages in stable jobs, or see if we can install pluging-priorities before the other packages as jaosorior suggests | 11:49 |
| hewbrocca | sigh | 11:49 |
| jaosorior | NVR hight? | 11:49 |
| hewbrocca | or we could get real package repositories | 11:49 |
| bandini | remix_tj: it's likely yes | 11:50 |
| derekh | jaosorior: NVR - name/version/release , and hight should have been higher | 11:50 |
| remix_tj | grazie bandini | 11:52 |
| *** Goneri has quit IRC | 11:53 | |
| bandini | ;) | 11:53 |
| derekh | hewbrocca: real package repositories? | 11:54 |
| hewbrocca | where we don't have to use hacks like yum-plugin-priorities | 11:55 |
| hewbrocca | I realize that is entirely impossible | 11:55 |
| *** devvesa has joined #tripleo | 11:55 | |
| hewbrocca | but I don't have to like it... | 11:55 |
| *** thrash|g0ne is now known as thrash | 11:56 | |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-quickstart: Add master configuration https://review.openstack.org/321494 | 11:56 |
| derekh | hewbrocca: ahh, crazy talk ;-), actually I think this was an aim for RDO, its why we put the timestamps into the release numbers in the delorean packages so the the NVR would always be higher but that seems to have gone wrong here | 11:57 |
| derekh | trown|outtypewww: isn't that correct? ^ | 11:57 |
| openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates: Fix ceph keyring setting for gnocchi https://review.openstack.org/321133 | 11:59 |
| gfidente | anyone wants to merge https://review.openstack.org/#/c/318246/ ? | 12:00 |
| *** jaosorior has quit IRC | 12:00 | |
| *** jaosorior has joined #tripleo | 12:00 | |
| derekh | jaosorior: bandini to reproduce(if you want), http://paste.openstack.org/show/505625/ | 12:01 |
| slagle | lucasagomes: dtantsur : hey, wondering if i can get some help on https://bugs.launchpad.net/tripleo/+bug/1585641 | 12:03 |
| openstack | Launchpad bug 1585641 in tripleo "CI: jobs failing with Message: No valid host was found. There are not enough hosts available., Code: 500"" [High,Triaged] - Assigned to James Slagle (james-slagle) | 12:03 |
| *** julim has quit IRC | 12:03 | |
| slagle | lucasagomes: dtantsur : a fair amount of tripleo-ci jobs have been failing with the the dreaded "No Valid Host" error, and it appears to be some inconsistency between the nova scheduler and ironic | 12:03 |
| bandini | derekh: oh I see | 12:04 |
| slagle | lucasagomes: dtantsur : from what i can tell, i think it's related to this nova commit: https://review.openstack.org/#/c/306670/, b/c i'm seeing that some nodes are suddenly reporting memory_mb=0 in the compute resource tracker, even though they aren't in use | 12:04 |
| lucasagomes | slagle, hi there. Lemme take a look | 12:05 |
| *** pradk has joined #tripleo | 12:05 | |
| dtantsur | I remember someone complaining about that.. | 12:05 |
| lucasagomes | slagle, there was a bug recently at how the resource manager does compute the available resources | 12:05 |
| slagle | i think there is something racey going on, b/c it doesnt happen all the time in our CI | 12:06 |
| bandini | derekh: so who do you think is best suited of the rdo overlords to help out here? | 12:06 |
| lucasagomes | slagle, https://bugs.launchpad.net/nova/+bug/1572555 | 12:06 |
| openstack | Launchpad bug 1572555 in OpenStack Compute (nova) "Nova reports memory_mb=0 for available Ironic node" [High,In progress] - Assigned to Anton Arefiev (aarefiev) | 12:06 |
| lucasagomes | slagle, the bug was introduced by https://review.openstack.org/#/c/279313/ | 12:06 |
| * bandini not all too familiar with how/who does what in packaging rdo | 12:06 | |
| derekh | bandini: probably worth asking number80 on #rdo | 12:06 |
| lucasagomes | slagle, here's the fix for nova, it's not merged yet tho: https://review.openstack.org/#/c/316031/ | 12:07 |
| bandini | number80: since you're here too ;) we have the following issue in liberty ci https://bugs.launchpad.net/tripleo/+bug/1585937 | 12:07 |
| openstack | Launchpad bug 1585937 in tripleo "liberty ci is broken (undercloud install fails)" [Undecided,New] | 12:07 |
| derekh | bandini: if there isn't a solution in #rdo we can just remove that list of pre installed packages for stable jobs | 12:07 |
| * number80 looking | 12:07 | |
| *** zoli|wfh is now known as zoli|lunch | 12:08 | |
| slagle | lucasagomes: hmm, yea, that might be the issue | 12:08 |
| number80 | bandini: there's maybe a problem w/ the delorean build of ceilo | 12:09 |
| bandini | derekh: so basically in our jobs we always assume that "delorean/liberty packages have later NVR than the ones in delorean-deps", correct? or the other way around? | 12:09 |
| derekh | number80: essentially the problem is that the NVR of openstack-ceilometer in http://trunk.rdoproject.org/centos7-liberty/current/ is lower then the package in http://buildlogs.centos.org/centos/7/cloud/x86_64/openstack-liberty/ | 12:09 |
| bandini | righto | 12:09 |
| * derekh goes for lunch | 12:10 | |
| slagle | lucasagomes: i think we see the issue after the first nodes get successfully scheduled to, so this sounds like it might be the same problem | 12:10 |
| jaosorior | went for coffee | 12:10 |
| *** trown|outtypewww is now known as trown | 12:10 | |
| trown | derekh: bandini, unfortunately we can not guarantee NVR in deps is lower than DLRN, hence the need for yum plugin priorities | 12:11 |
| number80 | bandini: ok, it tries to install openstack-ceilometer-alarm from delorean but the one in rdo-liberty-testing is newer | 12:12 |
| number80 | but what I don't get is why puppet doesn't take the openstack-ceilometer-alarm from rdo-liberty-testing? | 12:12 |
| *** yamahata has quit IRC | 12:12 | |
| *** jayg|g0n3 is now known as jayg | 12:12 | |
| trown | the telemetry project in particular has had other issues with release tagging that have caused this before | 12:13 |
| bandini | I think derekh mentioned that it depends on the when yum-plugin-priorities is instaled | 12:13 |
| trown | the solution is just to always install yum-plugin-priorities in its own transaction before anything else | 12:13 |
| jaosorior | number80, bandini: So the thing is that we do pre-installment of packages for the images here http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/toci_instack.sh#n222 | 12:14 |
| bandini | derekh: can we do what trown ^ suggests for the stable jobs at least? | 12:14 |
| *** fultonj has joined #tripleo | 12:14 | |
| trown | bandini: right if you install ypp in the same transaction as a bunch of other packages, it wont have affect | 12:14 |
| trown | for that transaction | 12:14 |
| number80 | bandini: yes, but then it should take all subpackages from the same repo | 12:14 |
| number80 | not mixing them | 12:14 |
| trown | hmm effect rather (I think) | 12:14 |
| trown | number80: well, not if later you update, since priorities is then in effect | 12:15 |
| bandini | number80: yes I think the reason is the line that jaosorior mentioned in the link above, by preinstalling some stuff before we effectively take stuff from two separate repos | 12:15 |
| lucasagomes | slagle, yes exactly. Once nova subtract the resources when a node was pick, it them subtract for other nodes too | 12:15 |
| lucasagomes | slagle, what I observed was, now memory_mb, local_gb and vpcus lives in the inventory database table in nova | 12:16 |
| openstackgerrit | Merged openstack/tripleo-quickstart: Add master configuration https://review.openstack.org/321494 | 12:17 |
| lucasagomes | slagle, and if you have one or more nodes with the exactly same amount of a given resource they will have the same id in that inventory table | 12:18 |
| lucasagomes | slagle, so once nova sets it to 0 because it's taken, it mistakenly update other nodes | 12:18 |
| *** dprince has joined #tripleo | 12:19 | |
| jtomasek | rbrady: do you agree that the proposed solution is a good one? | 12:19 |
| trown | bandini: derekh, I am 100% sure the only solution to that issue is installing yum plugin priorities before anything else | 12:21 |
| rbrady | jtomasek: what proposed solution? | 12:22 |
| trown | since RDO could hack something for ceilometer, but that just kicks the can down the road to the next time a project does something wonky with a release | 12:22 |
| jtomasek | rbrady: to make the changes so the instackenv.json can match the ironic nodes structure | 12:23 |
| trown | note that is how the RDO image is made: https://github.com/redhat-openstack/ansible-role-tripleo-image-build/blob/master/templates/repo-centos7-delorean.sh.j2#L10 | 12:23 |
| rbrady | jtomasek: I think it would be good to investigate to see if anything else depends on that file first. | 12:24 |
| EmilienM | slagle: about release -- how to we handle release notes usually? | 12:24 |
| slagle | EmilienM: we haven't | 12:26 |
| openstackgerrit | Merged openstack/tripleo-heat-templates: composable heat services https://review.openstack.org/313577 | 12:26 |
| slagle | EmilienM: we don't track anything, so it would be a manual process of scouring the changelog to see what has changed | 12:27 |
| jistr | marios: reviewed the upgrade script change, just found a few issues, but the overal approach is very nice | 12:28 |
| EmilienM | slagle: ok -- in the future we might want to use reno | 12:28 |
| *** MaxPC has joined #tripleo | 12:29 | |
| slagle | EmilienM: yea, agreed we probably need to | 12:30 |
| jtomasek | rbrady: problem I am facing now is how/where do I get the information about what drivers we support and what are the properties I need to set for each of the driver? If it was using Ironic, that would be clear, but for some reason instackenv.json nodes format is different | 12:30 |
| marios | jistr: thx for looking - still early days but i am trying to make it work with 'what we know'. for example once we know exactly which services _are_ pcmk we could be more explicit/exact | 12:31 |
| marios | jistr: thanks for reminding me about the ${@[*]} wanted to sanity check it | 12:31 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Pass MysqlVirtualIP via EndpointMap https://review.openstack.org/305751 | 12:34 |
| *** rodrigods has quit IRC | 12:34 | |
| *** rodrigods has joined #tripleo | 12:34 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Pass MysqlVirtualIP via EndpointMap https://review.openstack.org/305751 | 12:38 |
| *** myoung has joined #tripleo | 12:46 | |
| *** olap has quit IRC | 12:47 | |
| panda | derekh: what packages are taken from master to create triplo-ci images, even when testing stable branches ? | 12:47 |
| *** rlandy has joined #tripleo | 12:48 | |
| *** julim has joined #tripleo | 12:49 | |
| openstackgerrit | Emilien Macchi proposed openstack/puppet-tripleo: loadbalancer: make sure controller_* are array-typed https://review.openstack.org/321206 | 12:49 |
| openstackgerrit | Emilien Macchi proposed openstack/puppet-tripleo: Explode loadbalancer role in 2 sub-roles https://review.openstack.org/321201 | 12:50 |
| panda | derekh: because using the images "as is" it's showin some weird behaviours. For example I'm hitting https://bugs.launchpad.net/tripleo/+bug/1572467 while testing mitaka | 12:50 |
| openstack | Launchpad bug 1572467 in tripleo "Cannot load scheduler driver from configuration" [Critical,Fix committed] - Assigned to John Trowbridge (trown) | 12:50 |
| d0ugal | jtomasek: Does the ironic format cover the virt case? | 12:51 |
| jtomasek | d0ugal: not sure I follow, virt case is some driver type? | 12:53 |
| *** myoung has quit IRC | 12:56 | |
| *** paramite has quit IRC | 12:56 | |
| *** trown is now known as trown|mtg | 12:59 | |
| derekh | panda: the cached image is for master only, you shouldn't be using it for mitaka | 13:01 |
| *** liverpooler has quit IRC | 13:01 | |
| *** lucasagomes is now known as lucas-hungry | 13:03 | |
| derekh | bandini: trown|mtg if yum-plugin-priorities can't easily be installed first we can just remove the list of packages for the stable jobs. | 13:03 |
| *** myoung has joined #tripleo | 13:04 | |
| *** pradk has quit IRC | 13:05 | |
| trown|mtg | derekh: it seems like we would just want to add `yum install -y yum-plugin-priorities` to the repo setup step in tripleo.sh | 13:05 |
| trown|mtg | it is a hard requirement for dlrn repos | 13:05 |
| derekh | trown|mtg: the problem is that its happening inside DIB here http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/toci_instack.sh#n222 , we'de have to add a yum-plugin-priorities element or something | 13:07 |
| trown|mtg | derekh: or we inject repos into the base image before running dib | 13:07 |
| trown|mtg | that is how it is done in RDO | 13:08 |
| panda | derekh: thanks, I was missing that part of information. I though that all periodic jobs were creating an image, and indeed I did not understand what image would land on the repository since periodic jobs for stable are launched too | 13:08 |
| *** rbrady_ has joined #tripleo | 13:08 | |
| *** rbrady has quit IRC | 13:08 | |
| trown|mtg | which is I guess is what an element would do too... I just am biased against adding more DIB elements | 13:08 |
| *** Goneri has joined #tripleo | 13:09 | |
| derekh | trown|mtg: the repos are availble and use when DIB is run, anyways will continue after meeting | 13:11 |
| *** tzumainn has joined #tripleo | 13:15 | |
| *** rbrady_ has quit IRC | 13:15 | |
| *** rbrady_ has joined #tripleo | 13:15 | |
| *** MaxPC has quit IRC | 13:16 | |
| *** MaxPC has joined #tripleo | 13:16 | |
| *** akshai has joined #tripleo | 13:18 | |
| *** pradk has joined #tripleo | 13:18 | |
| openstackgerrit | Carlos Camacho proposed openstack/tripleo-heat-templates: Configure ComputeServices via resource chains https://review.openstack.org/319873 | 13:21 |
| sshnaidm | myoung, hi | 13:22 |
| myoung | sshnaidm: hi :) | 13:23 |
| sshnaidm | myoung, did you try to build images on promoted undercloud.qcow2 ? like from here : http://artifacts.ci.centos.org/artifacts/rdo/images/master/delorean/stable/ | 13:24 |
| bnemec | derekh: trown|mtg: We already are installing yum-plugin-priorities in tripleo.sh. It's a gross hack that needs to go away. | 13:24 |
| bnemec | http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/scripts/tripleo.sh#n423 | 13:24 |
| sshnaidm | myoung, just running "overcloud build images --all" | 13:24 |
| bnemec | IMHO it should just be added to the image build command, but I got pushback when I proposed that. | 13:24 |
| derekh | bnemec: yes we are but the problem we're trying to solve is in the indercloud install | 13:25 |
| bnemec | derekh: We're also doing it there: http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/scripts/tripleo.sh#n250 | 13:26 |
| bnemec | That one is slightly less gross since it's actually documented that way. | 13:26 |
| derekh | bnemec: the reason its in tripleo.sh and not anywhere else is because devs and CI are the only cases we're layer repositories the we we do | 13:26 |
| bnemec | More than slightly, in fact. tripleo.sh should just reflect what's documented, so it's not gross at all. | 13:26 |
| derekh | bnemec: users who consume tripleo shoudl only be using one repository | 13:26 |
| *** masco has quit IRC | 13:27 | |
| derekh | bnemec: but having said all that I got no objection to a cleaner solution | 13:27 |
| *** absubram has quit IRC | 13:27 | |
| myoung | sshnaidm: no...not sure I have context... | 13:27 |
| *** akshai_ has joined #tripleo | 13:33 | |
| sshnaidm | I was able to build overcloud images with tripleo.sh only, including repo-setup though.. But building images on undercloud fails, it seems like a bug... | 13:33 |
| sshnaidm | trown|mtg, derekh ^^ | 13:33 |
| *** lucas-hungry is now known as lucasagomes | 13:36 | |
| *** gfidente has quit IRC | 13:36 | |
| *** akshai has quit IRC | 13:37 | |
| *** gfidente has joined #tripleo | 13:37 | |
| ramishra | hi guys, what's this error "ERROR: The Stack (overcloud-VipConfig-df2v3qligao4) could not be found." for gate-tripleo-ci-f22-nonha. I don't see any error in the heat engine log of undercloud. | 13:38 |
| openstackgerrit | Carlos Camacho proposed openstack/tripleo-heat-templates: Configure ComputeServices via resource chains https://review.openstack.org/319873 | 13:38 |
| ramishra | http://logs.openstack.org/98/286098/22/experimental-tripleo/gate-tripleo-ci-f22-nonha/7a5fa0e/console.html#_2016-05-26_13_16_16_078 | 13:39 |
| ramishra | derekh: hey any idea? | 13:39 |
| *** lblanchard has joined #tripleo | 13:41 | |
| *** zoli|lunch is now known as zoli|mtg | 13:45 | |
| *** rbrady_ has quit IRC | 14:01 | |
| *** rbrady has joined #tripleo | 14:01 | |
| derekh | ramishra: I don't see anything obvious, is the problem consistently happening on that patch? | 14:04 |
| derekh | sshnaidm: whats the error? | 14:04 |
| ramishra | derekh: yeah twice | 14:04 |
| jaosorior | EmilienM: Hey dude, in this CR https://review.openstack.org/#/c/305751/ I'm trying to pass the mysql_virtual_ip as a string, since in IPv6 we're using brackets for the addresses | 14:07 |
| derekh | ramishra: hmm, maybe its possible to get "openstack overcloud deploy" to output more debug info, but if there is I don't know it <--- anybody know ? | 14:07 |
| jaosorior | and that seems to fail down in https://github.com/openstack/puppet-openstacklib/blob/master/manifests/db/mysql.pp | 14:07 |
| jaosorior | the address ends up something like this in the hieradata that's persisted in the controller: | 14:08 |
| *** trown|mtg is now known as trown | 14:08 | |
| jaosorior | mysql_virtual_ip: '[fd00:fd00:fd00:2000::11]' | 14:08 |
| openstackgerrit | Carlos Camacho proposed openstack/tripleo-heat-templates: Composable roles within services - NTP https://review.openstack.org/310421 | 14:08 |
| jaosorior | and the error I'm seeing is: | 14:08 |
| jaosorior | Error: Puppet::Parser::AST::Resource failed with error ArgumentError: Invalid resource type openstacklib::db::mysql::host_access at /etc/puppet/modules/openstacklib/manifests/db/mysql.pp:68 on node overcloud-controller-0.localdomain | 14:08 |
| jaosorior | any idea what could be causing that? | 14:08 |
| EmilienM | jaosorior: it only happens during upgrades? | 14:09 |
| jaosorior | EmilienM: Yes, because we only use IPv6 in the upgrades jobn | 14:09 |
| openstackgerrit | Carlos Camacho proposed openstack/tripleo-heat-templates: Composable roles within services - MongoDB https://review.openstack.org/315051 | 14:09 |
| openstackgerrit | Merged openstack/puppet-tripleo: Add lookup_hiera_hash function https://review.openstack.org/320112 | 14:11 |
| d0ugal | jtomasek: hmm, I think I am confusing myself. | 14:11 |
| d0ugal | jtomasek: So nevermind, I should look at how Ironic does it. Do you have a link handy? | 14:11 |
| openstackgerrit | Carlos Camacho proposed openstack/puppet-tripleo: Composable roles within services - NTP https://review.openstack.org/310725 | 14:11 |
| EmilienM | jaosorior: I don't have any clue after a quick look, I'll have to investigate a bit further, let me finish my current task first | 14:12 |
| jaosorior | EmilienM: No problem, just wanted to know if you had an idea or had seen something like that before | 14:14 |
| jtomasek | d0ugal: yes, one sec | 14:15 |
| EmilienM | jaosorior: I'll first look why it worked before | 14:15 |
| *** egafford has joined #tripleo | 14:16 | |
| *** MaxPC has quit IRC | 14:17 | |
| jtomasek | d0ugal: this is kind of helpful http://docs.openstack.org/developer/ironic/webapi/v1.html#Node | 14:17 |
| jtomasek | d0ugal: but doing ironic node-show is better imo | 14:18 |
| d0ugal | jtomasek: oh, I see, so just the API format. | 14:18 |
| jtomasek | d0ugal: yes, important parts are driver and driver_info | 14:18 |
| jtomasek | d0ugal: because that is what changes | 14:19 |
| jtomasek | d0ugal: dependent on selected driver, the driver_info properties are changing | 14:19 |
| jtomasek | d0ugal: http://paste.openstack.org/show/505594/ | 14:19 |
| *** MaxPC has joined #tripleo | 14:20 | |
| jtomasek | d0ugal: so the instackenv.json would need to use attribute names that Ironic uses and specified driver_info json object with properties based on selected driver | 14:21 |
| *** tserong has quit IRC | 14:22 | |
| jtomasek | d0ugal: in the same manner for manual nodes registration, GUI could provide means to display list of available drivers (provided by Ironic API) and get the list of driver_info properties (again using Ironic API) so no driver related logic would be hardcoded either on the backend or frontend side | 14:23 |
| jaosorior | EmilienM: It worked before cause it didn't have the brackets | 14:23 |
| jaosorior | so it looked exactly like this | 14:23 |
| jaosorior | mysql_virtual_ip: fd00:fd00:fd00:2000::11 | 14:23 |
| jaosorior | but the change towards getting the mysql virtual IP in the endpoint map kinda makes us use them | 14:23 |
| d0ugal | jtomasek: Right, changing this makes sense, but I guess it isn't totally trivial | 14:24 |
| *** noslzzp has joined #tripleo | 14:24 | |
| jtomasek | d0ugal: I agree, probably not. I'll need to adjust GUI to use attributes defined in instackenv.json, but I am not sure what is the list of drivers and relevant driver properties supported by tripleocommon/utiles/nodes.py which I can let user choose in the GUI | 14:25 |
| *** eggmaster has joined #tripleo | 14:26 | |
| *** cwolferh has quit IRC | 14:26 | |
| *** noslzzp has quit IRC | 14:27 | |
| *** noslzzp has joined #tripleo | 14:27 | |
| openstackgerrit | Thomas Herve proposed openstack-infra/tripleo-ci: DNM: check heat translation patch https://review.openstack.org/321577 | 14:31 |
| openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates: Gnocchi composable roles https://review.openstack.org/318413 | 14:33 |
| d0ugal | jtomasek: http://docs.openstack.org/developer/tripleo-docs/environments/baremetal.html?highlight=instackenv#instackenv-json | 14:33 |
| d0ugal | jtomasek: That may help | 14:33 |
| hoobaman | why is there also a br-ex bridge being attached on every compute node? | 14:34 |
| hoobaman | i did not make any dvr configurations | 14:35 |
| jtomasek | d0ugal: thanks | 14:37 |
| *** dmacpher has joined #tripleo | 14:39 | |
| openstackgerrit | Emilien Macchi proposed openstack/puppet-tripleo: loadbalancer: make sure controller_* are array-typed https://review.openstack.org/321206 | 14:39 |
| openstackgerrit | Brad P. Crochet proposed openstack/tripleo-heat-templates: Composable Sahara services https://review.openstack.org/318840 | 14:39 |
| dtantsur | folks, we've put together a spec on ironic in overcloud, can someone please take a look: https://review.openstack.org/320995 | 14:40 |
| dtantsur | dprince, ^^^ | 14:40 |
| dtantsur | hewbrocca, ^^ | 14:40 |
| openstackgerrit | Emilien Macchi proposed openstack/puppet-tripleo: Explode loadbalancer role in 2 sub-roles https://review.openstack.org/321201 | 14:41 |
| openstackgerrit | Emilien Macchi proposed openstack/instack-undercloud: loadbalancer: explode hieradata to separate HAproxy & keepalived https://review.openstack.org/321210 | 14:41 |
| dprince | dtantsur: nice | 14:43 |
| *** myoung has quit IRC | 14:43 | |
| *** oshvartz has quit IRC | 14:45 | |
| egafford | gfidente: Saw your feedback on the Trove patches; sounds reasonable. I'm noting though that the CI jobs are consistently failing at the moment, in ways that don't seem related to the patch. Is this par for the course at the moment? | 14:48 |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates: Rework the pacemaker_common_functions for M..N upgrades https://review.openstack.org/321027 | 14:49 |
| *** dc has quit IRC | 14:53 | |
| openstackgerrit | Gabriele Cerami proposed openstack/tripleo-quickstart: Add options to download and handle overcloud images https://review.openstack.org/321057 | 14:53 |
| *** myoung has joined #tripleo | 14:55 | |
| *** myoung has quit IRC | 15:00 | |
| *** zoli|mtg is now known as zoli|gone | 15:01 | |
| *** zoli|gone is now known as zoli_gone-proxy | 15:02 | |
| *** dtrainor has quit IRC | 15:03 | |
| *** dmk0202 has quit IRC | 15:04 | |
| *** dc has joined #tripleo | 15:07 | |
| *** ebarrera has quit IRC | 15:08 | |
| *** sshnaidm has quit IRC | 15:11 | |
| *** itamarl has quit IRC | 15:15 | |
| *** rcernin has quit IRC | 15:19 | |
| *** penick has joined #tripleo | 15:22 | |
| d0ugal | mandre: I think it would be better to squash https://review.openstack.org/#/c/320984/ | 15:24 |
| d0ugal | into the parent, just to keep things green | 15:24 |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates: Rework the pacemaker_common_functions for M..N upgrades https://review.openstack.org/321027 | 15:25 |
| d0ugal | mandre: actually, it might be best to just squash all three of those, as they are all really just the initial cookiecutter | 15:29 |
| mandre | d0ugal: sure, I'll do that | 15:30 |
| *** tesseract has quit IRC | 15:34 | |
| *** trozet has quit IRC | 15:36 | |
| openstackgerrit | Martin André proposed openstack/tripleo-validations: Initial Cookiecutter Commit. https://review.openstack.org/320973 | 15:36 |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui: Refacor input error and description https://review.openstack.org/321620 | 15:37 |
| bnemec | hoobaman: The bridge may not be necessary in all configurations, but I believe it is in some and since it doesn't hurt anything to have it, that's the default configuration. | 15:37 |
| bnemec | If you're using network isolation you can certainly remove it from the compute template if you want. | 15:37 |
| *** trozet has joined #tripleo | 15:38 | |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui: Update FormErrorList component https://review.openstack.org/320498 | 15:38 |
| gfidente | dprince, cephstorage chain https://review.openstack.org/#/c/321092/ pls | 15:39 |
| hoobaman | bnemec: ok thx | 15:40 |
| *** dc has quit IRC | 15:41 | |
| *** mgarciam has joined #tripleo | 15:41 | |
| *** ccamacho|lunch is now known as ccamacho | 15:43 | |
| *** mikelk has quit IRC | 15:47 | |
| *** noslzzp has quit IRC | 15:48 | |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Configure CephStorage services via resource chains https://review.openstack.org/321092 | 15:51 |
| *** dc has joined #tripleo | 15:53 | |
| *** Guest45481 has quit IRC | 15:55 | |
| *** noslzzp has joined #tripleo | 15:55 | |
| *** dprince has quit IRC | 15:57 | |
| *** dc has quit IRC | 15:58 | |
| *** dprince has joined #tripleo | 15:58 | |
| *** dc has joined #tripleo | 15:58 | |
| *** sshnaidm has joined #tripleo | 16:03 | |
| *** yamahata has joined #tripleo | 16:03 | |
| *** pradk has quit IRC | 16:07 | |
| *** trown is now known as trown|lunch | 16:07 | |
| *** mcornea has quit IRC | 16:09 | |
| *** jaosorior has quit IRC | 16:11 | |
| *** aufi has quit IRC | 16:14 | |
| *** sambetts is now known as sambetts|afk | 16:16 | |
| *** ifarkas has quit IRC | 16:16 | |
| *** noslzzp has quit IRC | 16:18 | |
| *** numans has quit IRC | 16:28 | |
| *** dtrainor has joined #tripleo | 16:29 | |
| *** anshul has joined #tripleo | 16:31 | |
| *** anshul is now known as Guest69263 | 16:31 | |
| *** dprince has quit IRC | 16:35 | |
| openstackgerrit | Michele Baldessari proposed openstack/tripleo-heat-templates: Set rabbitmq loopback_users explicitely https://review.openstack.org/321650 | 16:35 |
| *** dprince has joined #tripleo | 16:35 | |
| *** devvesa has quit IRC | 16:35 | |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/puppet-pacemaker: WIP: integrate PCS provider in the merge. https://review.openstack.org/310713 | 16:40 |
| *** lucasagomes is now known as lucas-afk | 16:41 | |
| *** pradk has joined #tripleo | 16:43 | |
| *** panda has quit IRC | 16:48 | |
| *** panda has joined #tripleo | 16:48 | |
| greghaynes | ianw: re: https://review.openstack.org/#/c/303227/2 - Something that would be neat to protect us even further is we could run all of our 'outside the chroot' phases inside the (or a differente) chroot but bind mount the rootfs in to it. That would require all operations to have the target_dir prefix or risk nuking the chrooot rather than the build host | 16:52 |
| *** absubram has joined #tripleo | 16:55 | |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/puppet-tripleo: WIP: integration of the new puppet pacemaker. https://review.openstack.org/309069 | 16:55 |
| *** derekh has quit IRC | 16:56 | |
| *** absubram has quit IRC | 16:56 | |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-heat-templates: WIP: integration of the new puppet pacemaker. https://review.openstack.org/302409 | 16:56 |
| *** florianf has quit IRC | 16:57 | |
| *** qasims has quit IRC | 16:58 | |
| *** absubram has joined #tripleo | 16:59 | |
| *** dtrainor has quit IRC | 17:01 | |
| *** diazjf has joined #tripleo | 17:01 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/os-apply-config: Updated from global requirements https://review.openstack.org/321698 | 17:03 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/os-cloud-config: Updated from global requirements https://review.openstack.org/313748 | 17:03 |
| pkovar | does one still need to enable epel when setting up environment? http://tripleo.org/environments/environments.html#preparing-the-virtual-environment-automated | 17:03 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/os-collect-config: Updated from global requirements https://review.openstack.org/317184 | 17:03 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/os-refresh-config: Updated from global requirements https://review.openstack.org/303137 | 17:03 |
| *** julim_ has joined #tripleo | 17:03 | |
| *** cwolferh has joined #tripleo | 17:04 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/python-tripleoclient: Updated from global requirements https://review.openstack.org/321735 | 17:05 |
| *** julim has quit IRC | 17:07 | |
| *** numans has joined #tripleo | 17:09 | |
| *** Guest69263 has quit IRC | 17:14 | |
| *** fragatina has joined #tripleo | 17:15 | |
| *** cwolferh has quit IRC | 17:19 | |
| *** fragatina has quit IRC | 17:20 | |
| *** cwolferh has joined #tripleo | 17:20 | |
| remix_tj | a very simple question: does pacemaker need to be already installed on the image overcloud-full? because i'm experiencing issue on deploying the cluster and i see that no packages are installed | 17:20 |
| *** trown|lunch is now known as trown | 17:20 | |
| *** dtrainor has joined #tripleo | 17:21 | |
| trown | panda: wrt https://review.openstack.org/#/c/321057 do you plan to submit a patch to tripleo-ci to create md5's on the mirror server? | 17:23 |
| dtrainor | should I expect to be able to use heat templates from Kilo in Liberty? how about introducing just what i changed from kilo in to liberty instead of *all* the templates? How does one approach this? | 17:24 |
| trown | I can do it if not, I can | 17:24 |
| trown | whoa | 17:24 |
| * trown kicks brain | 17:24 | |
| dtrainor | haha | 17:24 |
| *** isq has quit IRC | 17:25 | |
| *** isq has joined #tripleo | 17:25 | |
| trown | dtrainor: the heat templates are very coupled to the puppet modules, I doubt that Kilo templates could deploy Liberty | 17:25 |
| panda | trown: already did | 17:27 |
| *** dprince has quit IRC | 17:28 | |
| trown | dtrainor: how much did you change? Ideally, changes are included via extra environment files rather than direct modification of the templates, and those are supposed to work from release to release | 17:28 |
| dtrainor | yay i got more info about what i'm asking. we're still using extra environment config files which is great. | 17:29 |
| dtrainor | very cool, thanks | 17:29 |
| *** mgould is now known as mgould|afk | 17:29 | |
| trown | that said, I know the interface in Kilo days was less than stable, so I would not be surprised if some environments for Kilo have non existent variables in later versions | 17:29 |
| dtrainor | just trying to get the temperature on such a subject | 17:29 |
| dtrainor | thank you | 17:30 |
| trown | ya I am pretty sure extra environment files from n-1 is *supposed* to work on n | 17:30 |
| openstackgerrit | Merged openstack/diskimage-builder: Add a best-effort sudo safety check https://review.openstack.org/303227 | 17:31 |
| dtrainor | i'll start crossing fingers | 17:31 |
| dtrainor | :) | 17:31 |
| *** diazjf1 has joined #tripleo | 17:32 | |
| *** ebalduf has joined #tripleo | 17:33 | |
| trown | easy tripleo-ci review: https://review.openstack.org/#/c/320906 | 17:33 |
| *** diazjf has quit IRC | 17:33 | |
| *** noslzzp has joined #tripleo | 17:33 | |
| chem`` | hi, I've got this error Could not find data item mysql_clustercheck_password in any Hiera data file ... I cannot find where this is supposed to be set, could someone point me in the right direction ? | 17:35 |
| *** noslzzp has quit IRC | 17:36 | |
| *** electrofelix has quit IRC | 17:41 | |
| *** david-lyle has quit IRC | 17:47 | |
| *** david-lyle has joined #tripleo | 17:47 | |
| *** pkovar has quit IRC | 17:50 | |
| *** athomas has quit IRC | 17:53 | |
| *** lblanchard has quit IRC | 18:05 | |
| *** lblanchard has joined #tripleo | 18:06 | |
| *** mgarciam has quit IRC | 18:07 | |
| openstackgerrit | Honza Pokorny proposed openstack/tripleo-common: Add node assignment workflow https://review.openstack.org/320459 | 18:09 |
| dkelson | chem``, look in /etc/puppet/hieradata/ either on your undercloud or overcloud box | 18:10 |
| EmilienM | slagle, trown: do we have a logstash query for the no valid host found issue? | 18:16 |
| EmilienM | in elasticrecheck thing | 18:16 |
| EmilienM | I'm working on it | 18:17 |
| bnemec | EmilienM: http://logstash.openstack.org/#/dashboard/file/logstash.json?query=build_name:%20*tripleo-ci*%20AND%20build_status:%20FAILURE%20AND%20message:%20\%22Message:%20No%20valid%20host%20was%20found.%20There%20are%20not%20enough%20hosts%20available.,%20Code:%20500\%22 | 18:21 |
| bnemec | From https://etherpad.openstack.org/p/tripleo-ci-logstash-queries | 18:22 |
| EmilienM | bnemec: looking at system/config modules/openstack_project/files/logstash/jenkins-log-client.yaml | 18:22 |
| EmilienM | it's really outdating, I'm proposing a change | 18:22 |
| bnemec | EmilienM: sshnaidm had a change up to improve that. Let me see if I can find it. | 18:22 |
| bnemec | EmilienM: https://review.openstack.org/#/c/312985/ | 18:22 |
| *** noslzzp has joined #tripleo | 18:23 | |
| EmilienM | bnemec: ok I was about proposing the same kind of patch, but with a bit more things, it's fine do go with this one though | 18:23 |
| openstackgerrit | John Trowbridge proposed openstack-infra/tripleo-ci: Change DLRN promote method https://review.openstack.org/321801 | 18:24 |
| *** dtrainor has quit IRC | 18:27 | |
| *** chem`` has quit IRC | 18:28 | |
| dkelson | I need some help some custom configuration failing, see: https://paste.fedoraproject.org/371232/14642873/raw/ | 18:29 |
| *** chem`` has joined #tripleo | 18:29 | |
| *** gfidente has quit IRC | 18:34 | |
| slagle | EmilienM: yea i filed a bug and linked to the logstash query | 18:35 |
| EmilienM | cool | 18:35 |
| slagle | EmilienM: lucas also helped me identify the patch in nova that will hopefully fix it | 18:36 |
| slagle | it's not landed yet, but it sounds like our issue | 18:36 |
| EmilienM | slagle: do you have url handy? | 18:36 |
| EmilienM | https://review.openstack.org/#/c/306670/ | 18:37 |
| slagle | EmilienM: here is the tripleo bug: | 18:37 |
| slagle | > OpenStack use to be more of a product then it is now. When there were common problems to be solved, there was pressure applied to solve them in a way everyone (OpenStack Project, OpenStack Users, and Openstack Operators) would benefit from. | 18:37 |
| slagle | > | 18:37 |
| slagle | grr sorry | 18:37 |
| slagle | bad copy/paste | 18:37 |
| slagle | obviously i was reading openstack-dev :) | 18:37 |
| bnemec | lol | 18:37 |
| EmilienM | hopefully it was SFW | 18:37 |
| slagle | the exciting golang thread! | 18:37 |
| EmilienM | ah, that | 18:37 |
| slagle | here is the bug https://bugs.launchpad.net/tripleo/+bug/1585641 | 18:38 |
| openstack | Launchpad bug 1585641 in tripleo "CI: jobs failing with Message: No valid host was found. There are not enough hosts available., Code: 500"" [High,Triaged] - Assigned to James Slagle (james-slagle) | 18:38 |
| slagle | the nova bug and patch is linked in a comment | 18:38 |
| EmilienM | https://review.openstack.org/316031 | 18:38 |
| slagle | yep, we need that to land, then we need a dlrn promote | 18:39 |
| EmilienM | slagle: I dropped a comment in the review | 18:39 |
| EmilienM | to wake up people a bit | 18:39 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Remove the delay resource and its constraints https://review.openstack.org/285207 | 18:39 |
| slagle | thanks | 18:40 |
| trown | larsks: do you by chance know a better way to hide a URL parameter using curl? https://review.openstack.org/#/c/321801/1/scripts/mirror-server/promote.sh | 18:43 |
| *** dsariel has joined #tripleo | 18:43 | |
| * larsks looks | 18:43 | |
| trown | larsks: I think what I have will be fine, because that token is very narrowly scoped, but if there is a better way... that would be better :) | 18:44 |
| pradk | i'm trying to test an upgrade heat template and getting this error ERROR: Failed to validate: : resources.UpdateWorkflow: : Value must be a string ... the error message is clear but i have no idea what or where its referring to | 18:45 |
| pradk | any suggestions? | 18:45 |
| larsks | trown: that seems sufficient. You could also just put the curl command in a separate script, and call out to that; the trace settings wouldn't propogate. | 18:45 |
| trown | awesome thanks | 18:47 |
| *** lucas-afk has quit IRC | 18:48 | |
| openstackgerrit | Lars Kellogg-Stedman proposed openstack/tripleo-quickstart: add NAT rules for external network when network_isolation=true https://review.openstack.org/321810 | 18:48 |
| larsks | ayoung: https://review.openstack.org/#/c/321810/ ought to be fix the problem with outbound network access from the overcloud nodes when using network_isolation=true. If you have the chance to test to see if that resolves your problem that would be awesome. | 18:48 |
| *** lucasagomes has joined #tripleo | 18:54 | |
| openstackgerrit | ayoung proposed openstack/tripleo-quickstart: add NAT rules for external network when network_isolation=true https://review.openstack.org/321810 | 18:57 |
| *** cwolferh has quit IRC | 18:57 | |
| ayoung | larsks, will do | 18:57 |
| *** paramite has joined #tripleo | 19:04 | |
| *** hoobaman has quit IRC | 19:05 | |
| *** saneax is now known as saneax_AFK | 19:06 | |
| openstackgerrit | Emilien Macchi proposed openstack/puppet-pacemaker: Add other-requirements.txt for bindep https://review.openstack.org/318802 | 19:13 |
| *** dbecker has quit IRC | 19:13 | |
| *** fragatina has joined #tripleo | 19:16 | |
| *** fragatina has quit IRC | 19:17 | |
| *** fragatina has joined #tripleo | 19:17 | |
| *** fultonj has quit IRC | 19:26 | |
| *** dbecker has joined #tripleo | 19:26 | |
| *** dbecker has quit IRC | 19:26 | |
| *** fzdarsky_ has quit IRC | 19:27 | |
| *** dbecker has joined #tripleo | 19:30 | |
| colonwq | EmilienM, I have made headway for my tripleo install of radosgw. Once I get it fully working, I'll need to make it work the new right way | 19:35 |
| EmilienM | colonwq: with composable roles? | 19:36 |
| colonwq | EmilienM, Yes. Currently it is the old way but I will rework it as needed. | 19:37 |
| dsneddon | slagle, I have a question. If I wanted to modify the 20-os-net-config script in tripleo-image-elements, what would I have to do to use the modified element in the next deploy? | 19:37 |
| dsneddon | slagle, Or would modifying the file in /usr/share be sufficient? | 19:37 |
| *** pradk has quit IRC | 19:38 | |
| dsneddon | trown, bnemec: ^^^ any idea? | 19:42 |
| bnemec | dsneddon: You would have to modify the one in /usr/share, then rebuild your images. | 19:43 |
| dsneddon | bnemec, That's what I thought, so for OSP I'd need to use virt-customize. Thanks. | 19:43 |
| bnemec | Although I thought there was work going on to stop using that script from the elements so it could be updated. | 19:44 |
| bnemec | dsneddon: Ah, for OSP I think that would be correct. | 19:44 |
| dsneddon | bnemec, Yeah, there's a review to replace it, but I don't know if it's merged yet. | 19:44 |
| bnemec | Yeah, and it definitely wouldn't be in OSP yet. | 19:44 |
| rbrady | hey trown, do you have any time this afternoon to talk about release tagging? | 19:45 |
| trown | rbrady: sure, I am not sure I will have the answers | 19:49 |
| EmilienM | rbrady: tripleo release? | 19:49 |
| rbrady | trown: I was asked to look into why mistral isn't tagged. when I asked the mistral folks, they asked why it would be important. I just don't have the answer. Did you see their comments in the email you were forwarded? | 19:50 |
| rbrady | Emillien: mistral | 19:50 |
| rbrady | trown: my mistake, the email you got wasn't detailed | 19:51 |
| rbrady | trown: just sent you the ongoing conversation | 19:52 |
| trown | checking | 19:52 |
| *** absubram has quit IRC | 19:55 | |
| openstackgerrit | Andreas Florath proposed openstack/diskimage-builder: partitioning-parted: extraced and adapted element https://review.openstack.org/313938 | 19:55 |
| trown | rbrady: that is pretty odd to me... every openstack project tags releases either manually or through the release:managed tag | 19:56 |
| *** pradk has joined #tripleo | 19:56 | |
| rbrady | trown: what I'm looking for is the justification to them of why they should use the release:managed tag | 19:57 |
| *** lblanchard has quit IRC | 19:58 | |
| trown | seems pretty odd to need justification for something every other project does... but the main reason is to have a clear indicator in the git repository corresponding to a release | 19:58 |
| trown | do they need justification on why to release software? | 19:58 |
| trown | I am pretty confused actually... mistral does not have pypi releases either? https://pypi.python.org/pypi?%3Aaction=search&term=mistral&submit=search | 19:59 |
| openstackgerrit | Andreas Florath proposed openstack/diskimage-builder: [WIP] Add: lvm element https://review.openstack.org/316529 | 20:01 |
| *** cwolferh has joined #tripleo | 20:01 | |
| *** jprovazn has quit IRC | 20:01 | |
| trown | rbrady: ok so they at least have tarballs on http://tarballs.openstack.org/mistral/ ... as a consumer of those though, I want to be able to look up in the git repo the commit that tarball was made from | 20:03 |
| trown | that is what a release tag in git is for | 20:03 |
| openstackgerrit | Ben Nemec proposed openstack/tripleo-heat-templates: Enable firewall by default on the overcloud https://review.openstack.org/321833 | 20:07 |
| *** ayoung has quit IRC | 20:07 | |
| *** qasims has joined #tripleo | 20:07 | |
| openstackgerrit | Merged openstack/instack-undercloud: Only run mistral commands if mistral is enabled https://review.openstack.org/320035 | 20:09 |
| rbrady | trown: I can see there are tags in the git repo that correspond to tarballs. why is it important for them to used release:managed tag? I'm not finding anything that describes the value or purpose at https://governance.openstack.org/reference/tags/release_managed.html | 20:10 |
| trown | rbrady: to me the fact that there is confusion around why releases are a good thing is a good argument that mistral folks should let experienced release engineers take care of it for them with the release:managed tag | 20:10 |
| *** absubram has joined #tripleo | 20:11 | |
| trown | although release:managed is not a requirement to tag releases | 20:12 |
| openstackgerrit | Dan Prince proposed openstack/instack-undercloud: Only validate SSL VIPs if SSL is enabled https://review.openstack.org/321836 | 20:16 |
| panda | I'm developing a CREATE_FAILED panic syndrome | 20:16 |
| vkmc | hey guys, I'm getting python-hardware-detect package not found when trying to build a Centos overcloud image | 20:17 |
| vkmc | precisely, when running "openstack overcloud image build --all" | 20:18 |
| vkmc | is that a known issue? is there a workaround for it? | 20:18 |
| openstackgerrit | John Trowbridge proposed openstack/tripleo-quickstart: Do not archive root directory https://review.openstack.org/321839 | 20:19 |
| vkmc | EmilienM, ^? | 20:22 |
| *** jayg is now known as jayg|g0n3 | 20:22 | |
| EmilienM | slagle, bnemec: could you review https://review.openstack.org/#/c/321210/ - https://review.openstack.org/#/c/321206/ and https://review.openstack.org/#/c/321201/ please? | 20:22 |
| EmilienM | vkmc: I'm not aware about it, let me check | 20:22 |
| EmilienM | vkmc: in your undercloud.conf, what is the value of "inspection_extras" ? | 20:24 |
| EmilienM | True requires python-hardware-detect | 20:24 |
| trown | rbrady: re-reading that email I am still confused is the question "what is the benefit of release:manged" or is it "why should we tag a release in git" | 20:24 |
| trown | vkmc is there a reason to not use pre-validated images? | 20:25 |
| EmilienM | vkmc: you might need to build with --agent-dib-extra-args | 20:25 |
| vkmc | it's commented out | 20:25 |
| vkmc | so I assume the default is true | 20:25 |
| EmilienM | see http://git.openstack.org/cgit/openstack/python-tripleoclient/tree/tripleoclient/v1/overcloud_image.py#n320 | 20:25 |
| vkmc | trown, not really, giving my first steps with tripleo | 20:26 |
| EmilienM | trown ++ why not using pre-built images? | 20:26 |
| rbrady | trown: well, maybe I should prepare with answers to both. the idea is to bring this subject up at the next mistral irc meeting | 20:26 |
| vkmc | I'm following this guide http://tripleo.org/basic_deployment/basic_deployment_cli.html#upload-images | 20:26 |
| EmilienM | trown: if you have time to review it also ^ | 20:26 |
| trown | vkmc: ok cool there are images from tripleo-ci on the mirror server for master: http://66.187.229.99/builds/current-tripleo/ | 20:27 |
| vkmc | thanks trown, EmilienM | 20:27 |
| vkmc | will try those | 20:27 |
| trown | vkmc: and there are images built in RDO for both master and older releases: https://ci.centos.org/artifacts/rdo/images/ | 20:27 |
| trown | vkmc: I think the docs around building images have fallen out of date from what is used in CI | 20:28 |
| openstackgerrit | Emilien Macchi proposed openstack/puppet-tripleo: Update profiles for loabalancer split https://review.openstack.org/320410 | 20:28 |
| vkmc | I see :) | 20:29 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: loadbalancer: update hiera parameters for HAproxy/keepalived split https://review.openstack.org/320411 | 20:29 |
| openstackgerrit | Qasim Sarfraz proposed openstack/puppet-tripleo: add plumgrid neutron profile https://review.openstack.org/317259 | 20:31 |
| *** julim_ has quit IRC | 20:33 | |
| trown | EmilienM: don't we need tests for https://review.openstack.org/#/c/321201 ? | 20:37 |
| rbrady | trown: I have to go for the day, but maybe we could pick this discussion up tomorrow? | 20:37 |
| EmilienM | trown: well, there are no tests for loadbalancer class | 20:38 |
| trown | EmilienM: ya just saw that... | 20:38 |
| EmilienM | but yes, we would need to write them one day (for haproxy & keepalived) | 20:38 |
| EmilienM | in this patch I did copy paste | 20:38 |
| trown | ya that is fair | 20:38 |
| trown | rbrady: sure, though I think it is probably a better thing to engage openstack-infra folks on | 20:39 |
| bnemec | rbrady: trown: I would suggest a discussion with #openstack-release | 20:39 |
| rbrady | trown: the request came from tripleo folks | 20:39 |
| trown | rbrady: from a packaging perspective it makes it much easier when a project has sane release practices, but I don't really have justification beyond that | 20:39 |
| EmilienM | bnemec: your comment is nice | 20:39 |
| *** jdob has quit IRC | 20:40 | |
| EmilienM | bnemec: why didn't I think about it? | 20:40 |
| rbrady | trown, bnemec: thanks | 20:40 |
| *** numans has quit IRC | 20:41 | |
| openstackgerrit | Merged openstack/puppet-tripleo: loadbalancer: make sure controller_* are array-typed https://review.openstack.org/321206 | 20:46 |
| *** panda has quit IRC | 20:48 | |
| *** panda has joined #tripleo | 20:48 | |
| larsks | trown: well, though I would check to see how our playbooks worked with ansible 2.1. | 20:49 |
| trown | I am not very optimistic :p | 20:50 |
| larsks | trown: well, it went about as well as expected :) There was a failure early on that I thought was another awful regression, but it's actually just a chance in the default type applied to module params that don't declare one explicitly. So I am going to add a line to our generate_macs module and try again... | 20:52 |
| openstackgerrit | Merged openstack/puppet-pacemaker: Add other-requirements.txt for bindep https://review.openstack.org/318802 | 20:54 |
| openstackgerrit | Gabriele Cerami proposed openstack/tripleo-quickstart: Add options to download and handle overcloud images https://review.openstack.org/321057 | 20:58 |
| openstackgerrit | Emilien Macchi proposed openstack/puppet-tripleo: Explode loadbalancer role in 2 sub-roles https://review.openstack.org/321201 | 20:58 |
| EmilienM | bnemec: better ? :-) | 20:58 |
| openstackgerrit | Emilien Macchi proposed openstack/puppet-tripleo: Update profiles for loabalancer split https://review.openstack.org/320410 | 21:00 |
| *** Goneri has quit IRC | 21:03 | |
| *** paramite has quit IRC | 21:03 | |
| openstackgerrit | Emilien Macchi proposed openstack/instack-undercloud: loadbalancer: explode hieradata to separate HAproxy & keepalived https://review.openstack.org/321210 | 21:04 |
| *** apetrich has quit IRC | 21:08 | |
| *** MaxPC has quit IRC | 21:08 | |
| *** fragatina has quit IRC | 21:09 | |
| *** yamahata has quit IRC | 21:11 | |
| *** trown is now known as trown|outtypewww | 21:11 | |
| ianw | yolanda: where's the fedora-atomic element coming from? | 21:12 |
| ianw | i'm guessing magnum ... let me see if i can repro | 21:12 |
| *** rhallisey has quit IRC | 21:18 | |
| openstackgerrit | Lars Kellogg-Stedman proposed openstack/tripleo-quickstart: make generate_macs.py compatible with ansible 2.1 https://review.openstack.org/321863 | 21:26 |
| larsks | trown|outtypewww: with https://review.openstack.org/321863 the quickstart seems to work okay. | 21:26 |
| *** myoung has joined #tripleo | 21:27 | |
| *** myoung has quit IRC | 21:28 | |
| *** myoung has joined #tripleo | 21:28 | |
| *** fragatina has joined #tripleo | 21:29 | |
| *** jrist has quit IRC | 21:32 | |
| *** jrist has joined #tripleo | 21:45 | |
| *** akshai_ has quit IRC | 21:48 | |
| *** fultonj has joined #tripleo | 21:51 | |
| *** noslzzp has quit IRC | 21:53 | |
| *** dc has quit IRC | 22:00 | |
| *** fultonj has quit IRC | 22:03 | |
| *** ayoung has joined #tripleo | 22:12 | |
| *** Goneri has joined #tripleo | 22:23 | |
| Erming_ | larsks: Are your already off now? Just wondering if there is a dashboard in the virtual overcloud? how to access it? | 22:23 |
| Erming_ | larsks: I tried nearly the possible IPs but none work. | 22:25 |
| *** noslzzp has joined #tripleo | 22:26 | |
| *** qasims has quit IRC | 22:30 | |
| ayoung | Erming_, yes there is | 22:33 |
| ayoung | you need to set up some way to access it. | 22:33 |
| ayoung | By default it is not reachable from outside (assuming you are doing quickstart) | 22:34 |
| Erming_ | ayoung: I am trying with the default IPs, but don't find one that works for me. I knew it's not accessible from outside. so I use a vnc to the host but still doesn't work | 22:34 |
| ayoung | Erming_, I had used sshuttle | 22:34 |
| Erming_ | ayoung: is it https or http? | 22:35 |
| ayoung | shuttle -e "ssh -F $HOME/.quickstart/ssh.config.ansible" -r undercloud -v 192.168.82.9/24 | 22:35 |
| ayoung | Erming_, I'm not sure...one sec | 22:35 |
| ayoung | Erming_, heh...not running on mine. It is suppsed to though | 22:36 |
| ayoung | Erming_, nothing is running on mine | 22:36 |
| Erming_ | ayoung: is shuttle a X terminal? | 22:36 |
| ayoung | EmilienM, nah. Just something you run from one that sets up routes | 22:37 |
| ayoung | http://goodsquishy.com/2016/02/sshuttle-where-have-you-been-all-my-life/ | 22:37 |
| EmilienM | wesh | 22:37 |
| Erming_ | ayoung: is 192.168.82.9/24 is what? | 22:37 |
| ayoung | Erming_, that is the subnet I want to route via sshuttle | 22:38 |
| ayoung | controlplan | 22:38 |
| ayoung | ctlplane | 22:38 |
| Erming_ | ayoung: I think you are talking to Erming instead of EmilienM :-) | 22:38 |
| ayoung | Heh Good evening EmilienM .... | 22:38 |
| EmilienM | I'm going to run brb | 22:39 |
| Erming_ | >> iptables -t nat -A sshuttle-12300 -j RETURN --dest 127.0.0.0/8 -p tcp | 22:42 |
| Erming_ | ayoung: it gets stuck here: | 22:42 |
| *** diazjf1 has quit IRC | 22:42 | |
| ayoung | Erming_, so...is HTTPD even running on your overcloud, cuz mine isn't | 22:42 |
| *** Goneri has quit IRC | 22:43 | |
| Erming_ | ayoung: it's running | 22:44 |
| ayoung | Erming_, so you should be able to start by hitting it with curl locally | 22:45 |
| ayoung | probably curl $HOSTNAME or summat | 22:45 |
| Erming_ | ayoung: Connection refused, neither lftp works. But telnet hostname 80 works | 22:46 |
| ayoung | try 0.0.0.0 | 22:46 |
| Erming_ | ? | 22:47 |
| Erming_ | ayoung: from one controller or from undercloud? | 22:47 |
| ayoung | curl http://0.0.0.0 | 22:47 |
| Erming_ | ayoung: better. works on undercloud. | 22:48 |
| Erming_ | but doesn't make sense? | 22:48 |
| ayoung | Erming_, what is the overcloud HTTPD server listening on? Its at the top of /etc/httpd/conf/httpd.conf | 22:49 |
| Erming_ | ayoung: overcloud-controller-1.localdomain 172.16.2.7 | 22:50 |
| Erming_ | ayoung: but I tried this IP too. | 22:50 |
| ayoung | firewall? | 22:50 |
| Erming_ | ayoung: even on the localhost if I access the httpd | 22:51 |
| ayoung | weird | 22:51 |
| ayoung | vi 172.16.2.7? | 22:51 |
| ayoung | what does route or traceroute give you? | 22:52 |
| Erming_ | ayoung: sorry. it works locally. | 22:52 |
| ayoung | and is the port open? | 22:52 |
| Erming_ | ayoung: maybe I should find a way to access this IP | 22:52 |
| Erming_ | ayoung: from host to undercloud to a control (172.16.2.7), I should build a path for this port. | 22:53 |
| ayoung | Erming_, from undercloud can you curl 172.16.2.7? | 22:53 |
| Erming_ | ayoung: no | 22:54 |
| Erming_ | ayoung: that's what confuses me | 22:54 |
| ayoung | ok, looks like the undercloud httpd instance is listening on an interface that does not have routes to the outside world | 22:54 |
| Erming_ | ayoung: it goes back to my original test: instead of using a specific IP, we should use the VIP. However, even I could telnet/curl from undercloud, I failed to use brower to access to it. | 22:57 |
| ayoung | Erming_, I think it is a side effect of network isolation. He's attemptingto put it on its own network, just that network is misconfigured? | 22:58 |
| ayoung | Maybe? | 22:58 |
| Erming_ | ayoung: I will try you sshuttle later. gotta go to daycare now. Thank you so much! | 22:59 |
| *** cwolferh has quit IRC | 23:05 | |
| *** myoung has quit IRC | 23:07 | |
| *** ayoung has quit IRC | 23:07 | |
| *** absubram has quit IRC | 23:07 | |
| *** ayoung has joined #tripleo | 23:07 | |
| *** noslzzp has quit IRC | 23:13 | |
| *** tosky has quit IRC | 23:22 | |
| *** cwolferh has joined #tripleo | 23:23 | |
| *** rhallisey has joined #tripleo | 23:28 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Export die() function https://review.openstack.org/321892 | 23:29 |
| *** yamahata has joined #tripleo | 23:32 | |
| *** julim has joined #tripleo | 23:33 | |
| *** ayoung has quit IRC | 23:34 | |
| *** thrash is now known as thrash|g0ne | 23:38 | |
| openstackgerrit | Brad P. Crochet proposed openstack/puppet-tripleo: Add Sahara profiles https://review.openstack.org/318839 | 23:39 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!