*** penick has quit IRC | 00:01 | |
*** penick_ has joined #tripleo | 00:01 | |
*** shivrao has quit IRC | 00:04 | |
*** jrist has joined #tripleo | 00:04 | |
*** penick_ has quit IRC | 00:45 | |
*** penick has joined #tripleo | 00:56 | |
*** eil397 has quit IRC | 00:57 | |
*** shivrao has joined #tripleo | 01:00 | |
*** penick has quit IRC | 01:09 | |
*** shivrao has quit IRC | 01:12 | |
*** weshay has quit IRC | 01:36 | |
*** shivrao has joined #tripleo | 01:57 | |
*** masco has joined #tripleo | 02:08 | |
*** lazy_prince has joined #tripleo | 02:08 | |
*** Goneri has joined #tripleo | 02:09 | |
*** masco has quit IRC | 02:18 | |
*** lazy_prince has quit IRC | 02:22 | |
*** Goneri has quit IRC | 02:24 | |
*** trozet has joined #tripleo | 02:28 | |
*** lazy_prince has joined #tripleo | 02:29 | |
*** cwolferh has quit IRC | 02:30 | |
*** tzumainn has quit IRC | 02:53 | |
openstackgerrit | Merged openstack/tripleo-heat-templates: Allow a user to specify a comma separated list of ntp servers https://review.openstack.org/261017 | 03:04 |
---|---|---|
openstackgerrit | Merged openstack/tripleo-heat-templates: Introduce domain configuration through parameter https://review.openstack.org/261018 | 03:04 |
*** cwolferh has joined #tripleo | 03:04 | |
openstackgerrit | Merged openstack/tripleo-heat-templates: Align some defaults with with ControllerCount default https://review.openstack.org/261019 | 03:04 |
openstackgerrit | Merged openstack/tripleo-heat-templates: Rename overcloud-without-mergepy to overcloud. https://review.openstack.org/260157 | 03:05 |
*** lazy_prince has quit IRC | 03:28 | |
*** yamahata has quit IRC | 03:33 | |
*** olap_ has quit IRC | 03:36 | |
*** sthillma has quit IRC | 03:39 | |
*** sthillma has joined #tripleo | 03:51 | |
*** links has joined #tripleo | 03:52 | |
*** lazy_prince has joined #tripleo | 03:52 | |
*** sthillma has quit IRC | 03:52 | |
*** anande has joined #tripleo | 03:57 | |
*** anande has quit IRC | 03:58 | |
*** yamahata has joined #tripleo | 04:01 | |
*** trozet has quit IRC | 04:05 | |
*** rook has quit IRC | 04:06 | |
*** rook has joined #tripleo | 04:09 | |
*** shivrao has quit IRC | 04:21 | |
*** sthillma has joined #tripleo | 04:51 | |
*** lazy_prince has quit IRC | 04:52 | |
*** stendulker has joined #tripleo | 04:54 | |
*** sthillma_ has joined #tripleo | 04:55 | |
*** sthillma has quit IRC | 04:56 | |
*** sthillma_ is now known as sthillma | 04:56 | |
*** sthillma has quit IRC | 05:13 | |
*** penick has joined #tripleo | 05:16 | |
*** julim has joined #tripleo | 05:24 | |
*** masco has joined #tripleo | 05:24 | |
*** jaosorior has joined #tripleo | 05:37 | |
*** coolsvap|away is now known as coolsvap | 06:02 | |
*** jaosorior has quit IRC | 06:05 | |
*** coolsvap is now known as coolsvap|away | 06:17 | |
*** coolsvap|away is now known as coolsvap | 06:20 | |
*** jaosorior has joined #tripleo | 06:26 | |
*** shivrao has joined #tripleo | 06:50 | |
*** shivrao has quit IRC | 06:58 | |
openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/tuskar: Replace deprecated LOG.warn with warning https://review.openstack.org/263596 | 07:05 |
*** ukalifon has joined #tripleo | 07:11 | |
*** devvesa has joined #tripleo | 07:18 | |
*** devvesa has quit IRC | 07:29 | |
*** tremble has joined #tripleo | 07:36 | |
*** tremble has joined #tripleo | 07:36 | |
*** sthillma has joined #tripleo | 07:43 | |
*** devvesa has joined #tripleo | 07:44 | |
*** devvesa has quit IRC | 07:51 | |
*** jprovazn has joined #tripleo | 07:53 | |
*** jprovazn has quit IRC | 07:55 | |
*** jprovazn has joined #tripleo | 07:56 | |
*** coolsvap is now known as coolsvap|away | 08:00 | |
*** jtomasek has joined #tripleo | 08:02 | |
*** ifarkas has joined #tripleo | 08:07 | |
*** bvandenh has joined #tripleo | 08:13 | |
*** liverpooler has joined #tripleo | 08:13 | |
*** tremble has quit IRC | 08:23 | |
*** aufi has joined #tripleo | 08:24 | |
*** shardy has joined #tripleo | 08:33 | |
*** mcornea has joined #tripleo | 08:34 | |
*** sthillma has quit IRC | 08:38 | |
*** nico_auv has joined #tripleo | 08:48 | |
*** devvesa has joined #tripleo | 08:50 | |
*** sthillma has joined #tripleo | 08:54 | |
*** sthillma has quit IRC | 09:13 | |
openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/tripleo-common: Replace deprecated LOG.warn with LOG.warning https://review.openstack.org/263330 | 09:15 |
*** coolsvap|away is now known as coolsvap | 09:17 | |
openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/tuskar: Replace deprecated LOG.warn with LOG.warning https://review.openstack.org/263596 | 09:19 |
*** jistr has joined #tripleo | 09:21 | |
*** jcoufal has joined #tripleo | 09:22 | |
*** athomas has joined #tripleo | 09:25 | |
*** Marga_ has joined #tripleo | 09:38 | |
*** akrivoka has joined #tripleo | 09:40 | |
*** paramite has joined #tripleo | 09:41 | |
*** jaosorior has quit IRC | 09:44 | |
*** jaosorior has joined #tripleo | 09:44 | |
jistr | shardy: hi, just noticed your comment about the StructuredDeployments deprecation https://review.openstack.org/#/c/245314/8/puppet/swift-storage-post.yaml -- looks like a patch for this isn't yet submitted, can i submit it right now? should be a simple search & replace. (I'd do SoftwareDeployments rename too, according to http://docs.openstack.org/developer/heat/template_guide/unsupported.html) | 09:46 |
shardy | jistr: Hi, no please don't yet - I was planning to but then I realized there's a heat bug, ref: | 09:47 |
shardy | https://bugs.launchpad.net/heat/+bug/1528958 | 09:47 |
openstack | Launchpad bug 1528958 in heat "SoftwareDeployments -> SoftwareDeploymentGroup update is destructive" [High,New] - Assigned to Rico Lin (rico-lin) | 09:47 |
shardy | jistr: In reality, it will probably work OK, but it'll be safer to wait until that is fixed | 09:48 |
jistr | shardy: ack, thanks for the info | 09:48 |
*** mkovacik has joined #tripleo | 09:50 | |
*** mgould has joined #tripleo | 09:52 | |
*** penick has quit IRC | 09:55 | |
*** Marga_ has quit IRC | 09:55 | |
*** jaosorior has quit IRC | 09:56 | |
*** jaosorior has joined #tripleo | 09:56 | |
*** oshvartz_ has joined #tripleo | 09:56 | |
*** oshvartz_ is now known as oshvartz | 09:57 | |
*** oshvartz has quit IRC | 09:57 | |
*** oshvartz has joined #tripleo | 09:58 | |
*** krickwix has quit IRC | 09:58 | |
*** Marga_ has joined #tripleo | 10:06 | |
*** paramite is now known as paramite|afk | 10:07 | |
*** Marga_ has quit IRC | 10:09 | |
*** yamahata has quit IRC | 10:10 | |
*** bvandenh has quit IRC | 10:11 | |
*** yuanying has quit IRC | 10:11 | |
*** yuanying has joined #tripleo | 10:11 | |
openstackgerrit | Matthew Thode proposed openstack/diskimage-builder: Initial add of gentoo support for diskimage-builder https://review.openstack.org/263349 | 10:14 |
openstackgerrit | Merged openstack/tripleo-heat-templates: Wait for cluster to settle in yum_update.sh https://review.openstack.org/259009 | 10:19 |
*** dtantsur|afk is now known as dtantsur | 10:28 | |
*** yuanying has quit IRC | 10:28 | |
*** yuanying has joined #tripleo | 10:29 | |
shardy | Simple stable review needs +A if anyone has a moment: https://review.openstack.org/#/c/263356/ | 10:43 |
openstackgerrit | Jaume Devesa proposed openstack/tripleo-heat-templates: Adding ManagementIpSubnet to linux bridge net conf https://review.openstack.org/263669 | 10:43 |
openstackgerrit | Jiri Stransky proposed openstack/tripleo-heat-templates: Wait for cluster to settle in yum_update.sh https://review.openstack.org/263673 | 10:53 |
openstackgerrit | Merged openstack/tripleo-puppet-elements: controller: install python-redis https://review.openstack.org/263356 | 11:02 |
*** penick has joined #tripleo | 11:02 | |
*** lucas-dinner is now known as lucasagomes | 11:03 | |
openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates: Add option to add metadata for the overcloud nodes https://review.openstack.org/260159 | 11:07 |
openstackgerrit | Jaume Devesa proposed openstack/tripleo-heat-templates: Add a default proxy shared secret on MidoNet env https://review.openstack.org/263679 | 11:10 |
*** penick has quit IRC | 11:10 | |
*** yuanying has quit IRC | 11:15 | |
openstackgerrit | Jaume Devesa proposed openstack/tripleo-puppet-elements: Use a stable version of MidoNet Neutron Plugin https://review.openstack.org/263681 | 11:19 |
*** paramite|afk is now known as paramite | 11:19 | |
openstackgerrit | Jaume Devesa proposed openstack/tripleo-puppet-elements: Update deric-zookeeper reference in midonet image https://review.openstack.org/263684 | 11:26 |
*** stendulker has quit IRC | 11:26 | |
*** mbound has joined #tripleo | 11:29 | |
openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates: Pacemaker maintenance mode for the duration of Puppet run on update https://review.openstack.org/257976 | 11:48 |
openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates: Add fixup for pcs order constraints after update to new templates https://review.openstack.org/258495 | 11:48 |
*** coolsvap is now known as coolsvap|away | 11:54 | |
openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates: Pacemaker maintenance mode for the duration of Puppet run on update https://review.openstack.org/257976 | 11:57 |
*** mbound_ has joined #tripleo | 12:01 | |
*** mbound has quit IRC | 12:02 | |
*** rhallisey has joined #tripleo | 12:11 | |
openstackgerrit | Dmitry Tantsur proposed openstack/os-cloud-config: [WIP] Rewrite node creating and updating code https://review.openstack.org/263309 | 12:12 |
*** olap_ has joined #tripleo | 12:15 | |
*** Marga_ has joined #tripleo | 12:27 | |
*** devvesa has quit IRC | 12:32 | |
*** Marga_ has quit IRC | 12:33 | |
openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates: Allow for usage of pre-allocated IPs for the controller nodes https://review.openstack.org/260160 | 12:33 |
hewbrocca | marios: while you're in there... have you seen bandini latest on systemd timeouts? | 12:36 |
openstackgerrit | Jiri Stransky proposed openstack/tripleo-heat-templates: Add TimeZone parameter for all node types https://review.openstack.org/249886 | 12:36 |
marios | hewbrocca: hi, have not ... mail thread? looking | 12:37 |
*** slagle has quit IRC | 12:38 | |
hewbrocca | forwarding | 12:40 |
bandini | hewbrocca, marios: I will file bugs this afternoon and will reply to the thread with the BZ numbers | 12:41 |
* bandini off to lunch | 12:41 | |
hewbrocca | bandini: thank you sir | 12:41 |
*** trown|outttypeww is now known as trown | 12:43 | |
marios | hewbrocca: bandini ok thanks | 12:45 |
*** thrash|g0ne is now known as thrash | 12:45 | |
*** gfidente has joined #tripleo | 12:45 | |
*** gfidente has quit IRC | 12:45 | |
*** gfidente has joined #tripleo | 12:45 | |
hewbrocca | marios: I'm just thinking as long as you're working on the upstream patches, let's use the latest timeout numbers | 12:45 |
hewbrocca | then at least we'll have it right in future versions | 12:45 |
marios | hewbrocca: sure no problem, i haven't seen/checked the timeouts (up/down) for a while (am looking at stable/liberty branch cherry picks atm) but can have a look - the timeouts my michele seem sound and beekhof agrees so... | 12:48 |
marios | s/my/by | 12:48 |
*** devvesa has joined #tripleo | 12:51 | |
*** dprince has joined #tripleo | 12:54 | |
*** devvesa has quit IRC | 12:59 | |
*** akrivoka has quit IRC | 13:07 | |
*** bvandenh has joined #tripleo | 13:07 | |
*** fandrieu_ has joined #tripleo | 13:07 | |
*** links has quit IRC | 13:10 | |
*** Marga_ has joined #tripleo | 13:10 | |
openstackgerrit | Merged openstack/tripleo-heat-templates: Network Isolation support for containerized compute https://review.openstack.org/254304 | 13:10 |
openstackgerrit | Jiri Stransky proposed openstack/tripleo-heat-templates: Add resources for major upgrade in Pacemaker scenario https://review.openstack.org/253276 | 13:11 |
jistr | gfidente: ^ that could fix it | 13:11 |
*** Marga_ has quit IRC | 13:12 | |
*** masco has quit IRC | 13:12 | |
*** Marga_ has joined #tripleo | 13:12 | |
gfidente | jistr, I think we have to use OS::Heat::SoftwareDeployments | 13:13 |
gfidente | jistr, appending just the trailing s | 13:14 |
jistr | yeah, though that's just deprecated name for the same thing (which i just learned this morning), the new name is SoftwareDeploymentGroup | 13:14 |
jistr | we should eventually switch all our instances of SoftwareDeployments into SoftwareDeploymentGroup | 13:15 |
gfidente | ah right, just seen it, great thanks | 13:15 |
gfidente | so let me try a recheck on the one which triggers | 13:15 |
jistr | but there's a bug that if it's done with an existing SoftwareDeployments, it can be destructive | 13:15 |
jistr | gfidente: one sec, i'll just rebase the top patch, i left it out | 13:16 |
gfidente | yeah | 13:16 |
*** rbrady has quit IRC | 13:16 | |
openstackgerrit | Jiri Stransky proposed openstack/tripleo-heat-templates: TEST: this will help testing major upgrades https://review.openstack.org/260456 | 13:16 |
jistr | gfidente: ^ alright, done | 13:16 |
gfidente | ok takes a while but looks like we're on to something | 13:17 |
*** akrivoka has joined #tripleo | 13:19 | |
*** rbrady has joined #tripleo | 13:20 | |
*** aufi has quit IRC | 13:22 | |
openstackgerrit | Ryan Hallisey proposed openstack/tripleo-heat-templates: Use new heat-docker-agents images https://review.openstack.org/253515 | 13:24 |
*** Marga_ has quit IRC | 13:25 | |
*** pradk has joined #tripleo | 13:27 | |
gfidente | jistr, thinking about it, the upstream test might hit a timeout during upgrade | 13:28 |
gfidente | assuming it gets that far | 13:29 |
jistr | gfidente: yeah i thought of that too. Initially i thought the upgrade test would be a separate job. | 13:29 |
*** mbound_ has quit IRC | 13:30 | |
jaosorior | jistr: Hey man, is this workaround still needed? https://github.com/jistr/inlunch/blob/master/answers.yml.example#L70 | 13:31 |
jistr | jaosorior: hm good question, i'm not sure. /me looks if there was a stable delorean bump for tripleo | 13:32 |
openstackgerrit | Merged openstack/tripleo-heat-templates: Add option to add metadata for the overcloud nodes https://review.openstack.org/260159 | 13:33 |
*** morazi has joined #tripleo | 13:33 | |
*** weshay_xchat has joined #tripleo | 13:34 | |
*** jayg|g0n3 is now known as jayg | 13:34 | |
jistr | jaosorior: yeah http://trunk.rdoproject.org/centos7/current-tripleo/ still has openstack-ironic-4.2.1-dev242.el7 so i think the workaround is still needed | 13:34 |
*** weshay_xchat is now known as weshay | 13:34 | |
*** slagle has joined #tripleo | 13:35 | |
jaosorior | jistr: alright. Though it's pretty strange, for some reason inlunch seems to get stuck there | 13:35 |
*** mbound has joined #tripleo | 13:38 | |
*** ukalifon has quit IRC | 13:38 | |
jistr | jaosorior: hmm though without the workaround i don't think ironic would work. Can you try executing those by hand and see what's the scoop? https://github.com/jistr/inlunch/blob/master/answers.yml.example#L72-L78 | 13:39 |
jistr | jaosorior: btw did you run it first time or re-run it? i think i didn't make that workaround in an idempotent manner for re-execution (i surely didn't test for it). The whole step would get skipped if undercloud install succeeded, but if it didn't succeed and you'd re-run inlunch, that could be an issue. | 13:41 |
jaosorior | funky, gets stuck in | 13:41 |
jaosorior | sudo patch -r /tmp/ironic-pxe-fix.rej -p2 /tmp/ironic-pxe-fix.patch | 13:41 |
jaosorior | first time | 13:41 |
jistr | jaosorior: does rpm -q openstack-ironic print the version i pasted above? | 13:42 |
*** Goneri has joined #tripleo | 13:42 | |
jaosorior | what... | 13:43 |
jaosorior | uh | 13:43 |
jaosorior | for a very weird reason | 13:43 |
jaosorior | package openstack-ironic is not installed | 13:43 |
openstackgerrit | Ryan Brady proposed openstack/tripleo-common: Organizes Swift exceptions into Swift Storage Backend https://review.openstack.org/258516 | 13:43 |
jistr | uhm that's probably just src package / meta package, sorry. Try openstack-ironic-common | 13:43 |
jistr | jaosorior: ^ | 13:43 |
jaosorior | openstack-ironic-common-4.2.1-dev242.el7.centos.noarch | 13:44 |
openstackgerrit | Merged openstack/tripleo-heat-templates: Pacemaker maintenance mode for the duration of Puppet run on update https://review.openstack.org/257976 | 13:44 |
jistr | hmm funky indeed | 13:44 |
jistr | jaosorior: could you see if /usr/lib/python2.7/site-packages/ironic looks like this patch has already been applied? https://review.openstack.org/gitweb?p=openstack/ironic.git;a=patch;h=f968e1a22d3ed928278a04f0da9124678474ff0a | 13:45 |
*** miles has joined #tripleo | 13:46 | |
*** aufi has joined #tripleo | 13:46 | |
*** shardy has quit IRC | 13:47 | |
openstackgerrit | Athlan-Guyot sofer proposed openstack/puppet-tripleo: Haproxy has non-working Horizon session persistence. https://review.openstack.org/258483 | 13:47 |
jaosorior | jistr: it hasn't | 13:47 |
*** chem has joined #tripleo | 13:49 | |
*** shardy has joined #tripleo | 13:49 | |
chem | hi cores, if you have 2 min to review a very simple one https://review.openstack.org/#/c/258483/ | 13:50 |
chem | it fixes bug https://launchpad.net/bugs/1526786 | 13:50 |
openstack | Launchpad bug 1526786 in tripleo "Haproxy has non-working Horizon session persistence." [Undecided,New] | 13:50 |
jistr | jaosorior: oh my... my bad /o\ | 13:50 |
jistr | jaosorior: try sudo patch -r /tmp/ironic-pxe-fix.rej -p2 < /tmp/ironic-pxe-fix.patch | 13:51 |
jistr | missing '<' | 13:51 |
jaosorior | jistr: that worked, thanks dude :D | 13:52 |
jistr | jaosorior: cool, no problem :) so i think you can restart the ironic services, comment out the whole workaround in the answer file and re-run inlunch | 13:53 |
jaosorior | jistr: actually re-running inlunch had some problems with the setting of the repos | 13:53 |
jaosorior | last time I tried | 13:53 |
*** dtantsur is now known as dtantsur|brb | 13:54 | |
*** trozet has joined #tripleo | 13:54 | |
jistr | that does ring a bell too | 13:55 |
jistr | hmm i thought i fixed that but apparently i didn't | 13:55 |
jistr | one sec | 13:55 |
jistr | jaosorior: perhaps this, though i didn't test yet https://github.com/jistr/inlunch/commit/8ec18ea22faa7459c9c0ae8f4c612ce8b3c8652e | 13:57 |
jaosorior | I | 13:58 |
jaosorior | I' | 13:58 |
jistr | git fails when the destination already exists, so that would surely prevent a re-run. Not sure if there's another idempotency problems somewhere | 13:58 |
jaosorior | I'll try that in a bit | 13:58 |
jaosorior | although I whiped the environment already :P | 13:58 |
dprince | meeting time in 1 minute | 13:59 |
*** lucasagomes is now known as lucas-hungry | 14:00 | |
*** miles has quit IRC | 14:01 | |
*** mgould has quit IRC | 14:01 | |
*** b3nt_pin is now known as beagles | 14:01 | |
gfidente | chem, thanks! | 14:03 |
*** mgould has joined #tripleo | 14:03 | |
*** eggmaster has joined #tripleo | 14:03 | |
chem | gfidente: you're welcome | 14:03 |
prometheanfire | hi | 14:07 |
*** trozet has quit IRC | 14:10 | |
*** trozet_ has joined #tripleo | 14:12 | |
*** oshvartz has quit IRC | 14:12 | |
*** devvesa has joined #tripleo | 14:13 | |
gfidente | jistr, how about reusing timeout= here https://review.openstack.org/#/c/247014/5/extraconfig/tasks/pacemaker_resource_restart.sh ? | 14:20 |
openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates: Add fixup for pcs order constraints after update to new templates https://review.openstack.org/258495 | 14:20 |
*** rpothier has joined #tripleo | 14:22 | |
*** links has joined #tripleo | 14:24 | |
*** oshvartz has joined #tripleo | 14:28 | |
openstackgerrit | Ryan Brady proposed openstack/tripleo-common: Organizes Swift exceptions into Swift Storage Backend https://review.openstack.org/258516 | 14:31 |
*** cmyster has quit IRC | 14:31 | |
jistr | gfidente: nah that's too strict for keystone | 14:32 |
gfidente | jistr, well it gets is as argument | 14:33 |
jistr | gfidente: crm_resource --wait in the "start keystone" phase will probably only exit when all resources are up, and the timeout for keystone starting is 300 secs | 14:33 |
*** kbyrne has quit IRC | 14:33 | |
*** masco has joined #tripleo | 14:34 | |
jistr | gfidente: ok alternatively we could do it, but we'd have to bump the timeouts a notch up. (the keystone timeout bump shoul be significant then, e.g. 1800 or so) | 14:35 |
gfidente | uhm | 14:35 |
gfidente | it's getting weird because if we use 1800 in the function then the outer 300 would not be honored | 14:35 |
gfidente | :( | 14:35 |
*** masco has quit IRC | 14:37 | |
*** masco has joined #tripleo | 14:37 | |
*** cmyster has joined #tripleo | 14:38 | |
*** cmyster has quit IRC | 14:38 | |
*** cmyster has joined #tripleo | 14:38 | |
jistr | gfidente: yeah i meant use a single value for both and make it 1800... i don't think it's likely that both the timeouts would be excercised to their full value, totalling an hour. Either everything's going to be fine, or keystone will not start -> using the first timeout mechanism and failing without going through the 2nd one, or something post-keystone will not start -> using the 2nd timeout mechanism while the 1st one probably passed | 14:39 |
jistr | through relatively quickly. | 14:39 |
*** kbyrne has joined #tripleo | 14:40 | |
jistr | gfidente: but perhaps it would be best to use the solution from the other patch anyway | 14:40 |
jistr | and not merge those two timeouts? | 14:40 |
gfidente | I vote for single | 14:40 |
gfidente | longer | 14:40 |
gfidente | seems to represent better the caller expectations | 14:42 |
jistr | single value used in both timeout mechanisms? or getting rid of the `pcs status | grep` entirely and only using crm_resource --wait? The added benefit of the grep is that it actually checks that the resource really is running, not sure if crm_resource does that too (maybe it would exit even if the start up failed, as failed cluster is a 'stabilized' cluster too? i don't know :) ) | 14:43 |
jistr | gfidente: ^ | 14:43 |
*** dtantsur|brb is now known as dtantsur | 14:44 | |
gfidente | yeah we need the grep | 14:44 |
gfidente | same timeout for both | 14:44 |
*** lblanchard has joined #tripleo | 14:44 | |
openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates: Bump the pacemaker service op_params to 185s for start and stop https://review.openstack.org/263751 | 14:45 |
marios | hewbrocca: ^^ | 14:46 |
jistr | gfidente: alright, fine with me. Regardless of how many timeouts we have in the deployments, only one of them will trigger a deployment failure (or at least one per machine, unless we made a mistake somewhere), so i'm not concerned very much that the timeout for the grep mechanism is perhaps a bit longer than it needs to be. WFM | 14:46 |
hewbrocca | marios: well done | 14:49 |
*** links has quit IRC | 14:49 | |
marios | gfidente: jistr hmm we also need to bump those timeouts in yum_update.sh too seems we set 100s there explicitly | 14:49 |
gfidente | marios, right! | 14:49 |
jistr | yea | 14:49 |
hewbrocca | marios: do the HA guys agree that it should be 185s for all services, or do we have to pick and choose or something? | 14:49 |
marios | hewbrocca: well i just go tthe review out for now, we can tweak as necessary | 14:50 |
hewbrocca | marios: excelent | 14:50 |
hewbrocca | excellent | 14:50 |
marios | hewbrocca: didn't get a bz/upstream bug out yet or whatever | 14:50 |
hewbrocca | marios: cool, can do that once bandini gets it filed | 14:50 |
gfidente | marios, also see my comment about mongodb | 14:51 |
marios | hewbrocca: yeah thatn's the plan | 14:51 |
marios | gfidente: thanks | 14:51 |
*** tzumainn has joined #tripleo | 14:51 | |
gfidente | marios, actually I think the start timeout for mongodb is even higher than 185 | 14:51 |
* jistr would gladly accept a greater difference between pcmk vs. systemd than 5 seconds (say 15 or so) | 14:52 | |
*** trozet_ has quit IRC | 14:52 | |
* jistr is a bit skeptical after bumping the timeouts so many times :D | 14:52 | |
*** kbyrne has quit IRC | 14:52 | |
hewbrocca | jistr: +1 :) | 14:53 |
*** trozet_ has joined #tripleo | 14:53 | |
*** kbyrne has joined #tripleo | 14:53 | |
openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Ensure cluster remains stable during services restarts https://review.openstack.org/247014 | 14:54 |
gfidente | ^^ some more timeouts, getting out of control :( | 14:54 |
*** kbyrne has quit IRC | 14:55 | |
*** sbadia has left #tripleo | 14:57 | |
* marios errand biab | 14:58 | |
*** kbyrne has joined #tripleo | 15:00 | |
*** kbyrne has quit IRC | 15:01 | |
*** kbyrne has joined #tripleo | 15:02 | |
*** slagle has quit IRC | 15:02 | |
trown | dprince: dtantsur, I tested https://review.openstack.org/#/c/263351 and modulo the commit message it worked for me | 15:02 |
dprince | trown: cool, sound like we can move forwards then | 15:02 |
trown | dprince: I wonder if it would be possible to publish the rpms we create in CI in the job artifacts... would make local testing much easier | 15:03 |
openstackgerrit | Dmitry Tantsur proposed openstack/python-tripleoclient: Allow to use the old deploy ramdisk https://review.openstack.org/263351 | 15:03 |
dtantsur | trown, dprince ^^ | 15:04 |
bnemec | trown: I'm actually publishing rpms for some of the projects: https://repos.fedorapeople.org/repos/openstack-m/jenkins/rpm-build/ | 15:04 |
dprince | trown: it is probably possible, but I don't think we have anywhere setup currently to store these RPMs | 15:04 |
bandini | hewbrocca, marios: https://bugzilla.redhat.com/show_bug.cgi?id=1295830 | 15:04 |
openstack | bugzilla.redhat.com bug 1295830 in openstack-tripleo-heat-templates "Increase timeout defaults for pacemaker" [High,New] - Assigned to jason.dobies | 15:04 |
bnemec | Strangely this is the second time in the past hour this has come up. | 15:04 |
bandini | hewbrocca: do I just clone it for 7.x ? | 15:04 |
bnemec | I don't think I'm building tripleoclient right now though. | 15:04 |
trown | bnemec: really? | 15:04 |
dtantsur | I do wonder why we should rerun the whole CI on the commit message change >_< | 15:04 |
trown | :( | 15:04 |
bnemec | dtantsur: If it passed CI before the commit message change we can just merge it as soon as the check jobs come back. | 15:04 |
bnemec | This very much falls under the expedited approval rules. | 15:05 |
dtantsur | ok :) | 15:05 |
trown | thanks dtantsur for the quick patch | 15:05 |
*** lucas-hungry is now known as lucasagomes | 15:05 | |
bnemec | trown: I can add tripleoclient easily. | 15:06 |
trown | bnemec: cool, so I would just look up the change-id there... pretty neat | 15:07 |
bnemec | trown: Yeah. On gerrithub I had a bot account that actually left a link on the review, but I haven't gotten around to figuring out how to make that happen in upstream gerrit. | 15:08 |
*** sthillma has joined #tripleo | 15:09 | |
trown | bnemec: cool, manually finding it (now that I know it exists) is still way easier than installing/setting up delorean for a single package build | 15:10 |
bnemec | trown: Yeah, that was pretty much my reasoning too. :-) | 15:11 |
bnemec | It should be building on future patch sets now: https://repos.fedorapeople.org/repos/openstack-m/jenkins/rpm-build/263351-2/ | 15:11 |
trown | bnemec: awesome, thanks! | 15:11 |
*** mbound has quit IRC | 15:12 | |
*** paramite is now known as paramite|afk | 15:19 | |
*** trozet_ is now known as trozet | 15:20 | |
openstackgerrit | Merged openstack/tripleo-heat-templates: Remove deleted Nova rows https://review.openstack.org/207909 | 15:21 |
*** bvandenh has quit IRC | 15:23 | |
hewbrocca | bandini: that all looks great. I just dev-acked them both | 15:26 |
hewbrocca | thanks | 15:26 |
hewbrocca | marios, jistr if you guys have any further questions on the timeouts, bandini is the guy to ask :) | 15:27 |
openstackgerrit | Brent Eagles proposed openstack/tripleo-heat-templates: Enable configuration of Neutron QoS https://review.openstack.org/247056 | 15:27 |
*** jobewan has joined #tripleo | 15:27 | |
bandini | :) | 15:28 |
*** olap_ has quit IRC | 15:31 | |
openstackgerrit | Brad P. Crochet proposed openstack/tripleo-heat-templates: Making nova parameters configurable for nuage-metadata-agent https://review.openstack.org/263773 | 15:33 |
*** oshvartz has quit IRC | 15:33 | |
*** egafford has joined #tripleo | 15:33 | |
jistr | bandini: hi, i was just wondering if it would be a bit better to make the 'scheduling delta' higher than 5 seconds (say 15) to play it safe(r). Sometimes we're deploying on VMs where things are kinda slow, so i wonder if this could result in a mild time skew. E.g. systemd tries for 90 seconds, but during high machine load it might only start the 2nd attempt 92 seconds after the first one or so, etc. (I don't know if this can happen for real, | 15:35 |
jistr | but i wouldn't be very surprised if it did.) We constantly see warnings from pacemaker that 'machine load is high' on VMs. | 15:35 |
*** fandrieu__ has joined #tripleo | 15:35 | |
*** shardy has quit IRC | 15:35 | |
*** akrivoka has quit IRC | 15:37 | |
jistr | (i didn't mean time skew of system clock, but rather the "unmeasured" operations between the timeouts simply taking too long) | 15:37 |
bandini | jistr: that is a very valid question, to which only trial and error can give an answer. My five seconds came entirely from a gut feeling ;) I think a few seconds more won't really hurt, so 10 or 15 is okay for me (as long as it is not much more than that) | 15:37 |
jistr | yeah 15 seconds is also just my gut feeling :) | 15:37 |
*** fandrieu_ has quit IRC | 15:38 | |
bandini | I think we will improve the spurious fencing situation by a lot even with 185, anything above that won't hurt | 15:39 |
bandini | I am fine with 190 or 195 too | 15:39 |
bandini | if we want to play it safer and possibly change less in the future let's go for 195 and call it a day ;) | 15:39 |
bnemec | marios: Reviewed https://review.openstack.org/#/c/241167 | 15:40 |
jistr | bandini: alright thanks :) | 15:40 |
*** ayoung has quit IRC | 15:41 | |
*** shardy has joined #tripleo | 15:44 | |
egafford | Hi all. Rebasing and modifying the Sahara and Trove patches, against the possibility that there may be some time for review coming up. Question, though: I'm building a stack from master as per http://docs.openstack.org/developer/tripleo-docs/, and I'm seeing the Controller resource fail: http://paste.openstack.org/show/483035/ | 15:45 |
openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Manage keystone initialization directly in t-h-t manifests https://review.openstack.org/244162 | 15:45 |
*** paramite|afk is now known as paramite | 15:45 | |
*** absubram has joined #tripleo | 15:45 | |
openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: endpoint_map: bump nova api from 2.0 to 2.1 https://review.openstack.org/263366 | 15:46 |
openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: WIP - Deploy Aodh services, replacing Ceilometer Alarm https://review.openstack.org/241408 | 15:46 |
egafford | I noted a bug closed in August, which may be relevant, or at least presents the same symptom: https://bugzilla.redhat.com/show_bug.cgi?id=1226957 | 15:46 |
openstack | bugzilla.redhat.com bug 1226957 in rhel-osp-director "CREATE_FAILED for overcloud deploy on baremetal, 1 node active, 1 in error state: "BadRequest: Expecting to find username or userId in passwordCredentials"" [High,Closed: errata] - Assigned to jason.dobies | 15:46 |
egafford | Any thoughts? | 15:46 |
marios | bnemec: appreciated | 15:51 |
*** akrivoka has joined #tripleo | 15:51 | |
bnemec | egafford: The doc flow is unfortunately broken _a lot_ because developers mostly don't use it. I think tripleo.sh has some workarounds you may need: http://docs.openstack.org/developer/tripleo-docs/advanced_deployment/tripleo.sh.html | 15:52 |
*** pblaho has quit IRC | 15:53 | |
egafford | bnemec: Rock rock on. Thanks! | 15:53 |
egafford | bnemec: Looks easy enough to inject my dev stuff into. Much appreciated, again. | 15:54 |
bnemec | egafford: Yeah, I believe the important ones to use the script for are repo-setup, undercloud, and overcloud images. The other stuff should work as documented. | 15:56 |
bnemec | Although I haven't actually done a deployment in weeks, so take my advice with a grain of salt. :-) | 15:56 |
egafford | bnemec: I'll take anything that's more up to date than what I had. | 15:57 |
egafford | Beyond that, question: what's currently the state of the art for actually modifying the codebase within this framework? Is there any built-in ability to have local changes propagate into your cloud instead of remote ones? | 15:58 |
egafford | bnemec: ^ | 15:58 |
*** weshay_xchat has joined #tripleo | 15:58 | |
*** bvandenh has joined #tripleo | 16:00 | |
*** weshay has quit IRC | 16:01 | |
bnemec | egafford: One option is to push your changes and use the pending review functionality documented here: http://docs.openstack.org/developer/tripleo-docs/basic_deployment/basic_deployment_cli.html#get-images | 16:01 |
egafford | bnemec: Yeah, saw that; does that work for repos used outside of the image build process? | 16:02 |
*** devvesa has quit IRC | 16:03 | |
egafford | It clearly works for Puppet, which is great as pushing puppet modules into an image was pretty janky before that feature. (Thanks!) | 16:03 |
*** rebrego has joined #tripleo | 16:03 | |
*** regebro is now known as Guest49355 | 16:03 | |
*** rebrego is now known as regebro | 16:04 | |
bnemec | egafford: Depends on what you're trying to do. The undercloud install has the same thing, but I'm not sure if that method works for non-puppet things. | 16:04 |
egafford | bnemec: Ack. Will poke, and will probably just end up doing my push-repos-in-from-my-laptop-after-undercloud thing. | 16:05 |
egafford | Thanks for the hand. | 16:05 |
*** slagle has joined #tripleo | 16:05 | |
bnemec | egafford: In case it's helpful, there are rpm builds for proposed changes here: https://repos.fedorapeople.org/repos/openstack-m/jenkins/rpm-build/ | 16:06 |
bnemec | I frequently use that for testing non-puppet things. | 16:06 |
*** aufi has quit IRC | 16:07 | |
*** pblaho has joined #tripleo | 16:07 | |
*** jobewan is now known as jobewan_away | 16:07 | |
*** pblaho has quit IRC | 16:10 | |
*** pblaho has joined #tripleo | 16:11 | |
*** shakamunyi has joined #tripleo | 16:14 | |
openstackgerrit | Merged openstack-infra/tripleo-ci: Don't build everything in periodic jobs https://review.openstack.org/244519 | 16:15 |
*** regebro has quit IRC | 16:15 | |
openstackgerrit | Merged openstack-infra/tripleo-ci: Set DELOREAN_REPO_URL for the master periodic job https://review.openstack.org/244526 | 16:15 |
*** barra204 has quit IRC | 16:15 | |
*** pblaho has quit IRC | 16:17 | |
*** pblaho has joined #tripleo | 16:17 | |
*** pblaho has quit IRC | 16:20 | |
*** pblaho has joined #tripleo | 16:20 | |
*** pblaho has quit IRC | 16:22 | |
*** pblaho has joined #tripleo | 16:22 | |
jaosorior | any reviews for these two? https://review.openstack.org/#/c/254771/ https://review.openstack.org/#/c/258634/ | 16:25 |
jaosorior | :D | 16:26 |
*** fandrieu__ has quit IRC | 16:26 | |
*** pradk has joined #tripleo | 16:26 | |
openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates: Bump the pacemaker service op_params to 200s for start and stop https://review.openstack.org/263751 | 16:30 |
*** alop has joined #tripleo | 16:32 | |
*** ayoung has joined #tripleo | 16:35 | |
*** olap_ has joined #tripleo | 16:36 | |
openstackgerrit | James Slagle proposed openstack/tripleo-heat-templates: Add ExtraConfig to cinder storage role https://review.openstack.org/253710 | 16:36 |
*** jobewan_away is now known as jobewan | 16:37 | |
openstackgerrit | James Slagle proposed openstack/tripleo-heat-templates: Add ExtraConfig to cinder storage role https://review.openstack.org/253710 | 16:38 |
openstackgerrit | Merged openstack/python-tripleoclient: Make the undercloud install test discoverable https://review.openstack.org/256410 | 16:39 |
openstackgerrit | Merged openstack/puppet-tripleo: Enable X-Forwarded-Proto header for keystone_public https://review.openstack.org/254771 | 16:43 |
*** bvandenh has quit IRC | 16:43 | |
*** weshay__xchat has joined #tripleo | 16:44 | |
*** pblaho has quit IRC | 16:44 | |
*** pblaho has joined #tripleo | 16:44 | |
*** dprince has quit IRC | 16:45 | |
openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates: Bump the pacemaker service op_params to 200s for start and stop https://review.openstack.org/263751 | 16:45 |
*** weshay_xchat has quit IRC | 16:47 | |
*** jobewan is now known as jobewan_away | 16:47 | |
*** dprince has joined #tripleo | 16:50 | |
openstackgerrit | James Slagle proposed openstack/tripleo-heat-templates: Add ExtraConfig to cinder storage role https://review.openstack.org/253710 | 16:52 |
*** pblaho has quit IRC | 16:54 | |
openstackgerrit | Merged openstack/puppet-tripleo: Enable X-Forwarded-Proto header for Heat and Nova https://review.openstack.org/258634 | 16:54 |
*** pblaho has joined #tripleo | 16:54 | |
*** pblaho has quit IRC | 16:56 | |
*** pblaho has joined #tripleo | 16:57 | |
openstackgerrit | Merged openstack/tripleo-heat-templates: Changes for configuring Nuage https://review.openstack.org/260397 | 17:02 |
*** ayoung has quit IRC | 17:03 | |
jaosorior | Thanks for the reviews :D | 17:07 |
*** pblaho has quit IRC | 17:07 | |
*** pblaho has joined #tripleo | 17:07 | |
*** jobewan_away is now known as jobewan | 17:08 | |
*** paramite is now known as paramite|afk | 17:09 | |
*** jistr has quit IRC | 17:19 | |
openstackgerrit | Merged openstack/puppet-tripleo: Sahara integration https://review.openstack.org/220859 | 17:21 |
*** paramite|afk is now known as paramite | 17:21 | |
openstackgerrit | Merged openstack/puppet-tripleo: Trove integration https://review.openstack.org/233232 | 17:23 |
*** liverpooler has quit IRC | 17:25 | |
*** mbound has joined #tripleo | 17:29 | |
*** tremble has joined #tripleo | 17:31 | |
*** tremble has joined #tripleo | 17:31 | |
*** tremble has left #tripleo | 17:33 | |
*** mbound has quit IRC | 17:34 | |
*** shardy has quit IRC | 17:34 | |
*** shardy has joined #tripleo | 17:36 | |
*** dtantsur is now known as dtantsur|afk | 17:42 | |
*** liverpooler has joined #tripleo | 17:42 | |
*** jaosorior has quit IRC | 17:44 | |
*** jaosorior has joined #tripleo | 17:44 | |
*** jaosorior has quit IRC | 17:45 | |
*** jaosorior has joined #tripleo | 17:45 | |
dprince | shardy: hey, could you review this one https://review.openstack.org/#/c/248500/ | 17:46 |
shardy | dprince: sure, will do | 17:47 |
*** liverpooler has quit IRC | 17:47 | |
shardy | dprince: it's weird that we're maintaining the v2.0 suffix for ironic/swift auth_url's | 17:48 |
dprince | shardy: my interest there is this is required for Mistral | 17:48 |
dprince | shardy: yeah, that was required to keep CI passing | 17:48 |
shardy | I thought everything was moving to unversioned endpoints, that's what we've done for Heat recently w/devstack for example | 17:48 |
dprince | shardy: I don't think Ironic inspector properly handles keystone v3 | 17:48 |
shardy | dprince: k, I guess we can work that out later then | 17:48 |
dprince | shardy: fwiw there is also one other v2 URL for the nova -> Ironic URL that I'm not switching in this patch | 17:49 |
dprince | shardy: will loop back around to fix those later... | 17:49 |
*** yamahata has joined #tripleo | 17:49 | |
shardy | dprince: it sounds like mistral is doing something wrong too, if it requires auth_uri to be versioned | 17:51 |
shardy | we had to add some version discovery keystoneclient stuff to heat to avoid that | 17:51 |
dprince | shardy: it doesn't | 17:51 |
dprince | shardy: it just requires keystone v3 | 17:51 |
shardy | keystone_auth_uri: http://{{LOCAL_IP}}:5000/v3 | 17:52 |
shardy | that's going to set the auth_token auth_uri to a versioned endpoint isn't it? | 17:52 |
dprince | shardy: yep, I see no harm in that | 17:52 |
dprince | shardy: it is the latest... and would get chosen if you unversioned it anyways | 17:53 |
shardy | dprince: well, it's wrong, we had a devstack hack workaround for heat which I recently removed | 17:53 |
jaosorior | it has to be versioned for now, as the default I think ends up being 2.0 anyway | 17:53 |
shardy | https://review.openstack.org/#/c/254755/1/lib/heat | 17:53 |
openstackgerrit | James Slagle proposed openstack/tripleo-image-elements: Don't set default install type for os-cloud-config to pip https://review.openstack.org/263828 | 17:53 |
*** trown is now known as trown|lunch | 17:54 | |
dprince | shardy: why is pinning to a version wrong | 17:54 |
shardy | dprince: I'm fine with landing it for now, but we need to look towards moving away from versioned endpoints, AFAIK it's not right to move from v2.0 to v3 endpoints | 17:54 |
shardy | dprince: because everything requires updating whenever an API version change happens | 17:54 |
openstackgerrit | Martin Mágr proposed openstack/tripleo-heat-templates: Keystone domain for Heat https://review.openstack.org/180566 | 17:54 |
shardy | which is why moving folks to keystone v3 has been *so* difficult | 17:55 |
shardy | there are hardcoded v2.0 references everywhere | 17:55 |
dprince | shardy: if you don't version then don't you risk breaking things if keystone bumps a new version then. Which may require client changes as well? | 17:55 |
shardy | the keystone folks fixed it in nearly all clients, by adding version discovery | 17:55 |
shardy | which handles detecting and using the latest version | 17:55 |
shardy | dprince: I think the idea is keystoneclient abstracts things sufficiently that you should, in many cases, require versioned interfaces | 17:56 |
shardy | auth_uri = http://192.168.1.57:5000 | 17:56 |
shardy | that's what my heat.conf auth_uri looks like on devstack FWIW | 17:57 |
shardy | dprince: I suspect the issue is mistralclient can't do version discovery, but I'm fine with landing this as an interim fix if it works I guess | 17:58 |
shardy | I'm just quite sensitive to the v2.0/v3 thing because it's been a real headache for heat, which has long relied on v3 features | 17:59 |
openstackgerrit | James Slagle proposed openstack/instack-undercloud: Use enable-packages-install element https://review.openstack.org/263829 | 17:59 |
shardy | it should be possible to e.g consume v4 features when they happen, just by discovering the verion is available - you shouldn't need to reconfigure every service in your cloud | 18:00 |
dprince | shardy: I'm open to using versionless if it works and gets us on v3 sooner | 18:00 |
dprince | shardy: I will confirm w/ keystone and update accordingly then | 18:01 |
shardy | dprince: kk, perhaps we should confirm with ayoung or jamielennox, AIUI versionless is the reccommendation but most keystone docs still reference hard-coded v2.0 everywhere | 18:01 |
openstackgerrit | Miles Gould proposed openstack/python-tripleoclient: Use Ironic API v1.11 to support ENROLL state https://review.openstack.org/263831 | 18:04 |
*** nico_auv has quit IRC | 18:07 | |
*** mgould has quit IRC | 18:07 | |
*** mkovacik has quit IRC | 18:11 | |
*** mcornea has quit IRC | 18:19 | |
*** paramite has quit IRC | 18:22 | |
*** eil397 has joined #tripleo | 18:25 | |
*** masco has quit IRC | 18:26 | |
*** athomas has quit IRC | 18:27 | |
*** ifarkas has quit IRC | 18:33 | |
*** sthillma has quit IRC | 18:34 | |
dprince | shardy: https://bugs.launchpad.net/ironic-inspector/+bug/1531236 | 18:35 |
openstack | Launchpad bug 1531236 in Ironic Inspector "ironic os_auth_url doesn't support keystone v3" [High,Triaged] | 18:35 |
*** sthillma has joined #tripleo | 18:35 | |
egafford | bnemec: Ping | 18:37 |
bnemec | egafford: pong | 18:37 |
openstackgerrit | Dan Prince proposed openstack/instack-undercloud: Switch default keystone auth_uri to versionless https://review.openstack.org/248500 | 18:38 |
openstackgerrit | Dan Prince proposed openstack/instack-undercloud: Install mistral (workflow) in the undercloud https://review.openstack.org/248506 | 18:38 |
dprince | shardy: Switched it to versionless... | 18:38 |
egafford | bnemec: Heyup. I'm using tripleo.sh now, and seem to be seeing much the same issue as earlier, or at least a very similar one. | 18:38 |
dprince | shardy: thanks for the help... | 18:38 |
egafford | At the moment, my Compute and Controller servers are stuck spinning on CREATE_IN_PROGRESS, and my ironic nodes are in provisioning state "wait call-back". If this continues to previous plan, I'll likely see the same error as I did with the docs in a (significant) while. | 18:39 |
egafford | Any known issues you're abreast of that might be indicated? | 18:40 |
bnemec | egafford: Sounds like an Ironic deployment issue. Can you get on the console of the ironic nodes? | 18:40 |
egafford | I think I should be able to rebase the Sahara and Trove heat templates reasonably smoothly if I can get an env up, but I'm loathe to push anything without seeing it all go. | 18:41 |
bnemec | It sounds like there is probably an issue with the deploy ramdisk talking back to Ironic. | 18:41 |
bnemec | egafford: I'm actually doing a deployment right now and didn't have that problem, and CI is passing so I don't think there's anything fundamentally broken. Is this a virt environment? | 18:43 |
*** jaosorior has quit IRC | 18:43 | |
*** akrivoka has quit IRC | 18:44 | |
egafford | bnemec: Yeah, this is an instack-virt-setup spanwed env. | 18:44 |
egafford | tripleo.sh from there. | 18:44 |
egafford | CentOS Linux release 7.2.1511 (Core) | 18:44 |
bnemec | egafford: Oh, CentOS. I think that may be a problem. | 18:44 |
egafford | bnemec: Ah... | 18:45 |
*** sthillma has quit IRC | 18:45 | |
egafford | What's the gold standard atm? | 18:45 |
egafford | I thought CentOS was the OS for upstream dev? | 18:45 |
bnemec | egafford: Yeah, but there was a problem because it had an old ipxe: https://bugs.launchpad.net/tripleo/+bug/1507738 | 18:45 |
openstack | Launchpad bug 1507738 in tripleo "ipxe fails on Centos 7 (inc: command not found)" [Critical,Triaged] | 18:45 |
egafford | Sweet. | 18:46 |
bnemec | It appears to be fixed in Ironic, but unless we've updated our CI repo pin we may not have it yet. | 18:46 |
egafford | bnemec: What distro and version would you recommend using? | 18:46 |
egafford | RHEL 7.2 with upstream TripleO? | 18:47 |
*** ayoung has joined #tripleo | 18:47 | |
bnemec | egafford: I'm not sure. I actually use fedora 21 as my host, but I'm running in OVB so that won't work for you (I'm pretty sure instack-virt-setup doesn't work on fedora). | 18:48 |
bnemec | I actually don't know what other people are using for development. :-/ | 18:48 |
egafford | bnemec: The CI is running a baremetal deploy, I assume, because that's sane and reasonable? | 18:49 |
bnemec | egafford: CI is running in some sort of custom virt environments. I suspect it's actually still on an old fedora because most of that was deployed a long time ago and hasn't been touched since. | 18:50 |
egafford | Okay. So what I'm hearing (no judgment, just reflection) is that there is no known good instack-virt-setup (single node) deployment path that spawns a functional overcloud from master at the moment. Is that true? | 18:52 |
openstackgerrit | Lucas Alvares Gomes proposed openstack/python-tripleoclient: Use the grub2 element instead of grub2-deprecated https://review.openstack.org/263846 | 18:53 |
lucasagomes | quick patch ^ | 18:53 |
egafford | bnemec: Given that I don't really have a ton of hardware, what's your suggestion for how to spawn a viable overcloud? | 18:55 |
bnemec | egafford: I'm not sure. I don't personally use instack-virt-setup, but I have to think most other developers are. | 18:55 |
bnemec | slagle: What does your dev environment look like? re: egafford's issues running on centos above. | 18:55 |
slagle | i use centos7 | 18:56 |
slagle | you're only going to get that ipxe error though if your vm's have mutliple nics, i dont think mine do | 18:56 |
slagle | to workaround it, you just need to use the latest http://boot.ipxe.org/undionly.kpxe | 18:57 |
bnemec | slagle: I don't think that's true. It does the inc the first time through the loop. | 18:57 |
slagle | hmm | 18:58 |
slagle | ok i probably just have http://boot.ipxe.org/undionly.kpxe then | 18:58 |
bnemec | Yeah, we definitely don't have the ironic fix in our current pinned repo. I just installed an undercloud this morning and it still has the old boot.ipxe. | 18:59 |
bnemec | slagle: So egafford is probably right that we don't have a working out of the box virt-setup right now. :-( | 19:00 |
egafford | bnemec, slagle: Ok, let's assume for a moment that I'm a Hadoop/Python/Java/database guy, and don't know how and where to inject that file into my env. How do I do? :) | 19:00 |
*** bswartz has joined #tripleo | 19:00 | |
slagle | well if you know Java, you're already smarter than me | 19:00 |
slagle | you'd just put undionly.kpxe under /tftpboot | 19:00 |
egafford | slagle: I'm gonna go out on a limb and suggest that they're separate and non-hierarchical skillsets. :) | 19:01 |
egafford | slagle: If I do that on my instack node, then respawn the overcloud, will that (to your knowledge) work? Do I need to repack images? Do I need to blow away my virsh domains and re-instack? | 19:02 |
slagle | not that i know of | 19:02 |
egafford | slagle: I made the mistake of asking too many questions. Let me ask one: Should I: | 19:06 |
egafford | A) Delete my overcloud heat stack, copy undionly.kpxe into my instack env, use my pre-existing undercloud and images to spawn a new overcloud. | 19:06 |
egafford | B) Delete my overcloud heat stack, copy undionly.kpxe into my instack env, delete and rebuild my glance images, spawn a new overcloud using my pre-existing undercloud. | 19:06 |
egafford | C) Delete my virsh domains on my baremetal server, use instack-virt-setup to recreate them, copy undionly.kpxe into my instack env and spawn a new undercloud. | 19:06 |
egafford | D) Delete my virsh domains on my baremetal server, copy undionly.kpxe into my baremetal server, use instack-virt-setup to recreate my env, copy undionly.kpxe into my instack node, proceed. | 19:06 |
egafford | There are a lot of layers of this onion, and test cycles are reasonably long. Thanks for the help. | 19:07 |
*** penick has joined #tripleo | 19:11 | |
egafford | Okay, I'll just start experimenting. Thanks. | 19:11 |
beagles | gfidente: is there any reason why backporting https://review.openstack.org/#/c/187506/ wouldn | 19:12 |
beagles | ugh | 19:12 |
beagles | gfidente: should've ended with "wouldn't be valid?" | 19:12 |
beagles | gfidente, as in backport to stable/liberty that is... I see someone already posted a patch in December, but its currently in merge conflict | 19:13 |
*** mkovacik has joined #tripleo | 19:19 | |
*** trown|lunch is now known as trown | 19:20 | |
slagle | egafford: try A, if that doesnt work, we'd likely need some ironic expertise when those folks are back online | 19:23 |
egafford | slagle: Cool; started with B, but that's a superset. Thanks! | 19:23 |
openstackgerrit | Merged openstack/tripleo-heat-templates: compute/ceilometer: use internalURL for os endpoint type https://review.openstack.org/263403 | 19:24 |
*** penick has quit IRC | 19:30 | |
*** mkovacik has quit IRC | 19:37 | |
slagle | EmilienM: i'd like to look into moving off the ironic-conductor element in instack-undercloud and use puppet-ironic exclusively | 19:37 |
slagle | EmilienM: one thing i wasnt sure of though was if puppet-ironic took care of this bit: https://github.com/openstack/tripleo-image-elements/blob/master/elements/ironic-conductor/install.d/69-ironic-tftp-support | 19:37 |
*** gfidente has quit IRC | 19:38 | |
openstackgerrit | Dan Sneddon proposed openstack/tripleo-heat-templates: Add IPv6 Support to Isolated Networks https://review.openstack.org/235423 | 19:43 |
*** jprovazn has quit IRC | 19:45 | |
*** jcoufal has quit IRC | 19:46 | |
EmilienM | slagle: that's a very good idea | 19:50 |
EmilienM | slagle: no, puppet-ironic only aims to manage ironic services | 19:50 |
EmilienM | for https://github.com/openstack/tripleo-image-elements/blob/master/elements/ironic-conductor/install.d/69-ironic-tftp-support -- we would need to write some custom manifests | 19:50 |
EmilienM | which is not a big deal | 19:50 |
EmilienM | slagle: I also have a question for you, can I drop all tuskar related puppet code in uc & oc ? | 19:51 |
openstackgerrit | James Slagle proposed openstack/instack-undercloud: Remove unused scripts https://review.openstack.org/263881 | 19:57 |
openstackgerrit | James Slagle proposed openstack/instack-undercloud: Remove unused elements https://review.openstack.org/263882 | 19:57 |
slagle | EmilienM: yea, you can kill all tuskar stuff | 19:58 |
slagle | EmilienM: do you think the puppet manifests for that ironic conductor stuff would go in puppet-ironic or puppet-stack-config in instack-undercloud? | 19:58 |
*** weshay__xchat is now known as weshay | 19:59 | |
*** lucasagomes is now known as lucas-dinner | 19:59 | |
*** ayoung has quit IRC | 19:59 | |
EmilienM | I'm thinking | 19:59 |
openstackgerrit | Merged openstack/python-tripleoclient: Passes to os-cloud-config Keystone{Admin,Internal}Vip https://review.openstack.org/259144 | 20:00 |
EmilienM | slagle: I would rather see it in puppet-tripleo for a first shoot. So we can use it from uc & oc (eventually one day). If we realize it can live in puppet-ironic, we'll move it but I think puppet-tripleo is a nice start. | 20:00 |
EmilienM | slagle: I can write that code if you want | 20:01 |
slagle | yea, puppet-tripleo makes sense, i didnt think of that | 20:01 |
*** mkovacik has joined #tripleo | 20:01 | |
EmilienM | ok I'll do it | 20:02 |
EmilienM | slagle: urgent or not urgent? | 20:02 |
slagle | EmilienM: not urgent | 20:02 |
slagle | was just poking around in instack-undercloud and looking to clean some old stuff up | 20:03 |
EmilienM | cool, I like cleaning too | 20:03 |
bnemec | EmilienM: slagle: Speaking of cleaning up old broken things: https://review.openstack.org/#/c/258028/ | 20:10 |
*** ayoung has joined #tripleo | 20:10 | |
slagle | lgtm | 20:13 |
egafford | slagle: Looks as though this issue may be persistent (for me) even with the new undionly.kpxe, suggesting to me that I may be missing another step somewhere. Still "wait call-back" indefinitely. | 20:25 |
openstackgerrit | Merged openstack/python-tripleoclient: Don't build images with rdo-release by default https://review.openstack.org/252073 | 20:28 |
*** sparkyco_ has joined #tripleo | 20:36 | |
*** weshay has quit IRC | 20:38 | |
*** jobewan is now known as jobewan_away | 20:42 | |
openstackgerrit | Dmitry Tantsur proposed openstack/python-tripleoclient: Install python-hardware package on the agent image by default https://review.openstack.org/263225 | 20:42 |
*** jobewan_away has quit IRC | 20:43 | |
slagle | egafford: ok, well there's the guy lurking who can help you ^^ | 20:45 |
slagle | egafford: to confirm, you have the boot.ipxe fix that uses inc? and are using the latest undionly.kpxe? | 20:47 |
slagle | if so, that should fix it, assuming it's the known issue | 20:47 |
slagle | otherwise, it's possible you're hitting a new issue | 20:47 |
egafford | slagle: I had not heard of a "boot.ipxe fix that uses inc" until that line, or at least, did not interpret it if I heard it. | 20:48 |
egafford | Is there any record of this issue and a workaround? | 20:49 |
egafford | I am using the latest undionly.kpxe. | 20:49 |
slagle | it's the bug bnemec pointed out earlier | 20:49 |
*** penick has joined #tripleo | 20:51 | |
slagle | egafford: do you see any error on the console of the vm's that it's trying to provision? | 20:51 |
egafford | slagle: Okay, cool, so I need to bring Ironic up to https://review.openstack.org/#/c/244530/ (though this has -1s at present.) Is that accurate? | 20:51 |
slagle | if you use that patch, you shouldn't need the newer undionly.kpxe | 20:52 |
egafford | I see, so the undionly.kpxe should be the only required fix. | 20:52 |
slagle | what do you have in /httpboot/boot.ipxe now? | 20:52 |
egafford | (That was my previous understanding, so that's (sort of) good.) | 20:52 |
slagle | can u pastebin it? | 20:52 |
egafford | I just started a fresh env; when I get there I'll let you know. | 20:53 |
*** weshay has joined #tripleo | 20:53 | |
slagle | i thought you said it was stuck in wait call-back? | 20:53 |
egafford | I'm trying a run in which I set absolutely no variables (including node_count and the like,) just to remove every imaginable confound. | 20:53 |
egafford | Yes, that's correct. After a half hour of waiting, though, I trashed the env and restarted. | 20:54 |
dprince | bnemec: hey so I'm just going to revert to /v3 for https://review.openstack.org/#/c/248500/ | 20:57 |
bnemec | dprince: I have no strong opinions on that so I'll +2 anything that passes CI. :-) | 20:59 |
dprince | bnemec: yeah, I switch to versionless to accomidate shardy | 21:00 |
dprince | bnemec: I think keystone would also prefer us to use versionless | 21:00 |
dprince | bnemec: but not all services support that yet apparently as we are discovering. I already filed a ticket on the inspector issue :/ | 21:01 |
bnemec | dprince: Yeah, I'm guessing nobody wants us on v2, so anything that gets us off that seems like a step in the right direction. | 21:01 |
openstackgerrit | Merged openstack/instack-undercloud: Create auth endpoints for Ironic Inspector https://review.openstack.org/260207 | 21:02 |
shardy | dprince: Sorry if versionless didn't work out, I'm fine with the v3 thing as an interim step, I can help look into the steps needed to move to versionless (perhaps service by service?) later | 21:02 |
*** ayoung has quit IRC | 21:02 | |
dprince | shardy: it is cool. I tried it :) | 21:02 |
shardy | dprince: it went pretty smoothly with Heat, but we've converted to keystonclient sessions & auth plugins | 21:03 |
dprince | shardy: I think I tried this a month ago as well. Just couldn't remember all the fallout | 21:03 |
shardy | if other services are still using the legacy keystonclient python iterfaces, I guess things may not work as easily | 21:03 |
shardy | dprince: ack, OK well sorry for the inconvenience, I just wanted to raise it as a potential issue | 21:04 |
dprince | shardy: np, I did file an issue on the ironic inspector stuff | 21:04 |
dprince | shardy: https://bugs.launchpad.net/ironic-inspector/+bug/1531236 | 21:04 |
openstack | Launchpad bug 1531236 in Ironic Inspector "ironic os_auth_url doesn't support keystone v3" [High,Triaged] | 21:04 |
dprince | shardy: no surprise there really... we still use keystoneclient.v2_0.client directly there | 21:05 |
shardy | dprince: ah OK, sessions and auth plugins are the "new" way, I can potentially help w/fixes if needed as I've been fairly involved in the heat->keystone stuff | 21:06 |
egafford | slagle: Possible theory: do I need to restart any ironic services in my undercloud after replacing undionly.kpxe? | 21:08 |
slagle | egafford: i dont know for sure | 21:08 |
egafford | slagle: That's fair. | 21:08 |
slagle | if anything, it would be httpd | 21:08 |
slagle | that's what is serving /httpboot | 21:09 |
slagle | err, actually it's under /tftpboot | 21:09 |
slagle | which is tftpd, which is xinetd | 21:09 |
slagle | i dont really know if any of that needs to be restarted | 21:09 |
openstackgerrit | Dan Prince proposed openstack/instack-undercloud: Switch default keystone auth_uri to v3 https://review.openstack.org/248500 | 21:12 |
openstackgerrit | Dan Prince proposed openstack/instack-undercloud: Install mistral (workflow) in the undercloud https://review.openstack.org/248506 | 21:12 |
dprince | shardy, bnemec, slagle: back to previous v3 https://review.openstack.org/248500. If you could ack it I would like to send once it passes CI | 21:13 |
dprince | it should pass as it is the same as the previous version... | 21:13 |
trozet | EmilienM: I see you have done work on puppet-aodh, any idea if it is supported or plan support in tht? | 21:13 |
EmilienM | trozet: I wrote the module :) | 21:14 |
EmilienM | trozet: https://review.openstack.org/241412 | 21:14 |
EmilienM | and https://review.openstack.org/241408 | 21:14 |
trozet | EmilienM: some folks on the Doctor project for OPNFV asking for it to be included in our next release | 21:14 |
EmilienM | I'm working on both overcloud & undercloud | 21:14 |
EmilienM | it will be | 21:14 |
trozet | EmilienM: cool, overcloud is all we really care about. Any time frame for this? we release on Feb 2nd | 21:15 |
EmilienM | trozet: doing my best to make it before then | 21:16 |
trozet | EmilienM: if you want, we could help you | 21:16 |
EmilienM | code is all writen but does not pass CI yet | 21:16 |
EmilienM | I'm working on that but yeah, feel free to help me on the overcloud | 21:16 |
EmilienM | apache does not start and I need to figure why | 21:16 |
trozet | EmilienM: OK I'll talk to the OPNFV folks then get back to you | 21:17 |
EmilienM | if you have a working setup, you can apply the patch | 21:17 |
trozet | EmilienM: thanks | 21:17 |
EmilienM | trozet: let me know if you can help on that | 21:18 |
EmilienM | thanks | 21:18 |
*** rhallisey has quit IRC | 21:18 | |
openstackgerrit | Merged openstack/tripleo-heat-templates: Add fixup for pcs order constraints after update to new templates https://review.openstack.org/258495 | 21:19 |
*** jobewan has joined #tripleo | 21:20 | |
*** chlong has quit IRC | 21:22 | |
*** weshay has quit IRC | 21:26 | |
*** weshay has joined #tripleo | 21:40 | |
egafford | slagle: I've now gotten back to the failing stage. In my instack node, I have no file in either /httpboot/boot.ipxe nor /tftpboot/boot.ipxe. | 21:41 |
egafford | There are files under /httpboot/{node-uuid}/config | 21:42 |
egafford | slagle: Ah, nvm; perms. Dumb. Contents of /httpboot/boot.ipxe: http://paste.openstack.org/show/483081/ | 21:45 |
egafford | There really is not a file /tftboot/boot.ipxe. | 21:45 |
*** thrash is now known as thrash|g0ne | 21:49 | |
slagle | egafford: do you see an error on the consoles of the vm? | 21:52 |
egafford | I haven't managed to succeed in ironic node-get-console, and I'm cmdline only atm (haven't routed UI through.) | 21:53 |
slagle | the console through ironic isn't going to work in a virt environment afaik | 21:53 |
slagle | so you'd have to use virt-manager or something | 21:53 |
*** mkovacik has quit IRC | 21:54 | |
openstackgerrit | Emilien Macchi proposed openstack/instack-undercloud: Deploy Logging on the undercloud with Puppet https://review.openstack.org/258653 | 21:56 |
*** rpothier has quit IRC | 21:58 | |
*** penick has quit IRC | 21:58 | |
*** sparkyco_ has quit IRC | 22:00 | |
*** penick has joined #tripleo | 22:00 | |
openstackgerrit | Emilien Macchi proposed openstack/instack-undercloud: Deploy Monitoring on the undercloud with Puppet https://review.openstack.org/253709 | 22:02 |
openstackgerrit | Emilien Macchi proposed openstack/instack-undercloud: Deploy Logging on the undercloud with Puppet https://review.openstack.org/258653 | 22:02 |
*** lblanchard has quit IRC | 22:03 | |
EmilienM | slagle: is it ok to land https://review.openstack.org/241412 for you? | 22:03 |
*** sparkyco_ has joined #tripleo | 22:05 | |
prometheanfire | anything more I need to do for this? https://review.openstack.org/#/c/263349/ | 22:05 |
bnemec | EmilienM: There's one thing missing in that patch. I left a comment. | 22:05 |
EmilienM | bnemec: for undercloud_aodh_password ? | 22:06 |
bnemec | EmilienM: Yes, all of those params should also be in https://github.com/openstack/instack-undercloud/blob/master/undercloud.conf.sample#L75 | 22:07 |
bnemec | Other than that it looks fine to me. | 22:07 |
EmilienM | damn | 22:07 |
EmilienM | I missed that | 22:07 |
* EmilienM doing a patchset | 22:07 | |
EmilienM | ok thx bnemec | 22:07 |
bnemec | I wonder if instack-undercloud has stable enough options that we could add a sample config check job. | 22:08 |
bnemec | I know it caused a lot of headaches in other projects, but we have a lot fewer options and don't expose any options from other libraries. | 22:08 |
openstackgerrit | Emilien Macchi proposed openstack/instack-undercloud: Deploy Aodh services, replacing Ceilometer Alarm https://review.openstack.org/241412 | 22:09 |
EmilienM | bnemec, dprince ^ | 22:09 |
*** trown is now known as trown|outttypeww | 22:09 | |
*** sparkyco_ has quit IRC | 22:10 | |
bnemec | prometheanfire: Left a couple more comments, but overall I'm good with it now. | 22:18 |
prometheanfire | and replied | 22:20 |
prometheanfire | I'll update the irc channel to #gentoo-virtual but that's it, the profiles are what we use upstream | 22:20 |
*** egafford has quit IRC | 22:21 | |
openstackgerrit | Matthew Thode proposed openstack/diskimage-builder: Initial add of gentoo support for diskimage-builder https://review.openstack.org/263349 | 22:22 |
prometheanfire | bnemec: ^ | 22:23 |
prometheanfire | actually removed the irc chan comment entirely | 22:23 |
greghaynes | prometheanfire: Hey, so I know I mentioned the workaround for install-packages of just exiting, but to get gentoo support added I'd like to see if we can make emerge work too | 22:26 |
*** jayg is now known as jayg|g0n3 | 22:26 | |
greghaynes | prometheanfire: is there a reason it'd be especially hard for that to work? I would expect it to 'just work'... | 22:27 |
openstackgerrit | Ben Nemec proposed openstack/instack-undercloud: Remove option of installing tuskar https://review.openstack.org/258028 | 22:30 |
EmilienM | bnemec: you'll need to patch puppet-elements too | 22:34 |
EmilienM | bnemec: see https://review.openstack.org/263919 | 22:34 |
openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: toci_gate_test/overcloud/non-ha: enable monitoring https://review.openstack.org/254787 | 22:37 |
bnemec | EmilienM: Yeah, I don't mess with puppet-elements. I'll make a note to send in the request to remove the module though. | 22:37 |
EmilienM | cool | 22:38 |
openstackgerrit | Ben Nemec proposed openstack/instack-undercloud: Remove option of installing tuskar https://review.openstack.org/258028 | 22:38 |
* bnemec forgot to regenerate the config :-) | 22:38 | |
slagle | i'm just going to keep +2'ing | 22:38 |
*** dprince has quit IRC | 22:40 | |
bnemec | slagle: I'm all about helping your review stats ;-) | 22:42 |
bnemec | Although not as much as I helped shardy's when I made reviewstats start counting stable reviews properly. | 22:42 |
*** zaneb has quit IRC | 22:47 | |
openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Allow to enable Monitoring agent on Overcloud Controller(s) https://review.openstack.org/254788 | 22:58 |
*** zaneb has joined #tripleo | 23:02 | |
*** absubram has quit IRC | 23:02 | |
*** yuanying has joined #tripleo | 23:03 | |
*** rhallisey has joined #tripleo | 23:04 | |
*** chlong has joined #tripleo | 23:12 | |
*** dmacpher has joined #tripleo | 23:13 | |
*** chlong has quit IRC | 23:14 | |
*** morazi has quit IRC | 23:18 | |
openstackgerrit | Merged openstack/instack-undercloud: Switch default keystone auth_uri to v3 https://review.openstack.org/248500 | 23:25 |
*** jobewan is now known as jobewan_away | 23:29 | |
*** mbound has joined #tripleo | 23:29 | |
*** jobewan_away has quit IRC | 23:29 | |
*** shardy has quit IRC | 23:30 | |
*** penick has quit IRC | 23:30 | |
*** chlong has joined #tripleo | 23:30 | |
*** weshay has quit IRC | 23:31 | |
openstackgerrit | Merged openstack/instack-undercloud: Deploy Aodh services, replacing Ceilometer Alarm https://review.openstack.org/241412 | 23:32 |
*** mbound has quit IRC | 23:34 | |
*** eil397 has quit IRC | 23:58 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!