| *** ccrouch has joined #tripleo | 00:01 | |
| *** jerryz has joined #tripleo | 00:02 | |
| openstackgerrit | greghaynes proposed openstack/tripleo-image-elements: Support common venv for os-*-config https://review.openstack.org/148755 | 00:02 |
|---|---|---|
| openstackgerrit | greghaynes proposed openstack/tripleo-image-elements: Support common venv for os-*-config https://review.openstack.org/148755 | 00:02 |
| *** Marga_ has quit IRC | 00:03 | |
| *** Marga_ has joined #tripleo | 00:03 | |
| *** Marga_ has quit IRC | 00:03 | |
| *** Marga_ has joined #tripleo | 00:04 | |
| *** Marga_ has quit IRC | 00:06 | |
| *** Marga_ has joined #tripleo | 00:07 | |
| *** Marga_ has quit IRC | 00:07 | |
| *** Marga_ has joined #tripleo | 00:08 | |
| *** Marga_ has quit IRC | 00:09 | |
| *** Marga_ has joined #tripleo | 00:09 | |
| *** dsneddon has quit IRC | 00:10 | |
| *** Marga_ has quit IRC | 00:11 | |
| *** dsneddon has joined #tripleo | 00:11 | |
| *** mandre has joined #tripleo | 00:11 | |
| *** Marga_ has joined #tripleo | 00:12 | |
| *** Marga_ has quit IRC | 00:14 | |
| *** Marga_ has joined #tripleo | 00:14 | |
| *** Marga_ has quit IRC | 00:14 | |
| *** Marga_ has joined #tripleo | 00:15 | |
| *** Marga_ has quit IRC | 00:19 | |
| *** Marga_ has joined #tripleo | 00:20 | |
| *** penick has quit IRC | 00:20 | |
| *** Marga_ has quit IRC | 00:21 | |
| *** Marga_ has joined #tripleo | 00:21 | |
| *** Marga_ has quit IRC | 00:22 | |
| *** Marga_ has joined #tripleo | 00:22 | |
| *** Marga_ has quit IRC | 00:22 | |
| *** Marga_ has joined #tripleo | 00:23 | |
| *** dsneddon has quit IRC | 00:31 | |
| openstackgerrit | Martin André proposed stackforge/kolla: Check for nova DB availability in dependent containers, refs #11370 https://review.openstack.org/148762 | 00:32 |
| *** dsneddon has joined #tripleo | 00:33 | |
| openstackgerrit | Martin André proposed stackforge/kolla: Check for nova DB availability in dependent containers https://review.openstack.org/148762 | 00:34 |
| TheJulia | greghaynes: yes, and it will make me very sad when I go back and fix stuff that has been pointed out in reviews. Just focusing on getting it working, luckilly, I'm almost there. Just running into issues standing things back up. :( | 00:35 |
| *** pradeep has joined #tripleo | 00:39 | |
| openstackgerrit | greghaynes proposed openstack/diskimage-builder: Add installtype support to package-installs https://review.openstack.org/148766 | 00:42 |
| greghaynes | slagle: ^ you might be interested in reviewing this | 00:42 |
| openstackgerrit | greghaynes proposed openstack/diskimage-builder: Add installtype support to package-installs https://review.openstack.org/148766 | 00:43 |
| greghaynes | TheJulia: \O/ for it almost workintg | 00:44 |
| *** dsneddon has quit IRC | 00:52 | |
| TheJulia | The CI pass rate is like 39% right now :( | 00:53 |
| *** dsneddon has joined #tripleo | 00:53 | |
| greghaynes | yep, its absurd | 00:53 |
| TheJulia | and the couple that I've looked at seem to indicate that the seed just webt *poof* | 00:54 |
| TheJulia | s/webt/went/ | 00:54 |
| openstackgerrit | greghaynes proposed openstack/tripleo-incubator: Make wait_for use getopt and add walltime support https://review.openstack.org/145566 | 00:58 |
| *** mandre has quit IRC | 01:04 | |
| *** rlandy has quit IRC | 01:05 | |
| *** ruagair has quit IRC | 01:05 | |
| *** mandre has joined #tripleo | 01:10 | |
| *** dsneddon has quit IRC | 01:13 | |
| *** dsneddon has joined #tripleo | 01:13 | |
| *** noslzzp has quit IRC | 01:15 | |
| *** noslzzp has joined #tripleo | 01:17 | |
| *** lucas-dinner has quit IRC | 01:18 | |
| *** Marga_ has quit IRC | 01:18 | |
| *** Marga_ has joined #tripleo | 01:19 | |
| *** zigo_ has joined #tripleo | 01:21 | |
| *** zigo has quit IRC | 01:21 | |
| *** rwsu has quit IRC | 01:23 | |
| *** Marga_ has quit IRC | 01:23 | |
| *** thrash is now known as thrash|g0ne | 01:25 | |
| *** radez is now known as radez_g0n3 | 01:25 | |
| *** zigo has joined #tripleo | 01:27 | |
| *** zigo_ has quit IRC | 01:27 | |
| *** dsneddon has quit IRC | 01:33 | |
| *** dsneddon has joined #tripleo | 01:34 | |
| *** pradeep1 has joined #tripleo | 01:46 | |
| *** zigo has quit IRC | 01:46 | |
| *** zigo has joined #tripleo | 01:47 | |
| *** pradeep has quit IRC | 01:48 | |
| *** dsneddon has quit IRC | 01:54 | |
| *** dsneddon has joined #tripleo | 01:54 | |
| *** zigo has quit IRC | 02:00 | |
| *** zigo has joined #tripleo | 02:00 | |
| *** Yanyanhu has joined #tripleo | 02:02 | |
| *** ChuckC has quit IRC | 02:07 | |
| StevenK | lifeless: Did you see my comment on https://review.openstack.org/131635 ? | 02:09 |
| *** mandre has quit IRC | 02:10 | |
| lifeless | StevenK: lets assume not | 02:14 |
| lifeless | gertty is up to 4430 pending syncs :/ | 02:14 |
| *** dsneddon has quit IRC | 02:14 | |
| StevenK | lifeless: Short form: wait_for_hypervisor_stats is terrible, I'm not sure what to do for multiple flavors, and am pondering rewriting the whole thing in Python | 02:15 |
| *** dsneddon has joined #tripleo | 02:15 | |
| lifeless | StevenK: that might work better | 02:16 |
| StevenK | lifeless: I'm happy to sink my teeth into rewriting it, but I'm still unclear about multiple flavors. | 02:17 |
| *** julim has joined #tripleo | 02:17 | |
| lifeless | StevenK: I suspect it all happened while I was on leave | 02:17 |
| lifeless | StevenK: so I don't have any context for it | 02:17 |
| lifeless | there was some discussion at the midcycle | 02:17 |
| lifeless | that I recall | 02:17 |
| StevenK | lifeless: Right. That review adds support for multiple flavors -- but by default we will only create one. If an operator adds two flavors, one for compute nodes and one for controllers, what should the vcpus and memory be looking for in wait_for_hypervisor_stats ? | 02:19 |
| lifeless | 'enough to deploy' | 02:19 |
| StevenK | As it stands, the operator can pass the values in | 02:19 |
| StevenK | But I'm sure that (flavor + flavor) * nodes is wrong | 02:20 |
| lifeless | *blink* | 02:20 |
| * greghaynes wishes we just didnt need wait_for_hypervisor_stats | 02:20 | |
| StevenK | greghaynes: That would be nice. | 02:20 |
| greghaynes | and with heat retrying I dont really get why we do | 02:20 |
| lifeless | hypervisor_stats is a short circuit to answer the question 'are there enough hosts' | 02:20 |
| lifeless | I don't see that flavor has anything to do with it | 02:21 |
| greghaynes | because things like vcpu count | 02:21 |
| StevenK | lifeless: wait_for_hypervisor_stats also supports waiting for enough vcpus or ram | 02:21 |
| lifeless | thats new | 02:21 |
| lifeless | see, told you I don't have context :) | 02:21 |
| lifeless | greghaynes: vcpu on real hardware? | 02:22 |
| greghaynes | yea, im fine if its really simple but ideally we just defer to nova scheduler for checking resources like that IMO | 02:22 |
| lifeless | I bet its being used to wait for the deploy cloud being actually 'usable' | 02:22 |
| greghaynes | lifeless: unsure, would make sense though | 02:22 |
| lifeless | greghaynes: hypervisor stats is a call to the scheduler, more or less | 02:22 |
| StevenK | lifeless: Support for that landed in caef5804, September | 02:22 |
| StevenK | As it stands, nothing in devtests passes in vcpus or mem, but I'm sure slagle had a plan when he landed it | 02:24 |
| greghaynes | heh | 02:24 |
| *** mandre has joined #tripleo | 02:25 | |
| *** Marga_ has joined #tripleo | 02:25 | |
| lifeless | so as greghaynes says | 02:26 |
| lifeless | the goal was 'don't fail to deploy' | 02:26 |
| lifeless | you could try using heat to deploy and once heat is doing its thing adding nodes one at a time and seeing if that works | 02:26 |
| lifeless | if so we could stop using it | 02:26 |
| lifeless | in principle | 02:26 |
| lifeless | but say you have 100 nodes | 02:26 |
| lifeless | mmm nvm | 02:26 |
| lifeless | registration isn't a sign of usability yet | 02:26 |
| *** pradeep has joined #tripleo | 02:31 | |
| greghaynes | SpamapS: maybe you can shed some light on this as our heat guru | 02:32 |
| *** pradeep1 has quit IRC | 02:32 | |
| greghaynes | AIUI with the heat retry stuff heat would just keep trying to create the nodes until the nodes are registered / available | 02:32 |
| greghaynes | which solves the fail to deploy problem that hypevisor stats exists for | 02:33 |
| *** dsneddon has quit IRC | 02:35 | |
| *** dsneddon has joined #tripleo | 02:36 | |
| *** Marga_ has quit IRC | 02:38 | |
| *** Marga_ has joined #tripleo | 02:39 | |
| *** pradeep1 has joined #tripleo | 02:43 | |
| *** pradeep has quit IRC | 02:45 | |
| *** yamahata has quit IRC | 02:46 | |
| *** ChuckC has joined #tripleo | 02:56 | |
| *** dsneddon has quit IRC | 02:56 | |
| *** dsneddon has joined #tripleo | 02:57 | |
| *** avozza is now known as zz_avozza | 03:04 | |
| *** noslzzp has quit IRC | 03:05 | |
| *** noslzzp has joined #tripleo | 03:07 | |
| *** ccrouch has quit IRC | 03:13 | |
| *** kbyrne has quit IRC | 03:16 | |
| *** pradeep has joined #tripleo | 03:17 | |
| *** ccrouch has joined #tripleo | 03:18 | |
| *** pradeep1 has quit IRC | 03:19 | |
| *** pradeep has quit IRC | 03:26 | |
| *** tchaypo has quit IRC | 03:30 | |
| *** ramishra_ has quit IRC | 03:30 | |
| greghaynes | TheJulia: The seed going *poof* fail you were seeing - did that happen on f20 job? | 03:37 |
| *** dsneddon has quit IRC | 03:37 | |
| greghaynes | Looke like theres whatever bug is causing that, and then also for some reason we fail to grab logs on the seed without any clear error in http://logs.openstack.org/66/145566/21/check-tripleo/check-tripleo-ironic-overcloud-f20-nonha/7d6af06/console.html#_2015-01-21_03_22_56_843 | 03:38 |
| *** dsneddon has joined #tripleo | 03:39 | |
| greghaynes | and http://logs.openstack.org/66/145566/21/check-tripleo/check-tripleo-ironic-overcloud-f20-nonha/7d6af06/logs/get_state_from_host.txt.gz | 03:39 |
| *** ramishra_ has joined #tripleo | 03:44 | |
| *** tchaypo has joined #tripleo | 03:45 | |
| *** masco has joined #tripleo | 03:46 | |
| *** tzumainn has quit IRC | 03:48 | |
| *** weshay has quit IRC | 03:50 | |
| *** ramishra_ has quit IRC | 03:51 | |
| *** tchaypo has quit IRC | 03:51 | |
| *** dsneddon has quit IRC | 03:58 | |
| *** dsneddon has joined #tripleo | 04:00 | |
| *** untriaged-bot has joined #tripleo | 04:00 | |
| untriaged-bot | Untriaged bugs so far: | 04:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1408818 | 04:00 |
| untriaged-bot | https://bugs.launchpad.net/tuskar/+bug/1387487 | 04:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1411497 | 04:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1409583 | 04:00 |
| *** untriaged-bot has quit IRC | 04:00 | |
| *** pradeep has joined #tripleo | 04:03 | |
| *** ramishra_ has joined #tripleo | 04:08 | |
| *** tchaypo has joined #tripleo | 04:12 | |
| *** pradeep has quit IRC | 04:16 | |
| *** dsneddon has quit IRC | 04:19 | |
| *** dsneddon has joined #tripleo | 04:20 | |
| *** yuanying_ has joined #tripleo | 04:37 | |
| *** masco has quit IRC | 04:38 | |
| *** dsneddon has quit IRC | 04:40 | |
| *** yuanying has quit IRC | 04:41 | |
| *** dsneddon has joined #tripleo | 04:42 | |
| *** pradeep has joined #tripleo | 04:44 | |
| *** yuanying_ has quit IRC | 04:48 | |
| *** yuanying has joined #tripleo | 04:48 | |
| *** yuanying has quit IRC | 04:58 | |
| *** yuanying has joined #tripleo | 04:59 | |
| *** dsneddon has quit IRC | 05:01 | |
| *** dsneddon has joined #tripleo | 05:02 | |
| *** pensu has joined #tripleo | 05:03 | |
| *** kbyrne has joined #tripleo | 05:12 | |
| *** yuanying has quit IRC | 05:17 | |
| *** yuanying has joined #tripleo | 05:19 | |
| *** dsneddon has quit IRC | 05:22 | |
| *** yuanying has quit IRC | 05:23 | |
| *** dsneddon has joined #tripleo | 05:24 | |
| *** Marga_ has quit IRC | 05:32 | |
| *** yuanying has joined #tripleo | 05:39 | |
| *** dsneddon has quit IRC | 05:43 | |
| *** dsneddon has joined #tripleo | 05:44 | |
| *** morazi has quit IRC | 05:49 | |
| *** dsneddon has quit IRC | 06:04 | |
| *** dsneddon has joined #tripleo | 06:06 | |
| *** ishant has joined #tripleo | 06:15 | |
| *** masco has joined #tripleo | 06:19 | |
| greghaynes | we have meeting tonight, right? | 06:22 |
| *** ukalifon1 has joined #tripleo | 06:26 | |
| *** pradeep has quit IRC | 06:30 | |
| greghaynes | tchaypo: ^ | 06:32 |
| greghaynes | oh, youre in EU | 06:32 |
| greghaynes | I sincerely hope the irony in this commit message was not missed https://review.openstack.org/#/c/147594/2 | 06:34 |
| *** pcrews has quit IRC | 06:42 | |
| *** lazy_prince has quit IRC | 06:43 | |
| *** mrunge has joined #tripleo | 06:44 | |
| *** dsneddon has quit IRC | 06:46 | |
| *** dsneddon has joined #tripleo | 06:47 | |
| SpamapS | greghaynes: it's not entirely ironic. One point of Delorean is to be able to use packages from a point in time. :) | 06:49 |
| greghaynes | SpamapS: Have you read about how coreos does os upgrades? | 06:50 |
| greghaynes | specifically the dual root partitions, write out the passive partition while still running then reboot into passive | 06:51 |
| *** eghobo has joined #tripleo | 06:53 | |
| greghaynes | doing it that way has a ton of benefits.... | 06:55 |
| *** mrunge has quit IRC | 07:01 | |
| *** mrunge has joined #tripleo | 07:06 | |
| *** dsneddon has quit IRC | 07:07 | |
| *** dsneddon has joined #tripleo | 07:09 | |
| *** ukalifon1 has quit IRC | 07:17 | |
| *** ukalifon1 has joined #tripleo | 07:18 | |
| *** ukalifon3 has joined #tripleo | 07:19 | |
| *** ukalifon1 has quit IRC | 07:22 | |
| *** yamahata has joined #tripleo | 07:24 | |
| *** zz_avozza is now known as avozza | 07:28 | |
| *** dsneddon has quit IRC | 07:28 | |
| *** dsneddon has joined #tripleo | 07:29 | |
| *** dshulyak_ has joined #tripleo | 07:31 | |
| *** avozza is now known as zz_avozza | 07:37 | |
| *** jtomasek has joined #tripleo | 07:37 | |
| *** zz_avozza is now known as avozza | 07:39 | |
| *** rdopiera has joined #tripleo | 07:40 | |
| tchaypo | greghaynes: let me consult the calendar | 07:41 |
| tchaypo | Yeah, on my calendar we have a meeting in ~15 minutes | 07:43 |
| tchaypo | on the openstack meetings ical feed it’s in 24 hours and 15 minutes | 07:43 |
| tchaypo | and there go all my things bingling to tell me it’s almost time for the meeting | 07:45 |
| *** chlong has quit IRC | 07:46 | |
| *** jprovazn has joined #tripleo | 07:46 | |
| *** rdopiera has joined #tripleo | 07:46 | |
| *** tchaypo1 has joined #tripleo | 07:47 | |
| *** tchaypo is now known as tchaypo_ | 07:47 | |
| *** tchaypo1 is now known as tchaypo | 07:47 | |
| tchaypo_ | so stuff I say here | 07:48 |
| tchaypo | does actually make it through to here | 07:48 |
| tchaypo_ | it just doesn’t show up here. that’s helpful. | 07:48 |
| marios | wat | 07:48 |
| rdopiera | splitbrain | 07:49 |
| *** dsneddon has quit IRC | 07:49 | |
| *** avozza is now known as zz_avozza | 07:49 | |
| *** dsneddon has joined #tripleo | 07:50 | |
| tchaypo_ | it’s awesome | 07:50 |
| tchaypo | okay, I'm just going to have to give up on irccloud for now | 07:50 |
| tchaypo | at least I still know how to use a real irc client | 07:50 |
| * SpamapS is awake at alt meeting time for once | 07:52 | |
| *** tzumainn has joined #tripleo | 07:53 | |
| marios | SpamapS: o/ is it 'sofa meeting' late or 'in bed meeting' late | 07:54 |
| SpamapS | marios: it's "deadline keeping me awake" late :) | 07:54 |
| SpamapS | it's serious enough I turned off redshift | 07:54 |
| SpamapS | which means my eyes think it is 3pm :-P | 07:55 |
| greghaynes | wow | 07:55 |
| tchaypo | for me it's "but i haven't even had coffee" meeting | 07:55 |
| SpamapS | Another serious thing is that devtest is so unreliable behind the corp firewall (even with a devpi) that I"m thinking about testing some of my patches with... <shock> devstack. | 07:56 |
| SpamapS | I had 2 cups of coffee 2 hours ago | 07:56 |
| SpamapS | should keep me up till 4am | 07:56 |
| SpamapS | :-P | 07:56 |
| tchaypo | SpamapS: what makes it unreliable? | 07:56 |
| SpamapS | tchaypo: so many fails | 07:56 |
| tchaypo | SpamapS: are you awake enough to run the meeting? | 07:56 |
| SpamapS | tchaypo: yes | 07:56 |
| tchaypo | SpamapS: downloading things? | 07:56 |
| tchaypo | SpamapS: yay :) | 07:56 |
| SpamapS | tchaypo: yeah d/l things is the chief problem | 07:56 |
| greghaynes | well, devtest isnt very reliable upstream right now | 07:56 |
| SpamapS | that too | 07:57 |
| StevenK | SpamapS: What about a distro mirror? | 07:57 |
| SpamapS | StevenK: have one | 07:57 |
| SpamapS | no problems with packages | 07:57 |
| StevenK | Oh, so it's git, then? | 07:57 |
| tchaypo | so it's mainly pythons? | 07:57 |
| SpamapS | Some of it is also just random pip fails as usual. | 07:57 |
| StevenK | I did ask Richard Jones why we can't have nice things at LCA | 07:58 |
| greghaynes | hrm, our pass rate is actually < 50% not including ha job | 07:58 |
| tchaypo | ffs | 07:58 |
| tchaypo | StevenK: BadRequest: Invalid input for operation: physical_network 'datacentre' unknown for VLAN provider network. | 07:58 |
| SpamapS | we still have some broken error handling btw | 07:58 |
| SpamapS | ctrl-c in dib results in stale temp mounts a lot | 07:58 |
| StevenK | tchaypo: I think I'm probably undistracted enough to look into that properly | 07:58 |
| StevenK | SpamapS: YES | 07:59 |
| StevenK | We need a bug for that | 07:59 |
| tchaypo | StevenK: cool. That error is in /home/shared/build_out_2015-01-20-11\:14 on the hp2 bastion. I've been using /home/shared/bin/build to try to build everything from the ground up | 08:00 |
| greghaynes | bahahaha, just ran this on my devtest box: mount | grep image | wc -l | 08:00 |
| greghaynes | result: 89 | 08:00 |
| SpamapS | my openstack meetings calendar says 24 hours from now | 08:00 |
| SpamapS | greghaynes: ew, it's worse than I thought then | 08:01 |
| tchaypo | SpamapS: yeah, that's what's in the ical feed | 08:01 |
| tchaypo | i don't know where that came from | 08:01 |
| SpamapS | ok so I don't want to run the meeting 24 hours early | 08:01 |
| tchaypo | https://wiki.openstack.org/wiki/Meetings/TripleO says MEETING TIME: Tuesdays at 19:00 UTC or Wednesday at 08:00UTC (alternating weeks) in #openstack-meeting-alt. | 08:02 |
| StevenK | steven@vanquished:~% date -u | 08:02 |
| StevenK | Wed Jan 21 08:02:07 UTC 2015 | 08:02 |
| marios | it should be now afaik | 08:02 |
| SpamapS | so the ical feed is wrong? | 08:02 |
| shardy | SpamapS: I started looking at launching tripleo templates on ironic enabled devstack a while back, it would be cool if we could get that working IMO | 08:02 |
| SpamapS | shardy: shouldn't be any reason that wouldn't work. | 08:02 |
| shardy | I got a bit stuck on the networking but I didn't spend a lot of time on it tbh | 08:02 |
| SpamapS | shardy: thats what we did before we had the seed. | 08:02 |
| shardy | SpamapS: cool, I added a tuskar option to devstack, having a tripleo one which sets things up in a convenient way for launching tripleo templates would be nice | 08:03 |
| StevenK | tchaypo: Except that the hp2 bastion does not love my key | 08:03 |
| SpamapS | Ok since we've advertised it as now everywhere except ical, I think we can just have it now. | 08:04 |
| tchaypo | StevenK: looging in as stack ? | 08:04 |
| shardy | phew, for a moment there I thought I started work without coffee for no reason ;) | 08:04 |
| tchaypo | StevenK: how would I find out what physical_networks are known for the VLAN provider network? | 08:04 |
| greghaynes | to the meetingtorium! | 08:04 |
| *** tzumainn has quit IRC | 08:07 | |
| *** pblaho has joined #tripleo | 08:08 | |
| *** dsneddon has quit IRC | 08:09 | |
| *** dsneddon has joined #tripleo | 08:11 | |
| *** pradeep has joined #tripleo | 08:13 | |
| openstackgerrit | Merged openstack/tripleo-incubator: Remove network config hardcode for overcloud fixed range https://review.openstack.org/140927 | 08:14 |
| tchaypo | https://review.openstack.org/#/c/140870/1 has two +2s and is passing tests | 08:20 |
| tchaypo | and I've reviewed it and I'm about to give a +2.. so I think I should just add the +A as well? | 08:21 |
| greghaynes | clearly it hasnt been in our CI in a while then ;) | 08:21 |
| greghaynes | tchaypo: yea, generally if its just not +A'd due to CI then go for it | 08:21 |
| *** zz_avozza is now known as avozza | 08:27 | |
| *** ifarkas has joined #tripleo | 08:29 | |
| *** mandre has quit IRC | 08:30 | |
| *** dsneddon has quit IRC | 08:30 | |
| *** yamahata has quit IRC | 08:31 | |
| *** dsneddon has joined #tripleo | 08:31 | |
| *** avozza is now known as zz_avozza | 08:37 | |
| tchaypo | it looks like StevenK has left the keyboard | 08:39 |
| StevenK | I was putting on some rice to cook, yes | 08:44 |
| tchaypo | ah, welcome back | 08:44 |
| tchaypo | so do you know how I can go find the known phsyical_networks? | 08:45 |
| *** jcoufal has joined #tripleo | 08:45 | |
| StevenK | tchaypo: I don't -- all I really did is rebase Rob's patch. If you give me a clue how to get the tools into my path, and where the code and rc files are, I was going to stare at code and have a poke at the seed tomorrow morning. | 08:46 |
| tchaypo | are you able to get onto the bastion as stack ? | 08:47 |
| StevenK | tchaypo: Yessum | 08:47 |
| tchaypo | excellent | 08:47 |
| StevenK | tchaypo: 'w' would have told that :-) | 08:47 |
| tchaypo | if you "tmux attach" you can see the output from the run I'm currently doing.. which if I've fixed build.sh, should abort as soon as devtest_seed.sh fails | 08:48 |
| tchaypo | everything I've been doing is encapsulated in /home/shared/bin/build | 08:48 |
| *** dsneddon has quit IRC | 08:51 | |
| *** dsneddon has joined #tripleo | 08:52 | |
| *** gfidente has joined #tripleo | 08:53 | |
| *** gfidente has quit IRC | 08:53 | |
| *** gfidente has joined #tripleo | 08:53 | |
| tchaypo | I've set tmux to use ^a just like screen, so you can ^a-c and ^a-n and so on as though it were screen | 08:54 |
| openstackgerrit | Merged openstack/diskimage-builder: Fallback to a boot_server kernel param if ip= not passed https://review.openstack.org/140870 | 08:56 |
| StevenK | tchaypo: Won't that just confuse me? :-P | 08:57 |
| *** jcoufal_ has joined #tripleo | 08:58 | |
| openstackgerrit | Merged openstack/diskimage-builder: Add install section to Ironic agent systemd service file https://review.openstack.org/142157 | 08:58 |
| *** jcoufal has quit IRC | 09:01 | |
| tchaypo | StevenK: there we go, we have an error! | 09:02 |
| *** dshulyak_ has quit IRC | 09:02 | |
| *** jang2 has joined #tripleo | 09:02 | |
| *** zz_avozza is now known as avozza | 09:03 | |
| *** pblaho_ has joined #tripleo | 09:03 | |
| StevenK | tchaypo: Huh, hitting ^a and stuff seems to have changed the file somehow? | 09:04 |
| *** pblaho has quit IRC | 09:04 | |
| tchaypo | changed the file? | 09:05 |
| StevenK | tchaypo: I ran tmux at, and it seems to be in vi or something? | 09:06 |
| tchaypo | huh | 09:08 |
| tchaypo | oh goodness | 09:08 |
| tchaypo | ^a : | 09:08 |
| tchaypo | then type "select-tree" and hit enter | 09:09 |
| tchaypo | actually, I think I just kille the vi session so you probably had tmux exiting.. if you attach again you should get to the right session | 09:09 |
| *** jistr has joined #tripleo | 09:09 | |
| *** jp_at_hp has joined #tripleo | 09:09 | |
| tchaypo | you have a tiny little screen | 09:10 |
| *** derekh has joined #tripleo | 09:10 | |
| *** dsneddon has quit IRC | 09:12 | |
| *** dsneddon has joined #tripleo | 09:14 | |
| *** mrunge has quit IRC | 09:20 | |
| *** mrunge has joined #tripleo | 09:23 | |
| *** mcornea has joined #tripleo | 09:23 | |
| *** pblaho__ has joined #tripleo | 09:24 | |
| *** bvandenh has joined #tripleo | 09:25 | |
| *** lucasagomes has joined #tripleo | 09:25 | |
| *** pblaho_ has quit IRC | 09:28 | |
| *** eghobo has quit IRC | 09:41 | |
| *** Marga_ has joined #tripleo | 09:43 | |
| *** pblaho has joined #tripleo | 09:50 | |
| *** pblaho__ has quit IRC | 09:50 | |
| *** jistr has quit IRC | 09:50 | |
| *** mrunge has quit IRC | 09:53 | |
| *** Yanyanhu has quit IRC | 09:53 | |
| *** mrunge has joined #tripleo | 09:55 | |
| *** jistr has joined #tripleo | 09:56 | |
| *** untriaged-bot has joined #tripleo | 10:00 | |
| untriaged-bot | Untriaged bugs so far: | 10:00 |
| untriaged-bot | https://bugs.launchpad.net/tuskar/+bug/1387487 | 10:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1411497 | 10:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1409583 | 10:00 |
| *** untriaged-bot has quit IRC | 10:00 | |
| openstackgerrit | Jiri Tomasek proposed openstack/tuskar-ui: Add Roles rest api https://review.openstack.org/148865 | 10:06 |
| *** pelix has joined #tripleo | 10:11 | |
| *** mkerrin has joined #tripleo | 10:12 | |
| *** dmellado has quit IRC | 10:13 | |
| openstackgerrit | Steven Hardy proposed openstack/os-net-config: Add mapping option to influence nicN mapping order https://review.openstack.org/147619 | 10:14 |
| *** pradeep has quit IRC | 10:27 | |
| openstackgerrit | Michael Kerrin proposed openstack/tripleo-image-elements: Store collectl logs in /mnt/state/var/log/collectl https://review.openstack.org/148868 | 10:28 |
| *** michchap_ has quit IRC | 10:45 | |
| *** michchap has joined #tripleo | 10:45 | |
| *** Marga_ has quit IRC | 10:58 | |
| openstackgerrit | Radomir Dopieralski proposed openstack/tuskar-ui: Added tests form tuskar_ui.infrastructure.flavors.utils https://review.openstack.org/148880 | 11:04 |
| *** akrivoka has joined #tripleo | 11:08 | |
| *** pensu has quit IRC | 11:22 | |
| *** pradeep has joined #tripleo | 11:27 | |
| *** dsneddon has quit IRC | 11:28 | |
| *** dsneddon has joined #tripleo | 11:29 | |
| *** dsneddon has quit IRC | 11:49 | |
| *** dsneddon has joined #tripleo | 11:50 | |
| *** tchaypo has quit IRC | 11:55 | |
| *** tchaypo_ is now known as tchaypo | 11:55 | |
| *** kbyrne has quit IRC | 11:58 | |
| *** kbyrne has joined #tripleo | 11:59 | |
| *** chlong has joined #tripleo | 12:02 | |
| *** dsneddon has quit IRC | 12:10 | |
| *** dsneddon has joined #tripleo | 12:11 | |
| openstackgerrit | Giulio Fidente proposed openstack/diskimage-builder: Switch manage_etc_hosts from True to localhost https://review.openstack.org/148668 | 12:24 |
| openstackgerrit | Radomir Dopieralski proposed openstack/tuskar-ui: Add tests for tuskar_ui.infrastructure.utils.utils https://review.openstack.org/148909 | 12:30 |
| *** dsneddon has quit IRC | 12:31 | |
| *** dsneddon has joined #tripleo | 12:33 | |
| *** masco has quit IRC | 12:35 | |
| *** ishant has quit IRC | 12:38 | |
| *** lucasagomes is now known as lucas-hungry | 12:42 | |
| *** rhallisey has joined #tripleo | 12:52 | |
| *** dsneddon has quit IRC | 12:53 | |
| *** dsneddon has joined #tripleo | 12:53 | |
| *** shadower has joined #tripleo | 12:55 | |
| *** thrash|g0ne is now known as thrash | 12:56 | |
| *** morazi has joined #tripleo | 12:59 | |
| *** jdob has joined #tripleo | 12:59 | |
| *** dprince has joined #tripleo | 13:13 | |
| *** weshay has joined #tripleo | 13:17 | |
| openstackgerrit | Radomir Dopieralski proposed openstack/tuskar-ui: Add tests for tuskar_ui.infrastructure.utils.utils https://review.openstack.org/148909 | 13:31 |
| *** killer_prince has joined #tripleo | 13:34 | |
| *** killer_prince is now known as lazy_prince | 13:34 | |
| *** tzumainn has joined #tripleo | 13:35 | |
| openstackgerrit | Andrea Frittoli proposed openstack/tripleo-image-elements: Do not override tempest.conf from pass-through https://review.openstack.org/148923 | 13:38 |
| *** dprince has quit IRC | 13:39 | |
| *** dprince has joined #tripleo | 13:40 | |
| *** lucas-hungry is now known as lucasagomes | 13:43 | |
| *** akrivoka has quit IRC | 13:44 | |
| *** akrivoka has joined #tripleo | 13:45 | |
| *** spzala has quit IRC | 13:46 | |
| openstackgerrit | Radomir Dopieralski proposed openstack/tuskar-ui: Tests for the plan validation https://review.openstack.org/148927 | 13:49 |
| *** dsneddon has quit IRC | 13:55 | |
| tchaypo | fungi has just approved 126513 so we should start seeing jobs back in hp1 now | 13:56 |
| * tchaypo is keeping an eye out for sadness | 13:56 | |
| *** dsneddon has joined #tripleo | 13:56 | |
| *** radez_g0n3 is now known as radez | 14:00 | |
| *** pradeep has quit IRC | 14:06 | |
| *** athomas has quit IRC | 14:12 | |
| *** dsneddon has quit IRC | 14:15 | |
| *** dsneddon has joined #tripleo | 14:17 | |
| *** jprovazn has quit IRC | 14:17 | |
| *** athomas has joined #tripleo | 14:18 | |
| *** pradeep has joined #tripleo | 14:26 | |
| *** pradeep1 has joined #tripleo | 14:30 | |
| *** pblaho_ has joined #tripleo | 14:30 | |
| *** pradeep has quit IRC | 14:30 | |
| *** pblaho has quit IRC | 14:31 | |
| *** Marga_ has joined #tripleo | 14:36 | |
| *** lblanchard has joined #tripleo | 14:36 | |
| *** dsneddon has quit IRC | 14:37 | |
| *** dprince has quit IRC | 14:37 | |
| *** dprince has joined #tripleo | 14:37 | |
| *** dsneddon has joined #tripleo | 14:38 | |
| *** pradeep has joined #tripleo | 14:39 | |
| *** jprovazn has joined #tripleo | 14:39 | |
| *** pradeep1 has quit IRC | 14:41 | |
| *** pradeep has quit IRC | 14:44 | |
| *** pradeep has joined #tripleo | 14:45 | |
| *** masco has joined #tripleo | 14:50 | |
| *** yamahata has joined #tripleo | 14:50 | |
| dprince | shardy: I've written a small book/reply here: https://review.openstack.org/#/c/147287/ | 14:56 |
| dprince | shardy: the primary thing we should agree on for this patchset is whether we are keeping controller things together, or splitting them out | 14:57 |
| *** rdopiera has quit IRC | 14:57 | |
| *** dsneddon has quit IRC | 14:57 | |
| *** masco has quit IRC | 14:58 | |
| *** dsneddon has joined #tripleo | 14:59 | |
| *** weshay is now known as weshay_mtg | 14:59 | |
| *** spzala has joined #tripleo | 15:00 | |
| *** mcornea has quit IRC | 15:01 | |
| shardy | dprince: Hi, thanks - I actually forgot I was replying to that earlier and never submitted the comment | 15:04 |
| shardy | dprince: Getting something which works then iterating on it is fine by me, switched to +1 | 15:04 |
| dprince | shardy: awesome, perhaps we can write a small book on this one ;) | 15:04 |
| dprince | shardy: cool. will look again | 15:05 |
| shardy | I'm still unsure of the heira input consolidation, but I don't speak puppet so I'll take your word for it if you say it makes things easier ;) | 15:05 |
| *** pradeep1 has joined #tripleo | 15:05 | |
| dprince | shardy: My stance (currently) is that splitting out software configs into separate files works great until you need to do something like create other software deployments (attached to the same server) inside of the split out config | 15:06 |
| *** pradeep has quit IRC | 15:07 | |
| dprince | shardy: we solve one of the issues by passing in the server, but that seemed to cause subtle ordering issues with regards to the signals. | 15:07 |
| dprince | shardy: at least it did for me w/ puppet, hiera, etc. | 15:07 |
| dprince | shardy: and doing it the way we've got it now isn't that much more verbose anyways. I'm happy with it for now I think. | 15:08 |
| dprince | shardy: I've made notes about some of your stylistic comments and will follow up with patches to address those | 15:08 |
| shardy | dprince: Ok, it would be good, in due course, to better understand those ordering issues, as in theory, you should be able to pass in the server ID and it should work exactly the same | 15:10 |
| shardy | The mistake before was leaving the deployment in a different stack to the config | 15:11 |
| shardy | but absolutely, whatever works at this point :) | 15:11 |
| *** mcornea has joined #tripleo | 15:11 | |
| dprince | shardy: when I moved them both I got an error about the server_id (resource) not existing when the nested stacks deployment resource was trying to create.... | 15:11 |
| dprince | shardy: tried, using dependencies. no help | 15:12 |
| shardy | dprince: isn't that the validation error I fixed recently? | 15:12 |
| dprince | shardy: it just seemed like it worked best if the server was created *in* the template where its deployment resources live | 15:12 |
| shardy | dprince: yup, cool, I just want to be sure if there's a bug that I fix it, as in theory it should work | 15:13 |
| dprince | shardy: not sure about the bug. do you have a link? | 15:13 |
| shardy | https://review.openstack.org/#/c/144766/ | 15:14 |
| shardy | dprince: ^^ | 15:14 |
| *** masco has joined #tripleo | 15:14 | |
| shardy | without that, validation would've failed saying the server didn't exist, before even trying to create the stack | 15:14 |
| shardy | I'm fine with moving the Server resource though, would just be good to confirm that was the bug vs some other issue | 15:15 |
| *** pblaho__ has joined #tripleo | 15:17 | |
| *** pblaho__ has quit IRC | 15:17 | |
| *** pblaho__ has joined #tripleo | 15:18 | |
| *** pblaho__ is now known as pblaho | 15:18 | |
| *** pradeep1 has quit IRC | 15:18 | |
| *** dsneddon has quit IRC | 15:18 | |
| *** pblaho has quit IRC | 15:18 | |
| *** pradeep has joined #tripleo | 15:19 | |
| *** dsneddon has joined #tripleo | 15:20 | |
| openstackgerrit | Alexis Lee proposed openstack/os-apply-config: Add --raw convenience https://review.openstack.org/148505 | 15:20 |
| *** pblaho_ has quit IRC | 15:21 | |
| openstackgerrit | Alexis Lee proposed openstack/os-apply-config: Add --raw convenience https://review.openstack.org/148505 | 15:22 |
| dprince | shardy: I don't think that fixed this issue for me. | 15:22 |
| dprince | shardy: it is probably worth trying to reproduce this again | 15:23 |
| *** pradeep1 has joined #tripleo | 15:23 | |
| *** pradeep has quit IRC | 15:24 | |
| TheJulia | tchaypo: by chance have you seen any keepalived weirdness recently? | 15:28 |
| *** pradeep1 has quit IRC | 15:28 | |
| tchaypo | I’m not sure that I’d ever notice any | 15:28 |
| tchaypo | urr | 15:28 |
| tchaypo | what i mean is - I don’t think I’ve ever knowingly noticed any | 15:29 |
| TheJulia | okay | 15:29 |
| *** pradk has joined #tripleo | 15:33 | |
| TheJulia | tchaypo: I'm basically seeing the vip on multiple controllers again but after doing a rebuild sequence with tripleo-ansible which is resulting in my cluster being... confused and grumpy | 15:35 |
| tchaypo | the only place I’d be likely to see that would be on hp2 | 15:35 |
| tchaypo | but at the moment I can’t even complete building the seed there | 15:36 |
| TheJulia | thats not good :( | 15:37 |
| *** dprince has quit IRC | 15:38 | |
| *** dprince has joined #tripleo | 15:39 | |
| *** dsneddon has quit IRC | 15:39 | |
| *** ukalifon3 has quit IRC | 15:41 | |
| *** dsneddon has joined #tripleo | 15:41 | |
| *** pblaho has joined #tripleo | 15:43 | |
| jpeeler | dprince: i'm to the point of the same error again, haven't had a chance to debug any: http://fpaste.org/172531/21854088/ | 15:46 |
| *** pcrews has joined #tripleo | 15:46 | |
| dprince | jpeeler: can you shell into one of your overcloud nodes and view the os-collect-config logs? | 15:58 |
| *** untriaged-bot has joined #tripleo | 16:00 | |
| untriaged-bot | Untriaged bugs so far: | 16:00 |
| untriaged-bot | https://bugs.launchpad.net/tuskar/+bug/1387487 | 16:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1411497 | 16:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1409583 | 16:00 |
| *** untriaged-bot has quit IRC | 16:00 | |
| *** dsneddon has quit IRC | 16:00 | |
| openstackgerrit | Merged openstack/tuskar-ui: header is changed to updating https://review.openstack.org/145152 | 16:02 |
| *** dsneddon has joined #tripleo | 16:03 | |
| *** masco has quit IRC | 16:05 | |
| *** spzala has quit IRC | 16:05 | |
| *** spzala has joined #tripleo | 16:05 | |
| *** mrunge has quit IRC | 16:06 | |
| *** jprovazn is now known as jprovazn_bbl | 16:15 | |
| *** Marga_ has quit IRC | 16:27 | |
| SpamapS | hm | 16:34 |
| SpamapS | | PublicVirtualIP | d52a1a29-b8a3-4b3c-bd85-8498df2a54af | NeutronClientException: Unable to find network with name 'public' | CREATE_FAILED | 2015-01-21T09:06:30Z | | 16:35 |
| *** dtantsur is now known as dtantsur|afk | 16:35 | |
| SpamapS | do we log the output of init-neutron? | 16:35 |
| SpamapS | s/init/setup/ | 16:38 |
| SpamapS | hrm | 16:38 |
| SpamapS | ValueError: metadata_server is required for physical networks | 16:41 |
| *** pradk has quit IRC | 16:41 | |
| *** dsneddon has quit IRC | 16:42 | |
| *** dsneddon has joined #tripleo | 16:43 | |
| * SpamapS tosses that red herring back in the sea | 16:44 | |
| *** jp_at_hp has quit IRC | 16:45 | |
| *** jang2 has quit IRC | 16:47 | |
| *** leseb_ has joined #tripleo | 16:49 | |
| gfidente | hi leseb_ , nice to see you around here :) | 16:49 |
| leseb_ | gfidente: hey | 16:49 |
| gfidente | so wanted to ask about this | 16:49 |
| gfidente | https://review.openstack.org/#/c/147135/2/elements/cinder-volume-ceph/os-refresh-config/post-configure.d/70-cinder-volume-ceph,cm | 16:49 |
| gfidente | I think I miss something about one of your comments, the key caps? what is wrong with those? | 16:50 |
| leseb_ | gfidente: ok, several questions | 16:51 |
| leseb_ | why do you use the 'rbd' default pool? best practice wants to use a dedicated pool for each openstack component | 16:51 |
| *** ChuckC_ has joined #tripleo | 16:51 | |
| *** ChuckC has quit IRC | 16:51 | |
| gfidente | leseb_, yeah I read that, you think I should create a new pool for the purpose instad of using 'rbd'? | 16:52 |
| gfidente | is there an actual benefit or is it just to not use the default one? | 16:53 |
| leseb_ | gfidente: definitely, because in ceph you can pin a pool to a specific rules that points to a specific set of hardware | 16:53 |
| SpamapS | weird | 16:53 |
| * SpamapS notices PublicVirtualNetwork is getting forced to 'public' in his overcloud-env.json but nowhere else. | 16:53 | |
| gfidente | leseb_, in which case I should change the caps to be applied to the $custom pool rather than rbd | 16:53 |
| leseb_ | gfidente: yup use a dedicated pool for cinder, glance and nova | 16:54 |
| gfidente | we only need one for cinder ATM as we only cover Cinder for now with Ceph; the only problem I have with creation of a new pool is that these scripts are potentially going to be executed multiple times | 16:54 |
| leseb_ | then you need to change the caps to also allow operations on other pool | 16:54 |
| gfidente | on which other pool? | 16:55 |
| leseb_ | glance and nova | 16:55 |
| gfidente | we don't have those | 16:55 |
| gfidente | I mean, we don't configure them with Ceph at all | 16:55 |
| leseb_ | well you're already managing the idempotence with shell tests so you can continue to do so to test if the pool exists or not | 16:55 |
| gfidente | so it's just one pool for Cinder | 16:55 |
| gfidente | yeah it doesn't look great but we can create it and manage idempotence yes | 16:56 |
| leseb_ | so VMs use a qcow2 disk and glance uses a fs? | 16:56 |
| gfidente | yeah so is it for now | 16:56 |
| SpamapS | oh duh devtest_overcloud.sh is using my old overcloud-env.json. | 16:56 |
| leseb_ | hum ok step by step process then | 16:56 |
| gfidente | yeah | 16:57 |
| gfidente | so I need a way to create a new pool and check if it doesn't exist already | 16:58 |
| gfidente | knowing that this could start almost concurrently from all the three controllers | 16:58 |
| leseb_ | yup | 16:59 |
| gfidente | I suspect I can let -mon taking care of the concurrency? | 16:59 |
| leseb_ | well you might get something like "pool already exists" | 16:59 |
| leseb_ | gfidente: let me check the return code for that | 17:00 |
| gfidente | but I am safe relying on the -mon in case two or more nodes attempt a creation almost simultaneously? | 17:01 |
| leseb_ | gfidente: ok this returns 0 so you can continue, this won't break | 17:01 |
| leseb_ | gfidente: sure | 17:01 |
| gfidente | great, let me try and update the change, thanks1 | 17:01 |
| leseb_ | np! | 17:01 |
| *** jdob has quit IRC | 17:03 | |
| *** jdob has joined #tripleo | 17:04 | |
| *** rwsu has joined #tripleo | 17:04 | |
| *** andreaf has joined #tripleo | 17:06 | |
| gfidente | hey guys this would need some plusvotes to close a bug! :) https://review.openstack.org/#/c/148668/ | 17:06 |
| gfidente | and I know you guys want to close bugs! | 17:06 |
| gfidente | ^_^% | 17:06 |
| jpeeler | dprince: logs of all overcloud nodes - https://jpeeler.fedorapeople.org/ | 17:13 |
| *** Marga_ has joined #tripleo | 17:16 | |
| SpamapS | lifeless: https://etherpad.openstack.org/p/kilo-tripleo-midcycle-meetup <-- you're not on the confirmed list btw. | 17:21 |
| *** dshulyak_ has joined #tripleo | 17:21 | |
| *** dsneddon has quit IRC | 17:23 | |
| SpamapS | https://review.openstack.org/#/c/114360/ <-- this one has been twisting in CI .. recent pass. Would appreciate prompt reviews. Thanks! | 17:25 |
| *** dsneddon has joined #tripleo | 17:26 | |
| gfidente | SpamapS, thanks; I wanted to come to mid-cycle :( | 17:27 |
| openstackgerrit | Steven Hardy proposed openstack/os-net-config: Add a persist_mapping option to the mapping file https://review.openstack.org/148996 | 17:27 |
| SpamapS | gfidente: budget blocked? | 17:28 |
| gfidente | SpamapS, yeah it's far from here | 17:28 |
| SpamapS | gfidente: You're not alone. :-/ | 17:29 |
| bnemec | gfidente: Strange. I would have thought https://github.com/openstack/tripleo-image-elements/blob/master/elements/hosts/os-refresh-config/configure.d/51-hosts#L45 would take care of that. | 17:29 |
| SpamapS | gfidente: We're going to brain storm on how we can have some high bandwidth interactions that don't require expensive travel. | 17:29 |
| gfidente | bnemec, it does | 17:29 |
| gfidente | but something in CI breaks | 17:29 |
| gfidente | probably something where we don't add the hosts element | 17:29 |
| gfidente | so setting to 'localhost' seemed a good mid-term | 17:30 |
| *** eghobo has joined #tripleo | 17:30 | |
| bnemec | I'm still not sure I understand though - if we don't include the hosts element then how do we get any hostname populated? | 17:30 |
| *** sdake has quit IRC | 17:31 | |
| gfidente | bnemec, we do in the overcloud | 17:31 |
| gfidente | CI was failing in UC and seed | 17:31 |
| bnemec | I guess it doesn't matter that much. Worst case scenario is we populate the template and then don't use it. | 17:31 |
| gfidente | that is in the logs for p#1 | 17:31 |
| bnemec | gfidente: I meant why it's broken now, not why false doesn't work. | 17:31 |
| gfidente | where I was removing the file entirely | 17:32 |
| gfidente | wait, I don't get the question then | 17:32 |
| bnemec | gfidente: Adding the entires to hosts.redhat|ubuntu.tmpl should cause cloud-init to include them when manage_etc_hosts is true, right? | 17:32 |
| bnemec | *entries | 17:32 |
| gfidente | ah yes | 17:33 |
| bnemec | So why isn't it? :-) | 17:33 |
| gfidente | oh that is yet another angle | 17:33 |
| bnemec | I'm not opposed to the change, I'm just trying to understand what's going on. | 17:33 |
| *** sdake has joined #tripleo | 17:34 | |
| *** sdake has quit IRC | 17:34 | |
| *** sdake has joined #tripleo | 17:34 | |
| *** radez is now known as radez_g0n3 | 17:35 | |
| TheJulia | tchaypo: So upon digging further investigation, looks like one of my vrrp instances crashed and restarted... and the vip never got stripped off the other node which caused havoc. Second time it has happened after running my upgrade sequence with tripleo-ansible. I suspect we may need to update the notify helper script | 17:36 |
| gfidente | bnemec, just got an updated deployment, let me reboot and see what happens | 17:36 |
| *** Marga_ has quit IRC | 17:37 | |
| *** athomas has quit IRC | 17:37 | |
| *** Marga_ has joined #tripleo | 17:37 | |
| SpamapS | TheJulia: and thus the argument for fencing. ;) | 17:38 |
| TheJulia | SpamapS: bingo | 17:38 |
| gfidente | bnemec, worked for me | 17:38 |
| SpamapS | I've only ever seen VIP flipping work two ways. 1) STONITH, 2) BigIP. | 17:38 |
| SpamapS | Don't know how the BigIP guys do it but they definitely get it right. | 17:38 |
| bnemec | gfidente: Which worked for you? | 17:39 |
| SpamapS | oh I lied, CARP on OpenBSD, but only for gateways, not for endpoints. | 17:39 |
| gfidente | the template, reprovisioned a good hosts file as well | 17:39 |
| SpamapS | (I assume VRRP on expensive routing/firewall gear is equally as successful as OpenBSD's CARP but at 20x the cost...) | 17:39 |
| TheJulia | heh. Keepalived has worked for me int he past, but honestly I think i ended up putting ip up/down scripts for actions instead of relying upon it using the netlink. The funny thing is, the note in the keepalived 1.2.13 release notes, it actually mentions that route as the more sane path. | 17:40 |
| SpamapS | TheJulia: hah, yeah, so basically "use this like you would use pacemaker" | 17:40 |
| SpamapS | which is one ilo script away from STONITH :) | 17:41 |
| *** ChuckC has joined #tripleo | 17:41 | |
| *** ChuckC_ has quit IRC | 17:41 | |
| TheJulia | SpamapS: kind of yeah... that or the userspace tools have enough ability to tear the node completely down and out of production. | 17:42 |
| * TheJulia lets things run again and goes in search of food | 17:42 | |
| SpamapS | Yeah it just depends on how much havoc split brain causes you. :) | 17:43 |
| SpamapS | for quick HTTP requests through haproxy to backends with one gateway router.. a flipping mac for the VIP should only be a little insane | 17:43 |
| SpamapS | but rabbit and mysql are where it starts to 'splode | 17:44 |
| *** pblaho has quit IRC | 17:44 | |
| *** dsneddon has quit IRC | 17:45 | |
| TheJulia | yeah, rabbit and the two active vip assignments is what has resulted in a dead stack for me twice this morning. :( | 17:45 |
| *** sdake has quit IRC | 17:46 | |
| SpamapS | Crazy idea: why not just use the multi-server option for rabbit built into kombu? | 17:46 |
| *** dsneddon has joined #tripleo | 17:46 | |
| *** Marga_ has quit IRC | 17:46 | |
| *** Marga_ has joined #tripleo | 17:47 | |
| *** avozza is now known as zz_avozza | 17:50 | |
| *** Marga_ has quit IRC | 17:51 | |
| *** sdake has joined #tripleo | 17:54 | |
| *** sdake has quit IRC | 17:54 | |
| *** sdake has joined #tripleo | 17:54 | |
| *** lucasagomes is now known as lucas-dinner | 17:55 | |
| *** jistr has quit IRC | 17:56 | |
| *** yamahata has quit IRC | 18:01 | |
| gfidente | SpamapS, TheJulia I am in for testing that | 18:02 |
| gfidente | I think I left everything in place for switching between list of nodes and haproxy | 18:02 |
| SpamapS | I've heard rumblings it is one of those "has to be used right in each project" problems | 18:02 |
| gfidente | so if you remove rabbit.host from template, config will fallback to rabbit_hosts | 18:03 |
| gfidente | at which point, if we're happy, we can remove the haproxy listener | 18:03 |
| SpamapS | to which I say http://38.media.tumblr.com/tumblr_lsxk1ndn7W1r2vs7so3_r3_500.gif | 18:03 |
| gfidente | SpamapS, before recent changes in rabbit things wouldn't work without it | 18:04 |
| *** sdake has quit IRC | 18:05 | |
| gfidente | https://bugs.launchpad.net/nova/+bug/856764/comments/70 | 18:05 |
| gfidente | SpamapS, ^^ | 18:05 |
| *** prad has joined #tripleo | 18:06 | |
| *** derekh has quit IRC | 18:08 | |
| *** sdake has joined #tripleo | 18:08 | |
| *** sdake has quit IRC | 18:08 | |
| *** sdake has joined #tripleo | 18:08 | |
| openstackgerrit | Merged openstack/tripleo-image-elements: Add an element to restore ssh keys from /mnt/state https://review.openstack.org/114360 | 18:09 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-image-elements: Update correct cloud-init hosts template file on RHEL. https://review.openstack.org/149016 | 18:14 |
| SpamapS | woooot | 18:19 |
| SpamapS | TheJulia: ^^ /mnt/state ssh key restorer finally merged | 18:19 |
| TheJulia | I just saw \o/ | 18:21 |
| * TheJulia dances | 18:21 | |
| *** jdob has quit IRC | 18:35 | |
| cinerama | oh that's great | 18:38 |
| *** bvandenh has quit IRC | 18:38 | |
| cinerama | pepperidge farm remembers when that got put up | 18:39 |
| *** jerryz has quit IRC | 18:40 | |
| *** mcornea has quit IRC | 18:47 | |
| *** sdake has quit IRC | 18:53 | |
| *** sdake has joined #tripleo | 18:56 | |
| *** sdake has quit IRC | 18:56 | |
| *** sdake has joined #tripleo | 18:56 | |
| *** akrivoka has quit IRC | 18:56 | |
| *** dshulyak_ has quit IRC | 19:00 | |
| *** Marga_ has joined #tripleo | 19:05 | |
| *** jcoufal_ has quit IRC | 19:10 | |
| *** spzala has quit IRC | 19:11 | |
| *** sdake has quit IRC | 19:14 | |
| *** eghobo has quit IRC | 19:20 | |
| *** dshulyak_ has joined #tripleo | 19:27 | |
| *** Marga_ has quit IRC | 19:27 | |
| *** Marga_ has joined #tripleo | 19:27 | |
| *** dshulyak_ has quit IRC | 19:30 | |
| *** jdob has joined #tripleo | 19:31 | |
| *** Marga_ has quit IRC | 19:32 | |
| *** pelix has quit IRC | 19:32 | |
| *** spzala has joined #tripleo | 19:36 | |
| tchaypo | pepperidge farm? | 19:54 |
| openstackgerrit | Merged openstack/diskimage-builder: Switch manage_etc_hosts from True to localhost https://review.openstack.org/148668 | 19:56 |
| *** Marga_ has joined #tripleo | 19:59 | |
| *** Marga_ has quit IRC | 20:04 | |
| *** Marga_ has joined #tripleo | 20:05 | |
| *** Marga_ has quit IRC | 20:09 | |
| *** lucas-dinner has quit IRC | 20:20 | |
| *** jtomasek has quit IRC | 20:22 | |
| *** jerryz has joined #tripleo | 20:25 | |
| greghaynes | shameless plug for reviews on https://review.openstack.org/#/c/145566/ | 20:27 |
| greghaynes | The last two CI runs together combined pass CI | 20:27 |
| *** andreaf has quit IRC | 20:29 | |
| *** akuznetsov has joined #tripleo | 20:34 | |
| *** andreaf has joined #tripleo | 20:44 | |
| *** yamahata has joined #tripleo | 20:44 | |
| *** sdake has joined #tripleo | 20:46 | |
| *** Marga_ has joined #tripleo | 20:48 | |
| bnemec | And the fact that it properly timed out is actually kind of a good thing... ;-) | 20:52 |
| *** Marga_ has quit IRC | 20:53 | |
| openstackgerrit | yogananth subramanian proposed openstack/diskimage-builder: ramdisk-image-create: add support for vmlinux file https://review.openstack.org/149045 | 20:53 |
| *** ifarkas has quit IRC | 20:55 | |
| greghaynes | yes! logs! | 21:00 |
| *** lblanchard has quit IRC | 21:04 | |
| *** akuznetsov has quit IRC | 21:05 | |
| *** zz_avozza is now known as avozza | 21:15 | |
| *** jdob_ has joined #tripleo | 21:24 | |
| *** jdob_ has quit IRC | 21:25 | |
| *** dprince has quit IRC | 21:25 | |
| *** Marga_ has joined #tripleo | 21:26 | |
| *** sdake has quit IRC | 21:26 | |
| openstackgerrit | Merged openstack/tuskar: Updated from global requirements https://review.openstack.org/145674 | 21:40 |
| *** jerryz has quit IRC | 21:42 | |
| *** dshulyak_ has joined #tripleo | 21:42 | |
| *** jprovazn_bbl has quit IRC | 21:43 | |
| *** david-lyle has joined #tripleo | 21:46 | |
| *** jdob has quit IRC | 21:52 | |
| *** thrash is now known as thrash|g0ne | 21:53 | |
| *** ccrouch1 has joined #tripleo | 21:55 | |
| *** ccrouch has quit IRC | 21:55 | |
| greghaynes | ok, the world has been released | 21:59 |
| *** untriaged-bot has joined #tripleo | 22:00 | |
| untriaged-bot | Untriaged bugs so far: | 22:00 |
| untriaged-bot | https://bugs.launchpad.net/tuskar/+bug/1387487 | 22:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1411497 | 22:00 |
| untriaged-bot | https://bugs.launchpad.net/diskimage-builder/+bug/1413362 | 22:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1409583 | 22:00 |
| *** untriaged-bot has quit IRC | 22:00 | |
| *** chlong has quit IRC | 22:05 | |
| *** ChuckC_ has joined #tripleo | 22:07 | |
| *** ChuckC has quit IRC | 22:08 | |
| *** david-lyle has quit IRC | 22:11 | |
| *** david-lyle has joined #tripleo | 22:11 | |
| *** ChuckC has joined #tripleo | 22:11 | |
| *** radez_g0n3 is now known as radez | 22:14 | |
| *** ChuckC_ has quit IRC | 22:15 | |
| *** david-lyle has quit IRC | 22:15 | |
| *** ChuckC has quit IRC | 22:17 | |
| *** gfidente has quit IRC | 22:19 | |
| *** dshulyak_ has quit IRC | 22:21 | |
| *** dshulyak_ has joined #tripleo | 22:40 | |
| *** dshulyak_ has quit IRC | 22:40 | |
| *** lblanchard has joined #tripleo | 22:54 | |
| *** david-lyle has joined #tripleo | 22:58 | |
| *** Marga_ has quit IRC | 23:01 | |
| *** Marga_ has joined #tripleo | 23:01 | |
| *** chuckc__ has joined #tripleo | 23:03 | |
| *** Marga_ has quit IRC | 23:04 | |
| *** Marga_ has joined #tripleo | 23:04 | |
| *** david-lyle has quit IRC | 23:09 | |
| *** sdake has joined #tripleo | 23:09 | |
| *** sdake has quit IRC | 23:09 | |
| *** sdake has joined #tripleo | 23:09 | |
| *** Marga_ has quit IRC | 23:12 | |
| *** Marga_ has joined #tripleo | 23:13 | |
| *** eghobo has joined #tripleo | 23:13 | |
| *** sdake has quit IRC | 23:19 | |
| *** lblanchard has quit IRC | 23:25 | |
| *** yamahata has quit IRC | 23:26 | |
| *** Marga_ has quit IRC | 23:29 | |
| *** prad has quit IRC | 23:29 | |
| *** Marga_ has joined #tripleo | 23:29 | |
| *** chlong has joined #tripleo | 23:33 | |
| *** Marga_ has quit IRC | 23:34 | |
| *** Marga_ has joined #tripleo | 23:41 | |
| *** morazi has quit IRC | 23:41 | |
| *** gothicmindfood has quit IRC | 23:42 | |
| *** chuckc__ has quit IRC | 23:43 | |
| *** gothicmindfood has joined #tripleo | 23:45 | |
| *** Marga_ has quit IRC | 23:45 | |
| *** Marga_ has joined #tripleo | 23:54 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
