*** david-lyle has quit IRC | 00:24 | |
*** dsneddon has quit IRC | 00:51 | |
*** Slower has quit IRC | 01:00 | |
*** Marga_ has quit IRC | 01:00 | |
*** Marga_ has joined #tripleo | 01:01 | |
*** saurabhs has left #tripleo | 01:04 | |
*** Slower has joined #tripleo | 01:07 | |
*** sthillma has quit IRC | 01:16 | |
*** cwolferh has joined #tripleo | 01:31 | |
*** Marga_ has quit IRC | 01:50 | |
*** david-lyle has joined #tripleo | 02:08 | |
*** cwolferh has quit IRC | 02:11 | |
openstackgerrit | Dan Sneddon proposed openstack/tripleo-heat-templates: Include Redis VIP in example environment https://review.openstack.org/192036 | 02:13 |
---|---|---|
*** spzala has quit IRC | 02:22 | |
*** yog__ has joined #tripleo | 02:50 | |
*** Marga_ has joined #tripleo | 02:51 | |
*** Marga_ has quit IRC | 02:52 | |
*** Marga_ has joined #tripleo | 02:53 | |
*** penick has joined #tripleo | 02:53 | |
*** untriaged-bot has joined #tripleo | 03:01 | |
untriaged-bot | Untriaged bugs so far: | 03:01 |
untriaged-bot | https://bugs.launchpad.net/diskimage-builder/+bug/1461615 | 03:01 |
openstack | Launchpad bug 1461615 in diskimage-builder "Yum vars directory doesn't exist" [Undecided,In progress] - Assigned to Abel Lopez (al592b) | 03:01 |
untriaged-bot | https://bugs.launchpad.net/diskimage-builder/+bug/1461601 | 03:01 |
openstack | Launchpad bug 1461601 in diskimage-builder "CentOS 6 not resizing" [Undecided,Fix committed] - Assigned to Abel Lopez (al592b) | 03:01 |
untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1459465 | 03:01 |
openstack | Launchpad bug 1459465 in tripleo "devtest fails to build ramdisk on Fedora or CentOS due to Yum multilib error" [Undecided,Triaged] | 03:01 |
untriaged-bot | https://bugs.launchpad.net/os-collect-config/+bug/1463409 | 03:01 |
openstack | Launchpad bug 1463409 in os-collect-config "Do not tolerate world readable os-collect-config.conf" [Undecided,New] | 03:01 |
untriaged-bot | https://bugs.launchpad.net/diskimage-builder/+bug/1463357 | 03:01 |
openstack | Launchpad bug 1463357 in diskimage-builder "missing symlink support for ppc64 Fedora deployment ramdisk " [Undecided,In progress] - Assigned to yogananth subramanian (anantyog-o) | 03:01 |
untriaged-bot | https://bugs.launchpad.net/diskimage-builder/+bug/1461631 | 03:01 |
openstack | Launchpad bug 1461631 in diskimage-builder "New Fedora 22 cloud images fail to build" [Undecided,New] | 03:01 |
untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1461984 | 03:01 |
openstack | Launchpad bug 1461984 in tripleo "VM element: /usr/sbin/grub2-install: error: will not proceed with blocklists." [Undecided,New] | 03:01 |
untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1464692 | 03:01 |
openstack | Launchpad bug 1464692 in tripleo "debian build broken" [Undecided,New] | 03:01 |
*** untriaged-bot has quit IRC | 03:01 | |
*** yuanying_ has joined #tripleo | 03:18 | |
*** yuanying has quit IRC | 03:20 | |
*** marios has quit IRC | 03:32 | |
*** sdake has quit IRC | 03:32 | |
*** marios has joined #tripleo | 03:32 | |
*** penick has quit IRC | 03:37 | |
*** yuanying_ has quit IRC | 03:47 | |
*** Marga_ has quit IRC | 03:48 | |
*** yuanying has joined #tripleo | 03:52 | |
*** radek_ has joined #tripleo | 04:14 | |
*** radek__ has quit IRC | 04:15 | |
*** sdake has joined #tripleo | 04:17 | |
*** panda has quit IRC | 04:47 | |
*** panda has joined #tripleo | 04:48 | |
*** ukalifon1 has joined #tripleo | 04:53 | |
*** lazy_prince has joined #tripleo | 05:24 | |
*** yamahata has joined #tripleo | 05:28 | |
openstackgerrit | Steve Baker proposed openstack/tripleo-heat-templates: Ensure present/latest for puppet driven package updates https://review.openstack.org/190918 | 05:30 |
*** ishant has joined #tripleo | 05:31 | |
*** gfidente has joined #tripleo | 05:36 | |
*** gfidente has quit IRC | 05:36 | |
*** gfidente has joined #tripleo | 05:36 | |
*** jtomasek has joined #tripleo | 05:57 | |
*** shadower has quit IRC | 06:06 | |
*** shadower has joined #tripleo | 06:06 | |
*** sdake has quit IRC | 06:14 | |
*** masco has joined #tripleo | 06:21 | |
*** dasm|afk is now known as dasm | 06:21 | |
*** pino|work has joined #tripleo | 06:30 | |
*** dtantsur|afk is now known as dtantsur | 06:32 | |
shardy | morning all! | 06:39 |
*** jprovazn has joined #tripleo | 06:49 | |
*** masco has quit IRC | 07:00 | |
*** ukalifon1 has quit IRC | 07:02 | |
*** greghaynes has quit IRC | 07:08 | |
shardy | anyone know why https://review.openstack.org/#/c/191726/ hasn't triggered CI? | 07:09 |
*** greghaynes has joined #tripleo | 07:10 | |
*** paramite has joined #tripleo | 07:10 | |
*** shardy_ has joined #tripleo | 07:15 | |
*** ukalifon has joined #tripleo | 07:16 | |
*** shardy has quit IRC | 07:16 | |
*** ifarkas has joined #tripleo | 07:17 | |
*** chlong has quit IRC | 07:18 | |
*** ifarkas has quit IRC | 07:21 | |
*** shardy_ has quit IRC | 07:21 | |
*** ifarkas has joined #tripleo | 07:21 | |
*** shardy has joined #tripleo | 07:21 | |
*** adrianopetrich has joined #tripleo | 07:29 | |
openstackgerrit | Merged openstack/tripleo-heat-templates: Isolate network traffic for Nova VNC proxy https://review.openstack.org/190409 | 07:38 |
*** regebro has quit IRC | 07:42 | |
*** regebro has joined #tripleo | 07:43 | |
openstackgerrit | Merged openstack/tripleo-heat-templates: Remove external bridge from Compute nodes https://review.openstack.org/190849 | 07:47 |
openstackgerrit | Merged openstack/tripleo-heat-templates: Add support for isolating swift storage nets https://review.openstack.org/188655 | 07:48 |
openstackgerrit | Merged openstack/tripleo-heat-templates: Flexible IP allocations for isolated nets https://review.openstack.org/191169 | 07:49 |
openstackgerrit | Merged openstack/tripleo-heat-templates: network/config storage role fixes https://review.openstack.org/188234 | 07:50 |
openstackgerrit | Merged openstack/tripleo-heat-templates: Add BondInterfaceOvsOptions parameter https://review.openstack.org/189827 | 07:50 |
jprovazn | Hi, anyone in review mood? If so, could you please take a look at https://review.openstack.org/#/c/178150/ and https://review.openstack.org/#/c/179862/ | 07:51 |
*** shardy_ has joined #tripleo | 07:52 | |
*** shardy has quit IRC | 07:53 | |
*** al has quit IRC | 07:56 | |
*** al has joined #tripleo | 07:57 | |
*** shardy_ has quit IRC | 07:57 | |
*** shardy has joined #tripleo | 07:58 | |
*** shardy has quit IRC | 07:58 | |
*** jcoufal has joined #tripleo | 08:00 | |
*** shardy has joined #tripleo | 08:01 | |
openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Allow control of hostname formatting https://review.openstack.org/191726 | 08:02 |
gfidente | shardy, ^^ rebased | 08:02 |
shardy | gfidente: aha, thanks, I was just about to do that! :) | 08:05 |
gfidente | ah okay I won't if you're keeping eyes on it | 08:05 |
gfidente | hopefully we don't need to do it anymore though | 08:06 |
*** Goneri has joined #tripleo | 08:07 | |
gfidente | shardy, actually I forgot to remove the tab as well ... so if you want to update it now that has no votes ... | 08:07 |
shardy | gfidente: the most recent revision didn't have that line in it at all? | 08:10 |
shardy | I'll rebase from my local branch | 08:10 |
*** derekh has joined #tripleo | 08:10 | |
gfidente | shardy, I downloaded last revision | 08:11 |
gfidente | so take care I think the tab will still be there | 08:11 |
shardy | gfidente: Ok let me check, your comment was on the n-1'th revision I think | 08:11 |
gfidente | ps#5 yes | 08:11 |
gfidente | well, comment was much older, but ps#5 still has the tab | 08:12 |
openstackgerrit | Steven Hardy proposed openstack/tripleo-heat-templates: Make puppet-applying *Post resources depend on hieradata https://review.openstack.org/191146 | 08:12 |
*** lucasagomes has joined #tripleo | 08:13 | |
openstackgerrit | Steven Hardy proposed openstack/tripleo-heat-templates: Allow control of hostname formatting https://review.openstack.org/191726 | 08:15 |
shardy | gfidente: thanks, that should fix it I think | 08:16 |
shardy | I'm not sure why CI hasn't been triggering on that change though? | 08:16 |
*** Marga_ has joined #tripleo | 08:21 | |
*** ukalifon has quit IRC | 08:23 | |
*** ukalifon has joined #tripleo | 08:24 | |
openstackgerrit | Derek Higgins proposed openstack-infra/tripleo-ci: Switch to using pypi https://review.openstack.org/192101 | 08:27 |
*** yamahata has quit IRC | 08:29 | |
*** ukalifon has quit IRC | 08:37 | |
*** ukalifon has joined #tripleo | 08:37 | |
*** AzherKhan has joined #tripleo | 08:39 | |
*** panda has quit IRC | 08:48 | |
*** panda has joined #tripleo | 08:48 | |
*** yamahata has joined #tripleo | 08:52 | |
*** dtantsur is now known as dtantsur|brb | 08:52 | |
*** yamahata has quit IRC | 08:57 | |
*** mcornea has joined #tripleo | 08:57 | |
*** chlong has joined #tripleo | 08:59 | |
*** untriaged-bot has joined #tripleo | 09:01 | |
untriaged-bot | Untriaged bugs so far: | 09:01 |
untriaged-bot | https://bugs.launchpad.net/diskimage-builder/+bug/1461615 | 09:01 |
openstack | Launchpad bug 1461615 in diskimage-builder "Yum vars directory doesn't exist" [Undecided,In progress] - Assigned to Abel Lopez (al592b) | 09:01 |
untriaged-bot | https://bugs.launchpad.net/diskimage-builder/+bug/1461601 | 09:01 |
openstack | Launchpad bug 1461601 in diskimage-builder "CentOS 6 not resizing" [Undecided,Fix committed] - Assigned to Abel Lopez (al592b) | 09:01 |
untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1459465 | 09:01 |
openstack | Launchpad bug 1459465 in tripleo "devtest fails to build ramdisk on Fedora or CentOS due to Yum multilib error" [Undecided,Triaged] | 09:01 |
untriaged-bot | https://bugs.launchpad.net/os-collect-config/+bug/1463409 | 09:01 |
openstack | Launchpad bug 1463409 in os-collect-config "Do not tolerate world readable os-collect-config.conf" [Undecided,New] | 09:01 |
untriaged-bot | https://bugs.launchpad.net/diskimage-builder/+bug/1463357 | 09:01 |
openstack | Launchpad bug 1463357 in diskimage-builder "missing symlink support for ppc64 Fedora deployment ramdisk " [Undecided,In progress] - Assigned to yogananth subramanian (anantyog-o) | 09:01 |
untriaged-bot | https://bugs.launchpad.net/diskimage-builder/+bug/1461631 | 09:01 |
openstack | Launchpad bug 1461631 in diskimage-builder "New Fedora 22 cloud images fail to build" [Undecided,New] | 09:01 |
untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1461984 | 09:01 |
openstack | Launchpad bug 1461984 in tripleo "VM element: /usr/sbin/grub2-install: error: will not proceed with blocklists." [Undecided,New] | 09:01 |
untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1464692 | 09:01 |
openstack | Launchpad bug 1464692 in tripleo "debian build broken" [Undecided,New] | 09:01 |
*** untriaged-bot has quit IRC | 09:01 | |
*** hewbrocca has joined #tripleo | 09:01 | |
hewbrocca | shardy: morning! | 09:02 |
hewbrocca | shardy: could you remind me of the location of your excellent write-up on debugging heat stack-deploy failures? | 09:02 |
shardy | hewbrocca: morning, sure, sec | 09:02 |
shardy | http://hardysteven.blogspot.co.uk/2015/04/debugging-tripleo-heat-templates.html | 09:02 |
hewbrocca | thanks! | 09:02 |
hewbrocca | jcoufal: when are we going to have an index of all this stuff, so I can stop asking shardy where they are? | 09:03 |
shardy | hewbrocca: tl;dr heat resource-list -n5 overcloud | grep FAILED | 09:03 |
shardy | then heat deployment-show <failing deployment ID> | 09:03 |
hewbrocca | Cool | 09:04 |
hewbrocca | thanks | 09:04 |
hewbrocca | marios: dobre rano | 09:04 |
jcoufal | hewbrocca: we synced with shardy yesterday, and as we discussed last week, you should have something by thursday | 09:04 |
hewbrocca | jcoufal: excellent | 09:04 |
shardy | jcoufal: FWIW I'd like to help refactor much of this stuff into proper docs at some point, but if linking the blog posts is the quickest approach for now that's probably fine | 09:07 |
*** chlong has quit IRC | 09:07 | |
jcoufal | shardy: depends on time pressure | 09:07 |
shardy | I need to finish the series on heat template internals, there's at least one more post required to complete the deconstruction | 09:07 |
jcoufal | for now I linked that blog post from the docs | 09:08 |
shardy | jcoufal: agreed, sounds good :) | 09:08 |
jcoufal | if you manage to transfer it to proper docs right away it would be better solution | 09:08 |
jcoufal | but really depends on the time conditions | 09:08 |
shardy | jcoufal: Yeah, I'm happy to do that but I was a bit put off after spending time then discovering we had overlap | 09:08 |
shardy | if we can get a matrix of docs gaps then assign them to folks that will help avoid future duplication :) | 09:09 |
jcoufal | shardy: sure | 09:09 |
*** pelix has joined #tripleo | 09:13 | |
*** chlong has joined #tripleo | 09:20 | |
hewbrocca | yeah, that matrix thing | 09:20 |
hewbrocca | that is *exactly* what we need | 09:21 |
hewbrocca | jcoufal: is working on it I do believe | 09:21 |
jcoufal | hewbrocca: yes, we agreed yesterday that I will send it to shardy for review | 09:21 |
hewbrocca | superb | 09:24 |
shardy | Wow jq really makes my head hurt | 09:25 |
hewbrocca | stupid javascript | 09:25 |
shardy | Anyone super expert at it around here fancy a challenge? | 09:25 |
* hewbrocca whistles, looks at floor | 09:26 | |
* shardy needs moar coffee | 09:26 | |
*** akrivoka has joined #tripleo | 09:27 | |
*** paramite is now known as paramite|afk | 09:34 | |
dasm | hello o/ | 09:47 |
hewbrocca | dasm: morning! | 09:49 |
*** ishant has quit IRC | 09:56 | |
*** ishant has joined #tripleo | 09:57 | |
*** Marga_ has quit IRC | 10:00 | |
*** Marga_ has joined #tripleo | 10:06 | |
*** dtantsur|brb is now known as dtantsur | 10:08 | |
akrivoka | we still have a lot of tuskarclient patches that are waiting to be merged https://review.openstack.org/#/q/status:open+project:openstack/python-tuskarclient,n,z | 10:13 |
*** Marga_ has quit IRC | 10:20 | |
hewbrocca | help, anyone? | 10:21 |
hewbrocca | ^^^ | 10:21 |
*** yog__ has quit IRC | 10:36 | |
*** paramite|afk is now known as paramite | 10:37 | |
openstackgerrit | Tan Lin proposed openstack/diskimage-builder: Add oat-client element https://review.openstack.org/192140 | 10:39 |
openstackgerrit | Merged openstack/python-tuskarclient: Handle creation of plan with existing name https://review.openstack.org/187149 | 10:44 |
*** AzherKhan has quit IRC | 10:46 | |
openstackgerrit | Merged openstack/python-tuskarclient: Updated from global requirements https://review.openstack.org/187333 | 10:47 |
*** yuanying has quit IRC | 10:47 | |
openstackgerrit | Merged openstack/python-tuskarclient: Enable SSL-related CLI opts https://review.openstack.org/190251 | 10:49 |
openstackgerrit | Merged openstack/python-tuskarclient: Filter and format parameters for plan role in OSC https://review.openstack.org/186728 | 10:53 |
akrivoka | gfidente: thanks! | 11:03 |
gfidente | the remaining two I am less comfortable with though | 11:04 |
akrivoka | gfidente: due to jenkins failing or some other reason? | 11:05 |
gfidente | the contents! | 11:05 |
akrivoka | regebro: ^ | 11:05 |
gfidente | oh that means I don't feel I understand that the minimum required to +2 | 11:07 |
gfidente | not that they are anyway broken | 11:08 |
akrivoka | ah fair enough :) | 11:08 |
*** lucasagomes is now known as lucas-hungry | 11:08 | |
regebro | gfidente: Anything I can do how to help? | 11:09 |
*** rhallisey has joined #tripleo | 11:14 | |
openstackgerrit | Derek Higgins proposed openstack-infra/tripleo-ci: Switch to using pypi https://review.openstack.org/192101 | 11:24 |
gfidente | derekh, ^^ I see jobs failing on getting the fedora cloud image as well with no route to host | 11:25 |
derekh | gfidente: ya thanks, dan was working on this yesterday we have a few problems still outstanding that I'm trying to work through | 11:27 |
gfidente | but you want to update pypi meantime anyway? | 11:27 |
derekh | gfidente: yup, we need to install a new geard broker, pypi index and proxy and probbaly add some compute nodes | 11:28 |
derekh | gfidente: I only pushed that patch back up with a mottit message change to see if gets pasted the download fedora image erorr | 11:29 |
derekh | gfidente: which I think should now be ok | 11:29 |
derekh | gfidente: it was a problem with the proxy server | 11:29 |
gfidente | that would be great | 11:29 |
*** dprince has joined #tripleo | 11:29 | |
derekh | dprince: morning, I think I now have squick working, moving onto te-broker | 11:31 |
dprince | derekh: cool. Want me to add your keys there? | 11:31 |
dprince | derekh: what did I do wrong to the previous squid? | 11:32 |
derekh | dprince: I was just going to delete and reinstall | 11:32 |
derekh | dprince not sure, I couldn't log in because I didn't have the key | 11:33 |
dprince | derekh: right, and now I can't login because I don't have the key ;) | 11:33 |
derekh | dprince: my running theory was that there was some complication due to it being started by a different tennant to usual | 11:33 |
dprince | derekh: to the squid that is... | 11:33 |
derekh | dprince: its the key on the seed host | 11:34 |
dprince | derekh: what tenant are you using? I was putting these in 'admin'. | 11:34 |
derekh | dprince: openstack-nodepool | 11:34 |
*** jcoufal has quit IRC | 11:35 | |
dprince | derekh: lets put them all in the same place then | 11:36 |
derekh | dprince: anyways I may have been wrong, when I redid a new squid instance, the dns server in /etc/resolv.conf was set to 192.168..... which wasn't working, I've manually set it to 8.8.8.8 | 11:36 |
dprince | derekh: so kill my te-broker and you fire up one then... | 11:36 |
dprince | derekh: all the traffic to the squid should be on the 192.168. network though. So inter-tenant traffic shouldn't be the issue here. That would be the 10. network.... | 11:37 |
dprince | derekh: btw. I think the root cause of all this was /mnt/state/var/lib/neutron/keystone-signing being exactly 1 year old or something. | 11:38 |
derekh | dprince: I have a feeling that securuty groups are getting applied, getting something similar with traffic to the tebroker, | 11:38 |
derekh | dprince: ahh ok | 11:39 |
derekh | dprince: want to debug the tebroker thing first to see if it is security groups ? | 11:39 |
derekh | dprince: or just blow it away and create a new one | 11:39 |
dprince | derekh: not really. I'd rather just put them in the same tenant anyway :) | 11:39 |
derekh | dprince: ok, will do it now | 11:40 |
dprince | derekh: 2 more computes would give us 12? shall I create those? | 11:44 |
derekh | dprince: yup, I think so, it will give us a bit of head room if we loose one | 11:45 |
dprince | derekh: the nodepool tenant doesn't ahve a keypair? How are you logging in? And if so I'd like my key added too... | 11:48 |
*** paramite is now known as paramite|afk | 11:49 | |
derekh | dprince: ahh, is keypair tied to user ? | 11:49 |
derekh | dprince: I'm using the tripleo-ci user in the openstack-nodepool teant , default key | 11:50 |
hewbrocca | what on earth are you characters up to | 11:50 |
dprince | derekh: that would explain why we aren't seeing the same things. | 11:50 |
dprince | hewbrocca: fixing the TripleO ci cloud. https://etherpad.openstack.org/p/tripleo-cloud-outage | 11:51 |
derekh | hewbrocca: fixing stuff | 11:51 |
dprince | derekh: I'm really enjoying the devtest_overcloud_derekhisawsom.sh btw. I've just made it depend on overcloud.yaml_dan btw ;) | 11:54 |
derekh | dprince: just confirmed that default key is /root/.ssh/id_rsa on the seed host, I've just copied it locally | 11:54 |
derekh | dprince: nice ;-) | 11:54 |
*** athomas has quit IRC | 11:56 | |
hewbrocca | well someone's got to do it | 11:56 |
dprince | derekh: did you delete my te-broker yet? | 11:57 |
hewbrocca | dprince: while you're in there -- did you guys ever get multi-network environments together to test the network isolation code? | 11:57 |
derekh | dprince: yup, new instance is up | 11:57 |
dprince | hewbrocca: I've been testing the network isolation stuff... but not in this environment | 11:57 |
derekh | dprince: manaully installing geard now, we didn't have an image built | 11:58 |
dprince | derekh: use used fedora, pip installed it, and then just ran in screen... | 11:58 |
hewbrocca | dprince: roger that. was just curious | 11:58 |
*** morazi has joined #tripleo | 11:59 | |
derekh | dprince: was going to run through whats in tripleo-image-elements/elements/geard so it comes up on reboot etc.. | 12:00 |
dprince | derekh: even better | 12:00 |
*** paramite|afk is now known as paramite | 12:04 | |
*** athomas has joined #tripleo | 12:05 | |
*** lucas-hungry is now known as lucasagomes | 12:08 | |
openstackgerrit | Merged openstack-infra/tripleo-ci: Switch to using pypi https://review.openstack.org/192101 | 12:11 |
*** trown|outttypeww is now known as trown | 12:15 | |
*** jayg|g0n3 is now known as jayg | 12:20 | |
derekh | dprince: tebroker is now running, I don't see any other problem at the moment but will be back in about 10 minutes to look around and start on the pypi mirror | 12:20 |
dprince | derekh: we've got 12 computes now | 12:21 |
*** pblaho has joined #tripleo | 12:36 | |
dprince | derekh: ./toci_devtest.sh: line 193: export: `https://pypi.python.org/simple/': not a valid identifier | 12:44 |
openstackgerrit | Dan Prince proposed openstack-infra/tripleo-ci: Fix PIP_EXTRA_INDEX_URL. https://review.openstack.org/192180 | 12:47 |
derekh | dprince: opps, sorry, are your fixing it or will I? | 12:47 |
dprince | derekh: just pushed https://review.openstack.org/192180 | 12:48 |
*** panda has quit IRC | 12:48 | |
dprince | derekh: might as well wait for it to complete I guess | 12:48 |
derekh | dprince: ack | 12:48 |
*** panda has joined #tripleo | 12:48 | |
dprince | derekh: if it gets us further we may go ahead and send it even if we don't get a full pass though... | 12:49 |
*** thrash|g0ne is now known as thrash | 12:50 | |
derekh | dprince: ack, yup, once it builds an image, or pip installs anything | 12:50 |
dprince | derekh: https://jenkins06.openstack.org/job/check-tripleo-ironic-overcloud-f21-nonha/68/console | 12:56 |
*** jehb has quit IRC | 13:00 | |
*** stendulker has joined #tripleo | 13:00 | |
*** yamahata has joined #tripleo | 13:01 | |
derekh | dprince: its looking good, I reckon we should send it | 13:05 |
dprince | derekh: agree, it is at least progress | 13:07 |
openstackgerrit | Merged openstack-infra/tripleo-ci: Fix PIP_EXTRA_INDEX_URL. https://review.openstack.org/192180 | 13:07 |
*** Marga_ has joined #tripleo | 13:17 | |
*** dasm is now known as dasm|afk | 13:20 | |
*** julim has joined #tripleo | 13:21 | |
dprince | derekh: it is hanging on connecting to the seed vm | 13:25 |
*** jehb has joined #tripleo | 13:27 | |
derekh | dprince: this could be a "regular" non cloud problem, it connected to the seed ok to crate the baremetal flavor | 13:28 |
* derekh checks the undercloud job | 13:28 | |
*** lblanchard has joined #tripleo | 13:31 | |
derekh | dprince: looks like its not stuck anymore, was just taking the breather for 5 minutes | 13:32 |
*** sdake has joined #tripleo | 13:32 | |
*** sdake has quit IRC | 13:32 | |
derekh | dprince: fyi, bandersnatch index is now being built but it will take a few hours to complete | 13:33 |
*** sdake has joined #tripleo | 13:33 | |
*** Marga_ has quit IRC | 13:34 | |
dprince | derekh: cool. would it help if we took a snapshot periodically? We'd likely just have space for just one or two though... | 13:36 |
*** Goneri has quit IRC | 13:37 | |
*** Marga_ has joined #tripleo | 13:39 | |
*** Marga_ has quit IRC | 13:40 | |
*** Marga_ has joined #tripleo | 13:40 | |
derekh | dprince: maybe, I'm trying to remember how big it is, iirc around 100G | 13:41 |
* derekh wonders if a volume would be more suitable for that | 13:42 | |
dprince | derekh: volume works too... but I'd soon like to tear all this down and rev it to a more recent codebase | 13:43 |
dprince | derekh: primarily, because I'd like to get a more recent Heat template structure | 13:43 |
derekh | dprince: ack, yup makes sense | 13:43 |
dprince | derekh: also, I think a working 'nova rebuild' would help our use case for stuff like this | 13:43 |
dprince | derekh: in that we could simply issue nova rebuilds for all the compute nodes... | 13:44 |
dprince | derekh: step at a time. But more recent code gives us a few new tricks to troublshoot with I think | 13:44 |
derekh | dprince: yup, it would be a lot better then having to being up a new stack of compute nodes | 13:44 |
*** Goneri has joined #tripleo | 13:44 | |
*** ishant has quit IRC | 13:45 | |
*** gfidente is now known as gfidente|afk | 13:52 | |
derekh | dprince: https://jenkins06.openstack.org/job/check-tripleo-ironic-undercloud-precise-nonha/340/ | 13:53 |
*** lazy_prince has quit IRC | 13:54 | |
openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Implement Advanced Firewalling support https://review.openstack.org/191195 | 13:59 |
EmilienM | gfidente|afk: ^ | 13:59 |
dprince | derekh: yep, I think we are back up again | 14:01 |
dprince | derekh: less than 24 hours almost this time. Maybe we just need to do it once a month so we don't forget things | 14:02 |
derekh | dprince: yup, although hopefully the new deployment should sort it out | 14:03 |
*** Marga_ has quit IRC | 14:03 | |
derekh | dprince: btw did you see the arp flood this time ? or was it just a traceback on a few compute nodes | 14:04 |
dprince | derekh: 4-5 compute hosts were un-pingable. | 14:06 |
dprince | derekh: all the ones I checked on the DRAC were in kernel panic | 14:06 |
openstackgerrit | Steven Hardy proposed openstack/tripleo-image-elements: Return derived config id when signalling deployments https://review.openstack.org/190282 | 14:06 |
dprince | derekh: I figure a bunch of neutron ports maybe couldn't get cleaned up because our neutron-server had the keystone auth issue | 14:07 |
shardy | dprince: ^^ after battling with jq for longer than I care to admit, that I think should now work for signalling the derived config ID via 99-refresh-completed | 14:07 |
derekh | dprince: ok, its sounds different to the reason we redeployed the computes nodes a few months ago | 14:07 |
dprince | derekh: then when I tried rebooting the servers I think we got ARP flooded, because the machines then wouldn't stay booted | 14:07 |
derekh | dprince: ahh ok | 14:08 |
dprince | derekh: so I think rebooting them causes problems... | 14:08 |
derekh | dprince: ack, not surprised | 14:08 |
dprince | derekh: at that point I was like... lets just spin up some fresh computes... | 14:08 |
derekh | dprince: yup, better then spending days trying to figure it all out | 14:09 |
dprince | derekh: well, I'm pretty sure the stale neutron keystone-signing dir was the root of all our problems this time | 14:09 |
derekh | dprince: ok | 14:10 |
dprince | derekh: because once the new computes were up I still couldn't manage ports or anything until I deleted that one | 14:10 |
dprince | derekh: I deleted a bunch of ports and reset the floatingips too | 14:11 |
dprince | derekh: just to keep things clean | 14:11 |
derekh | dprince: ack | 14:12 |
*** regebro has quit IRC | 14:21 | |
*** jprovazn has quit IRC | 14:22 | |
*** directxman12 has left #tripleo | 14:24 | |
*** Marga_ has joined #tripleo | 14:26 | |
openstackgerrit | Julia Kreger proposed openstack/diskimage-builder: Change simple-init to use a PATH variable https://review.openstack.org/192224 | 14:28 |
*** lazy_prince has joined #tripleo | 14:30 | |
*** Marga_ has quit IRC | 14:35 | |
*** Marga_ has joined #tripleo | 14:35 | |
*** stendulker has quit IRC | 14:50 | |
*** stendulker has joined #tripleo | 14:59 | |
*** cwolferh has joined #tripleo | 14:59 | |
*** rlandy has joined #tripleo | 15:00 | |
*** untriaged-bot has joined #tripleo | 15:01 | |
untriaged-bot | Untriaged bugs so far: | 15:01 |
untriaged-bot | https://bugs.launchpad.net/diskimage-builder/+bug/1461615 | 15:01 |
openstack | Launchpad bug 1461615 in diskimage-builder "Yum vars directory doesn't exist" [Undecided,In progress] - Assigned to Abel Lopez (al592b) | 15:01 |
untriaged-bot | https://bugs.launchpad.net/diskimage-builder/+bug/1461601 | 15:01 |
openstack | Launchpad bug 1461601 in diskimage-builder "CentOS 6 not resizing" [Undecided,Fix committed] - Assigned to Abel Lopez (al592b) | 15:01 |
untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1459465 | 15:01 |
openstack | Launchpad bug 1459465 in tripleo "devtest fails to build ramdisk on Fedora or CentOS due to Yum multilib error" [Undecided,Triaged] | 15:01 |
untriaged-bot | https://bugs.launchpad.net/os-collect-config/+bug/1463409 | 15:01 |
openstack | Launchpad bug 1463409 in os-collect-config "Do not tolerate world readable os-collect-config.conf" [Undecided,New] | 15:01 |
untriaged-bot | https://bugs.launchpad.net/diskimage-builder/+bug/1463357 | 15:01 |
openstack | Launchpad bug 1463357 in diskimage-builder "missing symlink support for ppc64 Fedora deployment ramdisk " [Undecided,In progress] - Assigned to yogananth subramanian (anantyog-o) | 15:01 |
untriaged-bot | https://bugs.launchpad.net/diskimage-builder/+bug/1461631 | 15:01 |
openstack | Launchpad bug 1461631 in diskimage-builder "New Fedora 22 cloud images fail to build" [Undecided,New] | 15:01 |
untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1461984 | 15:01 |
openstack | Launchpad bug 1461984 in tripleo "VM element: /usr/sbin/grub2-install: error: will not proceed with blocklists." [Undecided,New] | 15:01 |
untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1464692 | 15:01 |
openstack | Launchpad bug 1464692 in tripleo "debian build broken" [Undecided,New] | 15:01 |
*** untriaged-bot has quit IRC | 15:01 | |
*** jcoufal has joined #tripleo | 15:04 | |
*** Marga_ has quit IRC | 15:05 | |
*** paramite has quit IRC | 15:05 | |
*** Marga_ has joined #tripleo | 15:06 | |
*** Marga__ has joined #tripleo | 15:10 | |
*** Marga_ has quit IRC | 15:10 | |
*** yamahata has quit IRC | 15:12 | |
*** Marga__ has quit IRC | 15:19 | |
*** gfidente|afk is now known as gfidente | 15:24 | |
*** spzala has joined #tripleo | 15:25 | |
*** Goneri has quit IRC | 15:28 | |
*** Marga_ has joined #tripleo | 15:30 | |
*** rhallisey has quit IRC | 15:51 | |
*** rhallisey has joined #tripleo | 15:51 | |
openstackgerrit | Dan Prince proposed openstack/tripleo-heat-templates: Configurable vlan_id's for isolated networks https://review.openstack.org/189835 | 15:52 |
*** alop has joined #tripleo | 15:55 | |
*** sthillma has joined #tripleo | 15:57 | |
*** sthillma_ has joined #tripleo | 16:00 | |
*** sthillma has quit IRC | 16:02 | |
*** sthillma_ is now known as sthillma | 16:02 | |
*** daneyon has joined #tripleo | 16:11 | |
*** daneyon_ has joined #tripleo | 16:14 | |
*** daneyon has quit IRC | 16:17 | |
*** dasm|afk is now known as dasm | 16:19 | |
*** mcornea has quit IRC | 16:22 | |
*** daneyon_ has quit IRC | 16:25 | |
*** daneyon has joined #tripleo | 16:25 | |
*** masco has joined #tripleo | 16:27 | |
*** jcoufal has quit IRC | 16:28 | |
*** trown is now known as trown|lunch | 16:30 | |
*** Marga_ has quit IRC | 16:32 | |
*** Marga_ has joined #tripleo | 16:33 | |
*** alop has quit IRC | 16:33 | |
*** stendulker has quit IRC | 16:35 | |
*** penick has joined #tripleo | 16:36 | |
*** dtantsur is now known as dtantsur|afk | 16:40 | |
*** alop has joined #tripleo | 16:41 | |
*** Marga_ has quit IRC | 16:44 | |
*** Marga_ has joined #tripleo | 16:44 | |
*** Marga_ has quit IRC | 16:45 | |
*** Marga_ has joined #tripleo | 16:46 | |
*** hewbrocca has quit IRC | 16:47 | |
*** panda has quit IRC | 16:48 | |
*** panda has joined #tripleo | 16:49 | |
*** Marga_ has quit IRC | 16:49 | |
*** Marga_ has joined #tripleo | 16:49 | |
*** Marga_ has quit IRC | 16:49 | |
*** dsneddon has joined #tripleo | 16:50 | |
*** lucasagomes is now known as lucas-dinner | 16:56 | |
*** sthillma has quit IRC | 16:56 | |
*** penick has quit IRC | 16:58 | |
*** masco has quit IRC | 16:59 | |
EmilienM | dprince: I see a lot of overlap with what I'm doing on https://review.openstack.org/191195 and os-refresh-config/pre-configure.d/97-fedora-iptables | 17:02 |
EmilienM | gfidente: ^ | 17:02 |
EmilienM | do we have a status of the CI now? It looks broken to me | 17:03 |
dprince | EmilienM: should be up again now. We had a cloud outage yesterday and only got things back online around 10am this morning | 17:06 |
dprince | EmilienM: there is probably overlap between those two yes. The 97-fedora-iptables should go away if you are using puppet | 17:06 |
dprince | EmilienM: should probably go away that is | 17:06 |
*** derekh has quit IRC | 17:06 | |
EmilienM | dprince: ok I'll take care of it | 17:07 |
EmilienM | dprince: can I run recheck now? | 17:08 |
dprince | EmilienM: we've had a really bad week almost for TripleO CI. Hoping things will go better now. I'm still digging out of some of the issues from last week still | 17:08 |
dprince | EmilienM: yes, recheck away | 17:08 |
EmilienM | dprince: good luck :( | 17:08 |
*** Marga_ has joined #tripleo | 17:12 | |
openstackgerrit | greghaynes proposed openstack/diskimage-builder: Document only exectuables in phase dirs policy https://review.openstack.org/192292 | 17:16 |
*** Marga_ has quit IRC | 17:17 | |
*** sdake has quit IRC | 17:19 | |
*** sdake_ has joined #tripleo | 17:19 | |
openstackgerrit | OpenStack Proposal Bot proposed openstack/diskimage-builder: Updated from global requirements https://review.openstack.org/192296 | 17:19 |
*** Marga_ has joined #tripleo | 17:20 | |
*** Marga_ has quit IRC | 17:25 | |
openstackgerrit | OpenStack Proposal Bot proposed openstack/os-cloud-config: Updated from global requirements https://review.openstack.org/188483 | 17:25 |
openstackgerrit | OpenStack Proposal Bot proposed openstack/os-refresh-config: Updated from global requirements https://review.openstack.org/192302 | 17:25 |
*** pino|work has quit IRC | 17:26 | |
openstackgerrit | OpenStack Proposal Bot proposed openstack/tuskar-ui: Updated from global requirements https://review.openstack.org/192324 | 17:27 |
*** dsneddon has quit IRC | 17:28 | |
*** dsneddon has joined #tripleo | 17:34 | |
*** yamahata has joined #tripleo | 17:36 | |
*** pelix has quit IRC | 17:38 | |
*** athomas has quit IRC | 17:41 | |
*** Marga_ has joined #tripleo | 17:46 | |
*** trown|lunch is now known as trown | 17:52 | |
*** daneyon_ has joined #tripleo | 17:57 | |
*** marzif_ has joined #tripleo | 18:00 | |
*** daneyon has quit IRC | 18:00 | |
*** marzif has quit IRC | 18:01 | |
*** Marga_ has quit IRC | 18:07 | |
*** Marga_ has joined #tripleo | 18:07 | |
*** lazy_prince has quit IRC | 18:07 | |
*** sthillma has joined #tripleo | 18:08 | |
openstackgerrit | greghaynes proposed openstack/diskimage-builder: Add init-scripts directory support https://review.openstack.org/192341 | 18:16 |
*** dprince has quit IRC | 18:20 | |
openstackgerrit | greghaynes proposed openstack/diskimage-builder: Move simple-init data files out of phase subdirs https://review.openstack.org/192342 | 18:22 |
*** gfidente is now known as gfidente|afk | 18:24 | |
*** Marga_ has quit IRC | 18:25 | |
*** Marga_ has joined #tripleo | 18:26 | |
openstackgerrit | Dan Sneddon proposed openstack/tripleo-heat-templates: Fix allocation pool in external.yaml https://review.openstack.org/192349 | 18:28 |
*** rlandy has quit IRC | 18:32 | |
*** rlandy has joined #tripleo | 18:33 | |
*** penick has joined #tripleo | 18:35 | |
*** daneyon has joined #tripleo | 18:42 | |
*** daneyon_ has quit IRC | 18:45 | |
*** spredzy|afk is now known as spredzy | 18:50 | |
*** weshay has quit IRC | 18:50 | |
*** yamahata has quit IRC | 19:06 | |
*** Marga_ has quit IRC | 19:07 | |
*** sdake_ is now known as sdake | 19:13 | |
*** sdake has quit IRC | 19:15 | |
*** sdake has joined #tripleo | 19:15 | |
*** ukalifon has quit IRC | 19:27 | |
*** dasm is now known as dasm|afk | 19:42 | |
*** trown is now known as trown|outttypeww | 19:46 | |
EmilienM | I'm trying to debug https://review.openstack.org/#/c/191195/ | 19:52 |
EmilienM | but I don't see failure in logs | 19:52 |
EmilienM | any thoughts? | 19:52 |
slagle | EmilienM: you want to look in the os-collect-config logs | 19:54 |
slagle | http://logs.openstack.org/95/191195/3/check-tripleo/check-tripleo-ironic-overcloud-precise-nonha/d76d785/logs/overcloud-controller0_logs/os-collect-config.txt.gz | 19:54 |
slagle | looks like horizon failed to start: | 19:55 |
slagle | [ALERT] 166/191044 (9350) : Starting proxy horizon: cannot bind socket | 19:55 |
EmilienM | slagle: I did, but puppet is not run on ubuntu | 19:55 |
EmilienM | I'm looking at fedora logs | 19:55 |
slagle | ok... | 19:56 |
*** jcoufal has joined #tripleo | 19:57 | |
*** ifarkas has quit IRC | 19:58 | |
slagle | so in this one http://logs.openstack.org/95/191195/3/check-tripleo/check-tripleo-ironic-overcloud-f21puppet-ha/36574a0/console.html | 19:58 |
slagle | i see: | 19:58 |
slagle | 2015-06-16 18:20:56.002 | {"Status" : "SUCCESS","Reason" : "Configuration Complete","UniqueId" : "seedImageID","Data" : "Finished os-refresh-config."} | 19:58 |
slagle | 2015-06-16 18:20:56.003 | + wait_for -w 20 --delay 1 -- ping -c 1 192.0.2.1 | 19:58 |
slagle | 2015-06-16 18:21:17.021 | Timing out after 20 seconds: | 19:58 |
slagle | so it failed to ping the instance on the overcloud at the end | 19:58 |
slagle | and yea, i don't see any overcloud logs | 19:59 |
*** julim has quit IRC | 20:00 | |
*** Marga_ has joined #tripleo | 20:01 | |
*** sthillma has quit IRC | 20:12 | |
*** sdake_ has joined #tripleo | 20:16 | |
*** julim has joined #tripleo | 20:18 | |
*** sdake has quit IRC | 20:20 | |
*** jcoufal has quit IRC | 20:23 | |
*** spzala has quit IRC | 20:23 | |
*** jayg is now known as jayg|g0n3 | 20:30 | |
*** radek_ has quit IRC | 20:30 | |
openstackgerrit | imain proposed openstack/tripleo-incubator: Add support for using containers. https://review.openstack.org/188218 | 20:36 |
*** lblanchard has quit IRC | 20:40 | |
*** rlandy has quit IRC | 20:40 | |
lifeless | .oO | 20:42 |
*** rlandy has joined #tripleo | 20:46 | |
openstackgerrit | imain proposed openstack/tripleo-image-elements: Add support for full images. https://review.openstack.org/190303 | 20:51 |
*** akrivoka has quit IRC | 20:59 | |
*** penick has quit IRC | 21:00 | |
*** untriaged-bot has joined #tripleo | 21:01 | |
untriaged-bot | Untriaged bugs so far: | 21:01 |
untriaged-bot | https://bugs.launchpad.net/diskimage-builder/+bug/1461615 | 21:01 |
untriaged-bot | https://bugs.launchpad.net/diskimage-builder/+bug/1461601 | 21:01 |
untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1459465 | 21:01 |
untriaged-bot | https://bugs.launchpad.net/diskimage-builder/+bug/1463357 | 21:01 |
openstack | Launchpad bug 1461615 in diskimage-builder "Yum vars directory doesn't exist" [Undecided,In progress] - Assigned to Abel Lopez (al592b) | 21:01 |
untriaged-bot | https://bugs.launchpad.net/diskimage-builder/+bug/1461631 | 21:01 |
untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1461984 | 21:01 |
untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1464692 | 21:01 |
openstack | Launchpad bug 1461601 in diskimage-builder "CentOS 6 not resizing" [Undecided,Fix committed] - Assigned to Abel Lopez (al592b) | 21:01 |
*** untriaged-bot has quit IRC | 21:01 | |
openstack | Launchpad bug 1459465 in tripleo "devtest fails to build ramdisk on Fedora or CentOS due to Yum multilib error" [Undecided,Triaged] | 21:01 |
openstack | Launchpad bug 1463357 in diskimage-builder "missing symlink support for ppc64 Fedora deployment ramdisk " [Undecided,In progress] - Assigned to yogananth subramanian (anantyog-o) | 21:01 |
openstack | Launchpad bug 1461631 in diskimage-builder "New Fedora 22 cloud images fail to build" [Undecided,Fix committed] | 21:01 |
openstack | Launchpad bug 1461984 in tripleo "VM element: /usr/sbin/grub2-install: error: will not proceed with blocklists." [Undecided,Triaged] | 21:01 |
openstack | Launchpad bug 1464692 in tripleo "debian build broken" [Undecided,Triaged] | 21:01 |
*** penick has joined #tripleo | 21:02 | |
openstackgerrit | Steve Baker proposed openstack/tripleo-heat-templates: Ensure present/latest for puppet driven package updates https://review.openstack.org/190918 | 21:09 |
*** Marga_ has quit IRC | 21:09 | |
*** Marga_ has joined #tripleo | 21:10 | |
*** cwolferh has quit IRC | 21:20 | |
*** julim has quit IRC | 21:21 | |
openstackgerrit | greghaynes proposed openstack/diskimage-builder: Add test for centos 6 https://review.openstack.org/192440 | 21:26 |
*** Marga_ has quit IRC | 21:30 | |
*** Marga_ has joined #tripleo | 21:30 | |
*** Marga_ has quit IRC | 21:40 | |
*** Marga_ has joined #tripleo | 21:40 | |
*** gfidente|afk has quit IRC | 21:47 | |
*** rlandy has quit IRC | 21:51 | |
*** rlandy has joined #tripleo | 21:51 | |
*** penick has quit IRC | 21:59 | |
*** sdake_ is now known as sdake | 22:02 | |
*** penick has joined #tripleo | 22:18 | |
*** rhallisey has quit IRC | 22:22 | |
openstackgerrit | Steve Baker proposed openstack/tripleo-heat-templates: Allow control of hostname formatting https://review.openstack.org/191726 | 22:23 |
*** rlandy has quit IRC | 22:25 | |
*** rlandy has joined #tripleo | 22:26 | |
openstackgerrit | Ben Nemec proposed openstack/tripleo-incubator: Make sure configure-vm mac doesn't start negative https://review.openstack.org/192449 | 22:34 |
*** openstackgerrit has quit IRC | 22:38 | |
*** openstackgerrit has joined #tripleo | 22:39 | |
openstackgerrit | Merged openstack/tripleo-incubator: Generate multiple baremetal MACs in sequence https://review.openstack.org/183503 | 22:40 |
*** pbourke has quit IRC | 22:44 | |
*** pbourke has joined #tripleo | 22:44 | |
*** Marga_ has quit IRC | 22:55 | |
*** rlandy has quit IRC | 22:59 | |
*** daneyon has left #tripleo | 23:01 | |
*** shardy has quit IRC | 23:05 | |
*** Goneri has joined #tripleo | 23:11 | |
*** chlong has quit IRC | 23:15 | |
*** Marga_ has joined #tripleo | 23:19 | |
openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Update default Fedora to 22 https://review.openstack.org/185843 | 23:39 |
openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Cleanup yum downloading https://review.openstack.org/186620 | 23:39 |
*** Marga_ has quit IRC | 23:41 | |
*** rhefner has quit IRC | 23:41 | |
*** lucas-dinner has quit IRC | 23:41 | |
*** Goneri has quit IRC | 23:44 | |
*** Marga_ has joined #tripleo | 23:53 | |
openstackgerrit | imain proposed openstack/tripleo-heat-templates: Containerized TripleO Overcloud. https://review.openstack.org/178840 | 23:57 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!