Friday, 2014-07-04

*** yfaykya has joined #tripleo00:00
*** weshay has quit IRC00:02
derekh_SpamapS: ^^ may be the reason your patch for bug 1337230 isn't passing, so at this stage the revert might be the easiest way to sort out that problem so we can figure out what else is now wrong00:03
uvirtbotLaunchpad bug 1337230 in tripleo "Loosing connection to keystone during CI" [High,In progress] https://launchpad.net/bugs/133723000:03
derekh_anyways I gotta hit the hay, if its still a problem in the morning I'll start looking into it00:04
greghaynesderekh_: thats why were seeing the controller not boot?00:05
*** yfaykya has quit IRC00:05
greghaynesI was pretty confused as to how SpamapS' patch was causing that error, so that makes more sense ;)00:05
derekh_greghaynes: the bug I linked was causing intermitant failures all day, about 8 hours ago things look like they started getting worse00:06
greghaynesyes, I had been seeing that locally too00:06
derekh_greghaynes: all overcloud jobs have failed for the last 8 hours so I think we now have 2 problems00:06
*** yfaykya has joined #tripleo00:06
greghaynesthe recent one seems like overcloud is failing on boot00:07
greghaynessince we cant even ssh in to it to get logs :/00:07
*** eghobo has quit IRC00:08
derekh_greghaynes: so looks like that problem is earlier in the devtest process00:08
greghaynesoh?00:08
*** eghobo has joined #tripleo00:09
derekh_so basically I think we should merge https://review.openstack.org/#/c/104510/ to get ride of the earlier intermittent problem00:09
derekh_since it looks like it fixed that00:09
derekh_then we can figure out this new problem (overcloud failing on boot ? I havn't looked into it)00:10
greghayneshrm00:10
greghaynesre boot fail: http://logs.openstack.org/02/104602/1/check-tripleo/check-tripleo-overcloud-f20/a0b20c8/console.html#_2014-07-03_22_00_17_309 times out waiting for create complete and then after that cant tar up logs for controller node00:10
*** yfaykya has quit IRC00:11
greghaynesand pretty sure that is failing universally, consistently now00:12
greghayneslike you were saying :)00:12
* greghaynes kicks off fedora job to test00:12
*** yfaykya has joined #tripleo00:13
derekh_greghaynes: agreed, from looking at that log, looks like we can't talk to the controller00:14
*** jp_at_hp has joined #tripleo00:17
*** yfaykya has quit IRC00:17
*** lazy_prif has joined #tripleo00:18
derekh_ok, I'm off, will look in the morning if its still a problem00:19
*** lazy_prif is now known as lazy_prince00:19
*** yfaykya has joined #tripleo00:19
*** derekh_ has quit IRC00:19
greghaynesgn00:19
*** eghobo has quit IRC00:19
SpamapSwellll00:19
SpamapSI don't think we should rush off and merge the revert .. I'm reproducing "weirdness" locally now00:20
SpamapSbut by all means merge it if you guys think it will clear things up00:20
* SpamapS must step out for a bit00:20
greghaynesIt doesnt seem like it will actually get us anywhere since we the other bug which consistently fails ATM00:21
greghaynesso IMO fix that one first00:21
*** killer_prince has quit IRC00:21
greghaynes(as in, the consistently non-booting controller node first)00:22
*** yfaykya has quit IRC00:23
greghaynesin other news, there is a hilarious bug on trusty with my new hp hardware - my second interface gets named "rename3"00:24
*** yfaykya has joined #tripleo00:25
*** xuhaiwei has joined #tripleo00:27
openstackgerritSteve Kowalik proposed a change to openstack/tripleo-incubator: Move updating the BM networks to a new script  https://review.openstack.org/10472400:30
*** yfaykya has quit IRC00:30
*** yfaykya has joined #tripleo00:31
*** killer_prince has joined #tripleo00:32
openstackgerritSteve Kowalik proposed a change to openstack/tripleo-incubator: Move updating the BM networks to a new script  https://review.openstack.org/10472400:34
*** lazy_prince has quit IRC00:35
*** killer_prince is now known as lazy_prince00:35
*** yfaykya has quit IRC00:36
*** yfaykya has joined #tripleo00:37
*** yfaykya has quit IRC00:42
*** yfaykya has joined #tripleo00:43
openstackgerritSteve Kowalik proposed a change to openstack/tripleo-incubator: Move updating the BM networks to a new script  https://review.openstack.org/10472400:43
*** ramishra has joined #tripleo00:44
*** yfaykya has quit IRC00:47
*** ramishra has quit IRC00:49
*** yfaykya has joined #tripleo00:49
*** yfaykya has quit IRC00:54
*** yfaykya has joined #tripleo00:55
Nglifeless: right, few abortive attempts and the seed is up. I need to stop being up though, it is now bonkers late. I can turn backscroll into an undercloud tomorrow though, if you spew the bits at me00:56
openstackgerritSteve Kowalik proposed a change to openstack/tripleo-incubator: Move updating the BM networks to a new script  https://review.openstack.org/10472400:56
Ngoh, how we need an installer :)00:56
*** yfaykya has quit IRC01:00
*** yfaykya has joined #tripleo01:01
*** sseago has quit IRC01:03
lifelessNg: undercloud will be tricky, I haven't finished the vlan tweaks yet, will try over the weekend01:05
*** yfaykya has quit IRC01:06
*** yfaykya has joined #tripleo01:08
*** yamahata has joined #tripleo01:08
openstackgerritSteve Kowalik proposed a change to openstack/tripleo-incubator: Move updating the BM networks to a new script  https://review.openstack.org/10472401:08
*** yfaykya has quit IRC01:12
*** yfaykya has joined #tripleo01:14
*** sseago has joined #tripleo01:15
*** yfaykya has quit IRC01:19
*** gfidente has quit IRC01:20
*** yfaykya has joined #tripleo01:20
*** lazy_prince has quit IRC01:21
*** rwsu has quit IRC01:23
openstackgerritGhe Rivero proposed a change to openstack/tripleo-image-elements: Force heat_local collector on seed first boot  https://review.openstack.org/10473301:24
*** yfaykya has quit IRC01:25
*** yfaykya has joined #tripleo01:26
*** yfaykya has quit IRC01:31
*** yfaykya has joined #tripleo01:32
*** yfaykya has quit IRC01:37
*** yfaykya has joined #tripleo01:38
*** nosnos has joined #tripleo01:39
*** yfaykya has quit IRC01:43
*** yfaykya has joined #tripleo01:44
*** ramishra has joined #tripleo01:45
*** yfaykya has quit IRC01:49
*** ramishra has quit IRC01:49
*** yfaykya has joined #tripleo01:50
*** rcarrillocruz has joined #tripleo01:54
*** yfaykya has quit IRC01:55
*** rcarrill` has quit IRC01:55
*** yfaykya has joined #tripleo01:57
*** marun has quit IRC02:00
*** yfaykya has quit IRC02:01
*** yfaykya has joined #tripleo02:03
*** eghobo has joined #tripleo02:04
*** yfaykya has quit IRC02:08
*** yfaykya has joined #tripleo02:09
tchaypolifeless: StevenK: eta rhidesstation 12:3802:09
tchaypolifeless: The thing I want from you today is pictures02:10
jerryztchaypo: need some help on pxe boot02:11
tchaypojerryz: Ell me more02:12
tchaypoAlso tell me more02:12
jerryztchaypo: i started a vm as to boot from pxe, it got the dhcp offer but didn't honor it02:12
tchaypoDamn autocorrect02:12
jerryzcomplaining no filename found02:12
tchaypoHow don't public that it ...02:12
tchaypoAh02:12
StevenKtchaypo: Right, I'll leave in 5 or so02:12
StevenKtchaypo: (And likely still beat you)02:13
tchaypoAh right02:13
jerryztchaypo: i see in the dhcp offer, the filename and tftp server is in option field02:13
*** yfaykya has quit IRC02:13
tchaypoI'm 3 minutes I'll be in a train and can switch to a real keyboard02:13
tchaypoIn the meantime - does it say what file name was not found?02:14
tchaypoDoes the tftp server02:14
jerryztchaypo: i think it is expecting pxelinux.002:14
tchaypoHave any logs showing what was02:14
tchaypoRequested?02:15
*** yfaykya has joined #tripleo02:15
jerryztchaypo: i have a tcpdump02:15
jerryztchaypo: also i can capture the vnc screen02:15
tchaypotcpdump showing it making a request to the tftp server?02:18
tchaypoi guess if it was catching traffic coming in/out of the booting vm it would02:18
* tchaypo is on train now02:18
jerryztchaypo: no02:19
jerryztchaypo: soon after it got the dhcp offer, it stopped on an error02:19
jerryzno filename or root path specified02:19
tchayposo it never even tried to contact the tftp server?02:19
*** yfaykya has quit IRC02:19
jerryztchaypo: no, i think gpxe was not understanding the offer02:20
tchaypooh02:21
* tchaypo re-reads what you said02:21
tchaypoit didn't say "file not found" it said "no filename not found"02:21
*** yfaykya has joined #tripleo02:21
jerryztchaypo: no filename or root path specified02:22
tchaypohrm. odd.02:22
tchaypoI don't know what a valid bootp packet with pxe details looks like. do you have a sample from a working setup you can compare it with?02:22
jerryztchaypo: i don't have a working setup02:23
jerryztchaypo: i am using gpxe provided by qemu-kvm to simulate bm machines02:23
tchaypoI've just had a reminder of when SpamapS was interviewing me02:24
tchaypoand i said something about not being overly familiar with how pxe boot works02:24
tchaypoand spamaps said something like "don't worry, I'm a bit vague too, but don't worry, we rarely have to get that far down the stack"02:24
jerryztchaypo: do you have access to a working setup to check what gpxe version you are using?02:25
jerryztchaypo: mine is 0.9.702:25
*** jp_at_hp has quit IRC02:25
*** yfaykya has quit IRC02:26
tchaypoprobably02:26
tchaypothis is gpxe on the dhcp server, right?02:26
tchaypowhich in the case of my setup would be the seed vm02:26
jerryztchaypo: acctually on undercloud vm02:27
*** yfaykya has joined #tripleo02:27
jerryztchaypo: seed vm doesn't pxe boot, but i think in devtest, the TE vms are all the same, so it is fine02:28
tchaypoMy setup is --no-undercloud, so I'm fairly sure the seed is doing pxe for the uovercloud directly02:29
jerryztchaypo: yeah, the same for overcloud02:30
*** yfaykya has quit IRC02:31
*** yfaykya has joined #tripleo02:33
tchaypogetting off train and heading in to office02:36
jerryztchaypo: gPXE wants to find the boot filename in the dedicated slot in the reply packet.  dnsmasq does something clever/stupid (I’m not sure which) and puts it  someplace else. You have to use the –dhcp-no-override option to  dnsmasq to make gPXE see the filename.02:37
jerryztchapypo:  seems to be the problem02:37
*** yfaykya has quit IRC02:38
*** yfaykya has joined #tripleo02:39
*** ramishra has joined #tripleo02:41
*** yfaykya has quit IRC02:44
*** yfaykya has joined #tripleo02:46
*** yfaykya has quit IRC02:50
*** yfaykya has joined #tripleo02:52
*** yfaykya has quit IRC02:57
*** yfaykya has joined #tripleo02:58
*** eghobo has quit IRC02:58
*** untriaged-bot has joined #tripleo03:00
untriaged-botUntriaged bugs so far:03:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/133458103:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/133504203:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/133510503:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/133611003:00
untriaged-bothttps://bugs.launchpad.net/diskimage-builder/+bug/133593203:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/133611503:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/133747503:00
uvirtbotLaunchpad bug 1334581 in tripleo "os-collect-config.conf need support to Heat userdata" [Undecided,In progress]03:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/133501603:00
uvirtbotLaunchpad bug 1335042 in tripleo "diskimage-builder: extlinux installation failure" [Undecided,New]03:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/133364903:00
uvirtbotLaunchpad bug 1335105 in tuskar-ui "Devtest init script: sourcing tripleorc fails" [Undecided,New]03:00
uvirtbotLaunchpad bug 1336110 in tripleo "mysql does not start on boot post-upgrade" [High,New]03:00
untriaged-bothttps://bugs.launchpad.net/diskimage-builder/+bug/133245803:00
uvirtbotLaunchpad bug 1335932 in diskimage-builder "/bin/dib-run-parts fails for spanish locale" [Undecided,Fix committed]03:00
untriaged-bothttps://bugs.launchpad.net/diskimage-builder/+bug/133252103:00
uvirtbotLaunchpad bug 1336115 in tripleo "instances no longer running post-upgrade" [Undecided,In progress]03:00
uvirtbotLaunchpad bug 1337475 in tripleo "devtest can't move the gateway to another ip address other than the seed" [Undecided,In progress]03:00
*** untriaged-bot has quit IRC03:00
uvirtbotLaunchpad bug 1335016 in tripleo "diskimage-builder fails for Debian" [Undecided,New]03:00
uvirtbotLaunchpad bug 1333649 in tripleo "/etc/hosts entries written by hosts element doesn't survive reboot" [Undecided,In progress]03:00
uvirtbotLaunchpad bug 1332458 in diskimage-builder "Not able to build armhf or arm64 images on amd64 arch systems." [Undecided,In progress]03:00
uvirtbotLaunchpad bug 1332521 in diskimage-builder "failed to remove `/tmp/image.A2aupYJb/mnt/tmp/in_target.d': Device or resource busy" [Undecided,New]03:00
*** yfaykya has quit IRC03:02
*** yfaykya has joined #tripleo03:04
tchaypojerryz: back03:06
*** yfaykya has quit IRC03:08
*** ramishra has quit IRC03:08
tchaypojerryz: *facepalm*03:09
*** yfaykya has joined #tripleo03:10
openstackgerritTzu-Mainn Chen proposed a change to openstack/tuskar-ui: Code update for new Tuskar API  https://review.openstack.org/10466003:12
*** ramishra has joined #tripleo03:12
tchaypojerryz: at my last job i worked with a guy who in a previous role sat near one of the dnsmasq authors03:12
tchaypojerryz: he and i had lots of fun bickering over whether it was clever or stupid :)03:12
*** rushiagr_ has joined #tripleo03:14
*** yfaykya has quit IRC03:15
*** yfaykya has joined #tripleo03:16
*** nosnos has quit IRC03:17
*** lazy_prince has joined #tripleo03:20
*** ccrouch has joined #tripleo03:20
*** yfaykya has quit IRC03:21
*** yfaykya has joined #tripleo03:22
openstackgerritpatrick-crews proposed a change to openstack-infra/tripleo-ci: Alter how we grab and store machine files to use logstash indexing.  https://review.openstack.org/9925003:24
tchaypolifeless: are you on your way out here?03:25
*** yfaykya has quit IRC03:27
*** ajc_ has joined #tripleo03:28
*** ajc_ has quit IRC03:28
*** yfaykya has joined #tripleo03:29
*** rushiagr_ has quit IRC03:29
*** ajc_ has joined #tripleo03:30
*** yfaykya has quit IRC03:33
*** yfaykya has joined #tripleo03:35
StevenKtchaypo: http://logs.openstack.org/61/104461/1/check-tripleo/check-tripleo-overcloud-f20/f11765e/console.html03:36
*** yfaykya has quit IRC03:40
*** yfaykya has joined #tripleo03:41
*** yamahata has quit IRC03:42
*** yfaykya has quit IRC03:46
*** yfaykya has joined #tripleo03:47
openstackgerritTzu-Mainn Chen proposed a change to openstack/tuskar-ui: Code update for new Tuskar API  https://review.openstack.org/10466003:50
*** yfaykya has quit IRC03:52
*** yfaykya has joined #tripleo03:53
*** lazy_prince is now known as killer_prince03:55
*** yfaykya has quit IRC03:58
*** yfaykya has joined #tripleo03:59
*** yfaykya has quit IRC04:03
*** nosnos has joined #tripleo04:04
*** yfaykya has joined #tripleo04:05
openstackgerritTzu-Mainn Chen proposed a change to openstack/tuskar-ui: Code update for new Tuskar API  https://review.openstack.org/10466004:09
*** yfaykya has quit IRC04:10
*** yfaykya has joined #tripleo04:11
*** openstackgerrit has quit IRC04:14
greghayneshuh, os-c-c is spinning for me and spitting out "expected string or buffer"04:15
*** yfaykya has quit IRC04:16
*** yfaykya has joined #tripleo04:18
*** yfaykya has quit IRC04:22
*** yfaykya has joined #tripleo04:24
*** tzumainn has quit IRC04:28
*** yfaykya has quit IRC04:28
*** amitpp has joined #tripleo04:29
*** rushiagr_ has joined #tripleo04:30
*** yfaykya has joined #tripleo04:30
*** killer_prince has quit IRC04:32
tchaypoblame canada04:33
tchaypoand/or StevenK04:33
*** yfaykya has quit IRC04:34
greghayneshrmmm, thats a hard choice04:36
*** yfaykya has joined #tripleo04:36
StevenKgreghaynes: :-(04:38
*** yfaykya has quit IRC04:41
*** yfaykya has joined #tripleo04:42
adam_ghmm. whats the scoop with haproxy stealing port binds from services? os-c-c is failing b/c ceilometer-api is attempting to bind to host 0.0.0.0:8777 instead of (i assume) 192.0.0.24:8777. haproxy is listening on 192.0.0.25:8777. am i missing some recent change?04:43
greghaynesadam_g: theres several patches which should cause services to bind on localhost or node-ip and haproxy to bind on VIP04:44
adam_ggreghaynes, are the in flight or merged?04:45
greghaynesif they are competing sounds like youre missing the t-h-t or t-i-e patch which causes services to bind to non-wildcard04:45
adam_glooks like im getting haproxy on VIP, but services on 0.0.0.004:45
greghaynesadam_g: merged04:45
adam_ggreghaynes, cool, ill take a look04:45
*** yfaykya has quit IRC04:47
*** yfaykya has joined #tripleo04:48
tchaypog1104:51
*** yfaykya has quit IRC04:53
*** yfaykya has joined #tripleo04:54
adam_ggreghaynes, hmm. im running with it (assuming https://review.openstack.org/#/c/61376/6), but template is falling back to bind_host = {{local-ipv4}}04:56
*** dshulyak_ has joined #tripleo04:57
*** yfaykya has quit IRC04:59
greghayneslocal-ipv4 should be find, as long as its not wildcard04:59
greghayneshaproxy should be binding on vip04:59
*** yfaykya has joined #tripleo05:00
*** killer_prince has joined #tripleo05:04
*** killer_prince is now known as lazy_prince05:04
*** yfaykya has quit IRC05:05
adam_goh, is ee05:06
adam_gceilometer is being configured incorrectly05:06
*** yfaykya has joined #tripleo05:07
greghaynesincorrect config parameter?05:07
greghaynesWe definitely discovered that not all services use the same config param for bind_host05:07
adam_ggreghaynes, right, its in a different section / different flag. is there a bug opened or shall i?05:09
greghaynesI dont remember there being one for ceilometer05:09
*** dsneddon has quit IRC05:11
*** yfaykya has quit IRC05:11
adam_gwell, there is now :)05:11
greghayneswooo more bugs!05:12
adam_gwhere are templates stored within the booted image? ill hack that fix in now  and see what else is breaking05:12
greghaynes/usr/libexec/something05:12
*** yfaykya has joined #tripleo05:13
*** martyntaylor1 has joined #tripleo05:15
adam_gdo t-i-e bugs get tagged against tripleo?05:15
greghaynesyep05:15
adam_ggreghaynes, cool, now im failing on your issue:  2014-07-04 05:15:41.154 14250 CRITICAL os-collect-config [-] expected string or buffer05:16
greghayneswoo!05:16
*** dsneddon has joined #tripleo05:17
*** yfaykya has quit IRC05:17
*** yfaykya has joined #tripleo05:19
*** rushiagr_ has quit IRC05:19
*** rushiagr_ has joined #tripleo05:19
adam_ggreghaynes, https://review.openstack.org/#/c/104752/05:22
*** yfaykya has quit IRC05:24
*** yfaykya has joined #tripleo05:25
*** yfaykya has quit IRC05:30
*** yfaykya has joined #tripleo05:31
*** rcarrill` has joined #tripleo05:33
*** rcarrillocruz has quit IRC05:34
*** yfaykya has quit IRC05:36
adam_ggreghaynes, FYI the TypeError is raising from https://git.openstack.org/cgit/openstack/os-collect-config/tree/os_collect_config/cfn.py#n121, on the controller0.Metadata path.05:36
*** rakesh_hs has joined #tripleo05:36
adam_gi need to run. have a good 4th. hold onto enough fingers to type on monday. 'murica.05:37
greghayneshaha, gn05:37
*** yfaykya has joined #tripleo05:37
*** jtomasek_ has joined #tripleo05:40
*** ramishra_ has joined #tripleo05:40
*** ramishra has quit IRC05:41
*** yfaykya has quit IRC05:42
lifelesstchaypo: I am not05:43
*** yfaykya has joined #tripleo05:43
*** andreaf has joined #tripleo05:44
*** pcrews has quit IRC05:44
*** ramishra_ has quit IRC05:45
*** jcoufal has joined #tripleo05:47
*** yfaykya has quit IRC05:48
*** ramishra has joined #tripleo05:48
tchaypolifeless: so we gathered :p05:49
*** yfaykya has joined #tripleo05:49
tchaypoStevenK:05:51
tchaypoStevenK: https://review.openstack.org/#/c/104715/05:52
*** yfaykya has quit IRC05:54
*** yfaykya has joined #tripleo05:56
*** jprovazn has joined #tripleo05:56
*** lazy_prince is now known as killer_prince05:57
*** killer_prince is now known as lazy_prince05:58
lifelessmm, too late to ask greghaynes about the next HA patch :)05:59
greghayneslifeless: were blocked on CI fail :/05:59
lifelessgreghaynes: what are you doing awake :)05:59
lifelessgreghaynes: CI fail?05:59
greghaynesheh, tomorrow is holiday for us06:00
*** ramishra has quit IRC06:00
greghayneshttps://review.openstack.org/#/c/104510/ was causing spurious fails, since then there is a constant seeming unrelated fail going on with overcloud06:00
*** yfaykya has quit IRC06:00
greghayneshttp://logs.openstack.org/02/104602/1/check-tripleo/check-tripleo-overcloud-f20/a0b20c8/ is an example of the latter, seems live overcloud controller node just doesnt boot06:01
lifeless!06:01
*** yfaykya has joined #tripleo06:02
greghaynesfun times06:02
StevenKIt's only 11pm in Pacific time, so it's surely still early for greghaynes ...06:02
greghaynesYes, pretty early in GST06:02
lifelesshah06:02
StevenKGreg Standard Time, as opposed to Goods and Services Tax? :-P06:03
*** ramishra has joined #tripleo06:05
*** yfaykya has quit IRC06:06
*** yfaykya has joined #tripleo06:08
lifeless70M and gertty sync still running06:08
lifelessits not cheap.06:08
lifelessOTOH I'll have a nice juicy log file to look for redundant actions on etc when (if) it finishes06:08
*** rcarrillocruz has joined #tripleo06:10
*** rcarrill` has quit IRC06:12
*** yfaykya has quit IRC06:12
*** yfaykya has joined #tripleo06:14
*** dshulyak_ has quit IRC06:15
lifelessso06:16
lifelessvlan or pbr06:16
lifelessvlan I think06:16
*** pcrews has joined #tripleo06:17
*** rcarrill` has joined #tripleo06:17
*** mrunge has joined #tripleo06:18
*** yfaykya has quit IRC06:18
*** rushiagr_ has quit IRC06:19
*** rushiagr_ has joined #tripleo06:19
*** rcarrillocruz has quit IRC06:19
*** yfaykya has joined #tripleo06:20
*** yfaykya has quit IRC06:24
*** yfaykya has joined #tripleo06:26
*** yfaykya has quit IRC06:31
*** yfaykya has joined #tripleo06:32
*** yfaykya has quit IRC06:37
*** yfaykya has joined #tripleo06:38
boris-42lifeless hi man06:39
boris-42lifeless how are you?06:40
lifelessboris-42: hey, good. you ?06:41
boris-42lifeless good =) very good=)06:41
boris-42lifeless I finally finished work on osprofller06:41
boris-42lifeless and seems like everybody is happy with current API06:42
boris-42lifeless as I remember you were interested in this topic06:42
boris-42lifeless and I finished read me for this lib06:42
boris-42lifeless https://github.com/stackforge/osprofiler06:42
boris-42lifeless so it will be nice to get some feedback from you, before starting fully integrating it06:43
boris-42=)06:43
*** ramishra_ has joined #tripleo06:43
*** yfaykya has quit IRC06:43
*** yfaykya has joined #tripleo06:44
*** ramishra has quit IRC06:45
*** rcarrillocruz has joined #tripleo06:45
lifelessboris-42: I'll have a look early next week; if you haven't heard from me on wed please ping me!06:46
boris-42lifeless ok=)06:46
boris-42lifeless thanks06:46
*** rcarrill` has quit IRC06:46
lifelesshmm, gerritbot is quiet. somethings up :)06:46
boris-42=)06:48
*** ramishra_ has quit IRC06:48
*** ramishra has joined #tripleo06:48
StevenKlifeless: openstackgerrit is MIA06:50
*** yfaykya has quit IRC06:50
*** yfaykya has joined #tripleo06:51
tchaypolifeless: no, somethings down06:51
*** ramishra has quit IRC06:53
tchaypoI cant remember if I'd never seen https://bugs.launchpad.net/tripleo/+bug/1325114 before, or if it just makes more sense to me now06:53
uvirtbotLaunchpad bug 1325114 in tripleo "can't deploy underclouds with VLAN API endpoints" [High,In progress]06:53
lifelesstchaypo: 6/1 1/2 ot he other06:55
lifeless?06:55
*** yfaykya has quit IRC06:55
*** mrunge has quit IRC06:56
StevenKlifeless: I have put up a branch that switches ensure-bridge to using ip route replace06:57
*** yfaykya has joined #tripleo06:57
lifelesscool06:58
*** ramishra has joined #tripleo06:59
*** yfaykya has quit IRC07:01
*** mrunge has joined #tripleo07:01
*** yfaykya has joined #tripleo07:03
*** lucasagomes has joined #tripleo07:05
lifelesshmm07:05
lifelesswhy hasn't https://review.openstack.org/#/c/104052/ landed ?07:05
lifelesshmm gerttysync 1938. yay.07:06
*** yfaykya has quit IRC07:07
*** yfaykya has joined #tripleo07:09
*** andreaf has quit IRC07:13
*** yfaykya has quit IRC07:13
*** andreaf has joined #tripleo07:14
*** yfaykya has joined #tripleo07:15
*** pcrews has quit IRC07:17
*** yfaykya has quit IRC07:19
*** rdopieralski has joined #tripleo07:20
*** yfaykya has joined #tripleo07:21
*** ifarkas has joined #tripleo07:22
*** akrivoka has joined #tripleo07:22
*** yfaykya has quit IRC07:25
*** yfaykya has joined #tripleo07:27
proffalkenshadower - here? I just uploaded that last niggle on the check_mk auto-collect checks stuff :)07:30
shadowerproffalken: yup, I'll have a look07:31
shadowerproffalken: a link would speed things up :-)07:32
proffalkenawesome, thanks07:32
proffalkenlol07:32
proffalkenhttps://review.openstack.org/#/c/91324/07:32
proffalken:D07:32
shadowerthanks07:32
*** yfaykya has quit IRC07:32
proffalkenneed to pop out for the school run but happy to answer any other questions when I get back07:32
shadowercool07:32
*** yfaykya has joined #tripleo07:33
*** markmc has joined #tripleo07:34
*** markmc has quit IRC07:34
*** rcarrillocruz has quit IRC07:35
*** markmc has joined #tripleo07:35
*** yfaykya has quit IRC07:38
*** lsmola__ has joined #tripleo07:39
*** yfaykya has joined #tripleo07:40
*** rcarrillocruz has joined #tripleo07:40
*** yfaykya has quit IRC07:44
*** yfaykya has joined #tripleo07:46
*** yfaykya has quit IRC07:50
*** yfaykya has joined #tripleo07:52
*** ramishra has quit IRC07:52
*** ramishra has joined #tripleo07:53
*** rushiagr_ has quit IRC07:55
*** tzumainn has joined #tripleo07:56
*** yfaykya has quit IRC07:57
*** ramishra has quit IRC07:57
*** yfaykya has joined #tripleo07:58
*** shardy is now known as shardy_afk08:00
*** yfaykya has quit IRC08:03
*** yfaykya has joined #tripleo08:04
*** regebro has joined #tripleo08:07
*** yfaykya has quit IRC08:09
*** yfaykya has joined #tripleo08:10
*** akrivoka has quit IRC08:11
*** jp_at_hp has joined #tripleo08:13
*** gfidente has joined #tripleo08:13
*** akrivoka has joined #tripleo08:13
*** yfaykya has quit IRC08:15
*** derekh_ has joined #tripleo08:15
*** yfaykya has joined #tripleo08:16
*** ramishra has joined #tripleo08:17
*** openstackgerrit has joined #tripleo08:20
*** yfaykya has quit IRC08:21
*** yfaykya has joined #tripleo08:22
*** martyntaylor1 has left #tripleo08:23
*** rushiagr_ has joined #tripleo08:23
*** yfaykya has quit IRC08:27
*** yfaykya has joined #tripleo08:29
*** viktors has joined #tripleo08:31
*** viktors has left #tripleo08:31
*** tzumainn has quit IRC08:32
*** yfaykya has quit IRC08:33
*** yfaykya has joined #tripleo08:35
derekh_Has anybody figured out what going on with CI yet ?08:36
*** jtomasek_ has quit IRC08:38
*** jtomasek has joined #tripleo08:38
*** yfaykya has quit IRC08:39
*** yfaykya has joined #tripleo08:41
*** jcoufal has quit IRC08:42
*** yfaykya has quit IRC08:46
*** yfaykya has joined #tripleo08:47
*** pblaho has joined #tripleo08:51
*** yfaykya has quit IRC08:52
*** jcoufal has joined #tripleo08:53
*** yfaykya has joined #tripleo08:53
*** rcarrill` has joined #tripleo08:57
*** yfaykya has quit IRC08:58
*** yfaykya has joined #tripleo08:59
*** rcarrillocruz has quit IRC09:00
*** untriaged-bot has joined #tripleo09:00
untriaged-botUntriaged bugs so far:09:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/133458109:00
uvirtbotLaunchpad bug 1334581 in tripleo "os-collect-config.conf need support to Heat userdata" [Undecided,In progress]09:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/133504209:00
uvirtbotLaunchpad bug 1335042 in tripleo "diskimage-builder: extlinux installation failure" [Undecided,New]09:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/133510509:00
uvirtbotLaunchpad bug 1335105 in tuskar-ui "Devtest init script: sourcing tripleorc fails" [Undecided,New]09:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/133611009:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/133770709:00
untriaged-bothttps://bugs.launchpad.net/diskimage-builder/+bug/133593209:00
uvirtbotLaunchpad bug 1336110 in tripleo "mysql does not start on boot post-upgrade" [High,New]09:00
uvirtbotLaunchpad bug 1337707 in tripleo "overcloud fails to come up, ceilometer binding to 0.0.0.0 instead of local_ipv4 causing port conflict" [Undecided,In progress]09:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/133611509:00
uvirtbotLaunchpad bug 1335932 in diskimage-builder "/bin/dib-run-parts fails for spanish locale" [Undecided,Fix committed]09:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/133747509:00
uvirtbotLaunchpad bug 1336115 in tripleo "instances no longer running post-upgrade" [Undecided,In progress]09:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/133501609:00
uvirtbotLaunchpad bug 1337475 in tripleo "devtest can't move the gateway to another ip address other than the seed" [Undecided,In progress]09:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/133364909:00
uvirtbotLaunchpad bug 1335016 in tripleo "diskimage-builder fails for Debian" [Undecided,New]09:00
untriaged-bothttps://bugs.launchpad.net/diskimage-builder/+bug/133245809:00
uvirtbotLaunchpad bug 1333649 in tripleo "/etc/hosts entries written by hosts element doesn't survive reboot" [Undecided,In progress]09:00
uvirtbotLaunchpad bug 1332458 in diskimage-builder "Not able to build armhf or arm64 images on amd64 arch systems." [Undecided,In progress]09:00
untriaged-bothttps://bugs.launchpad.net/diskimage-builder/+bug/133252109:00
uvirtbotLaunchpad bug 1332521 in diskimage-builder "failed to remove `/tmp/image.A2aupYJb/mnt/tmp/in_target.d': Device or resource busy" [Undecided,New]09:00
*** untriaged-bot has quit IRC09:00
*** yfaykya has quit IRC09:04
derekh_Hmmm, node came up / gets IP / runs o-r-r / sends success  and then looses it IP address09:05
derekh_Jul 04 08:38:42 undercloud-undercloud-il6jftz7thss os-collect-config[711]: 2014-07-04 08:38:42.719 711 CRITICAL os-collect-config [-] expected string or buffer09:05
derekh_Jul 04 08:38:42 undercloud-undercloud-il6jftz7thss dhclient[3210]: Received signal 15, initiating shutdown.09:05
derekh_Jul 04 08:38:42 undercloud-undercloud-il6jftz7thss dhclient[3210]: DHCPRELEASE on br-ctlplane to 192.0.2.2 port 67 (xid=0x2a508744)09:05
*** yfaykya has joined #tripleo09:05
*** athomas has joined #tripleo09:09
*** yfaykya has quit IRC09:10
*** tserong has quit IRC09:10
*** yfaykya has joined #tripleo09:11
*** ramishra has quit IRC09:13
*** yfaykya has quit IRC09:16
*** yfaykya has joined #tripleo09:18
openstackgerrittherese-mchale proposed a change to openstack/tripleo-incubator: Setup MySQL database passwords  https://review.openstack.org/10462809:18
*** yfaykya has quit IRC09:22
*** tserong has joined #tripleo09:23
*** tserong has joined #tripleo09:23
*** yfaykya has joined #tripleo09:24
*** yfaykya has quit IRC09:28
*** derekh_ changes topic to "FIREDRILL https://bugs.launchpad.net/tripleo/+bug/1337772 | Using OpenStack to deploy OpenStack;meetings Tuesday 1900//0700 UTC in #openstack-meeting-alt"09:29
*** yfaykya has joined #tripleo09:30
*** yfaykya has quit IRC09:34
*** yfaykya has joined #tripleo09:36
openstackgerrittherese-mchale proposed a change to openstack/tripleo-incubator: Setup MySQL database passwords  https://review.openstack.org/10462809:40
*** yfaykya has quit IRC09:41
*** yfaykya has joined #tripleo09:42
*** yfaykya has quit IRC09:47
*** yfaykya has joined #tripleo09:48
openstackgerritNicholas Randon proposed a change to openstack/tripleo-incubator: Improve readability of JQ for novaBM and ironic  https://review.openstack.org/10455909:52
openstackgerritNicholas Randon proposed a change to openstack/tripleo-incubator: Improve readability of JQ for novaBM and ironic  https://review.openstack.org/10455909:52
*** yfaykya has quit IRC09:53
*** yfaykya has joined #tripleo09:54
*** rushiagr_ has quit IRC09:55
*** rushiagr_ has joined #tripleo09:55
*** yfaykya has quit IRC09:59
*** yfaykya has joined #tripleo10:00
*** yfaykya has quit IRC10:05
*** yfaykya has joined #tripleo10:07
*** ramishra has joined #tripleo10:09
*** yfaykya has quit IRC10:11
*** yfaykya has joined #tripleo10:13
*** yfaykya has quit IRC10:17
*** yfaykya has joined #tripleo10:19
*** yfaykya has quit IRC10:23
*** yfaykya has joined #tripleo10:25
*** yfaykya has quit IRC10:29
*** pelix has joined #tripleo10:30
*** yfaykya has joined #tripleo10:31
*** yfaykya has quit IRC10:36
*** ramishra has quit IRC10:37
*** yfaykya has joined #tripleo10:37
*** yfaykya has quit IRC10:41
*** yfaykya has joined #tripleo10:43
*** rushiagr_ has quit IRC10:47
*** xuhaiwei has quit IRC10:47
*** yfaykya has quit IRC10:48
*** yfaykya has joined #tripleo10:49
*** yfaykya has quit IRC10:54
*** yfaykya has joined #tripleo10:56
*** nosnos has quit IRC11:00
*** nosnos has joined #tripleo11:00
*** yfaykya has quit IRC11:00
*** yfaykya has joined #tripleo11:02
*** nosnos_ has joined #tripleo11:03
*** nosnos has quit IRC11:04
*** yfaykya has quit IRC11:06
*** yfaykya has joined #tripleo11:08
*** jp_at_hp has quit IRC11:12
*** yfaykya has quit IRC11:12
*** yfaykya has joined #tripleo11:14
*** yfaykya has quit IRC11:18
*** yfaykya has joined #tripleo11:20
*** rushiagr has joined #tripleo11:23
*** yfaykya has quit IRC11:25
*** yfaykya has joined #tripleo11:26
*** ifarkas has quit IRC11:29
*** yfaykya has quit IRC11:31
openstackgerritDerek Higgins proposed a change to openstack-infra/tripleo-ci: Test revert  https://review.openstack.org/10487311:32
*** yfaykya has joined #tripleo11:32
*** yfaykya has quit IRC11:37
*** dshulyak_ has joined #tripleo11:37
*** ramishra has joined #tripleo11:38
*** yfaykya has joined #tripleo11:38
*** dshulyak_ has quit IRC11:40
*** ramishra has quit IRC11:42
*** yfaykya has quit IRC11:44
*** yfaykya has joined #tripleo11:45
*** yfaykya has quit IRC11:49
*** openstackgerrit has quit IRC11:49
*** openstackstatus has quit IRC11:50
*** openstack has joined #tripleo11:51
*** openstackstatus has joined #tripleo11:52
*** ChanServ sets mode: +v openstackstatus11:52
*** jtomasek has quit IRC11:54
*** yfaykya has quit IRC11:55
*** yfaykya has joined #tripleo11:57
*** yfaykya has quit IRC12:01
*** yfaykya has joined #tripleo12:03
*** ajc_ has quit IRC12:07
*** yfaykya has quit IRC12:08
*** yfaykya has joined #tripleo12:09
*** yfaykya has quit IRC12:14
*** yfaykya has joined #tripleo12:15
*** yfaykya has quit IRC12:20
proffalkenHey all, if any cores are around to review https://review.openstack.org/#/c/91324/ that would be appreciated, I already have a +2, I just need the magic +2/+A... :)12:20
*** yfaykya has joined #tripleo12:21
*** yamahata has joined #tripleo12:22
proffalkenalso, if anyone knows how to get hold of Richard Su and can ask him to take a look at my comments on https://review.openstack.org/#/c/87223/ that would be appreicated.12:24
proffalkenand finally, if slagle has a few minutes can you revisit https://review.openstack.org/#/c/87226/ for me please and let me know what you think?12:25
*** yfaykya has quit IRC12:26
gfidenteproffalken, richard is rwsu12:27
gfidentebut he doesn't seem to be online ATM12:27
*** yfaykya has joined #tripleo12:27
proffalkengfidente: ah, cool, thanks12:31
*** yfaykya has quit IRC12:32
*** rlandy has joined #tripleo12:33
*** yfaykya has joined #tripleo12:34
Ngmorning12:34
*** yfaykya has quit IRC12:38
*** ramishra has joined #tripleo12:39
*** yfaykya has joined #tripleo12:40
*** yamahata has quit IRC12:40
*** yamahata has joined #tripleo12:41
*** ramishra has quit IRC12:42
*** ramishra has joined #tripleo12:43
*** yfaykya has quit IRC12:44
*** yfaykya has joined #tripleo12:46
*** tzumainn has joined #tripleo12:48
*** ramishra has quit IRC12:48
*** lucasagomes has quit IRC12:50
*** yfaykya has quit IRC12:50
*** yfaykya has joined #tripleo12:52
*** ifarkas has joined #tripleo12:53
*** yfaykya has quit IRC12:57
*** yfaykya has joined #tripleo12:58
*** amitpp has quit IRC13:01
*** athomas has quit IRC13:01
*** yfaykya has quit IRC13:03
*** yfaykya has joined #tripleo13:04
*** yfaykya has quit IRC13:09
*** yfaykya has joined #tripleo13:10
*** nosnos_ has quit IRC13:12
*** yfaykya has quit IRC13:15
*** yfaykya has joined #tripleo13:16
*** IvanBerezovskiy has joined #tripleo13:19
*** yfaykya has quit IRC13:21
*** yfaykya has joined #tripleo13:23
*** rcarrillocruz has joined #tripleo13:24
*** rcarrill` has quit IRC13:26
*** yfaykya has quit IRC13:27
*** yfaykya has joined #tripleo13:29
*** ccrouch has quit IRC13:31
shadowerproffalken: rwsu is on the west coast of the US iirc. So he should be online in a few hours13:32
*** yfaykya has quit IRC13:33
openstackgerritDerek Higgins proposed a change to openstack-infra/tripleo-ci: Test revert  https://review.openstack.org/10489113:35
*** yfaykya has joined #tripleo13:35
derekh_shadower: probably not  -   July 4th13:38
shadoweryeah was just writing that...13:38
*** rushiagr has quit IRC13:39
*** yfaykya has quit IRC13:39
*** yfaykya has joined #tripleo13:41
*** rcarrill` has joined #tripleo13:41
*** rcarrillocruz has quit IRC13:44
*** yfaykya has quit IRC13:45
*** yfaykya has joined #tripleo13:47
proffalkenshadower: no worries, I'll talk to him next week13:48
*** yfaykya has quit IRC13:52
*** yfaykya has joined #tripleo13:53
*** yfaykya has quit IRC13:58
*** mrunge has quit IRC13:59
*** yfaykya has joined #tripleo13:59
openstackgerritTomas Sedovic proposed a change to openstack/tripleo-specs: Remove merge.py from TripleO Heat Templates  https://review.openstack.org/9793914:02
*** yfaykya has quit IRC14:04
*** yfaykya has joined #tripleo14:05
openstackgerritNicholas Randon proposed a change to openstack/tripleo-incubator: Improve readability of JQ for novaBM and ironic  https://review.openstack.org/10455914:09
*** jprovazn has quit IRC14:10
*** yfaykya has quit IRC14:10
*** yfaykya has joined #tripleo14:12
*** moorryan_ has joined #tripleo14:12
moorryan_bnemec please revisit https://review.openstack.org/#/c/85082/14:12
*** moorryan has quit IRC14:13
*** yfaykya has quit IRC14:16
openstackgerritA change was merged to openstack/tripleo-incubator: Make a separation between --heat-env  https://review.openstack.org/9309114:17
*** yfaykya has joined #tripleo14:18
*** akrivoka has quit IRC14:18
*** rcarrill` has quit IRC14:20
*** rcarrillocruz has joined #tripleo14:22
*** yfaykya has quit IRC14:22
*** yfaykya has joined #tripleo14:24
openstackgerritNicholas Randon proposed a change to openstack/tripleo-incubator: Clean-up os-apply-config lines in devtest.  https://review.openstack.org/10128614:24
*** lsmola__ has quit IRC14:28
*** yfaykya has quit IRC14:28
*** yfaykya has joined #tripleo14:30
*** rakesh_hs has quit IRC14:34
openstackgerritjan grant proposed a change to openstack/tripleo-image-elements: Permit non-loopback guest access to Rabbit.  https://review.openstack.org/9281514:35
*** yfaykya has quit IRC14:35
*** yfaykya has joined #tripleo14:36
*** yfaykya has quit IRC14:41
*** yfaykya has joined #tripleo14:42
openstackgerritDerek Higgins proposed a change to openstack-infra/tripleo-ci: Test revert  https://review.openstack.org/10490914:43
*** rcarrill` has joined #tripleo14:45
*** yfaykya has quit IRC14:46
*** rcarrillocruz has quit IRC14:47
*** yfaykya has joined #tripleo14:48
*** yfaykya has quit IRC14:53
*** lazy_prince is now known as killer_prince14:53
*** yfaykya has joined #tripleo14:54
*** yfaykya has quit IRC14:59
*** untriaged-bot has joined #tripleo15:00
untriaged-botUntriaged bugs so far:15:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/133458115:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/133504215:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/133510515:00
uvirtbotLaunchpad bug 1334581 in tripleo "os-collect-config.conf need support to Heat userdata" [Undecided,In progress]15:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/133611015:00
uvirtbotLaunchpad bug 1335042 in tripleo "diskimage-builder: extlinux installation failure" [Undecided,New]15:00
uvirtbotLaunchpad bug 1335105 in tuskar-ui "Devtest init script: sourcing tripleorc fails" [Undecided,New]15:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/133770715:00
uvirtbotLaunchpad bug 1336110 in tripleo "mysql does not start on boot post-upgrade" [High,New]15:00
untriaged-bothttps://bugs.launchpad.net/diskimage-builder/+bug/133593215:00
uvirtbotLaunchpad bug 1337707 in tripleo "overcloud fails to come up, ceilometer binding to 0.0.0.0 instead of local_ipv4 causing port conflict" [Undecided,In progress]15:00
uvirtbotLaunchpad bug 1335932 in diskimage-builder "/bin/dib-run-parts fails for spanish locale" [Undecided,Fix committed]15:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/133611515:00
uvirtbotLaunchpad bug 1336115 in tripleo "instances no longer running post-upgrade" [Undecided,In progress]15:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/133747515:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/133501615:00
uvirtbotLaunchpad bug 1337475 in tripleo "devtest can't move the gateway to another ip address other than the seed" [Undecided,In progress]15:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/133364915:00
uvirtbotLaunchpad bug 1335016 in tripleo "diskimage-builder fails for Debian" [Undecided,New]15:00
uvirtbotLaunchpad bug 1333649 in tripleo "/etc/hosts entries written by hosts element doesn't survive reboot" [Undecided,In progress]15:00
untriaged-bothttps://bugs.launchpad.net/diskimage-builder/+bug/133245815:00
uvirtbotLaunchpad bug 1332458 in diskimage-builder "Not able to build armhf or arm64 images on amd64 arch systems." [Undecided,In progress]15:00
untriaged-bothttps://bugs.launchpad.net/diskimage-builder/+bug/133252115:00
uvirtbotLaunchpad bug 1332521 in diskimage-builder "failed to remove `/tmp/image.A2aupYJb/mnt/tmp/in_target.d': Device or resource busy" [Undecided,New]15:00
*** untriaged-bot has quit IRC15:00
*** yfaykya has joined #tripleo15:00
*** IvanBerezovskiy has left #tripleo15:02
*** yfaykya has quit IRC15:06
*** yfaykya has joined #tripleo15:07
*** ifarkas has quit IRC15:09
*** ramishra has joined #tripleo15:10
*** yfaykya has quit IRC15:11
*** yfaykya has joined #tripleo15:13
*** rcarrillocruz has joined #tripleo15:13
*** rcarrill` has quit IRC15:15
*** ramishra has quit IRC15:15
*** yfaykya has quit IRC15:17
*** yfaykya has joined #tripleo15:19
*** robsparker has quit IRC15:20
*** yfaykya has quit IRC15:23
*** yfaykya has joined #tripleo15:25
*** pblaho has quit IRC15:28
*** rdopieralski has quit IRC15:29
*** yfaykya has quit IRC15:30
*** rushiagr has joined #tripleo15:30
*** yfaykya has joined #tripleo15:31
*** ccrouch has joined #tripleo15:31
*** markmc has quit IRC15:33
*** robsparker has joined #tripleo15:33
*** yfaykya has quit IRC15:36
*** yfaykya has joined #tripleo15:37
*** yfaykya has quit IRC15:42
*** yfaykya has joined #tripleo15:43
*** yfaykya has quit IRC15:48
*** yfaykya has joined #tripleo15:49
*** gfidente has quit IRC15:51
*** yfaykya has quit IRC15:54
*** yfaykya has joined #tripleo15:56
*** rcarrillocruz has quit IRC15:59
*** yfaykya has quit IRC16:00
*** rcarrillocruz has joined #tripleo16:00
*** yfaykya has joined #tripleo16:02
*** yfaykya has quit IRC16:06
*** yfaykya has joined #tripleo16:08
*** killer_prince is now known as lazy_prince16:10
*** jcoufal has quit IRC16:10
*** ramishra has joined #tripleo16:12
*** yfaykya has quit IRC16:13
*** yfaykya has joined #tripleo16:14
*** ramishra has quit IRC16:16
*** dshulyak_ has joined #tripleo16:16
bnemecmoorryan_: It looks like the last CI run got lost, and none of the previous ones had passed either, so that needs to happen before it can merge.16:17
*** yfaykya has quit IRC16:19
* bnemec stops working on a holiday :-)16:20
*** yfaykya has joined #tripleo16:20
SpamapSwhoa..16:21
SpamapS| NovaCompute0AllNodes      | fb9d8bb8-fedb-44ff-a69a-f2481556cf60 | state changed                                                                                                                     | CREATE_COMPLETE    | 2014-07-04T00:51:36Z |16:22
SpamapS| controller0AllNodesConfig | e716bb72-831f-46a3-af7b-12da73366b7c | HTTPUnauthorized: ERROR: Authentication failed. Please try again with option --include-password or export HEAT_INCLUDE_PASSWORD=1 | CREATE_FAILED      | 2014-07-04T04:30:51Z |16:22
SpamapS|                           |                                      | Authentication required16:22
SpamapSnote the *3.5 hour gap*16:22
*** yfaykya has quit IRC16:25
*** yfaykya has joined #tripleo16:26
*** dshulyak_ has quit IRC16:29
SpamapSsuspicious..16:30
SpamapSJul  4 04:30:01 undercloud-undercloud-4nalwyucn422 CRON[4364]: (heat) CMD (heat-manage purge_deleted 1)16:30
*** rcarrillocruz has quit IRC16:30
*** yfaykya has quit IRC16:31
SpamapSI wonder if we have some kind of infinite loop in the allnodes config16:32
*** yfaykya has joined #tripleo16:32
SpamapS_weird_16:34
SpamapSall hosts made it onto compute.. then it's almost like heat just went into la-la land for 3.5 hours16:34
*** rcarrillocruz has joined #tripleo16:36
*** yfaykya has quit IRC16:37
*** dshulyak_ has joined #tripleo16:38
*** yfaykya has joined #tripleo16:38
SpamapShttp://paste.ubuntu.com/7747825/16:41
SpamapS1.1G of tokens16:42
SpamapS9000 tokens per hour16:42
SpamapSdutifully deleted, 18,000 at a time every 2 hours16:43
*** yfaykya has quit IRC16:43
SpamapSmorganfainberg: ^^ can haz no storage tokenzzz .. kthxbai16:43
SpamapSaaand now keystone crashed16:44
*** yfaykya has joined #tripleo16:45
SpamapSJul  4 16:44:19 undercloud-undercloud-4nalwyucn422 kernel: [58689.424335] keystone-all[28923]: segfault at 14 ip 0819674b sp bfaa0160 error 6 in python[8048000+2a9000]16:45
SpamapS[58689.764887] init: keystone main process ended, respawning16:45
SpamapSthanks upstart, for masking my problems. ;)16:46
SpamapS[58560.326792] init: neutron-openvswitch-agent main process (28853) killed by SEGV signal16:46
SpamapSruh roh16:46
SpamapSOOM16:47
*** yfaykya has quit IRC16:50
SpamapS2014-07-04 16:49:56.241 29153 TRACE keystone.common.wsgi OSError: [Errno 12] Cannot allocate memory16:50
*** yfaykya has joined #tripleo16:51
* SpamapS reboots undercloud w/ 3G RAM16:51
*** dshulyak_ has quit IRC16:54
morganfainbergSpamapS, hi16:54
derekh_ok, I've reproduced /topic locally, the problem goes away when I set heat back a few commits DIB_REPOREF_heat=706f7289fa859fa4561b3fbb163f60b63efbd74616:54
morganfainbergSpamapS, token table ick :(16:55
SpamapSderekh_: yeah something weird is going on in Heat16:55
derekh_but my test revert doesn't look like its going to work https://review.openstack.org/#/c/104909/16:55
morganfainbergSpamapS, yes yes working on non-persistent tokens!16:55
*** yfaykya has quit IRC16:55
derekh_hmm the double revert didn't work16:56
derekh_2014-07-04 15:00:35.523 | error: could not revert a60f272... Convert WaitConditionHandle to use handle_signal16:56
*** yfaykya has joined #tripleo16:57
derekh_ok, I gotta run, overcloud ci jobs still busted anything I have that relevant is in bug 133777217:00
uvirtbotLaunchpad bug 1337772 in tripleo "ci: Failing to connect to overcloud controller" [Critical,Triaged] https://launchpad.net/bugs/133777217:00
*** rcarrill` has joined #tripleo17:01
*** yfaykya has quit IRC17:02
*** rcarrillocruz has quit IRC17:03
*** yfaykya has joined #tripleo17:03
SpamapSderekh_: thanks for working on it17:03
openstackgerritDerek Higgins proposed a change to openstack-infra/tripleo-ci: Pin to a specifc version of Heat  https://review.openstack.org/10493717:06
derekh_SpamapS: no prob, ^^ is just a test to see if it works, not necessarily looking for it to be merged, unless things drag on for a long time17:06
SpamapSderekh_: looks like we regressed CFN api _again_ in Heat17:07
SpamapSderekh_: I started on a tempest test 2 weeks ago to stop that from happening but EDISTRACTIONS17:07
derekh_SpamapS: I know the feeling17:07
*** yfaykya has quit IRC17:08
derekh_SpamapS: anyways have a good weekend17:08
*** derekh_ has quit IRC17:08
* SpamapS vows to force derekh_ to have a bouncer the next time we are physically in the same place17:08
*** yfaykya has joined #tripleo17:09
*** yfaykya has quit IRC17:13
*** yfaykya has joined #tripleo17:15
*** yamahata has quit IRC17:17
*** yfaykya has quit IRC17:20
*** jang has joined #tripleo17:21
*** yfaykya has joined #tripleo17:21
*** yfaykya has quit IRC17:26
*** yfaykya has joined #tripleo17:27
SpamapSFYI, 3 reverts submitted to fix CI17:31
SpamapShttps://review.openstack.org/#/c/104940/17:32
* SpamapS goes to do holiday stuff17:32
*** yfaykya has quit IRC17:32
*** yfaykya has joined #tripleo17:34
*** yfaykya has quit IRC17:38
*** dshulyak_ has joined #tripleo17:38
*** yfaykya has joined #tripleo17:40
*** dshulyak_ has quit IRC17:44
*** yfaykya has quit IRC17:44
*** yfaykya has joined #tripleo17:46
*** yfaykya has quit IRC17:50
*** yfaykya has joined #tripleo17:52
morganfainbergSpamapS, have a good holiday!17:57
*** yfaykya has quit IRC17:57
*** yfaykya has joined #tripleo17:58
*** yfaykya has quit IRC18:03
*** yfaykya has joined #tripleo18:04
*** yfaykya has quit IRC18:09
*** yfaykya has joined #tripleo18:10
*** yfaykya has quit IRC18:15
*** yfaykya has joined #tripleo18:16
*** yfaykya has quit IRC18:21
*** yfaykya has joined #tripleo18:23
*** yfaykya has quit IRC18:27
*** pblaho has joined #tripleo18:27
*** yfaykya has joined #tripleo18:29
*** yfaykya has quit IRC18:33
*** pelix has quit IRC18:34
*** yfaykya has joined #tripleo18:35
greghayneshuh. http://aphyr.com/posts/315-call-me-maybe-rabbitmq18:36
greghaynesinteresting given that we just enabled auto partition handling... and we kind of rely on it18:37
*** bcrochet has quit IRC18:37
*** yfaykya has quit IRC18:40
greghaynesI wonder if we might want to eventually not auto heal partitions and do the haproxy drain / failover / restart dance18:41
*** yfaykya has joined #tripleo18:41
*** yfaykya has quit IRC18:45
*** yfaykya has joined #tripleo18:47
*** rushiagr has quit IRC18:48
*** yfaykya has quit IRC18:52
*** yfaykya has joined #tripleo18:53
*** yfaykya has quit IRC18:58
*** yfaykya has joined #tripleo18:59
openstackgerrityolanda.robla proposed a change to openstack/diskimage-builder: Remove cloud-init-datasources element from ubuntu dependencies.  https://review.openstack.org/10494919:02
*** yfaykya has quit IRC19:04
*** yfaykya has joined #tripleo19:05
*** yfaykya has quit IRC19:10
openstackgerritpatrick-crews proposed a change to openstack-infra/tripleo-ci: Alter how we grab and store machine files to use logstash indexing.  https://review.openstack.org/9925019:11
*** yfaykya has joined #tripleo19:12
*** pblaho has quit IRC19:12
*** ramishra has joined #tripleo19:14
*** yfaykya has quit IRC19:16
*** yfaykya has joined #tripleo19:18
*** ramishra has quit IRC19:18
greghayneslifeless: I figure you might want to comment on https://review.openstack.org/#/c/97283/19:22
*** yfaykya has quit IRC19:22
*** yfaykya has joined #tripleo19:24
*** pcrews has joined #tripleo19:24
lazy_princeplease review https://review.openstack.org/#/c/101577/ (support for uboot)19:29
*** yfaykya has quit IRC19:29
SpamapSgreghaynes: IIRC the public cloud folks do the drain/failover dance.19:30
*** yfaykya has joined #tripleo19:30
*** gcha has quit IRC19:34
*** yfaykya has quit IRC19:34
*** yfaykya has joined #tripleo19:36
openstackgerritGregory Haynes proposed a change to openstack-infra/tripleo-ci: Pin to a specifc version of Heat  https://review.openstack.org/10493719:38
*** yfaykya has quit IRC19:41
*** noslzzp has quit IRC19:41
*** yfaykya has joined #tripleo19:42
*** yfaykya has quit IRC19:47
SpamapS"is locked by host ubuntu, please retry after the current operation is completed"19:48
SpamapSironic log19:48
SpamapSknown bug?19:48
*** yfaykya has joined #tripleo19:48
greghaynesI know weve hit that before, forgot what the fix was19:49
greghayneshttps://bugs.launchpad.net/tripleo/+bug/130807919:49
uvirtbotLaunchpad bug 1308079 in tripleo "'No valid host' during overcloud build when using Ironic" [Medium,Triaged]19:49
SpamapSoh that19:51
*** yfaykya has quit IRC19:53
SpamapSgreghaynes: your stuff is still failing :(19:54
*** yfaykya has joined #tripleo19:54
greghaynesMy stuff?19:55
greghaynesI have lots of failing stuff :)19:55
greghaynesto which are you referring?19:55
greghaynesoh, toci19:57
*** yfaykya has quit IRC19:59
openstackgerritGregory Haynes proposed a change to openstack-infra/tripleo-ci: Pin to a specifc version of Heat  https://review.openstack.org/10493720:00
*** yfaykya has joined #tripleo20:01
*** yfaykya has quit IRC20:05
*** yfaykya has joined #tripleo20:07
*** yfaykya has quit IRC20:11
*** yfaykya has joined #tripleo20:13
*** ramishra has joined #tripleo20:15
*** yfaykya has quit IRC20:17
openstackgerritGregory Haynes proposed a change to openstack-infra/tripleo-ci: Pin to a specifc version of Heat  https://review.openstack.org/10493720:18
*** yfaykya has joined #tripleo20:19
*** ramishra has quit IRC20:19
*** yfaykya has quit IRC20:24
*** yfaykya has joined #tripleo20:25
*** yfaykya has quit IRC20:30
*** yfaykya has joined #tripleo20:31
*** yfaykya has quit IRC20:36
*** yfaykya has joined #tripleo20:37
*** yfaykya has quit IRC20:42
*** yfaykya has joined #tripleo20:43
*** lazy_prince is now known as killer_prince20:44
*** yfaykya has quit IRC20:48
*** yfaykya has joined #tripleo20:50
*** yfaykya has quit IRC20:54
*** yfaykya has joined #tripleo20:56
lifelessSpamapS: locked by happens because of the background power thread, its an in-progress issue via the async api spec. The patches mrda / NobodyCam have up should allow us to drop our racy power-off-on-rego code and make it better20:58
*** yfaykya has quit IRC21:00
*** untriaged-bot has joined #tripleo21:01
untriaged-botUntriaged bugs so far:21:01
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/133458121:01
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/133504221:01
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/133510521:01
uvirtbotLaunchpad bug 1334581 in tripleo "os-collect-config.conf need support to Heat userdata" [Undecided,In progress]21:01
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/133611021:01
uvirtbotLaunchpad bug 1335042 in tripleo "diskimage-builder: extlinux installation failure" [Undecided,New]21:01
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/133770721:01
uvirtbotLaunchpad bug 1335105 in tuskar-ui "Devtest init script: sourcing tripleorc fails" [Undecided,New]21:01
uvirtbotLaunchpad bug 1336110 in tripleo "mysql does not start on boot post-upgrade" [High,New]21:01
untriaged-bothttps://bugs.launchpad.net/diskimage-builder/+bug/133593221:01
uvirtbotLaunchpad bug 1337707 in tripleo "overcloud fails to come up, ceilometer binding to 0.0.0.0 instead of local_ipv4 causing port conflict" [Undecided,In progress]21:01
uvirtbotLaunchpad bug 1335932 in diskimage-builder "/bin/dib-run-parts fails for spanish locale" [Undecided,Fix committed]21:01
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/133611521:01
uvirtbotLaunchpad bug 1336115 in tripleo "instances no longer running post-upgrade" [Undecided,In progress]21:01
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/133747521:01
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/133501621:01
uvirtbotLaunchpad bug 1337475 in tripleo "devtest can't move the gateway to another ip address other than the seed" [Undecided,In progress]21:01
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/133364921:01
uvirtbotLaunchpad bug 1335016 in tripleo "diskimage-builder fails for Debian" [Undecided,New]21:01
untriaged-bothttps://bugs.launchpad.net/diskimage-builder/+bug/133245821:01
uvirtbotLaunchpad bug 1333649 in tripleo "/etc/hosts entries written by hosts element doesn't survive reboot" [Undecided,In progress]21:01
uvirtbotLaunchpad bug 1332458 in diskimage-builder "Not able to build armhf or arm64 images on amd64 arch systems." [Undecided,In progress]21:01
untriaged-bothttps://bugs.launchpad.net/diskimage-builder/+bug/133252121:01
uvirtbotLaunchpad bug 1332521 in diskimage-builder "failed to remove `/tmp/image.A2aupYJb/mnt/tmp/in_target.d': Device or resource busy" [Undecided,New]21:01
*** untriaged-bot has quit IRC21:01
*** yfaykya has joined #tripleo21:02
*** yfaykya has quit IRC21:06
*** yfaykya has joined #tripleo21:08
*** yfaykya has quit IRC21:13
*** yfaykya has joined #tripleo21:14
*** yfaykya has quit IRC21:18
*** yfaykya has joined #tripleo21:20
*** yfaykya has quit IRC21:25
*** yfaykya has joined #tripleo21:26
*** yfaykya has quit IRC21:31
*** yfaykya has joined #tripleo21:32
*** tzumainn has quit IRC21:35
*** yfaykya has quit IRC21:37
*** yfaykya has joined #tripleo21:38
*** yfaykya has quit IRC21:43
*** dsneddon has quit IRC21:43
*** jang has quit IRC21:43
greghayneslifeless: and other cores https://review.openstack.org/#/c/104937/21:44
*** yfaykya has joined #tripleo21:45
*** yfaykya has quit IRC21:49
*** yfaykya has joined #tripleo21:51
*** yfaykya has quit IRC21:55
greghaynesSpamapS: ^21:57
*** yfaykya has joined #tripleo21:57
*** yfaykya has quit IRC22:02
*** yfaykya has joined #tripleo22:03
*** yfaykya has quit IRC22:08
*** yfaykya has joined #tripleo22:09
*** yfaykya has quit IRC22:14
*** yfaykya has joined #tripleo22:15
*** yfaykya has quit IRC22:20
*** yfaykya has joined #tripleo22:21
*** andreaf has quit IRC22:22
*** andreaf has joined #tripleo22:24
*** yfaykya has quit IRC22:26
*** yfaykya has joined #tripleo22:27
*** boris-42 has quit IRC22:29
*** boris-42 has joined #tripleo22:29
*** yfaykya has quit IRC22:32
*** yfaykya has joined #tripleo22:34
*** yfaykya has quit IRC22:38
*** yfaykya has joined #tripleo22:40
*** yfaykya has quit IRC22:44
*** yfaykya has joined #tripleo22:46
*** yfaykya has quit IRC22:51
*** yfaykya has joined #tripleo22:52
*** yfaykya has quit IRC22:56
*** yfaykya has joined #tripleo22:58
*** yfaykya has quit IRC23:03
*** yfaykya has joined #tripleo23:04
*** yfaykya has quit IRC23:09
*** yfaykya has joined #tripleo23:10
*** yfaykya has quit IRC23:15
*** yfaykya has joined #tripleo23:16
*** ramishra has joined #tripleo23:17
*** rlandy has quit IRC23:21
*** yfaykya has quit IRC23:21
*** ramishra has quit IRC23:22
*** yfaykya has joined #tripleo23:23
*** yfaykya has quit IRC23:27
*** yfaykya has joined #tripleo23:29
*** yfaykya has quit IRC23:33
*** yfaykya has joined #tripleo23:35
*** yfaykya has quit IRC23:39
*** yfaykya has joined #tripleo23:41
*** yfaykya has quit IRC23:45
*** yfaykya has joined #tripleo23:47
*** yfaykya has quit IRC23:52
*** yfaykya has joined #tripleo23:53
*** yfaykya has quit IRC23:58
*** yfaykya has joined #tripleo23:59
*** dkehnx has quit IRC23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!