Friday, 2014-05-30

*** ddieterly has quit IRC00:04
*** jp_at_hp has quit IRC00:07
SpamapSlifeless: just change the state from FAILED to COMPLETE00:12
SpamapSlifeless: and then make sure reality matches database00:12
*** e0ne has quit IRC00:17
*** e0ne has joined #tripleo00:17
*** e0ne has quit IRC00:19
*** yamahata has joined #tripleo00:29
*** yamahata has quit IRC00:29
*** matsuhashi has joined #tripleo00:30
*** nati_ueno has quit IRC00:31
mordredSpamapS: that seems like it would make babies cry00:32
greghaynesI saw we just set the state to COMPLETE and call it good00:32
greghaynesnothing to see here, move along00:32
*** ddieterly has joined #tripleo00:33
*** yamahata has joined #tripleo00:33
*** nati_ueno has joined #tripleo00:35
lifelessgreghaynes: hey00:48
*** e0ne has joined #tripleo00:48
*** fandi has quit IRC00:48
*** e0ne has quit IRC00:52
*** Penick has quit IRC00:55
lifelessanyone else been seeing 'message:"Unable to locate package ubuntu-cloud-keyring"' as a failure ?01:00
StevenKThat is a little worrying01:00
lifelessit was on an f20 run01:00
lifelesswhich is frankly more worrying01:01
StevenKHm, the only that does that in incubator is install-dependencies01:02
StevenKSo ... what the01:04
lifelesswe might be building f20 on a fedora host01:04
lifelessnot sure01:04
lifelessdevananda: reminder to have dinner with hernan01:05
*** weshay has joined #tripleo01:05
lifelessdshulyak: please file a blueprint now your spec has been approved01:06
*** vinsh has joined #tripleo01:07
tchaypolifeless: yes, and for you too ;)01:11
SpamapSmordred: well babies don't buy clouds, so maybe thats ok? ;)01:13
*** cwolferh has quit IRC01:13
mordredSpamapS: hrm. if babies don't buy clouds, then some of my strategies may be flawed01:14
StevenKmordred: You also have a bridge you're sweetning the deal with?01:14
SpamapSmordred: this explains a lot01:15
SpamapSmordred: but in all seriousness.. modifying the database directly, while evil, may be a decent short term solution until we have a legitimate in-band way to recover01:17
lifelesswill we be getting the gerrit bot back ?01:27
*** nati_ueno has quit IRC01:31
*** saurabhs has quit IRC01:34
*** ddieterly has quit IRC01:43
*** ddieterly has joined #tripleo01:44
StevenKlifeless: Not after a review, but can you weigh in on ? I'd love a direction to take the patch to stop subprocess usage.01:44
*** nosnos has joined #tripleo01:47
*** ddieterly has quit IRC01:48
*** weshay has quit IRC01:50
*** eguz has joined #tripleo01:52
*** eguz has quit IRC01:52
*** eghobo has quit IRC01:56
lifelessStevenK: looking02:05
lifelessadam_g: greghaynes: either of you on the 405 rack atm ?02:07
adam_glifeless, im finishing up an overcloud now02:07
adam_glifeless, ill let you know when its done and then its yours, cool?02:07
StevenKlifeless: Ah ha, please don't block because it isn't perfect?02:07
lifelessadam_g: I was just going to upgrade the host to trusty02:08
lifelessadam_g: maybe you want to do that yourself at a convenient time?02:08
lifelessStevenK: roughly02:08
adam_glifeless, sure02:09
StevenKlifeless: I shall leave them alone, and hopefully portitions will land soon02:09
lifelessStevenK: whinge for reviews here02:10
lifelessits the done thing02:10
lifelessStevenK: but only after you're happy with them and you have green from CI02:10
StevenKlifeless: There are two in the stack that are already approved, but they need the two that lsmola -1'd02:11
tchaypoxuhaiwei: morning02:13
lifelessStevenK: theres a second -1'd one ?02:13
tchaypoxuhaiwei: have you seen again, or was that a once-off?02:13
uvirtbotLaunchpad bug 1323167 in tripleo "overcloud can't create an instance by neutron error" [Undecided,New]02:13
StevenKlifeless: was also -1'd for the same reason02:13
*** fandi has joined #tripleo02:14
xuhaiweitchaypo: sorry, i think it's not exactly true about my report, yes, I met that problem once, and when rebuild devtest I can run instances properly, but the neutron error still exists02:18
tchaypoxuhaiwei: I'm not understanding what you mean. Which neutron error still exists?02:19
xuhaiweitchaypo: in this report, i found there is error in neutron-server.log02:20
uvirtbotLaunchpad bug 1323167 in tripleo "overcloud can't create an instance by neutron error" [Undecided,New]02:20
xuhaiweitchaypo:and when i run a new instance, the error will happen again, but the instance can still run02:21
tchaypoxuhaiwei: Are you rebuilding with cached images? Have you tried in the last 12 hours? apparently landed in neutron this morning, and I'm wondering if it might have fixed the problem02:22
xuhaiweitchaypo: i am not using cached images, and i am building devtest now, I will check it later02:23
tchaypothanks. Could you ping me if you see it agian?02:24
tchaypoI'd like to at least be able to figure out enough about the bug that we can assign an importance02:24
tchaypolifeless: I've had a quick look at and I have no idea what problem you're reporting or how to reproduce it. Could you add some detail?02:25
uvirtbotLaunchpad bug 1320090 in tripleo "pxe_ephemeral_format': u'ext4 left on nodes after deploys deleted" [Undecided,New]02:25
xuhaiweitchaypo: ok, as soon as the environment building is over, i will check it and tell you02:25
xuhaiweitchaypo: when rebuilding the environment, I am not starting from the very beginning, i started from the undercloud building, i undefined the baremetal_0 firstly, and then try to delete undercloud instance in seed, but i the instance can't be deleted and the undercloud stack can't be deleted either02:27
xuhaiweitchaypo: i don't know if it is a bug02:28
tchaypoxuhaiwei: thanks. I'm building an environment too, maybe I'll hit it as well02:28
tchaypoxuhaiwei: I don't know if it's a bug either... but I'd rather have it reported and then we close it than have bugs go unreported :)02:29
xuhaiweitchaypo: so i will report it02:30
lifelesstchaypo: run ironic node-show $somenode02:31
StevenKlifeless: Is there some way to get that bug bot back in here, so tchaypo and I will know when we're done?02:32
lifelessdunno, rpodolyaka will know :)02:32
*** geerdest has quit IRC02:33
*** matsuhashi has quit IRC02:45
*** shakayumi has quit IRC02:58
*** untriaged-bot has joined #tripleo03:00
untriaged-botUntriaged bugs so far:03:00
uvirtbotLaunchpad bug 1324751 in tripleo "Can't delete baremetal instance when the baremetal vm is deleted in advance" [Undecided,New]03:00
uvirtbotLaunchpad bug 1314978 in tripleo "Cloud vm not pingable after overcloud upgrade " [Undecided,New]03:00
uvirtbotLaunchpad bug 1315355 in tripleo "Upgrade of overcloud failed with "Connection to neutron failed: Maximum attempts reached"" [Undecided,New]03:00
uvirtbotLaunchpad bug 1320090 in tripleo "pxe_ephemeral_format': u'ext4 left on nodes after deploys deleted" [Undecided,New]03:00
*** untriaged-bot has quit IRC03:00
lifelessadam_g: I've done a dist-upgrade to get latest packages, grub-pc whinged about changing device names :< but other than that it should be ready to do a do-release-upgrade anytime03:06
* StevenK hmmms at bug 132475103:15
uvirtbotLaunchpad bug 1324751 in tripleo "Can't delete baremetal instance when the baremetal vm is deleted in advance" [Undecided,New]
lifelesstchaypo: make more sense now ?03:19
*** eghobo has joined #tripleo03:19
*** ramishra has joined #tripleo03:19
lifelessStevenK: commented03:20
tchaypolifeless: no, but I'm getting there03:21
tchaypolifeless: it will once i have an env set up where i can run it03:21
*** ramishra has quit IRC03:23
*** shakayumi has joined #tripleo03:24
*** ramishra has joined #tripleo03:25
lifelessadam_g: btw - at the end03:25
lifelessgit review -X or something might make a clearer loop03:25
*** shakayumi has quit IRC03:28
tchaypolifeless: do you have comments on
lifelesstchaypo: not really, seems reasonable03:33
tchaypolifeless: that's a comment ;)03:35
*** nosnos has quit IRC03:40
*** matsuhashi has joined #tripleo03:47
lifelessadam_g: present for you
lifelessNg: can I get a DNS entry? ?03:53
*** shakayumi has joined #tripleo03:54
*** matsuhashi has quit IRC03:54
*** matsuhashi has joined #tripleo03:54
*** akuznetsov has joined #tripleo03:55
*** matsuhashi has quit IRC03:57
*** matsuhashi has joined #tripleo03:58
*** ramishra has quit IRC03:58
*** ramishra has joined #tripleo03:58
*** shakayumi has quit IRC03:59
*** TravT has quit IRC03:59
lifelessadam_g: greghaynes: bnemec: - needs some review love04:00
*** matsuhashi has quit IRC04:03
*** matsuhashi has joined #tripleo04:03
*** rpodolyaka1 has joined #tripleo04:05
tchaypoIn other news, what needs to happen with ?04:06
lifelesstchaypo: see in tripleo-cloud04:11
lifelesstchaypo: it describes ;)04:11
tchayposo what I'm going to do at this point is wait a lot04:14
lifelessand poke existing admins for reviews04:15
lifelessmy x201s is still plenty fast :)04:16
tchaypolifeless: poke poke04:17
tchaypoin other news04:19
tchaypoIf a bug has been sitting around for 30 days with no updates04:19
tchaypoI want to close it saying "it's been 30 days without any update so I'm assuming this hasn't been a problem lately", does that seem reasonable?04:19
tchaypoit can be reopened if it's actually still a problem, so..04:19
lifelesstchaypo: if its been INCOMPLETE with a question asked to the reporter, that seems reasonable04:20
tchaypoit  hasn't been incomplete, it's new with no response to the reporter at all04:20
tchayposo maybe a "does this still happen? I haven't seen it lately" response?04:21
lifelesstchaypo: so we've failed to triage it?04:21
tchaypoit was raised by cian_04:22
lifelesstchaypo: philosophically the question isn't whether there has been activity04:22
lifelessits whether the problem still exists // whether we care about it04:22
tchaypoIt looks to me as though cian was chasing it (he commented a day later with more information), but he hasn't updated it since...04:23
*** ramishra has quit IRC04:23
StevenKlifeless: While I remember, I did not file a bug against squid for the maximum_object_size docs because blah, why do I want another useless bugzilla account?04:23
tchayposo I think I'm going to mark it incomplete/low priority and assign it to him and ask him on the assumption that he's been chasing it himself and ask him to update it if that's not the case04:23
*** jcoufal has joined #tripleo04:28
lifelessStevenK: so mail the squid list perhaps ?04:31
lifelesstchaypo: if it doesn't look like its really a bug, INCOMPLETE is entirely appropriate04:32
lifelessI wish some of these internal blog posts were accessible publically04:35
lifelesswould be easier to click through when the local MD sends mails around04:35
*** nosnos has joined #tripleo04:42
lifelessNg: also please add all the new entries from the public networks section of the cloud CD Admin spreadsheet.04:44
tchaypolifeless: has 2 +2s and now jenkins is happy with it (although the overcloud-precise job seems to have gone missing). Could you +A?04:52
lifelesstchaypo: read -dev :)04:52
lifelesstchaypo: also nice try but that patch doth not :)04:53
tchaypolifeless: I'm assuming that it's related to you rebuilding HP region?04:53
lifelesstchaypo: yes04:53
tchaypourr, correct. I meant
tchaypoi like it. fewer gates means fewer chances for name resolution to fail ;)04:53
*** shakayumi has joined #tripleo04:54
*** lokesh184 has joined #tripleo04:55
*** shakayumi has quit IRC04:57
*** alexiz has joined #tripleo04:57
greghaynesWhat happened to the overcloud-precise job:
lifelessgreghaynes: we removed it when we dropped CI capacity by 66%04:58
lifelessgreghaynes: see openstack-infra/config and the CI spec in tripleo-specs04:59
*** alexiz has quit IRC05:02
*** tzumainn has quit IRC05:03
*** rakesh_hs has joined #tripleo05:04
*** lazy_prince has joined #tripleo05:20
*** rpodolyaka1 has quit IRC05:42
*** dshulyak_ has joined #tripleo05:47
*** jprovazn has joined #tripleo05:49
lifelessoh stab05:51
StevenKVery very simple index05:54
StevenKI think it's for people who miss gopher05:55
lifelessgo fer it05:56
*** alexiz has joined #tripleo05:58
SpamapSIts for scripting06:00
*** rpodolyaka1 has joined #tripleo06:01
*** alexiz has quit IRC06:03
xuhaiweitchaypo: hi, i have finished the environment, and I can see the neutron error is still there in neutron-server.log06:10
tchaypoxuhaiwei: see my comment on the bug :)06:11
tchaypoI can see it too, but I think it's unrelated to the demo vm not being present06:11
lifelesswe were using the router as a DNS server06:13
lifelessso wrong06:13
xuhaiweitchaypo: The demo doesn't fail building every time06:14
*** pblaho has joined #tripleo06:16
*** rpodolyaka1 has quit IRC06:18
tchaypoxuhaiwei: it seems to me that your bug was reporting two differnt issues then - one is the error seen in the logs, and one is that the demo vm sometimes isn't created. Does that sound right?06:18
xuhaiweitchaypo: yes06:19
StevenKrpodolyaka: Hai. Is there a way to get that untriaged bot to run again? I think tchaypo and I have emptied the list06:19
xuhaiweitchaypo: in my environment the 'demo' vm has been built this time06:19
*** alexiz has joined #tripleo06:24
*** alexiz has quit IRC06:29
*** matsuhashi has quit IRC06:30
*** matsuhashi has joined #tripleo06:31
*** jang has joined #tripleo06:36
*** gcha has joined #tripleo06:38
tchaypoxuhaiwei: yeah, i think it just worked on mine as well06:40
*** bogdando has quit IRC06:42
*** bogdando has joined #tripleo06:42
*** e0ne has joined #tripleo06:46
*** viktors has joined #tripleo06:56
lifelessgreghaynes: on patches like it might help to have an example of what happened (e.g. in a bug report)06:57
*** dshulyak_ has quit IRC06:59
GheRiveromorning all06:59
xuhaiweitchaypo: it belongs to the demo tenant07:01
lifelessnow if I can only find the review that setup bridging from brbm to a device07:01
xuhaiweitchaypo: but it seems i can't ssh the demo vm07:02
lifeless landed without a test07:02
lifelessnaughty reviewers!07:02
*** e0ne has quit IRC07:02
marioswasn't me07:02
*** e0ne has joined #tripleo07:02
tchaypoxuhaiwei: ah, that makes sense, thanks07:03
lifelessmarios: was too :)07:04
janglifeless: was one of Nick's attempts to do the brbm->phys bridging07:04
*** rdopiera has joined #tripleo07:05
lifelessjang: thanks07:05
lifelessoh right07:05
lifelessensure-bridge fixups needed07:05
lifelessjang: I'd love it if that arc can be completed07:06
lifelessnow, hopefully I didn't just break all my networking :)07:08
lifelessah yeah, I so did07:08
lifelessthis is going to be a DC ticket to fix07:10
lifelessNg: where art thou :(07:10
adam_glifeless, yikes
uvirtbotLaunchpad bug 1289977 in grub2 "Ubuntu 14.04 Update breaks grub, resulting in "error: symbol 'grub_term_highlight_color' not found"" [High,Triaged]07:10
lifeless(because the ilo details for that machine are on that machine)07:10
*** matsuhashi has quit IRC07:11
lifelessadam_g: did we just lose the 405 jump host too ?07:11
lifelessadam_g: I do have ilo details for the 405 jump host07:12
adam_glifeless, i have them too07:12
adam_glifeless, trusty upgrade gone bad. stuck at grub07:12
*** e0ne has quit IRC07:13
*** e0ne has joined #tripleo07:13
lifelessadam_g: I ran ensure-bridge on a running machine and it wedged07:14
lifelessadam_g: on the freecloud bastion host07:14
adam_glifeless, oh, this is unrelated.07:14
lifelessadam_g: yeah07:14
lifelessadam_g: just synchronicity07:14
adam_ganyway, i dont know how much longer i have tonight to get this back online07:15
*** matsuhas_ has joined #tripleo07:16
lifelessadam_g: fair enough too07:16
*** matsuhas_ has quit IRC07:16
*** e0ne_ has joined #tripleo07:17
*** matsuhashi has joined #tripleo07:17
*** e0ne has quit IRC07:18
*** e0ne_ has quit IRC07:19
jangensure-bridge isn't a great fit; it makes too many assumptions (like, you're not running it over ssh) that don't hold for a devtest-style use.07:19
marioslazy_prince: hey - minor nit on /#/c/96483/ (really wanted to +2 it so we could land /#/c/84600/ finally)07:19
xuhaiweitchaypo: i can ssh demo vm, it seems ok for me07:22
*** markmc has joined #tripleo07:22
adam_gwell, i was able to get a mini.iso attached as virtual media the other  rack07:24
*** alexiz has joined #tripleo07:25
*** IvanBerezovskiy has joined #tripleo07:27
lifelessjang: well, it was previously written to be very robust and using it over ssh was fine07:27
lifelessjang: the new one seems more rough07:27
lifelessadam_g: great; I would but I don't have the ilo details :(07:28
*** alexiz has quit IRC07:30
*** ifarkas has joined #tripleo07:34
* tchaypo sets mode +weekend07:34
mariostchaypo: happy friday matey07:36
*** eghobo has quit IRC07:39
lifelessoh, cipher suite zero time I guess. That should work07:41
*** spiffxp has quit IRC07:54
*** davidlenwell has quit IRC07:55
*** vipul has quit IRC07:55
*** jang1 has quit IRC07:55
*** davidlenwell has joined #tripleo07:55
*** kevinbenton has quit IRC07:56
*** dguerri has quit IRC07:56
*** NobodyCam has quit IRC07:56
*** echohead has quit IRC07:57
*** zaro has quit IRC07:57
*** devananda has quit IRC07:57
*** SlickNik has quit IRC07:57
*** jrist has quit IRC07:57
*** devananda has joined #tripleo07:58
*** vipul has joined #tripleo08:01
*** dguerri has joined #tripleo08:01
*** kevinbenton has joined #tripleo08:01
*** SlickNik has joined #tripleo08:02
*** spiffxp has joined #tripleo08:02
*** jrist has joined #tripleo08:03
*** echohead has joined #tripleo08:03
*** NobodyCam has joined #tripleo08:04
*** zaro has joined #tripleo08:04
*** lsmola has quit IRC08:07
*** lsmola has joined #tripleo08:12
*** derekh_ has joined #tripleo08:16
*** martyntaylor has joined #tripleo08:17
*** martyntaylor has quit IRC08:17
*** jistr has joined #tripleo08:19
derekh_lifeless: good morning/evening anything I can assist with on rack ? /me has put my name down on etherpad beside the burning task but can take on more08:19
*** martyntaylor has joined #tripleo08:23
*** alexiz has joined #tripleo08:26
lifelessderekh_: hi08:26
lifelessderekh_: gimme a sec08:26
lifelessderekh_: ok so08:28
*** shardy_afk is now known as shardy08:28
lifelessderekh_: ensure-bridge nuked my access to the jump host :(08:28
lifelessderekh_: I'm in on the ilo now, but it doesn't have stack:stack so I'm trying to find a password for it08:28
lifelesswhich reminds me08:28
lifelessSpamapS: NobodyCam: ng: ping - do you happen to remember your actual *password* for the freecloud bastion host ?08:29
lifelessEntropyWorks: ^08:29
lifelessits ilo is on .12; user robertc password foobarbaz [will be deleting the account once we've recovered it all]08:29
lifelessI could plug virtual media in I guess08:29
lifelesscan you do that over ssh ?08:30
*** alexiz has quit IRC08:31
*** jp_at_hp has joined #tripleo08:31
*** lucasagomes has joined #tripleo08:32
SpamapSlifeless: password? no08:33
*** lazy_prince2 has joined #tripleo08:33
derekh_o dear08:34
lxsliPlease could we get a +A on ? It has 5 +2s (!) and a clean CI run08:34
*** e0ne has joined #tripleo08:35
lifelessderekh_: yes, fun in the sun08:35
*** lazy_prince has quit IRC08:35
SpamapSlifeless: I use ILO via the web interface all the time.08:37
lifelessSpamapS: doesn't tht require java?08:37
SpamapSlifeless: yes. Works fine.08:37
*** lazy_prince2 has quit IRC08:37
SpamapSlifeless: you just need to setup a SOCKS proxy08:37
derekh_lxsli: you just need 2 more clean CI runs and a purple donkey to get a A+ :-)08:38
SpamapSlifeless: which ssh is quite happy to do for you :)08:38
lifelessSpamapS: java. no work fine :)08:38
derekh_lxsli: its merged, where is the merge info bot thingy ?08:38
lifelessderekh_: dead08:39
derekh_ahh well08:39
SpamapSlifeless: get over that. It works _FINE_08:39
*** lokesh184 has quit IRC08:40
*** lazy_prince has joined #tripleo08:40
SpamapSnot good, not well, not awesome, and not free.. but .. fine. ;)08:40
lifelessSpamapS: yeah, right now, on my machine, not so much08:40
lifelessSpamapS: in principle, sure.08:40
lifelessin theory I'm booting the jump host now08:40
SpamapSlifeless: oh right, you insist on having a "real developer machine" ;)08:40
lifelessSpamapS: not that, justborked.08:40
SpamapSmy almost-windows-xp stock 14.04 box wins again ;)08:41
lifelessSpamapS: and ETIME to fix08:41
lifelessderekh_: so08:41
*** lokesh184 has joined #tripleo08:41
lifelessderekh_: I've built seed and undercloud images08:41
*** e0ne has quit IRC08:41
lifelesspainful cause of network glitches08:41
lifelessI've got a network json, a nodes json and the start of an undercloud-env.json08:42
*** e0ne has joined #tripleo08:42
lifelessderekh_: next step is to deploy the seed (just using USE_CACHE=1
lifelessderekh_: then make sure we have all the params to deploy the undercloud with vlan'd public addresses etc08:43
lifelessstabbity stab stab graphics mode man...08:43
*** hashar has joined #tripleo08:44
lifelessrescue vga=77108:44
lifelessfingers crossed08:44
derekh_lifeless: this is on the bastion or a different seed host ?08:45
lifelessderekh_: bastion08:45
lifelessderekh_: which I'm recovering now08:45
lifelessderekh_: having had ensure-bridge nuke its running network config08:45
lifelessI just need to figure out how to boot rescue mode without graphics08:45
lifelessor fix my java08:45
lifelessor tunnel my windows machine to my laptop to the DC to aieeee08:46
*** e0ne has quit IRC08:47
lifelessah 769 might work better08:48
*** dtantsur|afk is now known as dtantsur08:49
lifelessgfxpayload=vga=normal apparently08:50
lifelessbut wow08:50
lifelessfigure that out08:50
*** cian_ has quit IRC08:54
*** martyntaylor has left #tripleo08:55
lparthcould someone please approve
*** untriaged-bot has joined #tripleo09:01
untriaged-botUntriaged bugs so far:09:01
uvirtbotLaunchpad bug 1320090 in tripleo "pxe_ephemeral_format': u'ext4 left on nodes after deploys deleted" [Undecided,New]09:01
*** untriaged-bot has quit IRC09:01
uvirtbotLaunchpad bug 1316475 in diskimage-builder "[SRU] CloudSigma DS for causes hangs when serial console present" [Undecided,In progress]09:01
*** lokesh184 has quit IRC09:03
*** lokesh184 has joined #tripleo09:05
*** lazy_prince has quit IRC09:06
*** lazy_prince2 has joined #tripleo09:07
xuhaiweilifeless: are you still there?09:11
*** hashar_ has joined #tripleo09:21
*** hashar has quit IRC09:25
*** alexiz has joined #tripleo09:27
*** giulivo has joined #tripleo09:27
*** lxsli has quit IRC09:27
*** lxsli has joined #tripleo09:28
*** lxsli has quit IRC09:28
*** lxsli has joined #tripleo09:28
*** lazy_prince2 has quit IRC09:31
*** alexiz has quit IRC09:31
*** lazy_prince has joined #tripleo09:31
*** hashar_ is now known as hashar09:32
*** pelix has joined #tripleo09:39
lifelessxuhaiwei: not really ;)09:49
xuhaiweilifeless:it must be late in your place09:50
xuhaiweilifeless: can i ask a question ?09:51
lifelessI don't know, can you? :)09:51
*** rlandy has joined #tripleo09:52
xuhaiweilifeless: i used stack-update to add a compute node to overcloud09:52
xuhaiweilifeless: stack-update seems right, but i can't run command in the overcloud now, it's a authentication error, seems the adminpasswd has been changed09:53
lifelessdo you mean ssh ? or nova commands?09:55
xuhaiweilifeless: stack-update should use the passwd kept in overcloud-env.json for the authentication, right?09:55
xuhaiweinova commands09:56
xuhaiweissh is ok09:56
lifelessmake sure you've sourced the right rc file09:56
xuhaiweii just ssh the controller node, and ssh the stackrc on home09:56
xuhaiweithat should work i think09:56
xuhaiweisorry source the stackrc09:57
lifelessrescue vga=normal fb=false nomode10:02
lifeless^ mini.iso incantatio10:02
lifelessxuhaiwei: I don't know sorry; maybe someone else here can answer your question10:02
lifelessI'm not awake enough to multitask10:03
xuhaiweilifeless: ok, thanks, you have been working too long, i think, every day i came here you are still here, when i'm leaving you are still here :)10:04
xuhaiweilifeless: good night! bye10:11
*** matsuhashi has quit IRC10:16
derekh_lifeless: any luck with the bastian, I'm not sure I have any way in without it10:21
*** matsuhashi has joined #tripleo10:21
lifelessderekh_: nearly10:22
*** alexiz has joined #tripleo10:27
*** matsuhashi has quit IRC10:27
*** matsuhashi has joined #tripleo10:28
* TheJulia rubs her eye in a futual attempt towake up10:28
lifelessruh roh10:28
*** alexiz has quit IRC10:32
lifelessderekh_: didn't have drivers in the ramdisk, trying a bigger image :)10:33
derekh_lifeless: k10:33
derekh_lifeless: btw, its a holiday here on monday so I probably wont be around10:34
derekh_R2 seems to be keeping up with the load ok so far today10:35
lifelesse too10:35
*** akrivoka has joined #tripleo10:35
derekh_although we havn't hit busy time yet10:35
lifelessderekh_: I suggest you work on patches to do the burnin automation10:35
lifelessderekh_: since we want it automated :)10:35
*** yamahata has quit IRC10:35
lifelessderekh_: did I link to you?10:36
derekh_lifeless: ok will concentrate on that and have a PATH ASAP10:36
derekh_lifeless: nope you didn't, looking now10:37
lifelessok great the main ubuntu server iso doesn't have the esc->textmode fallback in its boot loader10:38
* lifeless tries again10:38
*** dtantsur is now known as dtantsur|lunch10:43
*** BadCub has quit IRC10:44
*** hashar is now known as hasharEat10:45
*** BadCub has joined #tripleo10:45
*** akrivoka has quit IRC10:46
*** juice has quit IRC10:49
*** juice has joined #tripleo10:52
*** BadCub has quit IRC10:54
*** BadCub has joined #tripleo10:55
*** lokesh184 has quit IRC10:57
*** matsuhashi has quit IRC11:00
*** matsuhashi has joined #tripleo11:00
*** BadCub has quit IRC11:01
*** NobodyCam has quit IRC11:01
*** NobodyCam has joined #tripleo11:02
*** jp_at_hp1 has joined #tripleo11:03
lifelessok bastion recovered, rebooting to confirm its really stable11:04
*** BadCub has joined #tripleo11:04
*** jp_at_hp has quit IRC11:05
*** matsuhashi has quit IRC11:05
*** akuznetsov has quit IRC11:08
lifelessyup its up11:08
*** matsuhashi has joined #tripleo11:09
*** BadCub_ has joined #tripleo11:10
*** akuznetsov has joined #tripleo11:12
*** BadCub has quit IRC11:13
*** xuhaiwei has quit IRC11:13
lifelessjq: error: Cannot index string with number11:15
lifelessusage: ironic port-create -a <address> -n <node uuid> [-e <key=value>]11:15
lifelessironic port-create: error: argument -a/--address: expected one argument11:15
lifelessI'll poke at it later, its in the stack user screen session if anyone wants to look themselves11:15
*** lucasagomes is now known as lucas-hungry11:16
lifelessah, badly formed nodes.json11:16
*** akrivoka has joined #tripleo11:16
*** yamahata__ has quit IRC11:21
*** hasharEat is now known as hashar11:21
lifelessand now CONFLICT (content): Merge conflict in elements/keystone/os-apply-config/etc/keystone/keystone.conf11:21
*** tzumainn has joined #tripleo11:22
*** openstackgerrit has joined #tripleo11:26
*** jcoufal has quit IRC11:28
*** alexiz has joined #tripleo11:28
*** jcoufal has joined #tripleo11:28
*** hewbrocca has joined #tripleo11:33
*** alexiz has quit IRC11:33
*** ramishra has joined #tripleo11:39
*** yamahata has joined #tripleo11:49
*** e0ne has joined #tripleo11:55
*** matsuhashi has quit IRC12:00
*** matsuhashi has joined #tripleo12:00
*** dprince has joined #tripleo12:01
*** NobodyCam has quit IRC12:02
*** matsuhashi has quit IRC12:02
*** NobodyCam has joined #tripleo12:02
*** matsuhashi has joined #tripleo12:02
lsmolaNobodyCam: hello, are you around?12:04
*** NobodyCam has quit IRC12:08
Nglifeless: sorry, I do not have a record of the password for the freecloud bastion, I'm not sure I ever knew it (in that it's not in work/personal email, or hipchat or irc logs)12:08
openstackgerritA change was merged to openstack/python-tuskarclient: replaced e.message
Ngalso, morning12:08
*** NobodyCam has joined #tripleo12:09
*** ramishra has quit IRC12:12
*** matsuhashi has quit IRC12:12
*** matsuhashi has joined #tripleo12:13
*** matsuhashi has quit IRC12:13
openstackgerritAlexis Lee proposed a change to openstack/tripleo-specs: Promote HEAT_ENV
*** lxsli has quit IRC12:13
*** yamahata has quit IRC12:14
*** yamahata has joined #tripleo12:15
*** e0ne_ has joined #tripleo12:15
*** lucas-hungry is now known as lucasagomes12:17
*** e0ne has quit IRC12:19
*** rakesh_hs has quit IRC12:20
*** jdob has joined #tripleo12:20
dprincemarios: can you try setting the workflow on this back to 0?
openstackgerritA change was merged to openstack/os-cloud-config: PKI key/cert generation for use with Keystone
dprincemarios: when I commented it didn't seem to remove the workflow change you made12:21
*** openstackgerrit has quit IRC12:22
mariosdprince: sure, looking12:22
mariosdprince: sorry, i clicked the wrong one12:23
mariosfixed now12:23
mariosdprince: looks like i WIPed it rather than just -112:23
*** openstackgerrit has joined #tripleo12:23
dprincemarios: np, I figured that was the case12:23
*** lxsli has joined #tripleo12:26
*** dtantsur|lunch is now known as dtantsur12:27
*** alexiz has joined #tripleo12:29
StevenKjistr: Thanks for the recheck. You'll +A if CI loves it?12:31
jistrStevenK: yup. If i don't feel free to poke me about it.12:31
StevenKIt won't recheck due to the vote12:33
*** alexiz has quit IRC12:34
StevenKRight, *now* it's rechecking12:34
*** weshay has joined #tripleo12:35
dprincebnemec: question for you here
*** noslzzp has joined #tripleo12:36
openstackgerritA change was merged to openstack/os-apply-config: Updated from global requirements
*** nosnos has quit IRC12:40
*** noslzzp has quit IRC12:42
*** e0ne_ has quit IRC12:42
openstackgerritGonéri Le Bouder proposed a change to openstack/diskimage-builder: map-services: add openvswitch
*** e0ne has joined #tripleo12:42
openstackgerritAndrea Rosa proposed a change to openstack/tripleo-heat-templates: Adding the reserved host disk
*** hewbrocca has quit IRC12:46
*** e0ne has quit IRC12:47
openstackgerritGonéri Le Bouder proposed a change to openstack/diskimage-builder: introduce $SYSTEMD_SYSTEM_UNIT_DIR
*** hewbrocca has joined #tripleo12:49
*** noslzzp has joined #tripleo12:50
*** morazi has joined #tripleo13:03
ShrewsNg: just fixed the unit tests on that ephemeral patch. should be able to push it through today *crosses fingers*13:09
NgShrews: woot13:09
*** fandi has quit IRC13:10
*** e0ne has joined #tripleo13:10
*** julim has joined #tripleo13:13
marioslifeless: haha. not only is PUT /subnets/id allocation_pools :{} refusing to merge. It is now also spawning.
*** ramishra has joined #tripleo13:18
*** ramishra has quit IRC13:19
openstackgerritGonéri Le Bouder proposed a change to openstack/diskimage-builder: introduce $SYSTEMD_SYSTEM_UNIT_DIR
openstackgerritJiri Stransky proposed a change to openstack/tuskar: Regenerate tuskar.conf.sample
*** alexiz has joined #tripleo13:30
*** alexiz has quit IRC13:34
*** e0ne has quit IRC13:37
*** matty_dubs|gone is now known as matty_dubs13:37
*** e0ne has joined #tripleo13:37
*** e0ne_ has joined #tripleo13:40
*** e0ne has quit IRC13:41
openstackgerritGonéri Le Bouder proposed a change to openstack/tripleo-image-elements: make use of $SYSTEMD_SYSTEM_UNIT_DIR
openstackgerritGonéri Le Bouder proposed a change to openstack/tripleo-image-elements: use os-svc-enable to enable openvswitch
openstackgerritJan Provaznik proposed a change to openstack/os-cloud-config: Allow setup services endpoints
openstackgerritRyan Brady proposed a change to openstack/diskimage-builder: Add svc-map element.
*** yamahata has quit IRC13:56
*** yamahata has joined #tripleo13:56
*** jdob_ has joined #tripleo13:57
*** jdob has quit IRC14:00
*** jdob_ has quit IRC14:00
*** rdopiera has quit IRC14:02
openstackgerritGonéri Le Bouder proposed a change to openstack/diskimage-builder: introduce $SYSTEMD_SYSTEM_UNIT_DIR
*** pcrews has joined #tripleo14:10
*** jdob_ has joined #tripleo14:11
*** rwsu has joined #tripleo14:12
*** ddieterly has joined #tripleo14:18
*** ddieterl_ has joined #tripleo14:20
dprincerbrady: I see you are chipping away at svc-map...14:22
dprincerbrady: which is great!14:22
*** ddieterly has quit IRC14:22
dprincerbrady: wanted to get your take on a slight refinement to how the existing pkg-map stuff works.14:22
*** shakamunyi has joined #tripleo14:22
rbradydrpince: thanks.  I saw a patch for map-services come across this morning and it hit me that we needed svc-map too.  For now it's just a wholesale copy of pkg-map14:23
rbradydrprince: sure14:23
dprincerbrady: So what if we added a global pkg-map file to each repo as well (in addition to the local per element pkg-map files)14:24
dprincerbrady: not sure where that would live, but there may be some cases where we need to install the same package in multiple elements. Now mind you I don't want to encourage that case... but it may present itself from time to time.14:25
*** jcoufal has quit IRC14:25
dprincerbrady: I know I was opposed to this during the review but some others mentioned it. I was just talking it through w/ derekh again and he mentioned the same as well14:25
dprincerbrady: the common case should always be a package gets installed once by the single element though14:26
*** lazy_prince has quit IRC14:26
rbradydprince: so as an example, there would be a global package map in t-i-e or dib?14:27
*** pblaho has quit IRC14:27
*** e0ne_ has quit IRC14:28
dprincerbrady: Not sure. perhaps both?14:29
*** olaph has joined #tripleo14:29
dprincerbrady: not sure we even need this now, or ever14:29
dprincerbrady: I'm still convinced the local map case is much more important than any convienence provided from the global map14:30
dprincerbrady: just something to think about perhaps...14:30
rbradydprince: if the purpose of the local map is to decouple, aren't we re-coupling the two libraries with a global mapping?14:31
openstackgerritA change was merged to openstack/tripleo-image-elements: Add swiftclient library to swift
openstackgerritA change was merged to openstack/tripleo-image-elements: Ironic: set my_ip equal to local-ipv4 (if present)
dprincerbrady: yes and no. I would not encourage its use. Just like a software project globals should ideally be used sparingly (if ever). I'm just imagining a case where there are multiple elements installing a package. For example: libffi-dev14:33
dprincerbrady: which we could just solve by adding an explicit element14:34
dprincerbrady: yeah, I think I'm talking myself out of it again already :)14:34
rbradydprince: I agree about the explicit element.  if we add multiple methods for mapping, we will have folks using one or the other and have to support both14:36
*** openstackgerrit has quit IRC14:36
rbradydprince: maybe "There should be one-- and preferably only one --obvious way to do it." applies here14:36
*** pblaho has joined #tripleo14:36
dprincerbrady: mappings belong in each element14:37
dprincerbrady: so lets just go with that :)14:37
*** openstackgerrit has joined #tripleo14:37
rbradydprince: sounds good14:38
*** rcarrill` is now known as rcarrillocruz14:38
*** e0ne has joined #tripleo14:40
dprincegoneri: hi. So on your patch here now that it uses pkg-map you can drop the changes to the old map-packages...14:40
lxsliEyes please?
*** pblaho has quit IRC14:42
*** edmund1 has quit IRC14:45
*** e0ne has quit IRC14:46
*** e0ne has joined #tripleo14:48
*** edmund has joined #tripleo14:49
openstackgerritGonéri Le Bouder proposed a change to openstack/diskimage-builder: introduce $SYSTEMD_SYSTEM_UNIT_DIR
openstackgerritPhil Neal proposed a change to openstack/tripleo-heat-templates: Add CM signing directory to overcloud template
openstackgerritPhil Neal proposed a change to openstack/tripleo-heat-templates: Add CM signing directory to overcloud template
*** untriaged-bot has joined #tripleo15:00
untriaged-botUntriaged bugs so far:15:00
uvirtbotLaunchpad bug 1320090 in tripleo "pxe_ephemeral_format': u'ext4 left on nodes after deploys deleted" [Undecided,New]15:00
*** untriaged-bot has quit IRC15:00
*** rlandy has quit IRC15:01
openstackgerritAlexis Lee proposed a change to openstack/tripleo-incubator: Make a separation between --heat-env
*** TravT has joined #tripleo15:07
lxslidanpb: you're right, the double line should cross between the big t-i-e box and TE_DATAFILE, correct?15:15
TheJulia/w/win 715:17
*** giulivo has quit IRC15:19
*** gothicmindfood has quit IRC15:21
*** e0ne_ has joined #tripleo15:24
*** e0ne has quit IRC15:28
*** jprovazn is now known as jprovazn_afk15:28
*** rdopiera has joined #tripleo15:29
*** bnemec is now known as beekneemech15:37
*** jdob_ has quit IRC15:37
*** akrivoka has quit IRC15:37
*** jdob_ has joined #tripleo15:40
*** lsmola has quit IRC15:41
openstackgerritAlexis Lee proposed a change to openstack/tripleo-incubator: Add an overview diagram to the userguide
*** derekh_ has quit IRC15:46
*** eghobo has joined #tripleo15:50
openstackgerritThom Leggett proposed a change to openstack/tripleo-image-elements: Adds passthrough config for nova rootwrap
*** matty_dubs is now known as matty_dubs|lunch15:57
*** IvanBerezovskiy has left #tripleo16:02
*** eghobo has quit IRC16:03
*** eghobo has joined #tripleo16:03
*** hashar has quit IRC16:04
*** giulivo has joined #tripleo16:04
*** cwolferh has joined #tripleo16:05
*** shakamunyi has quit IRC16:20
*** julim has quit IRC16:21
*** ifarkas has quit IRC16:22
openstackgerritOpenStack Proposal Bot proposed a change to openstack/os-cloud-config: Updated from global requirements
openstackgerritOpenStack Proposal Bot proposed a change to openstack/os-collect-config: Updated from global requirements
*** jrist has quit IRC16:32
openstackgerritA change was merged to openstack/os-cloud-config: Loop for 600 seconds rather than checking ironic
openstackgerritA change was merged to openstack/os-cloud-config: Switch from subprocess to novaclient
openstackgerritA change was merged to openstack/os-cloud-config: Loop for 600 seconds rather than checking nova-bm
openstackgerritA change was merged to openstack/os-cloud-config: Switch from subprocess to keystoneclient
jistrStevenK: ^16:35
*** TravT has quit IRC16:37
openstackgerritDan Prince proposed a change to openstack/diskimage-builder: Name deploy-ironic and deploy-baremetal files uniq
openstackgerritDan Prince proposed a change to openstack/diskimage-builder: Add test to ensure we don't duplicate filenames
openstackgerritDan Prince proposed a change to openstack/diskimage-builder: Name 01-install-bin uniquely
openstackgerritDan Prince proposed a change to openstack/diskimage-builder: Name 10-rhel-cloud-image uniquely
openstackgerritDan Prince proposed a change to openstack/diskimage-builder: Name 99-setup-first-boot uniquely
openstackgerritDan Prince proposed a change to openstack/diskimage-builder: Delete redhat-common 00-usr-local-bin-secure-path
openstackgerritA change was merged to openstack/tripleo-heat-templates: ssl: Add support for a CA Certificate
*** david-lyle has joined #tripleo16:52
*** TravT has joined #tripleo16:54
*** jrist has joined #tripleo16:55
*** lucasagomes is now known as lucas-dinner16:57
*** jistr has quit IRC16:59
*** matty_dubs|lunch is now known as matty_dubs17:00
*** e0ne_ has quit IRC17:02
*** jdob_ has quit IRC17:03
*** jdob has joined #tripleo17:03
openstackgerritA change was merged to openstack/diskimage-builder: Set DISTRO_NAME in OS environment.d
*** martyntaylor has joined #tripleo17:25
*** julim has joined #tripleo17:26
*** giulivo has quit IRC17:27
*** jml has quit IRC17:28
openstackgerritJon-Paul Sullivan (jp_at_hp) proposed a change to openstack/tripleo-image-elements: Start openvswitch-switch early in upstart
*** shakayumi has joined #tripleo17:29
*** shakayumi has quit IRC17:29
*** eghobo has quit IRC17:31
*** dtantsur is now known as dtantsur|afk17:37
*** e0ne has joined #tripleo17:39
*** e0ne_ has joined #tripleo17:43
*** e0ne has quit IRC17:44
*** sseago__ has quit IRC17:44
*** e0ne_ has quit IRC17:47
*** martyntaylor has left #tripleo17:52
*** sseago has joined #tripleo18:06
*** akuznetsov has quit IRC18:07
*** dprince has quit IRC18:09
*** akuznetsov has joined #tripleo18:11
*** dshulyak_ has joined #tripleo18:11
*** gothicmindfood has joined #tripleo18:15
openstackgerritFlint Calvin proposed a change to openstack/tripleo-incubator: Making the necessary changes so that the value of metering_secret gets set appropriately in /etc/ceilometer/ceilometer.conf.
openstackgerritJon-Paul Sullivan (jp_at_hp) proposed a change to openstack/tripleo-incubator: Add the monitoring server and agents if requested
openstackgerritA change was merged to openstack/tripleo-image-elements: Glance: Enable passthrough configuration
lifelessmarios: :(18:44
lifelessNg: thats ok. Can I get DNS stuff done ?18:44
lifelessadam_g: I'm seeing elements/keystone/os-apply-config/etc/keystone/keystone.conf: needs merge18:45
lifelesserror: you need to resolve your current index first18:45
lifelessTraceback (most recent call last):18:45
Nglifeless: oh, nuts, yes18:48
greghayneslifeless: curious what you think of the comment on
*** pelix has quit IRC18:50
*** yamahata has quit IRC18:52
Nglifeless: done18:53
*** e0ne has joined #tripleo18:58
openstackgerritA change was merged to openstack/tripleo-incubator: Properly default MysqlInnodbBufferPoolSize (undercloud) v2
*** jp_at_hp1 has quit IRC18:59
lifelessNg: for clarity, what records did you add/update ?19:01 ->
adam_glifeless, context?19:01
Nglifeless: ah, I see there was more19:01
lifelessadam_g: prep_source trees19:01
NgI am finding my current IRC client+bouncer setup to be a bit lossy wrt scrollback, which is very sadmaking and will probably kill it for me19:01
* Ng reads logs instead19:02
lifelessNg: so on .319:02
*** e0ne has quit IRC19:02
lifelessNg: on .419:03
lifelessNg: on .519:03
*** e0ne has joined #tripleo19:06
*** morganfainberg is now known as network_destroye19:06
*** network_destroye is now known as morganfainberg19:06
*** Penick has joined #tripleo19:08
*** jang1 has joined #tripleo19:09
*** viktors has left #tripleo19:09
*** Penick_ has joined #tripleo19:11
*** noslzzp has quit IRC19:11
openstackgerritGregory Haynes proposed a change to openstack-infra/tripleo-ci: Add /mnt/state/var/log to log cleanup
*** Penick has quit IRC19:12
*** Penick_ is now known as Penick19:12
lifelessadam_g: so my tripleo-image-elements source tree isn't being setup properly19:13
lifelessadam_g: my refs yaml is the one from 24 hours ago, with one added ref:         # hp1 region configs
lifeless        - 66/96666/319:13
Nglifeless: done19:16
lifelessNg: can you also remove the unqualified endpoints ? e.g. ?19:16
Nglifeless: do we want to have a deprecation period before doing that?19:17
lifelessNg: update their ips then :P19:17
Nglifeless: done. only ci-overcloud was wrong19:19
lifelessadam_g: fixed by updating one of the refs19:20
lifelessadam_g: you've deleted all the comments ? :(19:21
lifelessoh, github being ridonkulous19:22
*** noslzzp has joined #tripleo19:24
adam_glifeless, ah yeah i was just going to suggest outdated refs.  i haven't put too much time into that script (obviously), it was just a quick 5 minute conversion of a shell script being used by nick. if its a valuable thing, it can definitely be improved. especially now that gerrit has a REST API19:25
lifelessadam_g: did you see the spec I kicked off ?19:26
lifelessadam_g: I'd love it if you or nick were to take that over19:26
lifelessadam_g: if you haven't, have a look in the tripleo-specs repo19:27
*** akuznetsov has quit IRC19:29
adam_glifeless, yeah read it last night. will take a closer look once im done wrestling these racks19:32
adam_glifeless, hmm now that we've upgraded jump host on 405 to trusty, im seeing the same tftp issues that were supposed to be resolved /w a mellanox driver update.. on nodes that were not having issues previously19:34
lifelessadam_g: oh hahahahahahahahahahahahahaahahahahahahahahaahhahhahahaha19:34
openstackgerritlifeless proposed a change to openstack/tripleo-incubator: Fix --heat-env option handling for undercloud.
greghaynesmellanox whack-a-mole19:36
vinshburn it all.19:36
openstackgerritlifeless proposed a change to openstack/tripleo-image-elements: Add a networks.json file for the hp1 region.
openstackgerritJames Slagle proposed a change to openstack/diskimage-builder: Check return code of element-info
lifelessgreghaynes: replied19:46
* SpamapS reads about more mellanox fail and sighs19:48
*** blamar has quit IRC19:51
lifelesshuh thats odd19:54
lifelesswhy doesn't keepalived look at the bridge?19:54
lifelesslots of use, should all be on the br-ex or br-ctlplane. sadface. much wow.19:55
*** noslzzp has quit IRC19:56
vinshoh ya... I saw this actually..19:56
vinshkeepalived was advertising on eth0 and br-ex..19:56
vinshbut would only really listen/reply on br-ex19:57
vinshMaybe kinda sorta related to what you are dealing with. Maybe.19:57
lifelesscould well be19:57
lifelessgeneral rule AIUI is 'never use the member devices directly cause they wont work'19:58
vinshIn the case I was looking at, I had 3 control nodes up .. using devtest.. and had keepalived and haproxy going.19:58
vinshrequired br-ex as keepalived interface19:58
vinshand haproxy required sysctl settings to bind to non local ports. (for floating virtual ip).19:58
openstackgerritlifeless proposed a change to openstack/tripleo-heat-templates: Add custom public IP support to the undercloud.
*** hewbrocca has quit IRC20:01
lifelessgreghaynes: I'm sympathetic to the idea of splitting out the user supplied env settings from ours20:03
lifelessgreghaynes: want to add that into lxsli's heat env spec?20:03
lifelessadam_g: how would you feel about making the git update stuff separate to the rc file20:09
adam_glifeless, sure20:10
lifelessadam_g: I realise we have to do the various calculations, but opening up a new terminal and connecting to a .. coo20:10
lifelessbah. Typing fail. I should go do saturday.20:10
lifelessyay. stab. Monitor is in graphics mode or an unsupported text mode.20:12
*** tteggel_ has joined #tripleo20:16
*** casanch1 has joined #tripleo20:18
greghayneslifeless: yep, will do20:20
*** SpamapS_ has joined #tripleo20:20
*** jeblair_ has joined #tripleo20:20
*** SpamapS has quit IRC20:21
*** tteggel has quit IRC20:21
*** jeblair has quit IRC20:21
morganfainberglifeless, what is your view on using caching in triple-o for keystone.20:21
*** SpamapS_ is now known as SpamapS20:21
*** jeblair_ is now known as jeblair20:22
*** SpamapS has quit IRC20:22
*** SpamapS has joined #tripleo20:22
SpamapSmorganfainberg: which caching?20:22
morganfainbergSpamapS, the cache-layer stuff.20:22
morganfainberge.g. domain, project,20:22
lifelessmorganfainberg: if keystone recommends it, we'll figure out how to do it. That said, I'm skeptical of caching as a workaround for poor design/code :).20:23
morganfainbergrequires memcache or redis (or similar) running.20:23
morganfainberglifeless, i'll look over the triple-o stuff and see if i can propose some stuff to get it in there over the weekend20:23
lifelessmorganfainberg: SQL is extremely fast when used well, and it has a big cache and so forth20:23
SpamapSThe only performance problems I've seen in keystone for tripleo is in getting revocation lists and deleting users.20:23
lifelesse.g. redis has all the IO characteristics of MySQL20:23
morganfainbergSpamapS, lifeless , the revocation list is on the caching list.20:24
SpamapSmorganfainberg: but what happened to revocation events?20:24
morganfainbergSpamapS, well, we're working on it. don't want to leave you all high-and dry if deployed w/o events20:24
morganfainbergSpamapS, i expect events to be ready by juno, but doesn't mean everyone will be using them. -- but since keystone is run in eventlet, anything we can offload from the DB (at times) can help.20:25
SpamapSyeah, silly eventlet20:25
SpamapSHad a few engineers corner me yesterday and basically ask "WHY EVENTLET?! WHY!!"20:26
*** e0ne has quit IRC20:26
SpamapSof course, their thinking was "this will suck in the future when we have 200 core CPUs"20:26
*** e0ne has joined #tripleo20:27
lifelessmorganfainberg: don't we defer DB load to a native thread?20:27
morganfainberglifeless, not that i am aware.20:27
lifelessmorganfainberg: oh, well that would suck :)20:27
morganfainberglifeless, we rely on... eventlet magic when not under apache20:27
lifelessmorganfainberg: we could use the native python mysql driver I suppose ;)20:28
morganfainbergunder apache we can't rely on the external threading, we just rely on "we have a bunch of these things"20:28
morganfainbergSpamapS, I want eventlet to go away in keystone personally :P20:28
morganfainberggetting close to being able to gate on apache deployed keystone ^_^20:28
morganfainbergthen i can really claim it is the "Recommended deployment" :)20:29
SpamapSlifeless: Somebody did some testing w/ the pure python mysql module and the trade off in response time was negative.20:30
morganfainbergSpamapS, that is unfortunate20:31
SpamapSmorganfainberg: if you're not using eventlet for request multiplexing or backend multiplexing, then yeah, eventlet makes very little sense.20:31
morganfainberghas the token size bitten triple o much yet?20:32
SpamapSmorganfainberg: yes, when our token table gets to 4GB in just one day... I'd call that a bite. ;)20:32
morganfainbergSpamapS, no i mean the request-header stuff20:32
lifelessmorganfainberg: we don't deploy keystone behind apache20:33
lifelessmorganfainberg: I'm not sure I understand why you want to, TBH20:33
*** rpodolyaka1 has joined #tripleo20:33
SpamapSmorganfainberg: no the header stuff is all OPP (Other peoples' problems)20:33
morganfainbergSpamapS, the token table stuff
morganfainbergSpamapS, spec to resolve it20:33
SpamapSlifeless: you don't know why we'd want to deploy keystone inside a much more scalable thing than eventlet?20:33
morganfainberglifeless, SpamapS, and for SAML/federation support20:33
morganfainbergwe use apache modules to do that.20:34
lifelessSpamapS: a) your troll is weak. b) you can do multiple processes for scaling with eventlet (and we do), so ??? wtf20:34
lifelessmorganfainberg: ah, ok so thats a decent reason. What about folk that want to use nginx etc? Out of luck?20:34
morganfainberglifeless, once apache is stable, i will be working on nginx + unicorn/uwsgi etc20:35
morganfainberglifeless, there are modules to do cool things with other servers as well and i want all of them (i actually want the recommended deployment to be nginx, but, right now a dream)20:35
openstackgerritThom Leggett proposed a change to openstack/tripleo-image-elements: Adds passthrough config for nova rootwrap
SpamapSThe benchmarks I've seen show nginx no longer a win vs. Apache 2.4 w/ the event mpm20:37
*** e0ne has quit IRC20:38
morganfainbergSpamapS, good to know (esp with 14.04 deploying apache 2.4)20:39
morganfainbergwill need to see how well mpm event works w/ mod_wsgi20:39
devanandalifeless: for your BMC woes, i'm curious - have you guys tried using the ipminative driver?20:40
openstackgerritThom Leggett proposed a change to openstack/tripleo-image-elements: Adds passthrough config for nova rootwrap
*** jdob has quit IRC20:43
lifelessdevananda: nope20:43
devanandalifeless: it may or may not help -- but i'd be very curious if it does20:45
SpamapSdevananda: I am curious .. what is the difference?20:49
*** rdopiera has quit IRC20:49
devanandaSpamapS: totally different implementations of IPMI20:49
devanandalots more "intellitence" in ipminative -- in theory20:49
beekneemechlifeless: You might have been thinking of the horrible thread pool hack that requires an out-of-tree patch to eventlet to work:
*** e0ne has joined #tripleo20:52
lifelessbeekneemech: well, blockingin an event loop is itself horrible20:55
*** rpodolyaka1 has quit IRC20:55
SpamapSdevananda: ah, and done in native python instead of with ipmitool20:57
*** pcrews has quit IRC20:59
*** untriaged-bot has joined #tripleo21:00
untriaged-botUntriaged bugs so far:21:00
uvirtbotLaunchpad bug 1320090 in tripleo "pxe_ephemeral_format': u'ext4 left on nodes after deploys deleted" [Undecided,New]21:00
*** untriaged-bot has quit IRC21:00
*** openstackgerrit has quit IRC21:01
beekneemechlifeless: No argument here.  I just don't like that we have code in-tree that just plain doesn't work if you install the requirements we tell you should work.21:02
*** dshulyak_ has quit IRC21:03
*** gcha has quit IRC21:03
lifelessbeekneemech: I dislike that too :)21:04
lifelesshmm undercloud timeout (in instance)21:04
*** lucas-dinner has quit IRC21:13
*** casanch1 has quit IRC21:16
*** cwolferh_ has joined #tripleo21:18
*** cwolferh has quit IRC21:20
*** hashar has joined #tripleo21:21
*** e0ne has quit IRC21:22
lifelessugh, thats horribly broken21:22
lifelessI think the ensure-bridge rework has broken the vlan support21:22
lifelessyeah, its regressed.21:25
*** tteggel_ has quit IRC21:29
*** tteggel has joined #tripleo21:30
lifelessah maybe not21:31
lifelessmaybe bad data21:31
*** jang has quit IRC21:35
*** markmc has quit IRC21:37
*** chuckC has quit IRC21:40
*** chuckC has joined #tripleo21:41
*** matty_dubs is now known as matty_dubs|gone21:42
*** david-lyle has quit IRC21:50
*** morazi has quit IRC21:51
*** chuckC has quit IRC21:51
* SpamapS -> airpot22:00
lifelessreally ;)22:09
lifelessso vlan25 support is not quite right22:11
*** hashar has quit IRC22:12
*** chuckC has joined #tripleo22:16
uvirtbotLaunchpad bug 1325114 in tripleo "vlan support regression" [Critical,Triaged]22:19
lifeless^ anyone wanting to help get CI back up, fixing that bug would be a big step.22:19
*** openstackgerrit has joined #tripleo22:32
*** openstackgerrit has quit IRC22:35
*** openstackgerrit has joined #tripleo22:36
openstackgerritA change was merged to openstack/tripleo-image-elements: Use sysctl -p instead of --load
*** ddieterl_ has quit IRC22:45
*** sseago has quit IRC22:49
*** shardy has quit IRC22:52
*** blamar has joined #tripleo23:01
*** TravT has quit IRC23:06
jogolifeless: is it safe to run a recheck on
jogoand is there a bug I can use or just no bug23:06
jogolifeless: btw w/ respect to our conversation the other day: if you want graphite to plot failures by provider zuul will need updating AFAIK23:07
*** sseago has joined #tripleo23:12
*** jogo is now known as flashgordon23:18
*** sseago has quit IRC23:27
*** edmund has quit IRC23:32
*** sseago has joined #tripleo23:39

Generated by 2.14.0 by Marius Gedminas - find it at!