Monday, 2014-06-09

*** yamahata has quit IRC00:04
*** matsuhashi has joined #tripleo00:21
*** michchap has joined #tripleo00:50
*** nati_ueno has quit IRC00:53
openstackgerritGregory Haynes proposed a change to openstack/tripleo-image-elements: Add os-is-bootstrap-host element and script  https://review.openstack.org/8643501:09
*** jml has quit IRC01:16
*** nati_ueno has joined #tripleo01:19
*** jml has joined #tripleo01:20
*** nati_ueno has quit IRC01:20
*** yamahata has joined #tripleo01:23
*** nosnos has joined #tripleo01:43
*** pcrews_ has joined #tripleo01:49
*** pcrews has quit IRC01:52
*** yamahata has quit IRC01:54
*** yamahata has joined #tripleo01:54
*** pcrews_ has quit IRC02:16
*** julim has quit IRC02:26
*** untriaged-bot has joined #tripleo03:00
untriaged-botUntriaged bugs so far:03:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/132576203:00
uvirtbotLaunchpad bug 1325762 in tripleo "intermittent tftp issues using physical hardware" [Undecided,New]03:00
*** untriaged-bot has quit IRC03:00
*** tzumainn has quit IRC03:03
*** eghobo has joined #tripleo03:16
*** nosnos has quit IRC03:18
*** greghaynes has quit IRC03:51
*** greghaynes has joined #tripleo03:52
*** greghaynes has quit IRC03:52
*** greghaynes has joined #tripleo03:52
*** matsuhashi has quit IRC04:04
*** nosnos has joined #tripleo04:06
*** matsuhashi has joined #tripleo04:13
*** noslzzp has joined #tripleo04:16
*** rameshg87 has joined #tripleo04:58
*** sseago has quit IRC04:59
StevenKlifeless: You comment "You have it spot on" on my os-cloud-config logging branch is on line 1 of the base patchset -- are you agreeing with Tomas, or trying to say something else?05:01
lifelessagreeing with Tomas05:03
lifelesshmm, gertty fail I guess05:03
*** akuznetsov has joined #tripleo05:06
*** ramishra has joined #tripleo05:17
rameshg87hello tripleo-reviewers05:28
rameshg87request you to take a look at the reviews: https://review.openstack.org/97143 https://review.openstack.org/9730105:29
*** rakesh_hs has joined #tripleo05:32
*** jprovazn has joined #tripleo05:33
*** ramishra has quit IRC05:34
*** ramishra has joined #tripleo05:37
*** flashgordon is now known as jogo05:46
*** xuhaiwei_ has joined #tripleo05:47
*** sseago has joined #tripleo05:49
*** akuznetsov has quit IRC05:53
*** jcoufal has joined #tripleo05:57
*** rameshg87 has left #tripleo05:59
*** rdopiera has joined #tripleo06:01
*** akuznetsov has joined #tripleo06:13
*** eghobo has quit IRC06:14
*** eghobo has joined #tripleo06:14
*** eguz has joined #tripleo06:23
*** eghobo has quit IRC06:27
*** akuznets_ has joined #tripleo06:28
*** akuznetsov has quit IRC06:29
*** akuznetsov has joined #tripleo06:34
*** akuznets_ has quit IRC06:37
*** eguz has quit IRC06:54
*** nosnos has quit IRC06:57
*** lsmola_ has joined #tripleo07:05
*** e0ne has joined #tripleo07:16
openstackgerritRadomir Dopieralski proposed a change to openstack/tuskar-ui: Silence the deprecation warning about with statements  https://review.openstack.org/9870307:20
openstackgerritRadomir Dopieralski proposed a change to openstack/tuskar-ui: Silence the deprecation warning about with statements  https://review.openstack.org/9870307:22
*** akuznets_ has joined #tripleo07:25
*** ifarkas has joined #tripleo07:26
*** akuznetsov has quit IRC07:27
*** ramishra has quit IRC07:29
openstackgerritRadomir Dopieralski proposed a change to openstack/tuskar-ui: Don't use deprecated keystone module  https://review.openstack.org/9870507:39
SpamapS7/win 1307:43
rdopiera\/msg #tripleo /lose 2107:44
*** akuznets_ has quit IRC07:51
*** akuznetsov has joined #tripleo07:51
*** akuznets_ has joined #tripleo07:53
*** markmc has joined #tripleo07:53
*** akuznetsov has quit IRC07:56
*** pblaho has joined #tripleo07:57
openstackgerritClint 'SpamapS' Byrum proposed a change to openstack/tripleo-heat-templates: Replace nagios3 template with HOT rewrite  https://review.openstack.org/9066507:57
SpamapS^^ We might want to consider moving templates like that out of t-h-t into some other utility repo (or a contrib dir or something). It is mostly for our own infrastructure needs.07:58
*** jistr has joined #tripleo08:00
SpamapShrm.08:01
SpamapSI'd love to test https://review.openstack.org/#/c/97703/ locally..08:01
*** akuznets_ has quit IRC08:01
SpamapSbut I can't even build a working seed08:01
*** russell_h has quit IRC08:01
*** russell_h has joined #tripleo08:02
*** openstackgerrit has quit IRC08:02
*** athomas has joined #tripleo08:04
*** martyntaylor has joined #tripleo08:07
*** openstackgerrit has joined #tripleo08:07
elynnHi all, do you have time to review this patch? https://review.openstack.org/#/c/90887/08:07
SpamapSelynn: any reason you don't just put that image in ~/.cache/image-create ?08:08
*** derekh_afk is now known as derekh_08:09
elynnhi SpamapS , ubuntu element only support tar.gz now, not support qcow2 image.08:09
SpamapSelynn: also you can use a file:// path already08:09
SpamapSoh qcow2?08:09
openstackgerritJan Provaznik proposed a change to openstack/tripleo-heat-templates: Enable multi-node rabbitmq cluster  https://review.openstack.org/9870908:10
SpamapSwhy would you want to use that?08:10
elynnbecause my custom image is in qcow2 format...08:10
*** anteaya has quit IRC08:10
SpamapSelynn: ok, I'll leave my feedback in the review. thanks for clarifying.08:11
elynnthx for your review, SpamapS : )08:12
*** russell_h has quit IRC08:13
*** ramishra has joined #tripleo08:13
*** andreaf has joined #tripleo08:16
*** russell_h has joined #tripleo08:16
*** lucasagomes has joined #tripleo08:20
derekh_lifeless: did you spot the patch for https://bugs.launchpad.net/tripleo/+bug/132709008:20
uvirtbotLaunchpad bug 1327090 in tripleo "can't deploy ci-overclouds on Ubuntu - ensure-bridge wipes out /e/n/i" [High,In progress]08:20
*** russell_h has quit IRC08:21
lifelessderekh_: I have been (gosh, golly) reviewing today08:22
lifelessderekh_: but I saw it and had it on my to-try list08:23
lifelesslet me kick that now08:23
lifelessfirst off I have to test this new (hopefully fixed) node08:23
*** russell_h has joined #tripleo08:24
lifelessactually, looks like i don't need to test anything there, righto onto the patch08:25
lifelessderekh_: btw have you seen tripleo-end-to-end/repo_refs.yaml ?08:25
derekh_lifeless: cool, btw, I had noticed we weren't getting elastic recheck comments for signatures that I thought we should have been, so went digging https://review.openstack.org/#/c/98154/08:26
derekh_will talk to sdague later when he is on08:26
derekh_lifeless: no hadn't seen it08:26
*** ramishra has quit IRC08:27
lifelesswhen you import the seed.hp1.. rc file it automerges etc08:28
lifelessnifty and the inspiration for a spec ;)08:28
openstackgerritGonéri Le Bouder proposed a change to openstack/tripleo-image-elements: indent using 4 spaces (1/3)  https://review.openstack.org/9320608:29
derekh_lifeless: the spec about multiple refs in source-repositories ? will take a look08:29
lifelessrightyup08:29
lifeless40 node cloud deploy kicked off08:30
lifelessit will need manual intervention to 'ensure-bridge br-untagged'08:30
lifelessI don't want to take this cloud to production while such manual steps remain08:30
*** russell_h has quit IRC08:31
derekh_lifeless: ok, makes sense08:31
lifelesshow did you deploy the rh region without manual intervention for that step?08:32
lifelessjust needed the one bridge I guess?08:32
*** sseago has quit IRC08:33
*** russell_h has joined #tripleo08:33
SpamapSNo valid host was found. Reason: No conductor service registered which supports driver null. (HTTP 400)08:33
SpamapSargh08:33
*** sseago has joined #tripleo08:33
*** jtomasek has quit IRC08:33
*** jtomasek has joined #tripleo08:34
*** martyntaylor has left #tripleo08:34
SpamapSdoh08:34
SpamapS0 nodes08:34
derekh_lifeless: iirc, we did have manual intervention08:35
*** rameshg87 has joined #tripleo08:35
*** proffalken has quit IRC08:35
*** proffalken has joined #tripleo08:36
lifelessSpamapS: still lagged?08:38
*** russell_h has quit IRC08:38
*** akrivoka has joined #tripleo08:38
*** yamahata has quit IRC08:41
*** russell_h has joined #tripleo08:42
*** rlandy has joined #tripleo08:46
*** e0ne has quit IRC08:47
SpamapSlifeless: -ish ..08:50
SpamapSlifeless: much better now.. just mostly trying to get ahead of the flood of monday patches/reviews/etc. :-P08:50
lifelesscool08:54
*** rdopiera has quit IRC08:56
lifelessmordred: https://review.openstack.org/#/c/97583/ <- I think thats a bug in pbr08:57
*** russell_h has quit IRC08:58
*** russell_h has joined #tripleo08:59
lifelessmordred: i'm happy to put up a patch08:59
lifelessmordred: if you like the idea I just put in the review08:59
*** untriaged-bot has joined #tripleo09:00
untriaged-botUntriaged bugs so far:09:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/132576209:00
uvirtbotLaunchpad bug 1325762 in tripleo "intermittent tftp issues using physical hardware" [Undecided,New]09:00
*** untriaged-bot has quit IRC09:00
*** pelix has joined #tripleo09:01
*** russell_h has quit IRC09:02
*** IvanBerezovskiy has joined #tripleo09:02
*** pblaho has quit IRC09:02
*** rdopiera has joined #tripleo09:03
*** russell_h has joined #tripleo09:04
*** akrivoka_ has joined #tripleo09:11
*** russell_h has quit IRC09:14
*** akrivoka has quit IRC09:14
*** russell_h has joined #tripleo09:14
lifelesshmm09:15
lifelessthats amusing09:15
lifelessnow we're building all the computes09:15
lifelessbefore we try to spawn the control09:15
openstackgerritGonéri Le Bouder proposed a change to openstack/diskimage-builder: map-services: add openvswitch  https://review.openstack.org/9674909:17
openstackgerritGonéri Le Bouder proposed a change to openstack/diskimage-builder: map-services: add openvswitch  https://review.openstack.org/9674909:18
*** russell_h has quit IRC09:18
*** russell_h has joined #tripleo09:20
*** pblaho has joined #tripleo09:21
*** yfaykya has quit IRC09:25
*** russell_h has quit IRC09:26
*** athomas has quit IRC09:26
*** ci-overcloud has joined #tripleo09:28
ci-overcloud************** ci-overcloud complete status=1 ************09:28
*** ci-overcloud has quit IRC09:28
*** yfaykya has joined #tripleo09:30
*** matsuhashi has quit IRC09:31
*** matsuhashi has joined #tripleo09:31
*** yfaykya has quit IRC09:35
*** yfaykya has joined #tripleo09:36
*** athomas has joined #tripleo09:37
*** akuznetsov has joined #tripleo09:38
lifelesshmm09:38
lifeless| NovaCompute31            | 40c19b44-b33f-4040-af63-09a7ee4ae2ac | Error: Creation of server ci-overcloud-NovaCompute31-sfsoeoirxccn failed. | CREATE_FAILED      | 2014-06-09T09:28:10Z |09:39
*** matsuhashi has quit IRC09:39
*** matsuhashi has joined #tripleo09:40
lifelessderekh_: tag :) - I need to sleep; in principle its just diagnose and fix the issue, then09:40
lifelessUSE_CACHE=1 tripleo-image-elements/elements/tripleo-cd/deploy-ci-overcloud09:40
lifeless+ login to the controller before things timeout and run ensure-bridge br-untagged eth2 after checking that os-collect-config --print returns sane data etc.09:40
*** yfaykya has quit IRC09:41
lifelessderekh_: /09:41
lifelessderekh_: the cached images have your patch in them09:42
derekh_lifeless: ok, cool, will jump in in a few minutes09:42
*** yfaykya has joined #tripleo09:42
lifelessderekh_: I haven't looked at what caused that server to fail to deploy; if you put stuff into maintenance please add a note to machine-information-tab.txt09:42
derekh_lifeless: will do09:43
lifelessderekh_: and if it needs physical/DC intervention get ng or ghe or $whomever to file a ticket for you09:43
derekh_ok09:43
lifelesscool, thanks!09:43
derekh_np09:43
*** akrivoka_ has quit IRC09:46
*** yfaykya has quit IRC09:47
*** yfaykya has joined #tripleo09:48
*** yfaykya has quit IRC09:53
*** yfaykya has joined #tripleo09:54
*** e0ne has joined #tripleo09:55
*** yfaykya has quit IRC09:59
*** yfaykya has joined #tripleo10:01
*** russell_h has joined #tripleo10:01
*** yfaykya has quit IRC10:06
*** yfaykya has joined #tripleo10:07
*** anteaya has joined #tripleo10:11
*** yfaykya has quit IRC10:11
*** yfaykya has joined #tripleo10:13
*** yfaykya has quit IRC10:17
*** yfaykya has joined #tripleo10:19
*** yfaykya has quit IRC10:23
*** yfaykya has joined #tripleo10:25
*** e0ne has quit IRC10:29
*** yfaykya has quit IRC10:29
*** xuhaiwei_ has quit IRC10:30
*** yfaykya has joined #tripleo10:31
lxslilifeless: happy with https://review.openstack.org/#/c/94910/ ?10:33
*** yfaykya has quit IRC10:36
*** yfaykya has joined #tripleo10:37
*** yfaykya has quit IRC10:42
*** yfaykya has joined #tripleo10:43
*** yfaykya has quit IRC10:48
*** yfaykya has joined #tripleo10:50
*** matsuhashi has quit IRC10:51
*** rakesh_hs has quit IRC10:52
*** yfaykya has quit IRC10:54
*** yfaykya has joined #tripleo10:56
*** rameshg87 has left #tripleo10:56
*** matsuhashi has joined #tripleo10:58
*** yfaykya has quit IRC11:00
*** e0ne has joined #tripleo11:00
*** yfaykya has joined #tripleo11:02
*** e0ne has quit IRC11:04
*** e0ne has joined #tripleo11:06
*** yfaykya has quit IRC11:07
*** yfaykya has joined #tripleo11:08
*** akrivoka has joined #tripleo11:11
*** yfaykya has quit IRC11:12
*** yfaykya has joined #tripleo11:14
*** yfaykya has quit IRC11:18
*** matsuhashi has quit IRC11:18
*** matsuhashi has joined #tripleo11:19
*** yfaykya has joined #tripleo11:20
*** matsuhashi has quit IRC11:24
*** yfaykya has quit IRC11:25
*** yfaykya has joined #tripleo11:26
*** ramishra has joined #tripleo11:28
*** yfaykya has quit IRC11:31
*** e0ne has quit IRC11:32
*** yfaykya has joined #tripleo11:32
*** weshay has joined #tripleo11:37
*** yfaykya has quit IRC11:37
*** yfaykya has joined #tripleo11:39
*** DuncanT has quit IRC11:41
*** rlandy has quit IRC11:42
*** yfaykya has quit IRC11:43
*** morazi has quit IRC11:43
*** yfaykya has joined #tripleo11:45
*** yfaykya has quit IRC11:49
*** yfaykya has joined #tripleo11:51
*** lucasagomes is now known as lucas-hungry11:51
*** rlandy has joined #tripleo11:55
*** yfaykya has quit IRC11:56
*** yfaykya has joined #tripleo11:57
openstackgerritGhe Rivero proposed a change to openstack/diskimage-builder: Factor out get kernel image  https://review.openstack.org/9875611:58
openstackgerritGhe Rivero proposed a change to openstack/diskimage-builder: Add get-kernel-image element  https://review.openstack.org/9875711:58
openstackgerritGhe Rivero proposed a change to openstack/diskimage-builder: Use common element get-image-kernel  https://review.openstack.org/9875811:58
openstackgerritGhe Rivero proposed a change to openstack/diskimage-builder: Improve kernel detection in debian family systems  https://review.openstack.org/9875911:58
*** tzumainn has joined #tripleo12:00
*** yfaykya has quit IRC12:01
*** tzumainn has quit IRC12:02
*** tzumainn has joined #tripleo12:02
*** dprince has joined #tripleo12:02
*** yfaykya has joined #tripleo12:03
*** jdob has joined #tripleo12:07
*** giulivo has joined #tripleo12:07
*** yfaykya has quit IRC12:08
*** yfaykya has joined #tripleo12:09
*** rbrady has joined #tripleo12:13
*** yfaykya has quit IRC12:14
*** yfaykya has joined #tripleo12:15
*** yfaykya has quit IRC12:20
*** yfaykya has joined #tripleo12:21
*** yfaykya has quit IRC12:26
*** yfaykya has joined #tripleo12:27
openstackgerritJan Provaznik proposed a change to openstack/tripleo-image-elements: Set glance registry host from heat metadata  https://review.openstack.org/9877312:31
*** yfaykya has quit IRC12:32
*** yfaykya has joined #tripleo12:34
*** yfaykya has quit IRC12:38
*** yfaykya has joined #tripleo12:40
*** ccrouch has left #tripleo12:44
*** yfaykya has quit IRC12:45
*** yfaykya has joined #tripleo12:46
*** yfaykya has quit IRC12:51
*** lucas-hungry is now known as lucasagomes12:52
*** yfaykya has joined #tripleo12:52
d0ugaltzumainn: hi12:52
openstackgerritJan Provaznik proposed a change to openstack/tripleo-heat-templates: Use virtual ip for OS services  https://review.openstack.org/9877812:52
tzumainnd0ugal, hiya!12:52
openstackgerritA change was merged to openstack-infra/tripleo-ci: Create a dummy device on TE hosts  https://review.openstack.org/9133312:53
d0ugaltzumainn: I'm just getting my head around your comment.12:53
tzumainnd0ugal, it's just my understanding of what's going on, it's very possible others disagree12:53
d0ugaltzumainn: So - provider resources are fine, configurations files should be changed to environment files? and I need to add the master template concept?12:53
tzumainnd0ugal, I *think* so, but maybe discuss with jdob12:54
tzumainnI just wanted to state my understanding in one place rather than have it scattered in a hundred different comments12:54
d0ugalSure12:54
tzumainni.e., give people one place to disagree : )12:54
d0ugaland I want to make sure I understand your understanding because I figure out what my understanding is :)12:54
tzumainnlol12:54
jdobyes, the configuration is stored in the environment file and there's a master template that creates resources for the providers defined in the environment12:54
jdobalso realize though that, strictly speaking, the environment isn't just a config file as you may be referring to it12:55
jdob(not sure of the exact comment in question here)12:55
tzumainnjdob, https://review.openstack.org/#/c/97553/12:55
d0ugaljdob: bottom of https://review.openstack.org/#/c/97553/12:55
d0ugalah, too slow12:55
tzumainner, how do I link to a specific comment?12:55
tzumainnjdob, anyway, it's three from the bottom12:56
jdobyou misunderstood, I dont want to know the exact comment, I want to get out of this conversation12:56
jdob:)12:56
d0ugaljdob: there is no escape12:56
tzumainnlol12:56
*** yfaykya has quit IRC12:56
tzumainnjdob, you've gotten spicy in your old age12:57
jdobi shouldn't have gone to the doctor and just rode the black plague out from last week12:57
d0ugallol12:57
jdobi'm behind, four days of pretty much no interaction with people and all the shit talking is coming out at once12:57
tzumainnis it bad that I enjoy your agony?12:57
jdobone issue in this comment: "provider resource template"12:57
jdobits not just a single template12:58
jdobcheck out the tht repo right now, the config and resource def are split into two files12:58
tzumainnah, okay12:58
d0ugalalways two files?12:58
*** yfaykya has joined #tripleo12:58
jdobi dont think so12:58
tzumainnor is there the potential for more?12:58
tzumainnexciting!12:58
tzumainnmakes sense though12:58
jdobi'm trying to read the tht conversion spec now, i'm hoping it's covered in there12:58
d0ugalSo, this is the relationship thing?12:58
d0ugalso, this might be when we want shadower to join us :)12:59
*** julim has joined #tripleo13:01
shadowerm'ere13:02
shadowerd0ugal: I think I'm missing some context13:02
tzumainnshadower, we're trying to understand what it means to store "provider resource template(s)"13:02
tzumainnwhat exactly might that entail?  jdob does some clarification above ^13:03
*** yfaykya has quit IRC13:03
d0ugalshadower: I think the main issue is that we could have more than one and we are not sure how they are related.13:03
*** yfaykya has joined #tripleo13:04
shadowerright, so13:04
shadowerin general, you'd have 1 provider template per role. So e.g. nova-compute.yaml, controller.yaml, object-store.yaml, etc.13:05
shadowerthen you have one template that references puts tham all together (overcloud.yaml)13:05
shadowerand environment.yaml which is where the role -> resource type mapping (and possibly parameters) go13:06
shadowers/parameters/parameter values/13:06
tzumainnshadower, but jdob is saying that a role might be captured in multiple templates13:06
shadoweroh13:06
shadowermissed that bit13:06
jdobshadower: referring to the split between the resource def and the software config13:06
shadowerjdob: you mean one role could have multiple configs and we might want to put each config to its own file for better reuse?13:07
jdobnot exactly, I'm referring to the difference between swift-deploy.yaml and swift-source.yaml13:08
*** yfaykya has quit IRC13:09
shadowerI'm not fond of that setup. I'd prefer something like swift-source with the OS::Heat::Server and a corresponding deploy resource(s) and swift-config.yaml that contains the software config (and nothing else)13:10
*** yfaykya has joined #tripleo13:10
shadowerjdob: I don't see any value in splitting deploys and servers  (since they're closely tied to each other). Reusable configs are a different matter13:11
tzumainnshadower, but for purposes of storage - would it be correct to say that a role could be captured in multiple template files?13:11
openstackgerritAlexis Lee proposed a change to openstack/tripleo-incubator: Allow overcloud fixed range to be configurable  https://review.openstack.org/9762613:11
shadowertzumainn: yeah I would say so13:12
jdobshadower: ok, so what I see in tht with respect to that is changing?13:12
d0ugaland you'd want to pull them all out together? i.e. give me everything for 'swift'?13:12
shadowertzumainn: or rather, depends how you define "role"13:12
shadower:-)13:12
tzumainnshadower, compute, controller, object-store, etc :P13:12
tzumainnshadower, is there one "master template" in those cases that would reference everything else being used?13:13
tzumainnor would the various template files be independent?13:13
shadowertzumainn: I mean: you could say that role is just compute.yaml and then store configs separately. But the role would still reference & use the configs13:13
tzumainnshadower, okay, that makes sense13:13
jdobi'm hoping that the tuskar definition of a role is conceptual, but maps to all of the real pieces underneath13:14
tzumainnso it sounds like in general, when we talk about a template, it has the potential to refer to other templates, those other templates may refer to more templates, etc13:14
tzumainnlike a tree13:14
jdobso if I refer to the compute "role" as a user, Tuskar will translate if that means 1 file or 713:14
tzumainnbut there's always one primary template from which everything else flows?13:14
shadowertzumainn: each role should be standalone (heat stack-create -f compute.yaml + the assorted files should work) but we'd have a master template for the overcloud13:14
shadowertzumainn: yea, it can be a tree13:14
shadowertzumainn: but I don't think the storage system needs to capture that logic13:14
shadoweri.e. you should be able to just store a bunch of files13:15
tzumainnshadower, so you don't think it needs to capture the relationship(s) between the templates?13:15
*** yfaykya has quit IRC13:15
shadowertzumainn: I don't see why it should. The logic is captured within the templates themselves13:15
jdobshadower: but the problem comes in asking for them13:15
jdobso if I ask for a list of roles13:15
shadowerthe storage could store the whole bundle as .tar.gz for all we care13:15
jdobthat's a lot of inspection13:16
shadowerhm13:16
shadowerso13:16
jdobthat's why I think the tuskar storage layer isn't just a generic file storage mechanism13:16
jdobbecause it has to be optimized for certain questions13:16
jdoband since it's not a DB, it's not as simple as writing queries13:16
*** yfaykya has joined #tripleo13:16
d0ugalIf only it was a db... oh wait13:17
openstackgerritA change was merged to openstack/tuskar-ui: Don't use deprecated keystone module  https://review.openstack.org/9870513:17
shadowerI think the canonical operation should be "give me the overcloud I set up last Thursday" which should spit out all the templates you need for launching that thing13:17
shadowerimmutable & versioned (ideally)13:17
shadowerif different overclouds reference the same file in the background, that's fine13:17
openstackgerritA change was merged to openstack/tuskar-ui: Silence the deprecation warning about with statements  https://review.openstack.org/9870313:17
d0ugaljdob: It would be useful to build up a list of expected queries/questions13:18
tzumainnso. . . can a role be stored in some sort of folder construct13:18
jdobthat's what I thought your spec was doing (not to sound like an ass)13:18
jdobbut that's why I was expecting to see domain-specific APIs13:18
jdobthat defined those questions13:18
tzumainnand we just stick all the templates into that folder?13:18
jdoblist_roles instead of get(filename)13:18
d0ugaljdob: well, yes, but I mean from you as somebody that plans to use it13:18
shadowertzumainn: it could be but it's not at the moment and I'm not sure it should13:19
shadower(stored in a folder structure)13:19
openstackgerrittherese-mchale proposed a change to openstack/tripleo-image-elements: Use Percona Packages for Ubuntu  https://review.openstack.org/9013413:19
*** morazi has joined #tripleo13:19
jdobd0ugal: i hear ya, it's just a matter of time. faster for you to think it through and me to bitch it's not what I expected :)13:19
tzumainnshadower, what would be a reasonable alternative?13:19
jdobin all seriousness, I see your point, i just dont totally have the bandwidth to write it out all spec-like13:19
d0ugaljdob: sure, that's where I'm at I guess.13:19
d0ugaljdob: I didn't mean spec-like, just an ether or whatever - doesn't matter, it was just an idea.13:20
jdobno no, it was a valid one, don't let my punchiness today get in the way of that13:20
*** yfaykya has quit IRC13:21
*** yfaykya has joined #tripleo13:23
*** jang1 has quit IRC13:24
*** matty_dubs|gone is now known as matty_dubs13:24
shadowertzumainn: treat a deployment unit (e.g. the undercloud) as a set of templates that go together. No further structure needed (as far as I can see)13:25
*** ccrouch has joined #tripleo13:25
tzumainnshadower, the structure is needed because a role might be used in multiple overclouds13:25
tzumainnso we want to separate out "role templates" so we can stick them in the correct deployment units13:25
shadowertzumainn: but roles are a tripleo-specific concept. It's not necessarily useful to anyone else13:26
shadowerthe current direction seems to be towards tuskar being useful for describing othre things than openstack13:26
jdobnot sure I'd say it's only a tripleo-specfic concept13:27
shadowertzumainn: you should still be able to reuse files without introducing higher-level constructs such as roles13:27
jdobin any large-scale app, you might want to split your templates into multiple files like that13:27
*** yfaykya has quit IRC13:27
jdoband have a logical concept to refer to13:27
shadoweryeah I guess13:27
shadowertzumainn: btw I misspoke earlier, it's not a tree it's a directed acyclic graph13:28
shadower(two files could reference the same file)13:28
tzumainnshadower, yeah, you're right13:28
jdobthat sounds so much more intimidating than "tree" :)13:28
d0ugallol13:28
tzumainnsounds like it's time for jdob to pretend to be sick again!13:29
shadowerhence my preference to store them just as bundles13:29
shadowerworks for git, should work for us, too13:29
*** yfaykya has joined #tripleo13:29
tzumainnshadower, right, I guess my argument is just that it's useful to store the bundles at a 'role' level13:29
d0ugaltzumainn: Why?13:29
jdobhow else would you store them?13:29
jdobthe nice part about what tzumainn is saying is that the list of bundles is effectively the list of roles13:30
jdobi want the "compute" role, Tuskar pulls the compute bundle and starts handling it13:30
d0ugalbundle being a number of templates?13:30
*** ci-overcloud has joined #tripleo13:31
ci-overcloud************** ci-overcloud complete status=127 ************13:31
*** ci-overcloud has quit IRC13:31
jdobto be honest, I dunno13:31
jdobit kinda depends on what THT does13:31
*** howleyt has quit IRC13:32
jdoboriginally, I assumed each "role" would be split into a single template13:32
jdobbut that's not what I'm seeing in the tht repo right now13:32
shadoweryea me too13:32
jdobthe tht spec shadower wrote seems to allude to a single file13:32
jdoband I want to hash out in that spec if that's the plan or not13:32
jdoba lot is contingent on that13:32
shadowerjdob: I think I'll have to amend it13:32
d0ugalok13:32
jdobwhich is why I was so obnoxious in pushing for that spec to exist13:32
* shadower doesn't think we can rely on role/template having a 1-1 mapping13:33
jdobbecause it's hard to guess at given the current THT repo13:33
*** yfaykya has quit IRC13:33
openstackgerritGhe Rivero proposed a change to openstack/diskimage-builder: Use common element get-image-kernel  https://review.openstack.org/9875813:33
openstackgerritGhe Rivero proposed a change to openstack/diskimage-builder: Improve kernel detection in debian family systems  https://review.openstack.org/9875913:33
openstackgerritGhe Rivero proposed a change to openstack/diskimage-builder: Factor out get kernel image  https://review.openstack.org/9875613:34
openstackgerritGhe Rivero proposed a change to openstack/diskimage-builder: Add get-kernel-image element  https://review.openstack.org/9875713:34
d0ugalOkay, I'm going to go and spend a bit of time properly grokking that spec13:34
*** ramishra has quit IRC13:34
shadowerfwiw, my concern is this: all the structure necessary to figure out which templates to send to heat is in the template files themselves13:35
*** yfaykya has joined #tripleo13:35
shadowerin theory all you need is the root template13:35
shadowerso I'm wary of duplicating or restricting that logic higher up13:35
jdobunderstood, and that's awesome13:35
jdobthe only reason I'd say to duplicate13:35
jdoband i wouldnt even use that word13:35
d0ugalshadower: How does the root template refer to them? does it assume they are all on local disk?13:35
jdobcache might be more appropriate13:35
jdobis so that we can more easily ask the sort of tuskar questions we know we'll need to regularly ask13:36
shadowerd0ugal: via URIs13:36
jdobwithout loading everything up and parsing it every time13:36
shadowerd0ugal: they could theoretically be at different servers13:36
shadowerjdob: I'm fine with caching. But having a standalone Role resource (resource as in REST) at the *storage* level would limit things ihmo13:37
shadowerI'm perfectly fine with roles in Tuskar13:37
shadowerbut I'd like the storage to look more like git13:38
shadoweri.e. a bunch of templates that reference each other13:38
d0ugalshadower: Lots of one to one relationships?13:38
shadowercaching is fine but it'd be a transparent implementation detail kind of thing, rather than a bit of logic that we have to explicitly deal with13:38
shadowerd0ugal: ya13:38
shadowerwhich is what t-h-t does now and will do in the future13:39
d0ugalInteresting.13:39
jdobhrm, ok, that's fair. I'm approaching this very much as the dude writing the service code that needs to call into storage13:39
shadowerd0ugal: in theory, you wouldn't even need to store the 1-1 relationships (if you could live with parsing the templates)13:39
shadowerbut that's where the caching comes in I think13:39
d0ugalshadower: That's the relationship model being added to Glance Artifacts13:39
*** yfaykya has quit IRC13:39
*** pcrews_ has joined #tripleo13:40
shadowerjdob, d0ugal for the record, chances are that I won't be touching either side of that code much (storage and api) so whatever works for you guys is ultimately fine13:40
d0ugalha :)13:40
shadowerjust saying what feels right to me wrt the way heat and t-h-t work13:40
jdobi'd like to take a similar approach, I like the idea of the service saying what it needs and storage figures it out however they want13:41
jdobshadower: still, totally appreciate your input13:41
*** yfaykya has joined #tripleo13:41
jdobyou have a better idea of the templates so it's valuable to hear13:41
d0ugalshadower: yup, thanks!13:41
*** yfaykya has quit IRC13:45
*** yfaykya has joined #tripleo13:47
*** ifarkas has quit IRC13:47
*** e0ne has joined #tripleo13:51
openstackgerritJan Provaznik proposed a change to openstack/tripleo-image-elements: Update openstack services to listen on stunnel connect port  https://review.openstack.org/6137613:51
*** yfaykya has quit IRC13:51
*** yfaykya has joined #tripleo13:53
*** howleyt has joined #tripleo13:55
*** yfaykya has quit IRC13:57
*** yfaykya has joined #tripleo13:59
*** akrivoka has quit IRC14:00
*** beekneemech is now known as bnemec14:02
ccrouchping giulivo14:04
*** yfaykya has quit IRC14:04
*** rwsu has joined #tripleo14:04
*** yfaykya has joined #tripleo14:05
openstackgerritMatthew Macdonald-Wallace proposed a change to openstack/tripleo-image-elements: Install check_mk server  https://review.openstack.org/8722614:06
openstackgerritMatthew Macdonald-Wallace proposed a change to openstack/tripleo-image-elements: Auto-collect local checks  https://review.openstack.org/9132414:09
*** akrivoka has joined #tripleo14:09
*** yfaykya has quit IRC14:10
*** yfaykya has joined #tripleo14:11
*** yfaykya has quit IRC14:16
*** yfaykya has joined #tripleo14:18
*** ramishra has joined #tripleo14:18
*** e0ne has quit IRC14:20
*** yfaykya has quit IRC14:22
*** ci-overcloud has joined #tripleo14:23
ci-overcloud************** ci-overcloud complete status=127 ************14:23
*** ci-overcloud has quit IRC14:23
*** yfaykya has joined #tripleo14:24
*** jcoufal has quit IRC14:24
*** ramishra has quit IRC14:25
*** spiffxp_ has joined #tripleo14:25
*** SlickN1k has joined #tripleo14:26
openstackgerritAlexis Lee proposed a change to openstack-infra/tripleo-ci: Split very long line in toci_devtest.sh  https://review.openstack.org/9742914:27
*** ramishra has joined #tripleo14:27
*** yfaykya has quit IRC14:29
*** jistr has quit IRC14:29
*** rdopiera has quit IRC14:30
*** yfaykya has joined #tripleo14:30
*** akrivoka has quit IRC14:31
*** akrivoka has joined #tripleo14:31
*** jistr has joined #tripleo14:32
*** ramishra has quit IRC14:34
*** yfaykya has quit IRC14:34
*** yfaykya has joined #tripleo14:36
*** yfaykya has quit IRC14:41
*** yfaykya has joined #tripleo14:42
*** yfaykya has quit IRC14:47
*** yfaykya has joined #tripleo14:48
*** yfaykya has quit IRC14:53
*** e0ne has joined #tripleo14:53
*** yfaykya has joined #tripleo14:54
vinshhey.. whats the "key" to get the cluster virtualIP ..14:58
openstackgerrittherese-mchale proposed a change to openstack/tripleo-image-elements: Use Percona Packages for Ubuntu  https://review.openstack.org/9013414:58
vinshsay you  are on the undercloud.. or the tripleo-root.. and want to find the overcloud virtual ip14:58
vinshso that you can point devtest_overcloud at that.. vs the overcloud controller ip14:58
*** yfaykya has quit IRC14:59
*** untriaged-bot has joined #tripleo15:00
untriaged-botUntriaged bugs so far:15:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/132576215:00
*** jistr has quit IRC15:00
uvirtbotLaunchpad bug 1325762 in tripleo "intermittent tftp issues using physical hardware" [Undecided,New]15:00
*** untriaged-bot has quit IRC15:00
*** tzumainn has quit IRC15:00
*** jistr has joined #tripleo15:00
*** yfaykya has joined #tripleo15:00
*** tzumainn has joined #tripleo15:01
*** ci-overcloud has joined #tripleo15:02
ci-overcloud************** ci-overcloud complete status=1 ************15:02
*** ci-overcloud has quit IRC15:02
*** yfaykya has quit IRC15:05
*** yfaykya has joined #tripleo15:07
vinshgreghaynes, related to change 93554... haproxy.. any idea?15:08
vinshhow do we query the virtual IP from the overcloud?15:08
vinshI want to wedge that into devtest_overcloud.. vs the controller IP15:08
openstackgerritDerek Higgins proposed a change to openstack-infra/tripleo-ci: Remove temporary nova revert  https://review.openstack.org/9881615:10
*** yfaykya has quit IRC15:11
*** yfaykya has joined #tripleo15:13
*** jprovazn is now known as jprovazn_afk15:16
*** yfaykya has quit IRC15:17
jprovazn_afkvinsh, this is addressed in this patch: https://review.openstack.org/#/c/94354/1/scripts/devtest_overcloud.sh15:18
openstackgerritAndrea Frittoli  proposed a change to openstack/tripleo-image-elements: Configurable tests2skip file  https://review.openstack.org/9881715:18
*** morganfainberg_Z is now known as morganfainberg15:18
vinshjprovazn_afk, BINGO :) ty.15:19
*** yfaykya has joined #tripleo15:19
jprovazn_afkvinsh, np15:19
openstackgerritAndrea Frittoli  proposed a change to openstack/tripleo-image-elements: Configurable tests2skip file  https://review.openstack.org/9881715:20
*** yfaykya has quit IRC15:23
*** julim has quit IRC15:24
*** yfaykya has joined #tripleo15:25
openstackgerritBen Nemec proposed a change to openstack/tripleo-specs: TripleO on OpenStack  https://review.openstack.org/9264215:28
*** rlandy_ has joined #tripleo15:29
*** yfaykya has quit IRC15:30
*** yfaykya has joined #tripleo15:31
*** rlandy has quit IRC15:32
*** russell_h has quit IRC15:32
*** russell_h has joined #tripleo15:32
openstackgerrittom-howley proposed a change to openstack/tripleo-image-elements: Add keepalive option to rabbitmq config  https://review.openstack.org/8192015:33
*** yfaykya has quit IRC15:36
*** yfaykya has joined #tripleo15:37
openstackgerrittherese-mchale proposed a change to openstack/tripleo-image-elements: Use Percona Packages for Ubuntu  https://review.openstack.org/9013415:40
*** jcoufal has joined #tripleo15:41
*** yfaykya has quit IRC15:42
*** yfaykya has joined #tripleo15:43
*** geerdest has joined #tripleo15:44
*** athomas has quit IRC15:47
*** yfaykya has quit IRC15:48
*** yfaykya has joined #tripleo15:49
*** e0ne has quit IRC15:53
*** athomas has joined #tripleo15:53
*** e0ne has joined #tripleo15:54
*** yfaykya has quit IRC15:55
*** yfaykya has joined #tripleo15:56
*** matty_dubs is now known as matty_dubs|lunch15:57
*** e0ne has quit IRC15:58
*** andreaf has quit IRC16:00
*** yfaykya has quit IRC16:00
*** yfaykya has joined #tripleo16:02
*** IvanBerezovskiy has left #tripleo16:04
*** yfaykya has quit IRC16:06
*** yfaykya has joined #tripleo16:08
*** yamahata has joined #tripleo16:10
*** yfaykya has quit IRC16:12
*** yfaykya has joined #tripleo16:14
*** eghobo has joined #tripleo16:17
*** yfaykya has quit IRC16:19
*** yfaykya has joined #tripleo16:20
*** yamahata has quit IRC16:22
*** yamahata has joined #tripleo16:22
*** ifarkas has joined #tripleo16:24
vinshInteresting.. that the keystone url returned by heat output-show... isn't using the virtualIP in my case16:24
*** yfaykya has quit IRC16:24
*** sballe has joined #tripleo16:25
*** spzala has joined #tripleo16:25
*** pblaho has quit IRC16:26
*** yfaykya has joined #tripleo16:26
vinshoh neat.. I see where heat output-show comes from now. right on.16:27
vinsh"Outputs:"16:27
*** akuznetsov has quit IRC16:31
*** yfaykya has quit IRC16:31
*** yfaykya has joined #tripleo16:32
*** yamahata has quit IRC16:35
*** yamahata has joined #tripleo16:35
*** yfaykya has quit IRC16:37
*** yfaykya has joined #tripleo16:38
*** akuznetsov has joined #tripleo16:39
*** lucasagomes has quit IRC16:39
*** lucasagomes has joined #tripleo16:39
*** jcoufal has quit IRC16:40
*** rcarrill` has joined #tripleo16:42
*** akuznetsov has quit IRC16:42
lifelesso/16:42
lifelessvinsh: hi16:43
lifelessvinsh: I missed you before, haproxy change?16:43
vinshHey there16:43
*** yfaykya has quit IRC16:43
vinshI think things make good enough sense here.. just that my output from heat output-show for the keystone url.. does not match what I expect16:44
*** rcarrillocruz has quit IRC16:44
vinshwhere for example that same variable "ControllerVirtualIP" when referenced by the haproxy element does infact lay down the virtual ip16:44
*** jistr has quit IRC16:44
vinshthat same output from heat output-show though.. shows it to be the controller ip16:44
*** yfaykya has joined #tripleo16:45
vinshNot sure why.. as in the overcloud-source heat template.. they are called the same way for both items.16:45
lifelesshmm could be a bug in the template16:45
*** matty_dubs|lunch is now known as matty_dubs16:46
vinshalso.. the net_binds changes to haproxy can be a nightmare :)16:46
vinshreally.. you use net_binds or not16:46
vinshthere is not clean way to override like the readme claims16:46
derekh_lifeless: I managed to get the heat stack to go into complete state, but br-untagged doesn't seem to be getting an IP address, I've added some details to the bottom of the etherpad16:47
vinshThough, I am doing evil things here.. with controller nodes that are not homogeneous  (hosting different api services on each one)16:47
vinshso, no net_binds for me.16:48
derekh_lifeless: the undercloud doesn't seem to be seeing dhcp request from the ci-overcloud , I havn't been able to figure out why16:48
lifelessvinsh: I haven't read the recent haproxy changes16:49
*** yfaykya has quit IRC16:49
vinshbetter for your sanity :)16:49
lifelessvinsh: we should still totally be able to do haproxy->other-node balancing16:49
vinshoh I am16:49
lifelessvinsh: if we can't, thats a huge bug16:50
vinshwithout use of net_binds16:50
vinshnet_binds causes mustache overload16:50
vinshive got haproxy on control node, sending traffic to another control node that hosts ceilometer. and also sending traffic to swift proxy nodes16:51
*** yfaykya has joined #tripleo16:51
vinshwhich are spit out.16:51
vinshJust futzing with this keystone url.. which isn't right yet.16:51
*** spzala has quit IRC16:54
*** yfaykya has quit IRC16:55
*** yfaykya has joined #tripleo16:57
*** derekh_ has quit IRC16:59
*** yfaykya has quit IRC17:02
*** yfaykya has joined #tripleo17:03
*** akrivoka has quit IRC17:03
*** yfaykya has quit IRC17:07
*** marun has joined #tripleo17:07
*** yfaykya has joined #tripleo17:09
*** marun has quit IRC17:11
*** yfaykya has quit IRC17:14
openstackgerritA change was merged to openstack/tripleo-image-elements: indent using 4 spaces (1/3)  https://review.openstack.org/9320617:14
*** yfaykya has joined #tripleo17:15
*** yfaykya has quit IRC17:20
*** yfaykya has joined #tripleo17:21
*** yfaykya has quit IRC17:26
lifelessmordred: while you are around: https://review.openstack.org/#/c/97583/17:27
*** eghobo has quit IRC17:27
*** nati_ueno has joined #tripleo17:27
*** yfaykya has joined #tripleo17:27
*** spzala has joined #tripleo17:29
*** yfaykya has quit IRC17:32
*** yfaykya has joined #tripleo17:33
*** russell_h has quit IRC17:36
lifelessyayayaya the nova revert landed17:38
*** yfaykya has quit IRC17:38
*** russell_h has joined #tripleo17:39
*** ramishra has joined #tripleo17:39
*** yfaykya has joined #tripleo17:40
*** rcarrill` has quit IRC17:40
*** rcarrillocruz has joined #tripleo17:41
*** russell_h has quit IRC17:41
*** russell_h has joined #tripleo17:41
*** lucasagomes is now known as lucas-afk17:44
*** yfaykya has quit IRC17:44
*** eghobo has joined #tripleo17:45
*** yfaykya has joined #tripleo17:46
*** ramishra has quit IRC17:48
*** ramishra has joined #tripleo17:49
*** yfaykya has quit IRC17:50
*** yfaykya has joined #tripleo17:52
*** ramishra has quit IRC17:53
*** yfaykya has quit IRC17:57
*** yfaykya has joined #tripleo17:58
*** saurabhs has joined #tripleo17:59
*** ramishra has joined #tripleo18:00
*** jang has joined #tripleo18:01
*** e0ne has joined #tripleo18:01
*** yfaykya has quit IRC18:03
*** yfaykya has joined #tripleo18:04
*** ramishra has quit IRC18:05
openstackgerritA change was merged to openstack/tripleo-heat-templates: Replace nagios3 template with HOT rewrite  https://review.openstack.org/9066518:06
*** e0ne has quit IRC18:07
*** eguz has joined #tripleo18:07
*** eguz has quit IRC18:07
*** eguz has joined #tripleo18:07
*** yfaykya has quit IRC18:09
*** yfaykya has joined #tripleo18:10
*** eghobo has quit IRC18:11
*** yfaykya has quit IRC18:15
*** yfaykya has joined #tripleo18:16
*** athomas has quit IRC18:16
openstackgerritGhe Rivero proposed a change to openstack-infra/tripleo-ci: Add /mnt/state/var/log to the host state files  https://review.openstack.org/9362718:19
*** yfaykya has quit IRC18:21
*** yfaykya has joined #tripleo18:22
*** pelix has quit IRC18:26
*** e0ne has joined #tripleo18:26
*** yfaykya has quit IRC18:27
*** yfaykya has joined #tripleo18:29
*** nati_ueno has quit IRC18:31
*** yfaykya has quit IRC18:33
*** nati_uen_ has joined #tripleo18:34
*** yfaykya has joined #tripleo18:35
*** yfaykya has quit IRC18:39
*** e0ne has quit IRC18:39
*** e0ne has joined #tripleo18:40
*** yfaykya has joined #tripleo18:41
*** yfaykya has quit IRC18:45
*** yfaykya has joined #tripleo18:47
*** spzala has quit IRC18:48
*** yfaykya has quit IRC18:52
*** yfaykya has joined #tripleo18:53
*** weshay has quit IRC18:54
*** lucas-afk is now known as lucasagomes18:56
*** yfaykya has quit IRC18:58
*** yfaykya has joined #tripleo18:59
*** nati_ueno has joined #tripleo19:02
*** yfaykya has quit IRC19:03
*** yfaykya has joined #tripleo19:05
*** nati_uen_ has quit IRC19:06
*** rob_s_parker has quit IRC19:07
*** weshay has joined #tripleo19:09
*** rcarrill` has joined #tripleo19:10
*** yfaykya has quit IRC19:10
*** yfaykya has joined #tripleo19:11
*** rcarrillocruz has quit IRC19:12
*** jang1 has joined #tripleo19:13
*** spzala has joined #tripleo19:15
*** nati_ueno has quit IRC19:15
*** dprince has quit IRC19:15
*** yfaykya has quit IRC19:16
*** yfaykya has joined #tripleo19:18
jogolifeless: wow http://logs.openstack.org/08/96308/8/check-tripleo/check-tripleo-undercloud-precise/a879410/ logs are super sparse19:20
jogonova logs are useless19:20
greghaynesI think thats a somewhat recent change - just noticed when the nova bm scheduler issue came up that I didnt have tracebacks without turning debugging on19:21
greghaynesdefinitely wasnt the case a while ago19:21
lifelessjogo: yes, and this is why we run default, so you well, know.19:22
jogogreghaynes: hrm, I think sdague looked into that recently19:22
*** yfaykya has quit IRC19:22
jogolifeless: I added this to the list to investigate this19:23
*** yfaykya has joined #tripleo19:24
*** chuckC has quit IRC19:24
*** akuznetsov has joined #tripleo19:25
*** chuckC has joined #tripleo19:25
*** chuckC has quit IRC19:25
*** akuznetsov has quit IRC19:25
greghaynesdshulyak: jprovazn_afk: Any idea if https://review.openstack.org/#/c/92550/ is planning on being resurrected? Seems like it has some pretty bad conflicts going on...19:26
*** akuznetsov has joined #tripleo19:26
*** akuznetsov has quit IRC19:27
*** e0ne has quit IRC19:27
*** akuznetsov has joined #tripleo19:28
*** yfaykya has quit IRC19:28
*** yfaykya has joined #tripleo19:30
*** yfaykya has quit IRC19:34
*** e0ne has joined #tripleo19:35
*** yfaykya has joined #tripleo19:36
*** yfaykya has quit IRC19:41
*** yfaykya has joined #tripleo19:42
vinshlifeless, dunno what the deal was.. but on another run.. heat output-show gave me the correct ip for keystone (virtualip).19:46
*** yfaykya has quit IRC19:46
vinshMaybe a one-of failure. maybe not.19:46
greghaynesvinsh: Youre trying to run the ha overcloud patch stack?19:46
vinshIve got most of it yeah19:46
greghayneswhats the patches youre using?19:47
vinshonly am working on it in the split out of the CE context.19:47
greghaynesah19:47
vinshI'm using the latest haproxy patches from dimitri19:47
greghayneswhat about the one I asked about above?19:48
greghaynesIm getting the feeling that is starting to be superceded by some smaller patches19:48
vinshah, havn't hit that yet19:48
*** yfaykya has joined #tripleo19:48
vinshYeah, issue I have now.. is that haproxy binds to say.. glance api-port19:48
greghaynesyep19:48
vinshon virtualip:port19:48
*** noslzzp has quit IRC19:48
vinshso.. I think that might prevent glance from binding on anything:port19:49
vinshfor example.19:49
*** noslzzp has joined #tripleo19:49
greghaynesyep, the services need specific ip:port binds19:49
greghaynesas well19:49
vinshLooks like the story is to use stunnel for that issue19:50
vinshright?19:50
vinshOR modify all of the elements to change their ports. (extra work)19:50
greghaynesvinsh: Have you read the spec on this/19:51
vinshI have it up now19:51
greghaynesso services need to bind on localhost only19:51
greghaynesand then stunnel does the vip <-> localhost mapping19:51
greghayneser not vip19:51
greghaynes node <-> localhost19:51
vinshOK, got that19:52
*** yfaykya has quit IRC19:53
vinshwhere do the ports come from to fill in 92550?19:53
vinshThe ports I see in overcloud-source .. are for haproxy to bind to.. (standard ports)19:54
vinshfor 92550 we expect all service to bind to localhost only19:54
vinshi need to read that more.19:54
*** yfaykya has joined #tripleo19:54
greghaynesoh derp, ty for pointing that out19:54
greghaynesso that patch should probably be abandonded19:54
vinshmaybe not though..19:55
vinsh{{#nova.ec2.port}}3119:55
vinshec2_listen_port={{.}}3219:55
vinsh{{/nova.ec2.port}}19:55
*** marun has joined #tripleo19:55
vinshoh ya.19:55
openstackgerritJay Dobies proposed a change to openstack/tripleo-specs: Spec for Tuskar Juno REST API changes  https://review.openstack.org/9472019:55
vinshI was thinking IP19:55
lifelessyup19:56
vinshso what next?19:57
lifelesshaproxy is endpoint <-> node; stunnel is node<->localhost19:57
lifelesshaproxy should know all nodes19:57
lifelessstunnel only knows localhost19:57
lifelessservices only bind to localhost19:57
*** lucasagomes is now known as lucas-dinner19:57
greghaynesAFAICT were lacking a bing all the things to localhost patch19:57
vinshdoes a service bound to localhost:port prevent haproxy from binding to virtualip:sameport ?19:58
greghayness/bing/bind19:58
greghaynesno19:58
vinshk19:58
lifelesshaproxy should be binding to *:sameport19:58
*** yfaykya has quit IRC19:58
vinshthat would bind it to virtual ip.. and node ip19:59
vinshand local host?19:59
lifelessit binds it to *20:00
lifelesswhich is any incoming traffic without a more specific bind, IIRC.20:00
*** yfaykya has joined #tripleo20:00
greghaynesyea, was just looking up to confirm that, but I thought the same20:00
greghaynesthat specific binds override wildcard binds20:01
vinshfor the stunnel pipe/piece here.. is that the open review by Stuart? or do you guys have another WIP?20:01
vinshhttps://blueprints.launchpad.net/tripleo/+spec/tripleo-ssl-overcloud20:02
greghayneshttps://review.openstack.org/#/c/9566320:02
greghaynesis important20:02
vinshFrom what I understand, once a patch comes in to bind services to localhost.  the stunnel/ssl stuff is needed as well20:02
vinshOIC20:03
lifelessgreghaynes: speaking of important, where is the is-bootstrap-host tht patch at ?20:03
*** julim has joined #tripleo20:03
greghayneshttps://review.openstack.org/#/c/86435/20:03
greghayneslooks like about to merge once ci checks20:03
greghaynesonly 25 revisions later!20:04
vinshmust be bullet proof now 0_o20:04
greghaynesoh, I need to do some rebasing20:04
greghaynesof things depending on it20:04
*** spzala has quit IRC20:05
*** yfaykya has quit IRC20:05
vinshWould it be helpful for the ha-story if I were to generate a review for moving element api's to bind to localhost?20:05
vinshor is that duplicate effort?20:05
*** morganfainberg is now known as morganfainberg_Z20:06
vinshSuppose I will do it to test my local conifg anywho20:06
greghaynessounds like plan is to bind them in any addr20:06
greghayneswhich would be nice if possible - more defaults in configs ftw20:06
*** yfaykya has joined #tripleo20:06
vinshlifeless, was saying "services only bind to localhost"20:07
greghaynesoh, got that flipped20:07
* greghaynes rereads spec20:07
greghayneshrm, spec says we bind all the things on specific ip's.20:08
*** boris-42 has quit IRC20:08
vinshthe spec contradicts what we said here.20:11
openstackgerritGregory Haynes proposed a change to openstack/tripleo-image-elements: Extract mysql reset files from reset-db  https://review.openstack.org/9310420:11
openstackgerritGregory Haynes proposed a change to openstack/tripleo-image-elements: Add os-is-bootstrap-host element and script  https://review.openstack.org/8643520:11
vinshhas haproxy only bound to a vip, then it sends traffic out to node ip20:11
*** yfaykya has quit IRC20:11
vinshstunnel listens only on nodeip and sends to localhost20:11
vinshservices only listen on localhost20:11
vinshno haproxy bound to *.  (i'm probably being thick here)20:12
*** boris-42 has joined #tripleo20:12
vinshTime to test and see! :)20:12
jprovazn_afkgreghaynes, I think ports customization is not priority ATM (https://review.openstack.org/#/c/92550/)20:12
*** yfaykya has joined #tripleo20:13
greghaynesyea, looks like that just isnt needed at all now with the spec we went with?20:13
*** chuckC has joined #tripleo20:14
devanandaya'll may want to update the status of https://bugs.launchpad.net/tripleo/+bug/1326289 now that the fix/revert has landed in nova20:14
uvirtbotLaunchpad bug 1326289 in nova "Failing to launch instances : Filter ComputeCapabilitiesFilter returned 0 hosts" [Critical,Fix committed]20:14
openstackgerritGregory Haynes proposed a change to openstack/tripleo-image-elements: Improve percona cluster initialization  https://review.openstack.org/9303220:15
jprovazn_afkgreghaynes, yes20:15
*** jcoufal has joined #tripleo20:15
greghaynesjprovazn_afk: also, do you have any idea if theres a patch up for making services bind on localhost:20:17
vinshOr a patch to add stunnel to overcloud-source.yaml?20:17
jprovazn_afkgreghaynes, sure, that should be one of mine :)20:17
*** yfaykya has quit IRC20:17
jprovazn_afkhttps://review.openstack.org/#/c/61376/20:18
greghayneswow, thats an old one20:18
vinshI wish all of these were linked off the ha blue print :)20:18
jprovazn_afkvinsh, there is not stunnel patch for overcloud template20:18
*** akuznetsov has quit IRC20:18
*** yfaykya has joined #tripleo20:19
jprovazn_afkvinsh, right, I'll update commit messages tomorrow20:19
vinshOk. I get it.. stunnel is modified to talk nodeip-localhost on any needed port. so no config is needed to add stunnel?20:19
vinshoh. the stunnel element expects heat meatadata...20:20
vinshso?20:20
greghaynessounds like unfinished work :)20:21
greghaynesI also dont see a patch for that20:21
jprovazn_afkvinsh, there will be required a change to stunnel element (and/or overcloud template) to bind stunnel to the specified ports20:21
vinshOk, I am on the same page, understood.20:21
greghaynesI think https://review.openstack.org/#/c/95663 is all the needed element change20:21
greghaynes(also should link to blueprint)20:22
vinshsome greg guy -1'ed it though.20:22
jprovazn_afkwhich is different patch than setting binding addr for os services ;)20:22
greghaynesvinsh: yea, that jerk20:22
* greghaynes debates just editing commit msgs to add bp for these patches20:22
vinshDOIT DOIT DOIT20:23
*** yfaykya has quit IRC20:23
*** jdob has quit IRC20:25
*** jdob has joined #tripleo20:25
*** yfaykya has joined #tripleo20:25
jprovazn_afkFWIW, with pending patches, you should be able to run 3 control nodes overcloud and start instances in it20:25
greghaynes\O/20:25
jprovazn_afk(w/o stunnel ;)20:25
jprovazn_afkjust haproxy <-> service which worked for me20:26
jprovazn_afkbut still bunch of work remains :/20:26
SpamapSjprovazn_afk: are these 3 nodes partition tolerant?20:26
jprovazn_afkSpamapS, partition tolerant?20:27
lifelessjprovazn_afk: if there is a network break and we end up with two LANs, one with 2 nodes and one with 1, what happens20:27
lifelessjprovazn_afk: btw for someone afk, you're very lively20:28
*** jprovazn_afk is now known as jprovazn20:28
lifeless:)20:28
jprovaznSpamapS, lifeless: I did not get to further testing yet, just at the point of being able to start an instance in overcloud20:28
lifelessjprovazn: still, thats awesome20:29
lifelessSpamapS: I believe someone in HP said that keepalived doesn't have quorum support20:29
*** yfaykya has quit IRC20:29
lifelessSpamapS: and that pacemaker or corosync (I forget which and its not paged in) is needed in concert to deliver that20:29
hewbroccapacemaker + corosync are next on our agenda :)20:30
lifelesshewbrocca: cool20:31
*** yfaykya has joined #tripleo20:31
greghayneshrm, keepalived.conf does have a quorum field... no idea what tha does though :p20:31
greghaynesoh, looks like you set a min weight20:31
greghayneser, min total weight20:31
hewbroccaFabio Di Nitto would never forgive me if I didn't make sure there was at least a pacemker *option* in tripleo20:31
SpamapSSo without those, I believe we'll just have a loss-of-quorum failure on Galera20:31
SpamapSIIRC keepalived does not have any way to detect split brain.20:32
SpamapSWhich is perfectly fine for haproy20:32
SpamapShaproxy20:32
SpamapSwe don't have any state to preserve there20:33
greghaynesthis quorum field looks like it does that as long as you set it high enough20:33
greghaynesif you have 3 nodes which can each provide a weight of 5, set quorum to 1020:33
greghaynesyou need a total weight of 10 do do anything20:33
*** julim has quit IRC20:34
SpamapSanyway, We still have unresolved issues with leader election and failure detection, I'm sure.20:34
lifelessok so we just set that to floor(node_len/2 + 1) and weight one ?20:34
greghaynesyerp20:34
SpamapSbut just being able to spint it up is a nice accomplishment20:34
greghayneslifeless: Thats how the doc reads20:34
jprovaznregarding pacemaker...20:35
*** yfaykya has quit IRC20:35
jprovaznwe will need it anyway because of ceilometer which was merged recently20:36
jprovaznand ceilometer central agent *might be* A/A in Juno, there is a bp for it, but I' mscpetic it might be in near future20:37
*** yfaykya has joined #tripleo20:37
greghaynesjprovazn: for the ceilometer bit - how do we pick which node we run that on at the moment?20:38
jprovazngreghaynes, pacemaker does it for us, I suppose20:39
*** jcoufal has quit IRC20:39
*** nati_ueno has joined #tripleo20:40
jprovaznand with pacemaker it's more fun as fedora switched from crm to pcs (management tool), ubuntu/debian sticks (so far) with crm20:40
greghaynes\O/20:40
greghaynesjprovazn: what do you think of making fedora use mariadb by deafult?20:41
greghaynessomething that came up at summit - were not actually testing elements like that one which you have to opt in to20:42
*** yfaykya has quit IRC20:42
greghaynesand sounds like crm / pcs will be yet another of this20:42
jprovazngreghaynes, that makes sense, I was waiting for mariadb to be pulled into fedora repos20:42
greghaynesah, ok20:42
*** yfaykya has joined #tripleo20:43
jprovazngreghaynes, maybe with pcs/crm, it would worth try to use only one of them on all platforms if possible (=available on all platforms)20:44
openstackgerritHaneef Ali proposed a change to openstack/tripleo-image-elements: Enable token-cleanup for debian  https://review.openstack.org/9888720:45
greghaynesah, that would work too20:45
greghaynesIm not very experienced with those tools so id have to research some, but anything to keep from adding elements we dont CI :)20:46
*** ifarkas has quit IRC20:46
jprovazngreghaynes, +!20:46
jprovazn+1 actually20:46
greghaynes-> noms20:47
vinshbtw, in my multi control stack.. I have a separate node that runs only ceilometer.. (is build with that specifically)  but shares the database on other controlnodes20:47
*** yfaykya has quit IRC20:47
vinshits a static version of leader election. :-/20:48
greghaynesyea, IMO we should just use pacemaker or $leader_election to pick a node at runtime it runs on if it really cant do multinode20:48
*** yfaykya has joined #tripleo20:49
vinshhi-tech. one day.20:50
*** e0ne has quit IRC20:53
*** yfaykya has quit IRC20:54
*** yfaykya has joined #tripleo20:55
*** jprovazn has quit IRC20:57
openstackgerritGhe Rivero proposed a change to openstack/diskimage-builder: Use common element get-image-kernel  https://review.openstack.org/9875820:57
openstackgerritGhe Rivero proposed a change to openstack/diskimage-builder: Improve kernel detection in debian family systems  https://review.openstack.org/9875920:57
openstackgerritGhe Rivero proposed a change to openstack/diskimage-builder: Add get-kernel-image element  https://review.openstack.org/9875720:57
openstackgerritAdam Gandelman proposed a change to openstack/tripleo-incubator: Run apt-get update before installing UCA keyring  https://review.openstack.org/9889620:59
*** jdob has quit IRC20:59
*** yfaykya has quit IRC21:00
*** untriaged-bot has joined #tripleo21:00
untriaged-botUntriaged bugs so far:21:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/132826121:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/132576221:00
*** untriaged-bot has quit IRC21:00
uvirtbotLaunchpad bug 1328261 in tripleo "Keystone token cleanup is not enabled for debian" [Undecided,In progress]21:00
uvirtbotLaunchpad bug 1325762 in tripleo "intermittent tftp issues using physical hardware" [Undecided,New]21:00
*** yfaykya has joined #tripleo21:02
*** giulivo has quit IRC21:03
*** matty_dubs is now known as matty_dubs|gone21:05
*** yfaykya has quit IRC21:06
*** yfaykya has joined #tripleo21:08
*** morganfainberg_Z is now known as morganfainberg21:10
*** yfaykya has quit IRC21:12
*** yfaykya has joined #tripleo21:14
greghaynesvinsh: have you been running into issues with the apache2 ports.conf?21:15
*** yfaykya has quit IRC21:18
*** yfaykya has joined #tripleo21:20
*** jtomasek has quit IRC21:24
*** yfaykya has quit IRC21:25
*** yfaykya has joined #tripleo21:26
*** yfaykya has quit IRC21:31
*** yfaykya has joined #tripleo21:32
*** yfaykya has quit IRC21:37
*** yfaykya has joined #tripleo21:38
*** yfaykya has quit IRC21:43
*** yfaykya has joined #tripleo21:44
*** hewbrocca has quit IRC21:48
*** yfaykya has quit IRC21:49
*** yfaykya has joined #tripleo21:51
*** yfaykya has quit IRC21:55
*** andreaf has joined #tripleo21:57
*** yfaykya has joined #tripleo21:57
*** rlandy_ has quit IRC21:59
*** eguz has quit IRC22:01
*** yfaykya has quit IRC22:01
*** eghobo has joined #tripleo22:02
*** yfaykya has joined #tripleo22:03
*** jang1 has quit IRC22:06
*** yfaykya has quit IRC22:07
*** nati_ueno has quit IRC22:08
*** yfaykya has joined #tripleo22:09
vinshgreghaynes, where do those issues manifest?22:09
greghaynesin os-r-c, it seems like stunnel is defined in the current heat templates but stunnel.connect-ip is not. This results in most of the services having odd configurations22:10
vinshI need to pull in those template changes as well as, 95663 . and 6137622:12
vinshHave not seen that case yet.. bet I will soon!22:12
*** yfaykya has quit IRC22:13
*** yfaykya has joined #tripleo22:15
greghaynesI wonder if jprovazn is just running without the ssl tempalte22:17
*** yfaykya has quit IRC22:19
*** yfaykya has joined #tripleo22:21
vinshwonder how ports are configured in that case?22:23
greghayneshrm?22:23
vinshgreghaynes, which template are you using.. just to be sure?22:23
lifelesstemplates ;)22:23
vinshhah.. same :)22:23
greghaynesvinsh: https://review.openstack.org/#/c/93554/22:24
greghaynesrebased off master22:24
vinshah yeah. I know this one all to well.22:25
vinshI need to rebase off master though.22:25
*** yfaykya has quit IRC22:26
vinshI take it your next step is stubbing out those stunnel connect to ports?22:26
greghaynesI also left a comment on https://review.openstack.org/#/c/61376/ with my issue22:27
vinshack.22:27
greghaynesuhm, Id like to ensure that the fallback (no stunnel) behavior isnt horribly broken first22:27
vinshpshh22:27
*** yfaykya has joined #tripleo22:27
greghaynesbecause all signs are pointing to yes on that one22:27
*** michchap has quit IRC22:28
vinshIsn't this sort of an all or nothing ?22:29
vinshyou want haproxy using the spec.. you need the fleshed out template and the stunnel patches22:29
*** weshay has quit IRC22:29
lifelesswe need to get there from here22:30
vinshWhat I mean is: what is the flow /use case WITHOUT stunnel?22:30
lifelessno SSL22:30
lifelessthere is a patch to make throwaway SSL certs for CI22:30
lifelesswe need that to do SSL always22:30
lifelessand these patches need to land one at a time22:30
greghaynesThe templates have a {#stunnel}stuff with stunnel.connect_ip{^stunnel}other stuff{/stunnel}. The problem is AFAICT we have stunnel defined in ther overcloud templates, but no stunnel.connect_ip22:31
*** cody-somerville has joined #tripleo22:31
*** cody-somerville has joined #tripleo22:31
*** yfaykya has quit IRC22:31
greghaynesso we need to either fix the configs to check for stunnel.connect_ip or to have a way to undefine stunnel all together22:32
*** yfaykya has joined #tripleo22:33
vinshwithout stunnel its haproxy bound to * <--> service bound to node IP right?22:34
greghayneshaproxy bound to vip, service bound to node_ip22:34
vinshk.22:34
*** rainya has quit IRC22:37
*** cody-somerville has quit IRC22:38
*** yfaykya has quit IRC22:38
vinshHrmm..  [ALERT] 159/223813 (27312) : Starting proxy glance_api: cannot bind socket22:39
*** yfaykya has joined #tripleo22:40
vinshlisten glance_api22:40
vinsh  bind 192.0.2.33:929222:40
vinsh  mode tcp22:40
vinsh  balance source22:40
vinsh  option tcpka22:40
vinsh  option tcplog22:40
vinsh  server controller0 192.0.2.23:9292 check inter 2000 rise 2 fall 522:40
vinshhaproxy bound to the virtualip here ^^^22:40
vinshglance bound to the node ip22:40
vinshhaproxy unable to bind due to that22:40
vinshI'm missing something.22:40
lifelessnetstat -anp ?22:43
*** rainya has joined #tripleo22:43
vinsh netstat -anp | grep 929222:44
vinshtcp        0      0 0.0.0.0:9292            0.0.0.0:*               LISTEN      7482/python22:44
vinsh?22:44
*** yfaykya has quit IRC22:44
vinshah. is this due to 0.0.0.0 vs specific ip for the service bind?22:44
TheJuliait is22:44
TheJuliaI'm fairly sure you can't have both at the same time on the same system22:44
lifelessso our plan may need to have haproxy bound to the vip and restart it on vip failovers22:45
lifelessbut thats doable22:45
SpamapSCorrect, 0.0.0.0 pwns all22:45
TheJuliaheh22:45
*** yfaykya has joined #tripleo22:46
*** sballe_ has joined #tripleo22:46
*** nati_ueno has joined #tripleo22:47
lifelessrighto22:47
vinshlifeless, in my case. haproxy IS bound to the vip. its the face that the service is bound to 0.0.0.0 that sucks.22:47
lifelesswe need to update ze spec then22:47
*** sballe has quit IRC22:47
lifelessvinsh: yes, the service needs fixing22:47
vinsh+1 I follow.22:47
lifelessso I think a first set of patches is bind to node IP for the services22:47
lifelessthat gets haproxy in22:47
greghayneslifeless: The spec does say everything binds to a specific ip, nothing to anyaddr22:48
lifelessit will break our current public ip support temporarily22:48
lifelessgreghaynes: does it? cool.22:48
lifelessgreghaynes: not sure you can bind to a vip you don't own though22:48
greghaynesyou can :)22:48
vinshyou can22:48
greghaynestheres a patch for that too, somewhere22:49
vinshhttps://review.openstack.org/#/c/93121/ its merged22:49
greghayneslifeless: https://review.openstack.org/#/c/61376/ is the services bind to node_ip patch youre thinking of22:49
vinshjust add that sauce to your template "net.blah.."22:49
*** yfaykya has quit IRC22:50
*** vinsh is now known as vinsh_away22:51
greghaynesSpamapS: Curious what you think of my comment on ^ (looks like you added stunnel and +2'd that patch)22:51
*** yfaykya has joined #tripleo22:52
SpamapSgreghaynes: makes sense. We can probably play some games with conditional blocks to be able to operate fine without connect_ip.22:53
greghaynessweet22:54
SpamapSfine in that, we can go on living without haproxy working right too ;)22:55
*** yfaykya has quit IRC22:56
*** yfaykya has joined #tripleo22:58
*** yfaykya has quit IRC23:02
*** yfaykya has joined #tripleo23:04
*** yfaykya has quit IRC23:08
*** yfaykya has joined #tripleo23:10
*** yfaykya has quit IRC23:15
*** yfaykya has joined #tripleo23:16
*** yfaykya has quit IRC23:21
*** nati_ueno has quit IRC23:22
*** yfaykya has joined #tripleo23:22
*** morganfainberg is now known as morganfainberg_Z23:25
*** lucas-dinner has quit IRC23:26
*** geerdest has quit IRC23:27
*** yfaykya has quit IRC23:27
*** nati_ueno has joined #tripleo23:28
*** yfaykya has joined #tripleo23:28
*** nati_uen_ has joined #tripleo23:31
*** nati_ueno has quit IRC23:32
*** yfaykya has quit IRC23:33
*** yfaykya has joined #tripleo23:35
*** xuhaiwei has joined #tripleo23:37
*** yfaykya has quit IRC23:40
*** yfaykya has joined #tripleo23:41
*** morganfainberg_Z is now known as morganfainberg23:42
*** yfaykya has quit IRC23:45
StevenKlifeless, tchaypo: Looks like we have a winner for the sprint dates: July 21-2523:46
StevenKWhich makes me sad, since I'll miss Sarah's birthday23:47
*** yfaykya has joined #tripleo23:47
greghaynesoh, really?23:47
greghaynes:/23:47
*** eguz has joined #tripleo23:49
*** yfaykya has quit IRC23:51
StevenKgreghaynes: http://lists.openstack.org/pipermail/openstack-dev/2014-June/036974.html23:52
*** eghobo has quit IRC23:53
*** yfaykya has joined #tripleo23:53
SpamapSI may miss the whole thing entirely23:53
greghaynes:c23:54
SpamapSJuly is just bong for me.23:54
SpamapSAnd that first week of August is when all three of my kids are starting brand new schools in a span of 3 days.23:54
StevenKI told Sarah the dates before she ran out the door, and she's like "So, situation normal, then."23:54
SpamapSwe need to plan these further out23:54
greghaynesSpamapS: ++23:55
SpamapSI'd rather have it the week before feature freeze than only have 6 weeks to plan.23:55
StevenKSo a session at the summit about sprint dates? :-)23:55
SpamapSStevenK: and a session after that where we invite Captain Obvious to detail his plans for curing Polio.23:55
SpamapSI think a simple ML thread pleading with people "Please plan these 2 months in advance at least."23:56
pleia2StevenK: I enjoy the nice jewelry and delivered flowers that come from the husband being out to down for $important_day23:56
pleia2(fortunately husband birthday is near xmas, and no one schedules stuff around then, woo)23:57
SpamapShaha.. yeah the summit was right up against mother's day.. I was there for mother's day, but had to leave at 14:00 .. so there were extra flowers in the morning. ;)23:57
StevenKYeah, I flew out on Mother's Day, so my mother wasn't really pleased23:57
*** yfaykya has quit IRC23:57
pleia2yeah, I was planning on flying on mother's day, so I sent her jewelry in advance23:57
StevenKpleia2: Hmmmmm, flowers won't work for Sarah, but jewelry you say ... :-)23:58
pleia2StevenK: http://www.boutiqueacademia.com/ go to town :)23:59
StevenKjogo: "Hacking 0.9.1 has just been released (hacking 0.9.1 had a minor bug)."23:59
SpamapSbeautiful23:59
*** yfaykya has joined #tripleo23:59
StevenKpleia2: Oooooh23:59
SpamapS"Hacking 0.9.forklobster has been released to address the bugs in hacking 0.9.1 and hacking 0.9.1."23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!