Thursday, 2013-12-12

*** vipul-away is now known as vipul00:00
*** krotscheck has joined #tripleo00:09
lifelessSpamapS: all mails sent :)00:10
SpamapSlifeless: sweet00:14
SpamapSlifeless: left a comment on one of your rebuild reviews00:14
SpamapSlifeless: want me to take a crack at tests/commentary for 60285 ?00:15
SpamapSlifeless: also remember that Heat actually already does have rebuild support.00:16
lifelessSpamapS: doesn't have the 'ephemeral' bit set for it00:19
lifelessSpamapS: does it?00:19
lifelessit's an active opt-in behaviour00:19
SpamapSlifeless: oh right we'd have to add that bit.00:21
SpamapSlifeless: very very tiny change.00:21
lifelessyes00:22
SpamapSbasically just a property we set to true.00:22
lifelessyes00:22
lifelessSpamapS: and yes, crack on on the patch00:22
*** cd-undercloud has joined #tripleo00:22
cd-undercloud************** overcloud complete status=0 ************00:22
*** cd-undercloud has quit IRC00:22
*** AaronGr is now known as AaronGr_afk00:28
*** vipul is now known as vipul-away00:39
openstackgerritClint "SpamapS" Byrum proposed a change to openstack/tripleo-incubator: Make refresh-env more resilient for users  https://review.openstack.org/6161700:39
*** sdake_ has quit IRC00:49
*** morazi has quit IRC00:59
*** vipul-away is now known as vipul01:10
*** jcooley_ has quit IRC01:12
*** jergerber has quit IRC01:13
*** cd-undercloud has joined #tripleo01:15
cd-undercloud************** overcloud complete status=42 ************01:15
*** cd-undercloud has quit IRC01:15
*** jcooley_ has joined #tripleo01:33
*** nosnos has joined #tripleo01:35
*** tserong has quit IRC01:42
*** csd has quit IRC01:48
*** csd has joined #tripleo01:50
*** rongze has joined #tripleo01:52
*** krotscheck has quit IRC01:58
*** csd has quit IRC02:05
*** rcarrillocruz has joined #tripleo02:06
*** tserong has joined #tripleo02:07
*** cd-undercloud has joined #tripleo02:09
cd-undercloud************** overcloud complete status=1 ************02:09
*** cd-undercloud has quit IRC02:09
*** rcarrillocruz has quit IRC02:11
*** arata has joined #tripleo02:19
*** jcooley_ has quit IRC02:24
*** rongze has quit IRC02:28
*** rongze has joined #tripleo02:29
*** vkozhukalov has joined #tripleo02:38
*** krotscheck has joined #tripleo02:48
*** krotscheck has quit IRC02:49
*** julim has quit IRC02:57
*** cd-undercloud has joined #tripleo03:03
cd-undercloud************** overcloud complete status=0 ************03:03
*** cd-undercloud has quit IRC03:03
*** rpodolyaka has quit IRC03:12
*** rpodolyaka has joined #tripleo03:13
*** Haomeng has quit IRC03:20
*** Haomeng has joined #tripleo03:20
*** jcooley_ has joined #tripleo03:20
openstackgerritA change was merged to openstack/diskimage-builder: Add -U to pip install command in tox.ini  https://review.openstack.org/6150203:24
*** jcooley_ has quit IRC03:26
*** rongze_ has joined #tripleo03:33
*** rongze has quit IRC03:36
*** cd-undercloud has joined #tripleo03:55
cd-undercloud************** overcloud complete status=0 ************03:55
*** cd-undercloud has quit IRC03:55
*** sdake_ has joined #tripleo04:05
*** marun has joined #tripleo04:06
*** rushiagr has joined #tripleo04:06
*** AaronGr_afk is now known as AaronGr04:08
*** morazi has joined #tripleo04:09
*** AaronGr is now known as AaronGr_Zzz04:09
*** ccrouch has joined #tripleo04:09
*** rushiagr has quit IRC04:17
*** ohadlevy has quit IRC04:18
*** ohadlevy has joined #tripleo04:19
*** ohadlevy is now known as Guest8524504:19
*** jergerber has joined #tripleo04:25
*** Guest85245 has quit IRC04:37
*** jergerber has quit IRC04:39
*** ohadlevy_ has joined #tripleo04:45
*** cd-undercloud has joined #tripleo04:49
cd-undercloud************** overcloud complete status=0 ************04:49
*** cd-undercloud has quit IRC04:49
*** rongze_ has quit IRC04:52
*** rongze has joined #tripleo05:22
*** rongze has quit IRC05:27
*** rpodolyaka1 has joined #tripleo05:32
*** sdake_ has quit IRC05:32
rpodolyaka1morning tripleo05:32
lifelessmorning!05:33
rpodolyaka1lifeless: hey! I've a got a small update on rebuild. Our baremetal 'specialized' rebuild works again (without preserving ephemeral right now)05:35
lifelesscool05:36
rpodolyaka1lifeless: haven't tested preserving yet, because I didn't manage to boot a BM node with ephemeral >=0 mb05:36
rpodolyaka1lifeless: the scheduler couldn't find a node05:36
rpodolyaka1lifeless: going to debug this further today. But all we need is a different flavor with ephemeral_mb >=0, right?05:37
rpodolyaka1lifeless: our registered nodes should me modified too? haven't looked at how they are registered yet05:38
rpodolyaka1s/our/or05:38
lifelessrpodolyaka1: our automation scripts haven't been updated05:39
lifelessbut the cd-undercloud has ephemeral defined05:39
lifelessrpodolyaka1: what you need to do is subtract X space from the root and add it to ephemeral in the flavor definition05:39
lifelessthey should sum to the prior value05:39
*** cd-undercloud has joined #tripleo05:40
cd-undercloud************** overcloud complete status=0 ************05:40
*** cd-undercloud has quit IRC05:40
rpodolyaka1lifeless: I believe, this is exactly what I was doing. Ok, going to debug this today then05:41
lifelessrpodolyaka1: so that deploy ^ that just worked, has ephemeral volume05:41
rpodolyaka1hmm...05:42
lifelessrunning trunk05:42
lifelesstrunk05:42
*** tzumainn has quit IRC05:43
rpodolyaka1oh, and btw, the fix for https://bugs.launchpad.net/neutron/+bug/1254246 has been merged05:43
uvirtbotLaunchpad bug 1254246 in tripleo "somehow getting duplicate openvswitch agents for the same host" [Critical,In progress]05:43
lifelesscool05:43
rpodolyaka1though I think we have a new neutron issue with "vif binding failed" now05:44
lifelessthats a race condition with the agent05:44
rpodolyaka1saw it a few times yesterday05:44
lifelessSpamapS has a patch up to workaround it05:44
rpodolyaka1cool!05:44
lifelesssda      8:0    0   1.8T  0 disk05:45
lifeless├─sda1   8:1    0  1000G  0 part /mnt05:45
lifeless├─sda2   8:2    0   7.9M  0 part05:45
lifeless└─sda3   8:3    0   500G  0 part /05:45
lifeless^ lsblk on the deployed overcloud node05:45
*** rushiagr has joined #tripleo05:45
rpodolyaka1nice! let's make it preservable ;)05:46
lifelessthis is the flavor definition05:50
lifelesshttp://paste.ubuntu.com/6559674/05:50
rpodolyaka1thanks, will compare it to mine05:51
StevenKlifeless: Didn't know about lsblk, that looks handy06:05
*** jcooley_ has joined #tripleo06:06
*** rongze has joined #tripleo06:20
*** vipul has quit IRC06:22
*** vipul has joined #tripleo06:22
*** jcooley_ has quit IRC06:30
*** jcooley_ has joined #tripleo06:30
*** jcooley_ has quit IRC06:31
*** jcooley_ has joined #tripleo06:31
*** cd-undercloud has joined #tripleo06:32
cd-undercloud************** overcloud complete status=0 ************06:32
*** cd-undercloud has quit IRC06:32
*** lifeless has quit IRC06:48
*** vkozhukalov has quit IRC06:51
*** lifeless has joined #tripleo07:15
*** ChanServ sets mode: +v lifeless07:15
*** jcooley_ has quit IRC07:18
*** jcooley_ has joined #tripleo07:22
*** cd-undercloud has joined #tripleo07:26
cd-undercloud************** overcloud complete status=0 ************07:26
*** cd-undercloud has quit IRC07:26
*** jcoufal has joined #tripleo07:27
*** jprovazn has joined #tripleo07:39
*** boris-42 has quit IRC07:43
*** ohadlevy_ is now known as ohadlevy07:45
*** ohadlevy has joined #tripleo07:46
*** jcooley_ has quit IRC07:46
*** rcarrillocruz has joined #tripleo07:49
*** jcooley_ has joined #tripleo07:50
*** rpodolyaka1 has quit IRC07:56
*** ifarkas has joined #tripleo07:56
*** jcooley_ has quit IRC07:58
*** rongze has quit IRC08:02
*** rongze has joined #tripleo08:06
*** rongze has quit IRC08:11
lifelessjprovazn: hi08:14
lifelessjprovazn: can we chat about the haproxy patches a little?08:15
jprovaznlifeless: Hi, sure, I'm now reading your replies08:19
jprovaznlifeless: about haproxy and keepalive08:20
jprovaznboth elements need to know virtual ip08:20
*** cd-undercloud has joined #tripleo08:20
cd-undercloud************** overcloud complete status=0 ************08:20
*** cd-undercloud has quit IRC08:20
jprovaznlifeless: the mistake was to use haproxy.virtual_ip08:21
jprovaznlifeless: the config in template can be put outside of haproxy08:21
jprovaznI used haproxy.virtual_ip in keepalived just because I already had it in template, but you are right that it's misleading08:24
lifelessjprovazn: so I guess I want to talk at a meta level08:24
lifelesslike - why a virtual IP for haproxy? Whats the topology you're implementing (and how much variation do we want/need to support today?)08:25
lifelessI don't want to create a boil-the-oceans scenario for you08:25
lifelessbut equally I don't want to head down the wrong path early on!08:25
jprovaznok, so virtual ip is needed to make haproxy not being single point of failure08:27
jprovaznso the topology I wanted to use08:28
*** vkozhukalov has joined #tripleo08:28
lifelessjprovazn: oh, so previously I've just run N in active-active08:28
lifelesswith DNS round-robin08:28
lifelessdo we have clients that don't handle that, or does virtual IP offer faster failover ?08:28
jprovaznjust run haproxy+keepalived on all controller nodes, virtual ip would be used to point to active haproxy08:28
lifelessif there is keepalived everywhere, and a virtual ip that can move to any node, whats haproxy doing in that setup ?08:29
jprovaznlifeless: points to the specific running service08:30
jprovaznvirtual ip is used on node level, haproxy on service level (at least this is my understanding)08:30
lifelessso our services fall into two categories08:31
lifelessmmm three08:31
lifelessactive-active08:31
lifelessactive-passive08:32
lifelessand self-managing08:32
lifelessso nova-api is active-active08:32
lifelessrun as many as we want08:32
lifelessheat-engine today is active-passive - run one, start a new one when that fails08:32
lifelessself managing are things like extradb08:32
lifelesswe probably need to analyse these separately08:33
lifelessactive-active, if the virtual IP has moved, the service on the machine can just answer directly. Done - listen to 0.0.0.0 and it will Just Work.08:33
lifelessself-managing we don't want keepalived fiddling with at all.08:34
jprovaznmmt08:34
jprovazn"if the virtual ip has moved" - keepalived does this08:34
lifelessactive-passive, if the virtual IP has moved, wouldn't that also mean the origin machine is either being taken offline (manual move) or has failed (automatic move) ?08:34
lifelesssure, I'm rolling with using vip, whether it's needed or not is perhaps a separated iscussion08:35
lifelessyou've done work to get it working - cool.08:35
jprovaznlifeless: well, if VIP has moved it just means that the original one stopped responding08:35
lifelessrighto08:35
jprovaznwell08:36
jprovaznhttps://etherpad.openstack.org/p/tripleo-icehouse-ha-production-configuration08:36
lifelessso I'm just failing to understand when haproxy would send traffic to a difference machnie08:36
jprovazn*api instances08:36
jprovazn*virtual IP - keepalived + haproxy08:36
jprovaznIOW ,my understanding was08:36
lifelessand if haproxy isn't sending traffic to a different machine, what's it for :008:36
jprovaznok, so I can imagine 2 cases:08:37
*** akrivoka has joined #tripleo08:37
jprovaznsituation - I have 2 controller nodes, both run API services08:37
jprovazn1) on both I have openstack services configured to point to my virtual ip08:38
jprovaznvirtual ip points to one of these controller nodes08:38
jprovaznif nova-api on node1 fails, then haproxy on controller node where virtual ip points to just forwards nova requests to the second node08:39
jprovaznbut other services forwards as before08:39
jprovazn2) if my controller node where VIP points to blows out08:40
lifelessok08:40
jprovaznthe seond node takesover VIP and requests go through haproxy on this second machine08:40
lifelesslets roll with this08:40
lifelessthe implication is - let me check I have it right:08:40
lifeless - haproxy runs on set of VIP participating nodes [e.g. all, or maybe we build dedicated image]08:41
jprovaznyes08:41
lifeless - each haproxy needs connectivity to all backend apis08:41
lifelessso - here's what I propose08:42
jprovaznyes08:42
lifelessdon't fiddle with the binding port for any of the services.08:42
lifelessat least for openstack services other than the keystone discovery port.08:42
lifelesseverything else has the port encoded in the end point.08:42
lifelessSo lets make the super repeatable and predictable:08:43
lifeless - always install haproxy08:43
lifelessoh08:43
lifelesshmm hang on a second08:43
lifelesswhats our SSL strategy here?08:43
jprovaznI don't think we have SSL strategy so far08:44
jprovazndo we?08:44
lifelesswe're running SSL on the undercloud08:44
lifelessand it's a high pri ticket for the overcloud08:44
lifelessso yeah08:44
lifelessas you know, haproxy can do SSL two ways - tcp proxy or by having a ssl unwrapper in front of it08:44
lifelessBut if we unwrap it08:45
lifelesswe can't forward over the network.08:45
lifelessIf we don't unwrap it, we need a port per service.08:45
jprovaznwhich we have (port per service)08:45
lifelessbut we need two ports per service08:45
lifelesswe need one port for haproxy08:45
lifelessand one port for the service08:45
lifelesswhich you've achieved by manual binding08:45
lifelessat the likely cost of IPv6 support08:46
lifelessso anyhow, my suggestion is:08:46
lifeless - let everything bind as normal, the one exception being keystone's discovery port which we need to override08:46
*** ifarkas has quit IRC08:46
lifeless - pick some arbitrary new ports for each service, have haproxy listen on them08:47
lifeless - and we need to use tcp forwarding, not http, because all of the services will be running SSL before long08:47
lifelesswhat do you think?08:47
lifelessoh, tweaking that - with keystone, don't bind to a specific ip, jsut change the local port the native keystone service is running on, so that it still binds to *08:48
jprovaznmmt08:49
lifelessand always have haproxy there - it makes it easier to document, plus we can instrument things in one place08:49
jprovaznso keystone returns endpoints to haproxy, right?08:50
lifelessyeah08:50
jprovaznwhich one?08:50
jprovazn(if VIP is not used)08:50
lifelessthis is with VIP08:50
lifelessI'm not suggesting that we rework the design08:50
lifelessI asked some early questions about that08:50
*** ifarkas has joined #tripleo08:50
lifelessbut you can ignore them now08:50
jprovaznah, ok, so if you mean just port binding, that I think your idea makes sense08:51
lifelessso we install keepalived always, we install haproxy always, the pathologic case has nothing to fail over08:51
lifelesswe register the vip (well actually the DNS name for the cloud) + ports that haproxy is listening on in keystone08:52
lifelesswe could do interesting things in future like updating keystone when failover occurs rather than having haproxy08:52
lifelessbut that can wait08:52
jprovaznwell, it would be difficult - keystone would have to be notified that something went wrong08:53
jprovazn(which haproxy does pretty well since it's mature)08:53
jprovazn(I hope)08:53
lifelessThat why I say future ;)08:54
lifelessit might be a bad idea08:54
lifelessits late here and I'm sleep deprived :(08:54
jprovaznyour proposal sounds good, only one thing08:54
jprovaznI wonder if it's really better to use extra ports pointing to haproxy instead of bind openstack services to just local ip08:55
jprovaznI don't see the benefit TBH, I understand that it's alternative which works fine, but don't see the advantage08:56
lifelessok08:56
lifelessso here are the things I see08:56
lifelessa) operator confusion08:57
lifeless  two different tools both listening on port X is really confusing unless you explicitly know to look for it08:57
lifelessb) configuration complexity08:57
lifeless   we'll need to manually configure all the local ips - IPv4 and v6, and we haven't done any v6 work so far08:58
lifelessthats really about it08:58
lifelessa) is a big one for me08:58
jprovaznok, makes sense08:58
lifelessb) is meh - it's just some work to do, but a) would be stuck as part of the design08:58
*** rdopieralski has joined #tripleo08:59
lifelessok cool.08:59
jprovaznwill update the patch, thanks for feedback08:59
lifelessI'm going to go halt() for the night, thanks for  your time!08:59
jprovazngood night08:59
*** akuznetsov has joined #tripleo09:00
*** jistr has joined #tripleo09:03
*** derekh has joined #tripleo09:05
SpamapSwait why are we doing haproxy?09:09
SpamapSisn't that lbaas's thing?09:09
*** cd-undercloud has joined #tripleo09:15
cd-undercloud************** overcloud complete status=42 ************09:15
*** cd-undercloud has quit IRC09:15
Ngmorning all09:20
rpodolyakao/09:20
*** rushiagr has quit IRC09:26
jprovaznmorning09:26
jprovaznSpamapS: well, haproxy is used for the load balancing09:27
*** lucasagomes has joined #tripleo09:28
SpamapSjprovazn: LoadBalancer as a Service should be doing that.09:28
SpamapSjprovazn: we shouldn't need to invent our own :)09:29
SpamapShttps://wiki.openstack.org/wiki/Neutron/LBaaS/HowToRun09:29
SpamapSIt seems that it was implemented similar to the L3 agent, so it may not work with our hard wired provider network.09:31
jprovaznSpamapS: thanks, going to look09:31
jprovaznSpamapS: short answer why haproxy is used - as I understand it it's outcome of design summit: https://etherpad.openstack.org/p/tripleo-icehouse-ha-production-configuration also it's the default way suggested in openstack HA guide09:32
*** athomas has joined #tripleo09:32
SpamapSjprovazn: I appreciate that. I think we may end up having to do it if Neutron LBaaS won't fit into our network setup. But we should start with using the aaS's. :)09:33
SpamapSI'd like to eventually be using Trove too.09:33
*** marun has quit IRC09:36
SpamapSjprovazn: https://git.openstack.org/cgit/openstack/neutron/tree/neutron/services/loadbalancer/drivers/haproxy :)09:36
*** marun has joined #tripleo09:37
*** bauzas has joined #tripleo09:38
SpamapSinteresting, looks like they just spin haproxy up in a netns right on the network node09:40
SpamapSthat doesn't seem like something we could use unfortunately.09:43
openstackgerritDerek Higgins proposed a change to openstack/tripleo-image-elements: Add extra swift features  https://review.openstack.org/6150909:44
jprovaznSpamapS: didn't get to reasons why we can't use it yet, but I must admit that it looks extremely interesting09:46
jprovazn(at least according to doc :)09:46
SpamapSjprovazn: since we just have a hard wired network for baremetal nodes, it would be challenging to get the traffic into a namespace on the network node..09:48
SpamapSjprovazn: however, this is highly modular, so I do wonder if we could write one that just works without the netns stuff09:49
SpamapSof course then we have to find a way to get neutron to push that host as the gateway09:50
SpamapSjprovazn: one reason I like doing it with neutron's lbaas though is that vendors are writing hardware backends already09:50
SpamapSanyway.. -> sleep09:51
jprovaznSpamapS: would it make sense to do this in next step (set up haproxy is not so difficult)?09:51
jprovazngood night?09:51
jprovazns/?/./09:51
SpamapSjprovazn: yes it is fine to defer using neutron lbaas09:53
*** max_lobur_afk is now known as max_lobur10:00
*** rushiagr has joined #tripleo10:01
*** jtomasek has joined #tripleo10:05
*** martyntaylor has joined #tripleo10:09
*** cd-undercloud has joined #tripleo10:09
cd-undercloud************** overcloud complete status=0 ************10:09
*** cd-undercloud has quit IRC10:09
*** nosnos_ has joined #tripleo10:15
*** nosnos_ has quit IRC10:16
*** nosnos_ has joined #tripleo10:16
*** nosnos has quit IRC10:17
*** nosnos_ has quit IRC10:20
*** rongze has joined #tripleo10:41
*** rongze has quit IRC10:46
*** akuznetsov has quit IRC10:55
*** ProfFalken has joined #tripleo10:57
*** akuznetsov has joined #tripleo10:58
*** ProfFalken has left #tripleo10:58
*** cd-undercloud has joined #tripleo11:03
cd-undercloud************** overcloud complete status=1 ************11:03
*** cd-undercloud has quit IRC11:03
*** markmc has joined #tripleo11:17
*** boris-42 has joined #tripleo11:17
*** akuznetsov has quit IRC11:19
*** akuznetsov has joined #tripleo11:22
*** slagle has joined #tripleo11:49
*** cd-undercloud has joined #tripleo11:58
cd-undercloud************** overcloud complete status=42 ************11:58
*** cd-undercloud has quit IRC11:58
*** kushi_ has joined #tripleo12:05
kushi_I encounter an error while setup devtest env: https://bugs.launchpad.net/tripleo/+bug/126029212:13
uvirtbotLaunchpad bug 1260292 in tripleo "permission denied error in setup-network" [Undecided,New]12:13
kushi_Is there some suggestion for this issue ?12:14
slaglekushi_: you shouldn't need sudo there as you're already authenticated with virsh12:16
slaglekushi_: check permissions on /var/lib/libvirt/network/brbm.xml, and the directories leading up to it12:17
kushi_but the dir is not writable for common user by default, should we add write permission for it ?12:20
kushi_-rw------- 1 root root 499 Dec 12 03:24 /var/lib/libvirt/network/brbm.xml12:21
*** arata has left #tripleo12:21
slaglethat looks fine12:22
slagledrwx------. 2 root root 4096 Dec  4 07:53 /var/lib/libvirt/network/12:22
slagleodd that you could define the network but not start it12:22
kushi_run virsh with "sudo" or not will get different result on my Ubuntu12:23
kushi_drwxr-xr-x 2 root root 4096 Dec 12 03:24 /var/lib/libvirt/network//12:23
slagleubuntu 12 is pretty old isn't it?  would need to get someone more familiar with ubuntu to confirm if it's supposed to work12:24
slaglei think most of the work has been done on raring and after12:24
Ngcorrect12:27
Ngkushi_: 12.04 definitely won't work right now. You'd need to backport some stuff, but it would be much easier to go with 13.04 or 13.1012:28
kushi_Ng:  Ubuntu 13.04 is checked ?   We'd better add some comment in the wiki page: http://docs.openstack.org/developer/tripleo-incubator/devtest.html12:31
Ngkushi_: we probably should mention that 12.04 isn't supported12:34
kushi_Ng: agree12:35
*** rongze has joined #tripleo12:37
*** bcrochet has quit IRC12:38
* slagle updates bug12:40
*** rongze has quit IRC12:41
*** bcrochet has joined #tripleo12:43
*** cd-undercloud has joined #tripleo12:55
cd-undercloud************** overcloud complete status=0 ************12:55
*** cd-undercloud has quit IRC12:55
*** sdake_ has joined #tripleo12:58
openstackgerritDerek Higgins proposed a change to openstack/tripleo-image-elements: Add tempest element  https://review.openstack.org/5047713:01
*** panda has joined #tripleo13:01
*** panda_ has quit IRC13:04
*** jcoufal_ has joined #tripleo13:08
*** jcoufal has quit IRC13:10
*** rongze has joined #tripleo13:12
*** kushi_ has quit IRC13:12
*** rbrady has joined #tripleo13:19
*** rbrady has left #tripleo13:20
*** arata has joined #tripleo13:24
*** arata has left #tripleo13:25
*** rcarrillocruz1 has joined #tripleo13:32
*** rcarrillocruz has quit IRC13:33
*** jdob has joined #tripleo13:40
slaglejistr: thanks for adding my reply back to the openstack-dev list :)13:44
slaglegmail fail on my part13:44
*** cd-undercloud has joined #tripleo13:50
cd-undercloud************** overcloud complete status=0 ************13:50
*** cd-undercloud has quit IRC13:50
jistrslagle: hmm i think i didn't add it to openstack-dev, haha :)13:50
jistrslagle: but feel free to forward the whole thing there :)13:50
*** jprovazn has quit IRC13:51
*** dprince has joined #tripleo14:01
*** rongze_ has joined #tripleo14:01
*** blamar has quit IRC14:01
*** tzumainn has joined #tripleo14:02
*** rongze has quit IRC14:04
*** jayg|g0n3 is now known as jayg14:05
*** julim has joined #tripleo14:09
openstackgerritA change was merged to openstack/tripleo-image-elements: Add extra swift features  https://review.openstack.org/6150914:14
*** jprovazn has joined #tripleo14:16
*** athomas has quit IRC14:20
*** rongze has joined #tripleo14:20
*** rongze_ has quit IRC14:24
openstackgerritRalf Haferkamp proposed a change to openstack/diskimage-builder: Merge force-text-mode-console into the bootloader script  https://review.openstack.org/6175514:25
*** athomas has joined #tripleo14:27
*** sdake_ has quit IRC14:30
*** jdob has quit IRC14:34
*** jdob has joined #tripleo14:34
*** arata has joined #tripleo14:36
*** cd-undercloud has joined #tripleo14:42
cd-undercloud************** overcloud complete status=0 ************14:42
*** cd-undercloud has quit IRC14:42
*** rongze_ has joined #tripleo14:43
*** CaptTofu has quit IRC14:43
*** CaptTofu has joined #tripleo14:44
*** rongze has quit IRC14:44
*** blamar has joined #tripleo14:45
*** CaptTofu has quit IRC14:49
dprincederekh: I'm still seeing intermittent resizefs failures today (after the diskimage-builder fix yesterday)14:49
dprincederekh: similar to the ticket lifeless filed here https://lists.launchpad.net/kernel-packages/msg18566.html14:50
*** jergerber has joined #tripleo14:51
derekhdprince: looks like the error I was seeing , but the fix sorted out the problem for me, hmmm, looks like the size of the target disk is a factor14:56
*** martyntaylor1 has joined #tripleo14:57
*** jistr has quit IRC14:58
*** martyntaylor has quit IRC14:59
*** arata has left #tripleo14:59
*** akrivoka has quit IRC15:00
*** akrivoka has joined #tripleo15:01
*** jistr has joined #tripleo15:02
*** jcooley_ has joined #tripleo15:05
*** jtomasek has quit IRC15:05
*** matty_dubs|gone is now known as matty_dubs15:08
*** lsmola has quit IRC15:08
*** lsmola has joined #tripleo15:09
rcarrillocruz1hello guys, i'm willing to get my hands on tripleo code, install it and get involved. Where is the code hosted? github? there are quite a few projects with 'tripleO', not sure which ones I should pull...15:17
*** sdake_ has joined #tripleo15:18
*** sdake_ has quit IRC15:18
*** sdake_ has joined #tripleo15:18
Ngrcarrillocruz1: https://wiki.openstack.org/wiki/TripleO has a list :)15:18
Ngrcarrillocruz1: if you want to just get started with a demo TripleO setup, just grab the incubator one and see the docs link next to it, that will end up fetching most of the repos into ~/.cache/tripleo/ anyway15:19
Nghttp://docs.openstack.org/developer/tripleo-incubator/devtest.html15:19
*** jtomasek has joined #tripleo15:19
rcarrillocruz1ah, the incubator fetches other repos as deps15:20
rcarrillocruz1alright15:20
rcarrillocruz1thanks!15:20
*** rcarrillocruz1 is now known as rcarrillocruz15:20
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: Enable some services after installing daemon's...  https://review.openstack.org/5929915:25
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: Call os-svc-enable in post-configure.d  https://review.openstack.org/5929815:25
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: Enable services with os-svc-enable  https://review.openstack.org/5929715:25
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: Don't enable services in os-svc-install  https://review.openstack.org/5930015:25
Ngrcarrillocruz: well, specifically scripts/devtest.sh does. the incubator repo has a bunch of other stuff in it that's not ready for prime-time yet, but it's most useful for the devtest.sh demo setup :)15:26
NobodyCamgood morning TripleO15:27
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: Enable some services after installing daemon's...  https://review.openstack.org/5929915:28
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: Call os-svc-enable in post-configure.d  https://review.openstack.org/5929815:28
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: Enable services with os-svc-enable  https://review.openstack.org/5929715:28
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: Don't enable services in os-svc-install  https://review.openstack.org/5930015:28
NgDAN ALL THE THINGS15:29
dprinceNg: morning15:35
Nghey :)15:36
*** cd-undercloud has joined #tripleo15:42
cd-undercloud************** overcloud complete status=0 ************15:42
*** cd-undercloud has quit IRC15:42
*** ohadlevy has quit IRC15:43
*** vkozhukalov has quit IRC15:51
*** akrivoka has quit IRC15:52
*** akrivoka has joined #tripleo15:53
*** antonym has quit IRC15:54
*** homegrown has quit IRC15:55
*** Guest53164 has joined #tripleo16:00
*** Guest53164 has quit IRC16:07
SpamapSg'morning16:11
openstackgerritRalf Haferkamp proposed a change to openstack/diskimage-builder: Merge force-text-mode-console into the bootloader script  https://review.openstack.org/6175516:19
*** jtomasek has quit IRC16:24
*** spzala has joined #tripleo16:30
openstackgerritA change was merged to openstack/diskimage-builder: Quieten disk-image-get-kernel  https://review.openstack.org/6080416:31
*** bauzas has quit IRC16:32
*** AaronGr_Zzz is now known as AaronGr16:33
*** cd-undercloud has joined #tripleo16:38
cd-undercloud************** overcloud complete status=1 ************16:38
*** cd-undercloud has quit IRC16:38
*** jtomasek has joined #tripleo16:42
SpamapS| fault                                | {u'message': u'No valid host was found. ', u'code': 500, u'created': u'2013-12-12T12:09:40Z'} |16:42
SpamapSI just wonder if we should spin a few times on that nova boot16:42
*** Guest53164 has joined #tripleo16:43
*** martyntaylor1 has quit IRC16:44
*** martyntaylor has joined #tripleo16:44
*** jtomasek has quit IRC16:47
*** boris-42 has quit IRC16:48
*** martyntaylor has quit IRC16:49
*** jistr has quit IRC16:50
*** akrivoka has quit IRC16:51
*** akrivoka has joined #tripleo16:52
*** jtomasek has joined #tripleo17:00
*** rongze_ has quit IRC17:01
*** martyntaylor has joined #tripleo17:05
*** john-n-seattle1 has joined #tripleo17:05
derekhSpamapS: probably not related to that fault but test-env-testenv0-gvyeaxf2mb7k has been 30 minutes spawning, bad host on undercloud ? PXE deploy doesn't seem to have started17:07
*** matty_dubs is now known as matty_dubs|lunch17:08
*** jcoufal_ has quit IRC17:10
*** john-n-seattle1 has quit IRC17:12
rpodolyakahttp://paste.openstack.org/show/54907/ successful rebuild with preserve_ephemeral=True :)17:12
*** akrivoka has quit IRC17:13
rpodolyakalifeless: ^ update on rebuild17:13
*** jtomasek has quit IRC17:15
SpamapSderekh: baremetal will give up on one host and move on to another17:17
*** lsmola has quit IRC17:18
derekhok17:18
derekhSpamapS: thanks17:18
SpamapSderekh: nova-compute.log on cd-undercloud might have clues17:19
*** max_lobur is now known as max_lobur_afk17:19
derekhSpamapS: ahh yes, it seems to have timeout and started again17:20
SpamapSderekh: those boxes take 7-8 minutes per POST, so 16 minutes of POST alone is normal.17:21
derekhok17:21
SpamapSI do wish we could somehow link up console-log with the IPMI console17:23
*** markmc has quit IRC17:24
openstackgerritDerek Higgins proposed a change to openstack-infra/tripleo-ci: Update toci_gate_test.sh to use test_env  https://review.openstack.org/6182917:26
openstackgerritDerek Higgins proposed a change to openstack-infra/tripleo-ci: Adding testenv worker wrapper script  https://review.openstack.org/6105217:26
*** john-n-seattle1 has joined #tripleo17:28
derekhdprince: pleia2 ^^ so I had a test env working earlier this those working, dprince I also have an update for the heat template you proposed I'll send it to you so you can use it to help update your changeset17:30
*** cd-undercloud has joined #tripleo17:31
cd-undercloud************** overcloud complete status=0 ************17:31
*** cd-undercloud has quit IRC17:31
*** martyntaylor has quit IRC17:31
*** rdopieralski has quit IRC17:32
*** john-n-seattle1 has left #tripleo17:32
*** blamar has quit IRC17:33
pleia2derekh: great17:34
pleia2derekh: have some time for you and dprince to sync up with me on where we stand?17:34
derekhpleia2: I have about 20 minutes now if you want and dprince is around17:35
pleia2derekh: now is good :)17:35
derekhdprince: your there ?17:35
dprincederekh: yes17:36
derekhwanna chat, quick status update17:36
dprincepleia2/derekh: sure. I'll start a hangout...17:36
derekhcool17:36
Ngdprince: might be worth talking to lifeless about his -2 on 5929817:37
dprinceNg: will do17:39
Ngdprince: it seems like it can probably come off, but atm it's holding up all your changesets above it17:40
*** gothicmindfood has joined #tripleo17:43
*** gothicmindfood has left #tripleo17:43
SpamapSdprince: FWIW, now that I've been staring at all of this code this week, I understand why your noop would have been fine.17:54
SpamapSdprince: apologies for blocking your progress while I caught up.17:54
derekhdprince: pleia2 this is the command I was using to build a test-env image17:55
derekhELEMENTS_PATH=/root/tmp/tripleo-image-elements/elements:tripleo-ci/elements /opt/stack/diskimage-builder/bin/disk-image-create -o testenv local-config  stackuser ubuntu serial-console dhcp-all-interfaces testenv-worker17:55
*** ohadlevy has joined #tripleo17:56
*** bauzas has joined #tripleo17:58
*** athomas has quit IRC17:58
*** derekh has quit IRC17:58
SpamapSCores: https://review.openstack.org/#/c/59297/ wants your +217:59
SpamapShttps://review.openstack.org/#/c/60341/9 too17:59
dprinceSpamapS: no worries, I don't *really* care about the order. Just trying to do what is easiest18:05
SpamapSlifeless: https://review.openstack.org/#/c/59298/ <-- definitely ready for you to reevaluate.. I think the -2 comments are entirely addressed and this is ready to go now.18:05
SpamapSdprince: glad to hear it. Nonetheless, unintended consequence of my misunderstanding is that you have been delayed on this change for quite a while.18:06
*** vkozhukalov has joined #tripleo18:12
*** matty_dubs|lunch is now known as matty_dubs18:14
*** rpodolyaka1 has joined #tripleo18:17
*** lucasagomes has quit IRC18:24
*** cd-undercloud has joined #tripleo18:26
cd-undercloud************** overcloud complete status=0 ************18:26
*** cd-undercloud has quit IRC18:26
*** krotscheck has joined #tripleo18:37
*** rcarrillocruz1 has joined #tripleo18:37
*** rcarrillocruz has quit IRC18:37
lifelessSpamapS: dprince ack18:47
*** rushiagr has quit IRC18:47
dprincelifeless: morning.18:48
dprincelifeless: hey, I'm still hitting resizefs issues today. Lots of them. Similar to what you filed here: https://lists.launchpad.net/kernel-packages/msg18566.html18:48
dprincelifeless: I'm actually trying to get some more info on it because when it happens my system (bare metal) is totally hosed...18:48
dprincelifeless: currently, I'm building an image where the code in cloud-init to actually run the resizefs is just disabled. Then my plan was to log in and run it manually to see if I can get more info that way. Wondering if you had any more ideas on this one?18:49
*** blamar has joined #tripleo18:59
lifelessdprince: not offhand19:06
lifelessdprince: but - you might try using flavour that has a smallish / and a large ephemeral19:06
*** krotscheck has quit IRC19:12
*** ohadlevy has quit IRC19:16
*** ohadlevy has joined #tripleo19:17
*** ohadlevy is now known as Guest536119:17
*** cd-undercloud has joined #tripleo19:23
cd-undercloud************** overcloud complete status=0 ************19:23
*** cd-undercloud has quit IRC19:23
*** Guest5361 has quit IRC19:27
*** ohadlevy_ has joined #tripleo19:29
*** vipul is now known as vipul-away19:30
*** rcarrillocruz1 has quit IRC19:41
*** vipul-away is now known as vipul19:44
*** ohadlevy_ has quit IRC19:48
*** akuznetsov has quit IRC19:49
*** ohadlevy_ has joined #tripleo19:49
*** ohadlevy_ has quit IRC19:58
*** jprovazn has quit IRC19:58
*** ohadlevy_ has joined #tripleo20:00
*** akuznetsov has joined #tripleo20:09
*** vipul is now known as vipul-away20:13
*** vipul-away is now known as vipul20:13
*** jcoufal has joined #tripleo20:14
lifelessjcoufal: morning! well, evening!20:15
lifelesstzumainn: and hi20:15
jcoufallifeless: hey20:15
jcoufallifeless: tzumainn wrote that he might skip actually20:16
lifelessok20:16
jcoufaltzumainn: however if you want to join you are welcome20:16
*** ohadlevy_ has quit IRC20:19
*** cd-undercloud has joined #tripleo20:20
cd-undercloud************** overcloud complete status=42 ************20:20
*** cd-undercloud has quit IRC20:20
*** ohadlevy has joined #tripleo20:22
*** ohadlevy is now known as Guest5296420:22
*** vipul is now known as vipul-away20:28
tzumainnjcoufal, I don't think I need to, based on what we talked about earlier20:33
*** UtahDave has joined #tripleo20:33
*** bauzas has quit IRC20:44
*** vipul-away is now known as vipul20:49
*** jcooley_ has quit IRC20:49
*** cwolferh has joined #tripleo20:52
lifelessjcoufal: ironic has a key:value field per node20:53
lifelessjcoufal: so we can say rack=xyz in that field if we want.20:53
jcoufallifeless: which is actually what we need20:53
jcoufallifeless: that's great news20:53
lifelessjcoufal: it doesn't support searching it yet though, but yeah - we can stash per node data there as needed20:53
lifelessjcoufal: so the 'tags' field can let the user put what they want as 'foo' or as 'foo=bar', and we can store 'foo' as 'foo=""'20:54
lifelessIMO20:54
jcoufalawesome20:54
jcoufalwe can handle the search at the UI part - at least some basic filetring20:55
jcoufallifeless: thanks for checking20:55
lifelessanytime :)20:56
*** bauzas has joined #tripleo21:01
*** rpodolyaka1 has quit IRC21:03
*** boris-42 has joined #tripleo21:03
tzumainnjcoufal, does that let us find nodes with particular roles more easily?21:04
*** ccrouch has quit IRC21:05
jcoufaltzumainn: we can tag anything. so for the first iteration it might be useful for hardcoded roles21:05
*** rwsu has quit IRC21:05
*** bauzas has quit IRC21:06
tzumainnjcoufal, awesome - the only issue I'd potentially have is that - what happens if a node is manipulated outside of tuskar?  then the tag can become inaccurate21:06
jcoufaltzumainn: like somebody ssh to the node and change things manually?21:07
tzumainnor someone shuts down the service on the node manually, or something21:07
jcoufaltzumainn: well we will be checking the services if they are running anyway - rising warnings21:08
jcoufaltzumainn: however for the first steps I wouldn't complicate things that much21:09
*** epim has joined #tripleo21:09
tzumainnjcoufal, right, it just seems that we need to be careful, in that a tag on a node in ironic isn't a guarantee that the tag is correct21:09
tzumainnjcoufal, are you going to send out a summary of your conversation with lifeless?21:09
lifelessyes21:10
lifelessmmm, don't conflate tags - user data - with stuff we need21:10
lifelesswe can ask heat for what instances we have at any point in time21:10
lifelessno need for tags21:10
jcoufaltzumainn: yes, mostly trying to wrap up all the conversations, just not tonight, tomorrow morning21:10
tzumainnlifeless, yeah, that makes sense to me21:11
SpamapStzumainn: That is why arbitrary hard coded roles are problematic21:12
SpamapStzumainn: (re "what if something manipulates the node outside ironic")21:13
tzumainnSpamapS, yeah, agreed - I think what I was trying to get to in a roundabout way was that I don't think we should rely on ironic tags for stuff like that21:14
SpamapS:)21:14
*** cd-undercloud has joined #tripleo21:16
cd-undercloud************** overcloud complete status=1 ************21:16
*** cd-undercloud has quit IRC21:16
tzumainnlifeless, jcoufal, out of curiosity, was there agreement to push manual node allocation to a future consideration?21:17
jcoufaltzumainn: it will be ongoing discussion21:18
jcoufalso basically yes21:18
tzumainnjcoufal, okay, I think that as far as the requirements discussion, that was the biggest sticking point, so that's good to hear21:18
jcoufaltzumainn: need to run, it's getting late21:19
*** ccrouch has joined #tripleo21:19
tzumainnjcoufal, definitely, good night!21:19
jcoufalc'you tomorrow guys21:19
*** krotscheck has joined #tripleo21:36
openstackgerritClint "SpamapS" Byrum proposed a change to openstack/tripleo-incubator: Wait for Neutron L2 Agent on Compute Node  https://review.openstack.org/6160821:36
*** spzala has quit IRC21:37
openstackgerritBen Nemec proposed a change to openstack/tripleo-image-elements: Add tempest element  https://review.openstack.org/5047721:44
*** marun has quit IRC21:48
*** jdob has quit IRC21:50
*** cwolferh has quit IRC21:54
*** julim_ has joined #tripleo22:05
*** julim has quit IRC22:07
*** cd-undercloud has joined #tripleo22:11
cd-undercloud************** overcloud complete status=42 ************22:11
*** cd-undercloud has quit IRC22:11
*** boris-42 has quit IRC22:11
*** vkozhukalov has quit IRC22:15
*** dprince has quit IRC22:22
*** matty_dubs is now known as matty_dubs|gone22:32
*** noslzzp has quit IRC22:33
*** phschwartz is now known as phschwartz_aw22:46
*** phschwartz_aw is now known as phschwartz22:47
*** jayg is now known as jayg|g0n322:59
*** jergerber has quit IRC23:02
*** cd-undercloud has joined #tripleo23:07
cd-undercloud************** overcloud complete status=0 ************23:07
*** cd-undercloud has quit IRC23:07
*** phschwartz has quit IRC23:10
*** phschwartz has joined #tripleo23:11
*** jcoufal has quit IRC23:12
openstackgerritA change was merged to openstack/tripleo-incubator: Wait for Neutron L2 Agent on Compute Node  https://review.openstack.org/6160823:36
*** krotscheck has quit IRC23:49
*** ccrouch has quit IRC23:51
lifelesspleia2: nuts, was today our sync up day? or yesterday?23:52
openstackgerritClint "SpamapS" Byrum proposed a change to openstack/tripleo-incubator: Wait for nova-compute to claim to be up  https://review.openstack.org/6189423:57
SpamapSlifeless: ^^ ok that is the last spurious error thing I will work on.. unless more come up. :)23:57
SpamapSlifeless: now, on to preserve_ephemeral!23:57

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!