Friday, 2014-06-20

*** yamahata has quit IRC00:01
*** yamahata has joined #tripleo00:04
lifelesslets see if heat stack-delete overcloud; wait_for 30 10 ! heat stack-show overcloud; USE_CACHE=1 devtest_overcloud.sh00:08
lifeless finds it00:08
lifelessclarkb: so I think we need this 'ssh into test slave' thing00:10
lifelessclarkb: just a sleep the best way  you think, for now ?00:10
clarkbprobably but you will still have the test timeout00:10
clarkbas an alternative00:10
clarkbcould you just boot a test slave and run the test that way?00:10
*** ccrouch has quit IRC00:10
clarkbthen you avoid the timeout00:10
openstackgerritSteve Kowalik proposed a change to openstack/tripleo-image-elements: Add a sysctl-append-value script  https://review.openstack.org/10111700:14
openstackgerritSteve Kowalik proposed a change to openstack/tripleo-image-elements: Reserve the keystone high port out of ephemeral  https://review.openstack.org/10040700:15
lifelessclarkb: we have our own timeout00:16
lifelessclarkb: because defense in depth00:16
*** CaptTofu_ has quit IRC00:16
clarkbgotcha00:16
clarkbanyways if you point me at a running test (or arbitrary number of them) I can hold those nodes and add someones ssh key to them00:17
*** CaptTofu_ has joined #tripleo00:17
clarkbI am not sure what a good candidate is? maybe you want one precise and one f20?00:17
openstackgerritlifeless proposed a change to openstack-infra/tripleo-ci: Add a massive sleep to let debugging happen.  https://review.openstack.org/10137800:18
*** CaptTof__ has joined #tripleo00:18
lifelessclarkb: ^ the f20 job that that ends up running00:18
clarkbok00:19
lifelessplease  :)00:19
clarkbcan you point me at a public key to add to the jenkins authorised keys too?00:19
*** dkehn_ is now known as dkehnx00:20
clarkblifeless: I have held the node. just need a public key00:21
*** CaptTofu_ has quit IRC00:22
*** matsuhashi has joined #tripleo00:28
lifelessbnemec: can you tell me a little about your test environment?00:38
lifelessbnemec:  is it same-machine ?00:38
*** matsuhashi has quit IRC00:48
*** yamahata has quit IRC00:49
*** matsuhashi has joined #tripleo00:49
tchaypoI will be set up in about 10 minutes00:51
*** matsuhashi has quit IRC00:54
lifelessok seed up on this test00:56
lifelessnext up is OC00:56
bnemeclifeless: Yeah, same machine.00:57
bnemecI've tried it on two separate boxes - one new desktop and one old server.00:57
bnemeclifeless: Whoa, https://review.openstack.org/#/c/101349/ passed.00:59
* bnemec rechecks to make sure it wasn't just luck00:59
lifelessw.t.f.01:00
lifelesshow does one pull logs out of systemd?01:01
*** matsuhashi has joined #tripleo01:01
bnemeclifeless: journalctl01:02
bnemecI usually just do journalctl | grep whatever-i'm-looking-for01:02
lifelessournalctl --system01:02
lifelessNo journal files were found.01:02
lifelessoh ok01:02
bnemecYou can also do journalctl -u name-of-service01:02
*** CaptTof__ has quit IRC01:04
*** CaptTofu_ has joined #tripleo01:04
*** CaptTofu_ has quit IRC01:08
*** matsuhashi has quit IRC01:09
*** matsuhashi has joined #tripleo01:10
*** yjiang5 is now known as yjiang5_away01:11
lifelessbuilding oc compute now01:12
tchaypoas sson as StevenK stops distracting me with steam sales...01:23
openstackgerritDan Prince proposed a change to openstack-infra/tripleo-ci: Test revert of Neutron commit:  https://review.openstack.org/10138401:24
*** eguz has joined #tripleo01:25
lifelessheh01:29
lifelessdan needs to lurk here more :)01:30
*** eghobo has quit IRC01:30
*** nosnos has joined #tripleo01:30
lifelessbnemec: I particularly love the way -u accepts anything with no errors01:34
bnemeclifeless: Don't get me started. :-)01:34
lifelessplease, go right ahead :)01:34
lifelessok, overcloud spawning01:35
lifelessuser image built01:35
vinshwho? what?01:37
vinshnm.01:37
*** yamahata has joined #tripleo01:38
*** TravT has quit IRC01:41
lifelessok, tracing the pings01:42
lifelessthey hit the controller ok01:42
lifeless[   69.503632] cloud-init[265]: Cloud-init v. 0.7.2 running 'init-local' at Fri, 20 Jun 2014 01:39:58 +0000. Up 66.39 seconds.01:42
lifeless[  156.273807] cloud-init[439]: ci-info: !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!Route info failed!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!01:43
lifelessso indeed instance not glued up properly01:43
lifelesswe should get a nova console-log when this happens01:43
bnemecIndeed.01:44
*** CaptTofu_ has joined #tripleo01:44
bnemecIn my environment I could see the instance sending DHCP on the compute node's br-int, but couldn't find it getting anywhere past that.01:44
bnemecAlthough I know basically nothing about multi-node Neutron so I may have just missed it.01:45
lifelessok so I know whts wrong now01:49
lifeless        Port "gre-c0000203"01:49
lifeless            Interface "gre-c0000203"01:49
lifeless                type: gre01:49
lifeless                options: {df_default="true", in_key=flow, local_ip="192.0.2.6", out_key=flow, remote_ip="192.0.2.3"}01:49
lifelessthere's only one gre port on the hypervisor01:49
lifeless2.4 is missing01:50
lifelesswhich means DHCP from the instance isn't reaching the controller01:50
lifelessnovacompute0 is worse, its missing entirely01:51
lifelessnow thats setup by the ovs agent01:52
lifelessso in principle restarting it will fix it01:52
bnemecIt did for me.01:53
bnemecI think anyway, it was behaving weirdly.01:53
lifeless hah in fact01:55
lifeless               sudo ovs-vsctl show;01:55
*** CaptTof__ has joined #tripleo01:55
lifelessis already in the state dump01:55
*** CaptTofu_ has quit IRC01:57
lifeless\o/ for commit ff007a57efa82bc73cbe4a1e60b1cde42c7efa8201:58
lifelesswe can cleanup our compute config I think01:58
jogorandom question01:59
lifelessshoot02:00
jogodo you know if oslo.config normalizes group names by default in havana?02:00
lifelesscase normalisation ?02:00
lifelessI seem to recall a real compat mess with that02:01
jogolifeless: correct02:01
jogohmm I'll just test it out in devstack02:02
lifelessyeah,s orry02:03
lifelesswe restart the ovs-agent at 8002:03
lifelessthats well after we've made the bridge which we do in init-neutron-ovs02:04
lifelessor is it02:04
lifelessno, we do it there02:04
lifelesshmmm02:04
lifelessand we call --no-wait02:04
lifelessactually no02:05
lifelessconfigure/80 we make the bridge02:05
lifelesspost-configure/80 we restart02:05
lifelessoh I wonder02:07
lifelessbut its br-tun that the issue is on02:10
lifelessnot br-int02:10
bnemeclifeless: So https://review.openstack.org/#/c/101349/ appears about to pass again.02:10
lifelessbnemec: thats promising in that we can ask for a revert02:10
lifelessI want to understand why ;)02:11
bnemecWe'll see what dan's version does, but that's looking like the culprit.02:11
lifelessisn't dans identical to yours?02:11
lifelessoh you just means statistically02:11
bnemecYeah, but third time's a charm, right? :-)02:11
lifelessahahahaha02:12
lifelessI don't think neutron devs know their own codebase02:12
lifelessbnemec: look in neutron/plugins/openvswitch/agent/ovs_neutron_agent.py02:12
lifelessline 75002:12
lifelesstrace back from there02:12
lifelessoh nope, just confusing code02:13
vinshconfusingNode0.02:13
lifelessaiee02:14
lifeless    def reset_bridge(self):02:14
lifeless        self.destroy()02:14
lifeless        self.create()02:14
lifelessyou don't want to put anything precious on these bridges02:14
vinsh:D02:14
lifelessfdb_add handles the agent ports02:17
bnemeclifeless: Okay, you seem to have this well in hand now so I'm going to take off.  I won't be around much again tomorrow because funeral.02:17
lifelessbnemec: where was that debug-level run? did you make neutron be debug too?02:17
lifelessbnemec: sadness, empathy.02:18
bnemeclifeless: I think so.  Let me grab a link.02:18
bnemeclifeless: Thanks02:18
bnemeclifeless: https://review.openstack.org/#/c/100374/02:18
*** bnemec is now known as bnemec_away02:20
*** CaptTof__ has quit IRC02:22
*** CaptTofu_ has joined #tripleo02:22
lifelesstchaypo: ok so - vlans and public virtual ips02:24
*** rwsu has quit IRC02:24
lifelesstchaypo: do you have a good thread to pull on on that? we need to pull on that to get hp1 and hp2 up02:24
* tchaypo reloads state02:25
lifelesstchaypo: it is basically the second highest priority we have, highest being unwedging CI in the first place, which seems about done02:25
*** nati_ueno has quit IRC02:26
*** CaptTofu_ has quit IRC02:26
tchaypookay, the state that I've retrieved says that https://review.openstack.org/#/c/100151/2/specs/juno/virtual-public-ips.rst, although unapproved, is the current high-level spec for what we want02:27
lifelessyes.02:29
tchaypoand as for implementation - I saw the bit on the _overcloud.sh where we set $FLOATINGIP; it looks to me as though we need to grab another IP via the same method (possibly with a different network than ext-net?), get it set up, and then stick that value somewhere so things can look up what they need to bind to02:29
lifelessahhha I think I know the fault in neutron now02:29
lifelesshmm, nope02:30
*** eguz has quit IRC02:33
*** dkehn_ has joined #tripleo02:35
*** dkehn has quit IRC02:35
*** dkehnx has quit IRC02:36
lifelessgreghaynes: around ?02:37
lifelessgreghaynes: I can't talk to rabbit on localhost with rabbitmqctl02:37
lifelesslooking at the rabbit cluster setup I think I'm doing it right but it is all clamining nodedown02:37
*** dkehn has joined #tripleo02:37
lifelessso this is fun, the agent listens for messages based on the mac address of the integration bridge02:38
vinshI don't even...02:39
tchaypostevek and i are going for lunch. We have midcycle flights booked, yay.02:39
lifelessneutron/plugins/openvswitch/agent/ovs_neutron_agent.py02:39
lifelesslink 24402:39
lifeless        self.agent_id = '%s%s' % ('ovs', (mac.replace(":", "")))02:39
vinshi'm done with ovs and ovs_db. enough of it.02:39
vinsh;)02:39
lifelesswow02:40
lifeless    cfg.IntOpt('agent_down_time', default=75,02:40
lifeless'should be twice report_interval'02:41
lifelessreport_interval appears to default to 30002:41
lifeless...02:41
tchaypoasdfl;kjasdfglk;jasdfl;kjdasf02:41
*** ramishra has joined #tripleo02:41
tchaypolulz02:41
lifelessoh no, its worse, there are multiple report_intervals02:41
lifelesswith different dfaults02:41
lifelessanyhow the good news is none default 002:42
vinshso.. btw.. that 10-mysql file I was ranting about earlier. I ended up hacking it to pieces.. and it works-ish.  will patch once I consult with my mysql expert.02:44
vinshI blame hlinux.02:44
lifeless:)02:44
*** CaptTofu_ has joined #tripleo02:51
openstackgerritlifeless proposed a change to openstack-infra/tripleo-ci: Try reverting Neutron ovs-agent change  https://review.openstack.org/10134902:53
openstackgerritA change was merged to openstack-infra/tripleo-ci: Try reverting Neutron ovs-agent change  https://review.openstack.org/10134902:53
*** lifeless changes topic to "Using OpenStack to deploy OpenStack;meetings Tuesday 1900//0700 UTC in #openstack-meeting-alt"02:53
lifelesschuckC: hi - https://bugs.launchpad.net/neutron/+bug/1292105 - the neutron side of that may be something you want to run with ?02:54
uvirtbotLaunchpad bug 1292105 in tripleo "ovs tunnel state not syncing (failure pinging overcloud instance)" [Critical,In progress]02:54
lifelesschuckC: e.g. start by pushing up a revert02:54
lifelesschuckC: I'm popping into the neutron channel to discuss there now02:54
vinshhrm "+ os-db-create keystone keystone unset02:58
vinshERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)"02:58
clarkblifeless sounds like grabbing that node helped?02:58
lifelessclarkb: immenstely02:58
lifelesswe did have the info captured to identify remotely02:59
lifelessbut it wasn't obviously captured02:59
lifelessclarkb: thank you02:59
clarkbnp02:59
*** untriaged-bot has joined #tripleo03:00
untriaged-botUntriaged bugs so far:03:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/133193103:00
uvirtbotLaunchpad bug 1331931 in tripleo "NeutronControlPlaneID should not be set if building image only" [Undecided,In progress]03:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/133232103:00
uvirtbotLaunchpad bug 1332321 in tripleo "Undercloud horizon resource tab returns an error due to missing ceilometer components " [Undecided,New]03:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/133186703:00
uvirtbotLaunchpad bug 1331867 in tripleo "Building undercloud failed" [Undecided,New]03:00
*** untriaged-bot has quit IRC03:00
*** weshay has quit IRC03:00
lifelesschuckC: I've pushed up a revert for neutron to discuss :)03:02
*** matsuhashi has quit IRC03:02
*** matsuhashi has joined #tripleo03:03
tchaypodo we have a patch for config or toci up to get the revert applied? or should we wait a bit before we do that?03:06
lifelesstchaypo: see backscroll03:07
*** matsuhashi has quit IRC03:07
tchaypolifeless: oh, i see.03:07
lifelesstl;dr ben's revert is in, proper revert in neutron is pending; tomorrowish we can remove the toci revert03:08
tchaypolifeless: nice use of privilege.03:12
tchaypoit seems like the smallest useful unit of work for me is to get a patch that allocates a second vip03:13
tchaypopatches to make services bind to it can come later, yes?03:13
openstackgerritlifeless proposed a change to openstack-infra/tripleo-ci: Put host_info somewhere obvious. Its useful!  https://review.openstack.org/10140003:14
lifelesstchaypo: that seems like a patch that can be done straight away yes.03:14
lifelesstchaypo: another one that could be done straight away is to register the public network with the UC neutron03:14
lifelesstchaypo: (and likewise the seed, since seed == UC)03:15
tchaypoI was about to say that both _oc and _uc seem to call setup-neutron so maybe i should be doing it there?03:15
lifelessadding a new network would indeed happen there yes03:15
*** mestery has quit IRC03:17
tchaypolifeless: we didn't see your name on the midcycle etherpad03:17
*** nosnos has quit IRC03:18
tchaypoI see that there are two heat people coming, both of them Steves03:19
tchaypowe need to keep them away from our steve lest they steal him03:19
*** ccrouch has joined #tripleo03:20
tchaypominor confusion03:21
tchaypoconfusion resolved i think03:21
*** dkehn__ has joined #tripleo03:22
lifelessmine will be there03:22
lifelessETIME03:22
lifelessgate was down03:22
tchaypowe want the existing VIP to become the intra-cluster VIP; the new details we're passing in are for a public network which will have several VIPs allocated, for access to services in both under and overclouds?03:22
tchaypobut the public network is shared across both clouds?03:23
*** dkehn_ has quit IRC03:23
lifelessprobably03:23
lifelessat least as a first approximation03:23
lifelessthats how the ctlpane network workstoday03:23
*** dkehn has quit IRC03:24
StevenKlifeless: Now that CI is fixored, do you want to +A https://review.openstack.org/#/c/95396/ ?03:24
tchaypoif we03:24
*** dkehn has joined #tripleo03:24
lifelessdo dah do dah03:25
tchaypore going to pass in things like "vlan25" as being the public interface we're assuming that all nodes in that cloud will be using the same interface03:25
tchaypowhich seems like a reasonable assumption to be starting with03:25
lifelessyes, same as e.g. eth2 today.03:25
lifelessit sucks but its better than nothing03:25
StevenKlifeless: It ran through CI on the 17th, I'm fairly certain it will get a second run, but we can recheck to be sure before approval if you wish03:26
lifelessStevenK: indeed :P refres03:27
lifelessrefresh03:27
tchaypolifeless: so like, I think that means I need to tweak https://review.openstack.org/#/c/100078/5/toci_devtest.sh - it needs to pass in an interface name and a specific IP?03:34
*** tzumainn has quit IRC03:34
*** CaptTofu_ has quit IRC03:34
*** CaptTofu_ has joined #tripleo03:35
openstackgerritOpenStack Proposal Bot proposed a change to openstack/os-apply-config: Updated from global requirements  https://review.openstack.org/9908303:38
lifelesstchaypo: well, step back03:38
lifelesstchaypo: whats the first thing you're doing; we had two open candidates03:38
lifelessasking for a second VIP, or registering a new network on the seed and UC03:38
*** CaptTofu_ has quit IRC03:40
tchaypoactually for some reason I'd gone off on a tangent - I'd started thinking that the IP to use was going to be passed in because "clearly" (in my head) you can't have two neutrons managing the same ip range, so the allocation of the VIPs is going to happen outside03:41
*** jml has quit IRC03:41
tchaypoit sounds like you're saying we can just tell both neutrons about the network and then get them to allocate a VIP?03:41
lifelesstchaypo: so you're wrong :) the UC is allocated on the seed neutron the OC on the UC neutron from the same IP subnet, different ranges03:41
tchayposo for instance both would use 10.1.2.0/24 but the UC would take the first /25 and the OC takes the second /25?03:42
lifelessfor instance03:42
lifelessbut expressed at ip granularity, not prefix mask03:42
lifelesssee setup-neutron and the calls in _seed and _undercloud03:42
* StevenK was pondering moving setup-neutron to os-cloud-config03:43
tchayposo thinking about doing this as a singular run of devtest.sh or toci_devtest.sh - we'd want to pass in one range at the top and do some arithmetic to allocate half to the uc and half to the oc?03:43
StevenKSince I can't work out how pm_password gets allocated as null on CI03:43
*** jml has joined #tripleo03:44
lifelesstchaypo: my suggestion is to have a look at the current example first03:44
lifelessStevenK: what do you need to know to figure that out?03:44
StevenKlifeless: I've spent some time yesterday and Wednesday digging through the code, from what I can see, it should be an ssh key, and is locally. My second concern is if that ssh key will even work for power managing Ironic03:46
lifelessStevenK: it will03:46
lifelessStevenK: its present only in   "ssh-key": "-----BEGIN RSA PRIVATE KEY-----\nMIIEowIBAAKCAQEAulshU3UxW6vMbJnyOSs8CScX69ET9XG365twKUQNH9XUioKH\03:46
lifelessStevenK: in the testenvs on rh1.tripleo.org03:46
lifelessStevenK: I believe that just means they were deployed a while ago, as I said03:47
StevenKlifeless: Right, but create-nodes uses ["ssh-key"] for pm_password03:47
StevenKSo they should be the same03:47
lifelessStevenK: since *when*03:47
lifelessStevenK: create-nodes doesn't run IN CI03:48
lifelessStevenK: it runs when the testenvs are created.03:48
tchayporight, so the current call to setup-neutron reads everything from $TE_DATAFILE03:48
StevenKlifeless: So ssh-key is undefined when the nodes are created?03:50
lifelessStevenK: on a per-node basis, its defined globally03:50
lifelessStevenK: commit c2eaf8718a53349044e2210481a462a56f6ad81703:52
lifelessAuthor: Steve Kowalik <steven@wedontsleep.org>03:52
lifelessStevenK: should ring a bell03:52
tchaypolifeless: i lolled03:52
*** matsuhashi has joined #tripleo03:53
StevenKlifeless: Right, I get it. ssh-key in TE_DATAFILE is undefined when we call create-nodes03:54
lifelessStevenK: what/03:55
lifelessStevenK: no03:55
lifelessStevenK: tell me when you think create-nodes is being called ?03:55
StevenKlifeless: From devtest_testenv03:55
lifelessStevenK: and when do you think we call devtest_testenv03:55
StevenKlifeless: line 32, bin/ensure-test-env03:56
StevenKssh key is injected at line 4203:56
lifelessStevenK: and when do you think we call bin/ensure-test-env, in CI ?03:56
StevenKlifeless: We don't03:56
lifelessStevenK: bingo. Now, again, when do you think we call devtest_testenv ?03:56
StevenKCreated when the environment is03:56
lifelessStevenK: right.03:57
StevenKlifeless: The TE_DATAFILE is also created when the environment is03:57
lifelessStevenK: yes. So when do you think we create environments in CI ?03:57
StevenKWhen we re-deploy03:57
lifelessright. Does it make sense now?03:58
lifelessWe made the testenvs with much older code. Pre Feb 13th from the look of it.03:58
StevenKlifeless: It does, but ssh-key is still undefined when create-nodes is called.03:58
lifelessStevenK: NO03:58
tchaypolifeless: just to check - the place where we allocate the current vip is in _overcloud, where we set $FLOATINGIP via neutron floatingip-create ext-net .....03:59
lifelesstchaypo: no, I believe its set from the heat template as a port03:59
lifelessStevenK: *or* I'm fundamentally misunderstanding what you're saying04:00
lifelessStevenK: thats entirely possible since I've had 2 days without food and with fever etc.04:00
lifelessStevenK: its also why i"m unusually non-listeny, sorry.04:01
*** matsuhashi has quit IRC04:02
StevenKlifeless: In bin/ensure-test-env, on line 32 we call devtest_testenv, which calls create-nodes. devtest_testenv will set ssh-key to a path that doesn't exist, and then the nodes get created with null pm_password. 10 lines afterwards, we set ssh-key to its value04:02
*** matsuhashi has joined #tripleo04:02
lifelessStevenK: ok, now I understand.04:03
StevenKlifeless: So I think the way forward is to change devtest_testenv to have it possible to have an ssh key path passed in, and then change toci to do so.04:03
lifelessStevenK: so we need a fix to toci and a testenv redeploy, and/or a workaround in the incubator04:03
StevenKlifeless: I think a redeploy at this point will have the same issue04:04
lifeless'fix + redeploy' I said04:04
StevenKRight, ssh-key is always set by testenv, so it will require an -incubator patch04:05
lifelessStevenK: your approach sounds sane to me04:05
tchaypoi get greghaynes04:05
tchaypoknows04:05
lifelesstchaypo: ?04:05
tchaypobut of course today is the day he's not around04:05
tchaypos/get/bet/04:05
lifelesstchaypo: whats the q ?04:05
tchaypos/$/ knows/04:05
tchaypowhere the vip gets generated/allocated04:05
lifelesstchaypo: grep Port *.yaml04:06
tchaypowell actually i bet he knows everything04:06
StevenKlifeless: WRT to sickness, do you think you're getting better, or the weekend will be horrid too?04:06
*** matsuhashi has quit IRC04:06
lifelessStevenK: I'm *looking forward* to my flight, let me tell you04:07
StevenKHahahaha04:07
StevenKHow to cause 250 people to wish you were dead, yeah04:07
tchaypojust avoid going to the doctor04:07
tchaypoin case they decide you're unfit to fly04:07
tchaypoStevenK: most of them are going to wish he's dead because he's up the pointy end04:08
lifelessI'm not coughing04:08
StevenKAre you still contagious?04:08
lifelessunless they rub their hands down my throat or up my ass I think they'll be fine04:08
* StevenK goes to find the brain bleach04:08
* StevenK also goes to review his lunch04:09
tchaypoit seems to be contagious via text, as well as various forms of frottage04:09
*** ramishra has quit IRC04:09
lifelesstchaypo: pointy end - same travel policy...04:11
lifelesstchaypo: if I get my airpoints upgrade, then maybe :>04:11
*** nosnos has joined #tripleo04:11
tchaypoStevenK and i have just realised we were both at your wedding04:11
lifelessheh04:11
StevenKHaha04:12
StevenKLooks like tchaypo can not upgrade04:12
lifelessseriously though I have no reason to think I'm contagious04:14
lifelessairborne I mean04:14
lifelessjogo: your fi's are odd04:19
jogolifeless: ?04:20
*** ramishra has joined #tripleo04:20
lifelessjogo: if your hp mail, looks like a single stylish character04:21
tchaypounfortunately all my flights are on delta this time so the only upgrade I could do is the for-cash upgrade to economy comfort04:23
jogolifeless: I am still confused04:23
lifelessjogo: nvm I will show you on sunday :)04:25
jogoworks for me04:25
jogospeaking of which were are we meeting?04:26
lifelessjogo: the restaurant I think04:27
jogolifeless: which restaurant would that be?04:28
jogoall I know is 2pm Sunday04:28
lifelessheh04:29
lifelessgothicmindfood: I believe we said http://espetus.reachlocal.net/locations/san-francisco/ ?04:30
tchaypolifeless: yerp, there's a NeutronControlPlaneID parameter which is used to create an OS::Neutron::Port04:30
lifelesstchaypo: thats the point its allocated04:30
lifelesstchaypo: ports have ip addresses04:30
*** akuznetsov has joined #tripleo04:31
tchaypoit looks like the NeutronControlPlaneID parameter must be a network ID, I'm trying to figure out where that comes from04:31
lifelessgrep :)04:31
lifelesspicking up C04:31
tchaypolifeless: that link you pasted gothicmindfood makes me drool04:31
tchaypoI love a good churrascaria04:32
tchaypo(I've never encountered a bad one, or at least, not one that I still thougt was bad once I entered the meat-coma)04:32
tchaypoStevenK: http://www.braza.com.au/Braza-Leichhardt04:35
tchaypohttp://upload.wikimedia.org/wikipedia/commons/thumb/6/63/Churrasco_no_espeto.jpg/220px-Churrasco_no_espeto.jpg04:38
tchayposo much meat04:38
jogolifeless: hmmm sounds good04:39
jogosee you sunday04:39
*** eghobo has joined #tripleo04:40
*** CaptTofu_ has joined #tripleo04:44
*** CaptTofu_ has quit IRC04:48
*** ramishra has quit IRC04:53
*** openstackgerrit has quit IRC04:57
*** rakesh_hs2 has joined #tripleo05:02
*** dkehn_ has joined #tripleo05:23
*** dkehn has quit IRC05:26
*** dkehn__ has quit IRC05:27
*** lazy_prince has joined #tripleo05:27
*** dkehn has joined #tripleo05:28
*** dshulyak_ has joined #tripleo05:43
lifelessNg: could you try to move https://jira.hpcloud.net/browse/NET-4062 forward? be good to have no open tickets on the region05:52
*** loki184 has joined #tripleo05:53
lifelessSpamapS: I'd like a review on 100151 if you have time05:54
chuckClifeless: well, I was just about to turn in when I saw your messages.06:05
chuckClifeless: I'm reading through stuff now, but probably won't really get to it until the morning06:06
*** nati_ueno has joined #tripleo06:09
*** gcha has quit IRC06:09
*** eghobo has quit IRC06:11
xuhaiweihi, when building undercloud, i found heat failed in creating 99_undercloudDeployment, does anyone meet the same case?06:12
xuhaiweiI can't see the details why it failed in the log06:12
tchaypolifeless: so as far as I can tell, https://review.openstack.org/#/c/100078/5/toci_devtest.sh isn't going to be sufficient - I'm going to have to pass in allocation start/end points as well06:17
*** rcarrillocruz has joined #tripleo06:18
tchaypohow many params can I be adding to the _*cloud.sh scripts before we decide it's better to do this some other way, like maybe putting it in $TE_DATAFILE?06:18
tchaypomeh. I'm running ahead with lots of params for now, once I know how many we need we can move them06:18
tchaypoStevenK asserts we could already put it in the bm-networks fragment06:18
*** rcarrill` has quit IRC06:20
tchayposince the bot won't do it, I'll tell you manually that I pushed a new version of https://review.openstack.org/#/c/100078/506:23
chuckClifeless: well, never mind.  I guess the urgent stuff is well in hand.06:24
*** rdopieralski has joined #tripleo06:28
*** rdopieralski has quit IRC06:28
*** rdopieralski has joined #tripleo06:29
*** ramishra has joined #tripleo06:30
*** CaptTofu_ has joined #tripleo06:32
*** mrunge has joined #tripleo06:33
*** thegodfather is now known as fabbione06:36
chuckClooks like the revert has gotten a -1 review from gongysh, who is no longer on line.06:37
*** ramishra has quit IRC06:41
*** CaptTofu_ has quit IRC06:43
*** ramishra_ has joined #tripleo06:45
tchaypochuckC: i was about to say that "fortunately chuckc has responded"06:47
tchaypobut then I realised who I was talking to06:47
tchayposo glad I didn't make a fool of myself.06:48
chuckCtchaypo: not sure how fortunate anyway, looks like there's not too much left to do (for the TripleO issue, anyway)06:50
chuckCso, given the -1 review on lifeless's revert, can TripleO wait until tomorrow for the fix to land?06:51
chuckCnot sure it would have gotten enough core approvals anyway….06:51
chuckCin the next 8 hours or so, anyway06:52
tchaypochuckC: yes, we can06:52
tchaypowe have a manual revetrt in toci06:52
tchaypohttps://review.openstack.org/#/c/101349/06:53
StevenKchuckC: It's a -1 from a non-core, I don't think it will impact it at all.06:53
*** jprovazn has joined #tripleo06:53
tchaypothe sooner it lands the sooner we can revert to using undiluted upstream, but until upstream is fixed we can ignore it06:53
chuckCsome other time, I'll make it a point to learn about toci; but it looked like you had gotten things worked out06:54
chuckCI thought that any -1 would block a review.  But I suppose any new change and a couple of core approvals should be enough to erase the -1.06:55
StevenKchuckC: It's not a hard and fast rule.06:56
tchaypoif you check out http://www.nemebean.com/reviewstats/tripleo-30.txt the "disagreements" column on the right shows how often a person's -1 gets ignored and the patch lands anyway06:56
chuckCok.  I guess I haven't seen it in neutron reviews.  Posting new changes erases all prior reviews, though.06:59
StevenKUnless it's a trivial change, yes.06:59
chuckCStevenK: how does the tool define 'trivial'?07:01
StevenKJames uu07:01
tchaypohrmf.07:01
StevenKThat went well.07:01
StevenKWireless dropped out on the dot of 5:00pm. Suspicious.07:01
tchaypo"trivial" i think means "only the commit message change" or "automatic merge"07:01
tchaypos/change/changed/07:02
StevenKIt is marked on the review by the Trivial Rebase Bot or something similar.07:02
chuckCthanks, time to get some sleep….07:03
*** ramishra_ has quit IRC07:04
*** e0ne has joined #tripleo07:05
*** shardy has quit IRC07:12
*** dtantsur|afk is now known as dtantsur07:13
*** jtomasek has joined #tripleo07:13
lifelesstchaypo: so again, which of the two candidate first changes are you working on ?07:17
tchaypothe one where i update toci/devtest_overcloud/heat templates to add a public network range to the overcloud07:19
StevenKI'e been finding gems like:07:19
StevenK>>> help(n.create_network)07:19
*** rpodolyaka has quit IRC07:19
StevenKHelp on function with_params in module neutronclient.v2_0.client:07:19
*** bmahalakshmi has joined #tripleo07:20
tchaypoI've had to rework the toci_ change a bit because i think I'm going to need to provide explicit start/end points for the allocations07:20
lifelesstchaypo: ok so thats not the first change07:20
lifelesstchaypo: thats got dependencies07:20
lifelesstchaypo: defining the networks on the seed and the UC07:20
tchaypohrm.07:20
tchaypooh, because the traffic coming from the undercloud has to get routed out via the undercloud and then out the seed? *guesses*07:21
lifelesstchaypo: you're on a learning curve I think, you should pick something from the work items that has no dependencies and do that.07:21
tchaypos/under/over/07:21
lifelesstchaypo: no, because allocation of IP happens by asking neutron of cloud-107:21
tchaypoyeah, i picked the overcloud because it seemed simpler because it already has bits of this for me to duplicate;  seed and uc don't currently create a vip at all so there's more i need to copy..07:22
lifelesstchaypo: its simple because its layered wll07:22
lifelesstchaypo: you have to start at the bottom of the layer cake though.07:23
*** gcha has joined #tripleo07:23
lifelesstchaypo: I suggest starting with 'allocate a second 'public' VIP from the same network'07:23
lifelesstchaypo: and follow that with 'make haproxy bind appropriate ports on the public VIP07:23
tchaypofrom reading the code it looks to me as though the creation of the vip is happening in the overcloud - it's in the overcloud-source.yaml07:23
lifelesstchaypo: thats right07:24
lifelessallocating on the same network has no dependencies07:24
tchaypobut from talking to steve for a moment, I *think* that that IP iis being provisioned in a network the details of which came from the undercloud neutron?07:24
lifeless(other than having enough IP space available)07:24
lifelessthat is also right07:24
lifelessand appropriate07:24
tchayposo if I want to allocate another VIP in the same range it'll just be adding a new parameter in the heat template that references the same neutroncontrolplaneid?07:24
tchaypoand then doing something to haproxy.. i imagine that will happen in tripleo-image-elements?07:25
lifelessthats a second change07:25
lifelesslets get one passing CI first :)07:25
tchaypo12 hours ago we would have been happy to get *anything* passing ci :p07:25
tchaypobtw, can you scroll up to conversation with chuckc and make sure we didn't lie to him?07:25
lifelessI believe you steered him right07:27
lifelessI haven't read gongysh's comment though07:27
lifelessas you can see though, kyle has +2, which is significant07:28
tchaypoyes.07:28
tchaypowell, that's a patch.07:30
tchaypoand it's trivial enough that ?I'm confident it will pass. I'm less confident that it's useful.07:30
tchaypohttps://review.openstack.org/#/c/101438/07:30
tchaypogerritbot and uvirtbot both quiet07:31
tchaypos/h q/h seem q/07:31
tchaypoStevenK: http://english.stackexchange.com/questions/2119/in-which-cases-is-a-comma-period-placed-inside-or-outside-of-parentheses07:33
lifelesstchaypo: right, except -1 ;)07:36
lifelesstchaypo: sadly I suspect CI wouldn't catch the issue, because ENOTENOUGHCOVERAGE.07:36
tchaypowould you like to leave a comment explaining the issue?07:37
lifelesstchaypo: I did07:37
tchaypoI'm guessing it's relating to reusing ControlFixedIPs?07:37
StevenKlifeless: gertty ated your comment07:37
lifelessyou guess well kemosabe07:37
tchaypocould you check again? I'm guessing maybe you left a draft on revision 1 but then -1d revision 2?07:37
tchaypokemosabe is a reference to that grasshopper movie, right?07:38
tchaypokarate kid or something?07:38
lifelessfixed07:38
tchaypooh, it's lone range07:38
tchaypor07:38
lifelessits terribly culturalist07:38
tchaypooh07:39
tchaypothat list of fixed IPs must be a list of "here's the IP for rabbit, here's the IP for nova" etc then?07:39
lifelessno07:39
*** dshulyak_ has quit IRC07:39
lifelessread the parameter list for the neutron port object07:39
lifelessin the heat docs07:40
lifelessour reviewers let it in with a terrible description07:40
lifelesswe should fix that07:40
*** e0ne has quit IRC07:41
*** ramishra has joined #tripleo07:44
tchaypookay, one down07:44
tchaypoyou say "the paremeter list for the neutron port object" as though you expect me to know where to find that07:45
tchaypoand acutlaly I think I do07:45
tchaypohttp://docs.openstack.org/developer/heat/template_guide/openstack.html#OS::Neutron::Port07:45
tchaypoI'm not sure if I don't need to provide that list; or if I do, but I need to construct a new list containing the new VIP first07:47
lifelessIf you provide that list07:47
lifelessyou're telling neutron what IPs to give the prot.07:47
lifelesswhat will happen if you give the same list of IPs to two different ports?07:48
tchaypoin this case, when i said "that list" i meant "a value for that paremeter"07:48
tchaypoobviously I'm not going to reuse the same list of IPs again, that'll cause fun when it tries to route the same packet to two destinations07:49
tchaypooir more likely the second one it sees wins, or the second one will just fail07:49
lifelessactually it will error when neutron tells you to sod offf07:49
tchaypogood choice on neutron's part07:49
tchaypofail early, fail often07:49
*** athomas has joined #tripleo07:52
*** xuhaiwei has quit IRC07:54
*** jcoufal has joined #tripleo08:00
*** martyntaylor has joined #tripleo08:08
*** IvanBerezovskiy has joined #tripleo08:10
*** ramishra has quit IRC08:14
*** akrivoka has joined #tripleo08:19
*** CaptTofu_ has joined #tripleo08:21
*** lucasagomes has joined #tripleo08:23
*** CaptTofu_ has quit IRC08:26
*** e0ne has joined #tripleo08:26
tchaypoOf course https://review.openstack.org/100078 flailed the seed08:32
tchaypoWhy not? Everything else has failed at some point08:32
tchaypoIt was probably feeling lonely08:33
*** jang has quit IRC08:36
*** e0ne has quit IRC08:36
*** e0ne has joined #tripleo08:39
*** ramishra has joined #tripleo08:40
*** e0ne_ has joined #tripleo08:41
*** jp_at_hp has joined #tripleo08:43
*** e0ne has quit IRC08:45
*** ramishra has quit IRC08:47
*** ramishra has joined #tripleo08:53
*** ramishra has quit IRC08:55
*** ramishra has joined #tripleo08:56
*** ramishra has quit IRC08:56
*** ramishra has joined #tripleo08:57
*** untriaged-bot has joined #tripleo09:00
untriaged-botUntriaged bugs so far:09:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/133232109:00
untriaged-bothttps://bugs.launchpad.net/diskimage-builder/+bug/133245809:00
uvirtbotLaunchpad bug 1332321 in tripleo "Undercloud horizon resource tab returns an error due to missing ceilometer components " [Undecided,New]09:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/133193109:00
uvirtbotLaunchpad bug 1332458 in diskimage-builder "Not able to build armhf or arm64 images on amd64 arch systems." [Undecided,New]09:00
uvirtbotLaunchpad bug 1331931 in tripleo "NeutronControlPlaneID should not be set if building image only" [Undecided,In progress]09:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/133186709:00
uvirtbotLaunchpad bug 1331867 in tripleo "Building undercloud failed" [Undecided,New]09:00
*** untriaged-bot has quit IRC09:00
*** ramishra has quit IRC09:03
*** ramishra has joined #tripleo09:06
*** ramishra has quit IRC09:11
*** ramishra has joined #tripleo09:12
*** nati_ueno has quit IRC09:13
*** yamahata_ has quit IRC09:13
*** ramishra has quit IRC09:17
*** andreaf_ has joined #tripleo09:18
*** e0ne_ has quit IRC09:20
*** e0ne has joined #tripleo09:22
*** e0ne_ has joined #tripleo09:23
*** davidlenwell has quit IRC09:27
*** e0ne has quit IRC09:27
*** rakesh_hs2 has quit IRC09:38
*** rakesh_hs2 has joined #tripleo09:39
*** pelix has joined #tripleo09:42
*** davidlenwell_ has joined #tripleo09:52
*** davidlenwell_ is now known as davidlenwell09:52
*** ChanServ sets mode: +v davidlenwell09:52
*** davidlenwell is now known as davidlenwell_09:52
*** davidlenwell_ is now known as davidlenwell09:52
*** dguerri is now known as _dguerri09:53
*** _dguerri is now known as dguerri09:53
*** CaptTofu_ has joined #tripleo10:09
*** rakesh_hs3 has joined #tripleo10:13
*** rakesh_hs2 has quit IRC10:14
*** CaptTofu_ has quit IRC10:15
*** rakesh_hs4 has joined #tripleo10:46
*** rakesh_hs3 has quit IRC10:47
*** openstackgerrit has joined #tripleo10:56
*** dprince has joined #tripleo11:02
*** lparth has quit IRC11:02
*** yamahata_ has joined #tripleo11:02
*** nosnos has quit IRC11:03
*** dguerri is now known as _dguerri11:06
*** _dguerri is now known as dguerri11:07
openstackgerritLadislav Smola proposed a change to openstack/tripleo-incubator: Test if Undercloud Ceilometer monitors Overcloud  https://review.openstack.org/10147911:07
openstackgerritLadislav Smola proposed a change to openstack/tripleo-incubator: Test if Undercloud Ceilometer monitors Overcloud  https://review.openstack.org/10147911:11
*** jang has joined #tripleo11:13
openstackgerritLadislav Smola proposed a change to openstack/tripleo-incubator: Test if Undercloud Ceilometer monitors Overcloud  https://review.openstack.org/10147911:14
*** martyntaylor has quit IRC11:18
*** CaptTofu_ has joined #tripleo11:20
*** rcarrillocruz has quit IRC11:25
*** rakesh_hs4 has quit IRC11:28
*** dprince has quit IRC11:29
*** rcarrillocruz has joined #tripleo11:29
*** rakesh_hs4 has joined #tripleo11:30
*** rakesh_hs4 has quit IRC11:33
*** martyntaylor has joined #tripleo11:34
*** martyntaylor has quit IRC11:35
*** martyntaylor has joined #tripleo11:35
*** rakesh_hs4 has joined #tripleo11:36
*** ramishra has joined #tripleo11:47
*** lucasagomes is now known as lucas-lunch11:50
*** mrunge has quit IRC11:52
*** ramishra has quit IRC11:55
*** ramishra has joined #tripleo11:59
*** weshay has joined #tripleo12:01
*** dprince has joined #tripleo12:02
*** ccrouch has left #tripleo12:04
*** CaptTofu_ has quit IRC12:04
*** CaptTofu_ has joined #tripleo12:05
*** ramishra has quit IRC12:05
*** ramishra has joined #tripleo12:06
*** jdob has joined #tripleo12:07
*** CaptTofu_ has quit IRC12:09
*** athomas has quit IRC12:21
openstackgerritDan Prince proposed a change to openstack-infra/tripleo-ci: Revert "Try reverting Neutron ovs-agent change"  https://review.openstack.org/10151312:28
*** ramishra has quit IRC12:33
*** ramishra has joined #tripleo12:34
*** jcoufal has quit IRC12:35
*** ramishra has quit IRC12:38
*** dkehn_ is now known as dkehnx12:39
*** yamahata has quit IRC12:39
*** CaptTofu_ has joined #tripleo12:45
openstackgerritA change was merged to openstack/tripleo-incubator: Adding Undercloud Ceilometer config element  https://review.openstack.org/9463712:46
*** tzumainn has joined #tripleo12:47
openstackgerritA change was merged to openstack/tripleo-incubator: Generating of password for SNMPd  https://review.openstack.org/9483812:49
*** fandi has joined #tripleo12:50
*** tzumainn has quit IRC12:50
*** tzumainn has joined #tripleo12:50
openstackgerritStuart McLaren proposed a change to openstack/tripleo-image-elements: os-iptables-stateful: add help text  https://review.openstack.org/10151612:51
regebroHmm. I get this with Fedora: unsupported configuration: SATA is not supported with this QEMU binary12:54
*** lucas-lunch is now known as lucasagomes12:54
regebroGoogling shows that several people have had that problem before, and asked on this channel, but logs doesn't really hint at a solution...12:54
regebroFor me this is when trying to set up devtest.12:55
*** morazi has joined #tripleo12:58
*** ifarkas has quit IRC13:00
*** loki184 has quit IRC13:04
*** bmahalakshmi has quit IRC13:14
rcarrillocruzlifeless: hi, you already left, didn't you?13:15
*** noslzzp has quit IRC13:16
*** noslzzp has joined #tripleo13:16
openstackgerritStuart McLaren proposed a change to openstack/tripleo-image-elements: Add os-iptables-stateful script for managing iptables  https://review.openstack.org/8986013:16
*** noslzzp has quit IRC13:16
*** ccrouch has joined #tripleo13:23
*** jdob has quit IRC13:29
*** jdob has joined #tripleo13:29
yolandahi, finally i got my change passing the tests: https://review.openstack.org/#/c/99924/ , can i get some review?13:31
*** dricco has quit IRC13:34
rdopieralskiyolanda: how many tests did you have to remove ;)13:35
yolandardopieralski, not me, maybe someone removed a few :)13:35
*** ddieterly has joined #tripleo13:38
*** ddieterly has quit IRC13:45
lazy_princefor dib, is there a way to find if the element is being executed by ramdisk-image-create or by disk-image-create..?13:47
gfidenteregebro, I had that before, is misleading13:47
gfidenteI didn't have the KVM module loaded so it couldn't start the VM13:47
regebrogfidente: Aha. Interesting. I wonder why it's not loaded...13:48
gfidenteis it loaded for you? mine had the vt stuff disabled in the BIOS13:48
gfidenteso the kernel didn't load the module13:49
regebrogfidente: Well, I'm running it in a VM, and I've set up the vm to support vm's so it *should* be fine. :-)13:49
regebroBut I can investigate.13:49
gfidenteso if you're running it in a VM, your VM does not have VT capabilities unless you're using nested virtualization13:50
*** jdob_ has joined #tripleo13:50
gfidentewhich confirms the behaviour I've seen, it can't boot the instances supposed to run on top of your VM13:50
regebrogfidente: OK.13:51
*** fandi has quit IRC13:52
*** jdob has quit IRC13:53
gfidenteregebro, to use nested virtualization you've to pass nested=1 to the module on the baremetal and also configure the cpu flags for your vm to expose vmx13:54
gfidenteand yet, your undercloud/overcloud would be pretty slow so I wouldn't recommend it even though in theory it could work13:54
regebrogfidente: Aha. Well, as far as I can tell, the VM should expose VMX.13:55
regebrobut I need to pass in the nedsted=1 then.13:55
ccrouchall: can I just re-request that everyone who is planning to attend the Juno midcycle meetup for Tripleo in RDU, please make sure your name is down on here:13:56
ccrouchhttps://etherpad.openstack.org/p/juno-midcycle-meetup13:56
ccrouchalong with whether you require a hotel13:56
ccrouchi know not everyone is one there, e.g. lifeless13:56
regebrogfidente: But kvm is indeed not loaded.13:56
gfidenteregebro, half-good news then :)13:56
regebrogfidente: yup13:56
*** mkerrin has quit IRC13:56
StevenKccrouch: Myself and tchaypo require a hotel room, it's not clear how to say so on the etherpad.13:57
*** andrearosa has quit IRC13:57
*** andrearosa has joined #tripleo13:58
tchaypoPreferably one each13:58
*** mkerrin has joined #tripleo13:59
ccrouchStevenK: the default is requires a room.13:59
ccrouchso just annotate if you dont need one14:00
ccrouchwe're trying to get a group rate organized14:00
ccrouchwhich is the usual PITA14:00
ccrouchtchaypo: Stevek: if there are more HP guys that you expect to go, but aren't on the list I'd appreciate it if you could poke them14:02
ccrouchnot sure about Mirantis or Rackspace or others, it looks likes just HP and RH so far14:03
*** jrist has quit IRC14:04
*** rakesh_hs4 has quit IRC14:07
StevenKccrouch: The only one I'm reasonably confident about is lifeless.14:07
lxsliis there an agenda somewhere? or is it mostly hackathon?14:09
*** jrist has joined #tripleo14:20
*** jrist has quit IRC14:21
*** jtomasek has quit IRC14:21
*** jrist has joined #tripleo14:21
*** rdopieralski has quit IRC14:24
ccrouchthanks SteveK14:27
ccrouchlxsli: no agenda that I'm aware of yet, lots of hacking i expect14:27
*** regebro has quit IRC14:29
openstackgerritRyan Moore proposed a change to openstack/tripleo-heat-templates: Set vnc to use controller virtual_ip  https://review.openstack.org/10156314:30
*** dkehn_ has joined #tripleo14:31
*** dkehn has quit IRC14:31
*** dkehnx has quit IRC14:32
*** rcarrillocruz has quit IRC14:32
*** yamahata has joined #tripleo14:32
*** dkehn has joined #tripleo14:33
*** rwsu has joined #tripleo14:35
*** rcarrillocruz has joined #tripleo14:37
*** nati_ueno has joined #tripleo14:37
lxsliccrouch: ty14:39
*** jang1 has joined #tripleo14:42
*** dricco has joined #tripleo14:46
*** jomara has quit IRC14:48
*** jomara has joined #tripleo14:49
*** dkehn_ is now known as dkehnx14:50
*** cody-somerville has quit IRC14:51
*** cody-somerville has joined #tripleo14:54
*** cody-somerville has joined #tripleo14:54
*** martyntaylor has quit IRC14:57
*** akrivoka has quit IRC14:59
*** jang1 has quit IRC15:00
*** untriaged-bot has joined #tripleo15:00
untriaged-botUntriaged bugs so far:15:00
untriaged-bothttps://bugs.launchpad.net/diskimage-builder/+bug/133245815:00
uvirtbotLaunchpad bug 1332458 in diskimage-builder "Not able to build armhf or arm64 images on amd64 arch systems." [Undecided,New]15:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/133255415:00
uvirtbotLaunchpad bug 1332554 in tripleo "vnc to nova instances fail" [Undecided,In progress]15:00
untriaged-bothttps://bugs.launchpad.net/diskimage-builder/+bug/133252115:00
uvirtbotLaunchpad bug 1332521 in diskimage-builder "failed to remove `/tmp/image.A2aupYJb/mnt/tmp/in_target.d': Device or resource busy" [Undecided,New]15:00
*** untriaged-bot has quit IRC15:00
*** ifarkas has joined #tripleo15:15
openstackgerritDan Prince proposed a change to openstack/tripleo-incubator: Split ssh keys into files per user.  https://review.openstack.org/10157615:15
*** edmund has joined #tripleo15:16
openstackgerritOm Kumar proposed a change to openstack/diskimage-builder: Support for UBoot.  https://review.openstack.org/10157715:19
*** akuznetsov has quit IRC15:21
openstackgerritOm Kumar proposed a change to openstack/diskimage-builder: Support for UBoot.  https://review.openstack.org/10157715:23
*** akuznetsov has joined #tripleo15:25
openstackgerritPhil Neal proposed a change to openstack/tripleo-image-elements: Add CM signing directory for keystone client  https://review.openstack.org/9652715:26
openstackgerritOm Kumar proposed a change to openstack/diskimage-builder: Element to remove unwanted kernel/initrd.  https://review.openstack.org/10113015:27
*** fandi has joined #tripleo15:30
*** e0ne_ has quit IRC15:41
*** yjiang5_away is now known as yjiang515:45
*** eghobo has joined #tripleo15:52
*** jang1 has joined #tripleo15:52
*** ramishra has joined #tripleo15:58
*** akuznetsov has quit IRC16:04
*** TravT has joined #tripleo16:04
*** akuznetsov has joined #tripleo16:05
*** ramishra has quit IRC16:07
*** akuznetsov has quit IRC16:14
*** akuznetsov has joined #tripleo16:14
*** jogo is now known as flashgordon16:15
openstackgerritPhil Neal proposed a change to openstack/tripleo-incubator: Fix Ceilometer undercloud endpoint registration when UI not specified  https://review.openstack.org/10159616:19
*** ramishra has joined #tripleo16:21
*** akuznetsov has quit IRC16:26
*** akuznetsov has joined #tripleo16:26
*** akuznetsov has quit IRC16:30
*** akuznetsov has joined #tripleo16:30
*** fandi has quit IRC16:32
*** nati_ueno has quit IRC16:36
*** fandi has joined #tripleo16:44
*** akuznets_ has joined #tripleo16:45
*** ramishra has quit IRC16:46
*** akuznetsov has quit IRC16:46
*** ramishra has joined #tripleo16:46
*** yjiang5 is now known as yjiang5_afk16:50
nealphGheRivero:have updated https://review.openstack.org/#/c/96527/ to target ephemeral storage....feel free to take a look.16:53
*** yjiang5_afk is now known as yjiang516:53
*** mestery has joined #tripleo16:53
*** ramishra has quit IRC16:55
*** mestery has quit IRC16:59
*** mestery has joined #tripleo17:00
*** eghobo has quit IRC17:00
*** eghobo has joined #tripleo17:01
*** dtantsur is now known as dtantsur|afk17:02
*** lucasagomes is now known as lucas-dinner17:02
*** ramishra has joined #tripleo17:04
*** jp_at_hp has quit IRC17:04
*** mestery has quit IRC17:04
openstackgerritMichael Tupitsyn proposed a change to openstack/os-cloud-config: Do not create admin user if it exists already  https://review.openstack.org/10071917:05
*** dguerri is now known as _dguerri17:08
*** dshulyak_ has joined #tripleo17:15
*** yolanda has quit IRC17:17
*** nati_ueno has joined #tripleo17:17
openstackgerritOm Kumar proposed a change to openstack/tripleo-image-elements: Adds local storage Boot support on PXE failures.  https://review.openstack.org/7928917:19
*** rcarrill` has joined #tripleo17:19
*** rcarrillocruz has quit IRC17:20
*** ramishra has quit IRC17:25
*** ramishra has joined #tripleo17:25
*** ramishra has quit IRC17:25
*** markmc has quit IRC17:26
*** ramishra has joined #tripleo17:26
*** lazy_prince has quit IRC17:28
*** dshulyak_ has quit IRC17:32
*** ramishra has quit IRC17:38
*** ramishra has joined #tripleo17:39
*** ramishra has quit IRC17:43
*** e0ne has joined #tripleo17:43
*** yamahata has quit IRC17:48
*** jang1 has quit IRC17:51
*** rameshg87 has joined #tripleo17:57
*** e0ne has quit IRC18:00
*** e0ne has joined #tripleo18:01
*** akrivoka has joined #tripleo18:06
*** e0ne has quit IRC18:06
*** e0ne has joined #tripleo18:07
openstackgerritDan Prince proposed a change to openstack/tripleo-incubator: Add --users option to update-admin-ssh-keys  https://review.openstack.org/10161718:08
*** gcha has quit IRC18:09
*** fandi has quit IRC18:11
NobodyCamanyone good with neutron?18:12
*** blamar has joined #tripleo18:12
NobodyCamMy question is: I have a port created with a incorrect host_id I can issue a port-update with the host name but the port still shows as down, is there a way to bring it up?18:12
*** dkehn_ has joined #tripleo18:13
*** dkehn_ has quit IRC18:14
*** dkehn_ has joined #tripleo18:15
*** dkehn has quit IRC18:15
*** dkehnx has quit IRC18:15
*** dkehn has joined #tripleo18:17
*** robsparker has quit IRC18:18
*** ddieterly has joined #tripleo18:21
*** CaptTofu_ has quit IRC18:24
*** CaptTofu_ has joined #tripleo18:24
*** pelix has quit IRC18:24
*** dshulyak_ has joined #tripleo18:27
*** CaptTofu_ has quit IRC18:29
*** robsparker has joined #tripleo18:34
*** akrivoka has quit IRC18:39
*** devananda has quit IRC18:47
*** ddieterly has quit IRC18:48
*** dshulyak_ has quit IRC18:49
NobodyCamshould the overcloud's gateway be the undercloud (192.0.2.2) or the seed's (192.0.2.1)?18:50
*** dkehn__ has joined #tripleo18:53
*** dkehn__ is now known as dkehnx18:53
*** morazi has quit IRC18:54
*** dkehn_ has quit IRC18:54
*** dkehn has quit IRC18:54
*** devananda has joined #tripleo18:55
openstackgerritA change was merged to openstack/tripleo-heat-templates: Add custom public IP support to the undercloud.  https://review.openstack.org/9691718:55
openstackgerritA change was merged to openstack/tripleo-image-elements: Fix the nova.conf mustache template  https://review.openstack.org/9671118:56
openstackgerritA change was merged to openstack/tripleo-incubator: Add quotes to $NEW_JSON to preserve JSON formatting  https://review.openstack.org/9969318:56
*** CaptTofu_ has joined #tripleo18:57
*** dkehn has joined #tripleo18:58
*** dkehnx has quit IRC19:00
*** jang1 has joined #tripleo19:00
*** dkehn_ has joined #tripleo19:01
*** ddieterly has joined #tripleo19:03
Nglifeless: progress on NET-4062, the port had been shut down due to flapping and is now noshut :)19:05
*** rameshg87 has left #tripleo19:06
*** ddieterl_ has joined #tripleo19:08
*** ddieterly has quit IRC19:08
*** morazi has joined #tripleo19:12
*** ddieterl_ has quit IRC19:12
*** ddieterly has joined #tripleo19:12
*** ddieterl_ has joined #tripleo19:17
*** ddieterly has quit IRC19:17
*** dshulyak_ has joined #tripleo19:35
kevinbentondprince: ping re: the neutron bug that hit you19:36
dprincekevinbenton: on the phone, whats up?19:37
kevinbentondprince: i have the correct fix for it here: https://review.openstack.org/#/c/101447/19:38
kevinbentondprince: but i’m not sure how to verify it19:38
kevinbentondprince: i couldn’t find the individual node openvswitch agent logs from the tripleo check19:38
dprincekevinbenton: check experimental19:39
kevinbentondprince: yes, i did that19:39
*** blamar has quit IRC19:40
kevinbentondprince: overcloud precise failed but i can’t tell if it was because of the same problem19:40
dprincekevinbenton: I would want to see the overcloud job passing19:40
dprincekevinbenton: let me look at the logs...19:41
kevinbentondprince: http://logs.openstack.org/47/101447/3/experimental-tripleo/check-tripleo-overcloud-precise/7f1b2bb/19:41
*** mestery_ has joined #tripleo19:44
*** mestery_ is now known as mestery19:45
*** david-lyle has joined #tripleo19:50
*** CaptTofu_ has quit IRC19:50
*** Penick has joined #tripleo19:50
*** CaptTofu_ has joined #tripleo19:51
dprincekevinbenton: one of the Nova computes failed to come up. The seed's nova-compute logs shows some failures here: http://logs.openstack.org/47/101447/3/experimental-tripleo/check-tripleo-overcloud-precise/7f1b2bb/logs/seed_logs/nova-compute.txt.gz19:53
kevinbentondprince: okay. can i just launch another check with check experimental then?19:54
dprincekevinbenton: Lets run it again and see what happens19:54
dprincekevinbenton: The f20 overcloud jobs are passing. I would expect the ubuntu ones to pass as well (those are what we run in check experimental)19:54
kevinbentondprince: ok. just launched one19:55
kevinbentondprince: thanks for checking19:55
*** CaptTofu_ has quit IRC19:55
*** ddieterl_ has quit IRC19:57
*** ddieterly has joined #tripleo19:57
*** ddieterly has quit IRC19:58
*** jdob_ has quit IRC19:59
dprincekevinbenton: I'm running another check experimental job for comparison. If mine doesn't pass then we likely have another problem https://review.openstack.org/#/c/101617/120:00
dprincekevinbenton: I'll check back on these things in a few hours...20:00
kevinbentondprince: okay, sounds good20:00
*** ifarkas has quit IRC20:01
openstackgerritDan Prince proposed a change to openstack/diskimage-builder: dpkg: support pkg-map in bin/install-packages  https://review.openstack.org/9160120:02
openstackgerritDan Prince proposed a change to openstack/diskimage-builder: opensuse: support pkg-map in bin/install-packages  https://review.openstack.org/9160220:02
openstackgerritDan Prince proposed a change to openstack/diskimage-builder: Update base element to make use of pkg-map  https://review.openstack.org/9188020:02
openstackgerritPhil Neal proposed a change to openstack/tripleo-incubator: Fix Ceilometer undercloud endpoint registration when UI not specified  https://review.openstack.org/10159620:05
*** edmund has quit IRC20:09
*** dprince has quit IRC20:10
*** Penick has quit IRC20:11
*** jprovazn has quit IRC20:15
*** Penick has joined #tripleo20:16
*** e0ne has quit IRC20:18
*** e0ne has joined #tripleo20:19
*** akuznets_ has quit IRC20:23
*** mestery has quit IRC20:28
NobodyCamlifeless: happen to be around?20:35
*** ddieterly has joined #tripleo20:36
*** eghobo has quit IRC20:48
*** eghobo has joined #tripleo20:48
*** CaptTofu_ has joined #tripleo20:50
*** weshay has quit IRC20:54
*** CaptTofu_ has quit IRC20:55
*** untriaged-bot has joined #tripleo21:00
untriaged-botUntriaged bugs so far:21:00
untriaged-bothttps://bugs.launchpad.net/diskimage-builder/+bug/133245821:00
uvirtbotLaunchpad bug 1332458 in diskimage-builder "Not able to build armhf or arm64 images on amd64 arch systems." [Undecided,New]21:00
untriaged-bothttps://bugs.launchpad.net/diskimage-builder/+bug/133252121:00
uvirtbotLaunchpad bug 1332521 in diskimage-builder "failed to remove `/tmp/image.A2aupYJb/mnt/tmp/in_target.d': Device or resource busy" [Undecided,New]21:00
*** untriaged-bot has quit IRC21:00
*** ddieterly has quit IRC21:00
*** morazi has quit IRC21:04
*** ddieterly has joined #tripleo21:12
lifelessNobodyCam: ish21:29
lifelessSpamapS: ping21:29
*** eguz has joined #tripleo21:31
*** jang1 has quit IRC21:31
*** eghobo has quit IRC21:34
NobodyCamhey hey lifeless I saw https://review.openstack.org/#/c/96917 landed is theat releated to the issue I'm looking at for EE guys?21:34
NobodyCams/theat/that/21:35
lifelessNobodyCam: I don't know what issue you're looking at21:39
lifelessNobodyCam: but probably not21:39
NobodyCamundercloud not able to deploy overcloud getting tftp timeout21:41
*** dshulyak_ has quit IRC21:41
*** dshulyak_ has joined #tripleo21:42
*** dshulyak_ has quit IRC21:42
lifelessNobodyCam: no21:42
lifelessNobodyCam: they had the wrong ethernet port defined in their config21:42
lifelessso the DHCP traffic wasn't being received21:43
openstackgerritKiall Mac Innes proposed a change to openstack/diskimage-builder: VM element: Enable serial console on Debian  https://review.openstack.org/9617721:43
SpamapSlifeless: pong21:44
NobodyCamok in my case we are getting dhcp address looking at the packets with tcpdump shows next-server and boot file name set correctly21:44
NobodyCambut no tftp packets at all21:44
lifelessNobodyCam: oh21:45
lifelessNobodyCam: who are you working with?21:45
chuckCI've seen the same behavior in my devstack config, restarting ovs l2 agent on compute fixed it 4 me21:45
lifelessSpamapS: next week I'm in sunnyvale (assuming I don't push back the trip cause of this sick)21:45
NobodyCammital ... its jira CORE-70821:45
SpamapSlifeless: fantastic. I am 100% travel constrained until August.21:46
NobodyCamoh21:46
NobodyCamlet me check that21:46
lifelessSpamapS: was thinking it might be useful to do a Q&A session with bristol service team folk on how to approach problems with tripleo (design/arch/whatever)21:46
SpamapSWife at new job.. usual help on summer vacation .. extenuating circumstances unfortunately.21:46
chuckCnever mind, my comment is out of context21:46
SpamapSlifeless: yeah, as far as being in the same TZ, we're good.. should be good to have that conversation.21:47
lifelessSpamapS: so would you have a 0900 slot to do a phone conf with them ?21:47
lifelessSpamapS: yes, I know about ETOOMUCHTRAVEL21:47
SpamapSlifeless: yes, 0900 is entirely doable.21:47
lifelessNobodyCam: yeah, I can't log into work right now, its saturday and lynne is still away21:48
NobodyCamgoing to try what chuckC suggested21:51
lifelessNobodyCam: totally different issue21:51
Lotus907efinova, glance, neutron, ironic, and heat are "services" that run on the seed, undercloud, and overcloud (on controller node?)21:51
lifelesschuckC: that sounds like the issue we reverted yesterday21:52
Lotus907efiis swift also a service that runs in each of these layers?21:52
lifelessLotus907efi: we don't yet run swift in the deployment layers (seed, UC) but we will soon.21:52
chuckClifeless: I thought my issue was more devstack or config related since I've seen it for a while21:52
lifelessNobodyCam: if you're seeing DHCP request * and response*21:52
chuckCthought->think21:52
Lotus907efiare there other services runnin in the deployment layers besides nova, glance, neutron, ironic, and heat?21:52
lifelesschuckC: huh, well - bug of some sort21:53
lifelessLotus907efi: yes lots. Whats the big picture question here.21:53
* Lotus907efi is preparing a training class in the architecture of this and would like to have details correct & complete21:53
Lotus907efiright now I am making architecture drawings that will go on the slides21:54
lifelessLotus907efi: cool; uhmm - you may want to read through the source that builds the images etc to get a comprehensive answer21:54
lifelessbut offhand21:54
Lotus907efihehehehehe  "read the source" :)21:54
lifelessgalera, rabbit, keystone21:54
Lotus907efioh, ok  I forgot all about keystone21:55
Lotus907efithanks21:55
lifelessNobodyCam: PXE will only happen if the server recieves the dhcp response, and didn't get a dhcp response from a different dhcp server w/o pxe metadata.21:57
*** nati_ueno has quit IRC22:01
*** andreaf_ has quit IRC22:04
*** e0ne has quit IRC22:05
*** e0ne has joined #tripleo22:05
*** e0ne has quit IRC22:11
*** e0ne has joined #tripleo22:11
*** yamahata has joined #tripleo22:22
*** e0ne has quit IRC22:23
*** e0ne has joined #tripleo22:23
*** e0ne has quit IRC22:26
*** mestery has joined #tripleo22:31
*** mestery has quit IRC22:31
*** ddieterly has quit IRC22:32
*** mestery has joined #tripleo22:32
*** eguz has quit IRC22:38
*** eghobo has joined #tripleo22:38
*** morganfainberg is now known as morganfainberg_Z22:41
*** david-lyle has quit IRC22:44
*** e0ne has joined #tripleo22:54
*** Penick has quit IRC22:55
*** CaptTofu_ has joined #tripleo23:02
*** e0ne has quit IRC23:07
*** nati_ueno has joined #tripleo23:15
*** mestery has quit IRC23:21
*** e0ne has joined #tripleo23:33
*** TravT has quit IRC23:36
*** nati_ueno has quit IRC23:44
*** morganfainberg_L has joined #tripleo23:49
Lotus907efilifeless: is swift service only running on the overcloud right now?23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!