| *** raunak has quit IRC | 00:00 | |
| rm_work | we have to hotplug user networks | 00:00 |
|---|---|---|
| clarkb | but why? | 00:00 |
| clarkb | just l3 to them | 00:00 |
| rm_work | ok well... with VMs we need to hotplug user networks | 00:00 |
| rm_work | maybe with containers it can be done differently? | 00:00 |
| clarkb | I mean | 00:00 |
| *** zeih has joined #openstack-infra | 00:00 | |
| clarkb | we do lb just fine without that in VMs | 00:00 |
| fungi | nova boot, apt-get install haproxy, configure | 00:01 |
| rm_work | so, when a user adds a member to their LB, we examine the subnet of the VM and hotplug the network its on | 00:01 |
| clarkb | rm_work: right that shouldnt be necessary | 00:01 |
| rm_work | they can add members after create-time that are on arbitrary new subnets | 00:01 |
| *** thorst has joined #openstack-infra | 00:01 | |
| clarkb | alsoeans I cant load balance things not in my cloud tenant? | 00:01 |
| rm_work | we need to plug those networks, no? | 00:01 |
| clarkb | you shpuldnt need to if routing works | 00:02 |
| rm_work | err | 00:02 |
| fungi | granted our use for virtualized load balancers is maybe trivial? | 00:02 |
| rm_work | those subnets are isolated *by design* | 00:02 |
| clarkb | you only need to hotplug an interface if l2 connectivity is rrquired | 00:02 |
| rm_work | I *hope* routing doesn't just magically work between them | 00:02 |
| *** thingee has quit IRC | 00:02 | |
| clarkb | rm_work: you can isolate at l3 with firewalls | 00:02 |
| *** adam_g has joined #openstack-infra | 00:03 | |
| *** adam_g has quit IRC | 00:03 | |
| *** adam_g has joined #openstack-infra | 00:03 | |
| rm_work | ok... well, all i can say is that's not even remotely our design :P | 00:03 |
| rm_work | and i'm not enough of a low level networking expert to really dive too deep into it T_T | 00:03 |
| *** davidlenwell has quit IRC | 00:03 | |
| rm_work | but I trust the people who designed it | 00:04 |
| fungi | yeah, i have to assume octavia is addressing use cases far more complicated than "load balance traffic between our cloud servers" | 00:04 |
| clarkb | My biggest issue with that design is it assumes the load balancer is colocatable with the endpoints | 00:04 |
| rm_work | clarkb: it does not | 00:04 |
| clarkb | it does if it requires l2 connectivity to them | 00:04 |
| rm_work | we can plug a subnet that can route to the internet | 00:04 |
| fungi | because for that, you shouldn't even need a special separate service. just run load balancing software in virtual machines | 00:04 |
| rm_work | and you could put a home-server behind it if you want | 00:04 |
| rm_work | we just need to plug *some* subnet that can route to the member node | 00:05 |
| fungi | which in our case i guess is trivial because they're on the same network | 00:05 |
| *** zeih has quit IRC | 00:05 | |
| *** thingee has joined #openstack-infra | 00:06 | |
| fungi | and because we proxy balance rather than nat (so no triangulation problems or dsr to worry about) | 00:06 |
| rm_work | so if you have some set of servers on subnet E288BDE1-8668-43E6-BE56-7D339B5C0BE4 and then later you add another set on subnet B553EF7E-7F3B-460D-BD66-2F8C91863BD4 we need to have both plugged, right? | 00:07 |
| rm_work | because the Amphora can't just magically route from the first subnet to the other | 00:07 |
| rm_work | maybe I'm missing something :/ | 00:08 |
| fungi | it can if its default router has routes to those | 00:08 |
| *** thorst has quit IRC | 00:08 | |
| *** shashank_hegde has quit IRC | 00:08 | |
| fungi | again, we're probably assuming simple use cases involving everyone on globally-routed networks | 00:08 |
| clarkb | My suggestion would be t o test balancing with a simple.setup to 'internet' | 00:09 |
| *** sdake has quit IRC | 00:09 | |
| clarkb | and separately test your agents network plugiing | 00:09 |
| clarkb | then you can make the slow bits fast and the less slow bits less slow | 00:09 |
| clarkb | since I imagine the slow bits are botting nested VM not hitting neutron api | 00:10 |
| fungi | well, and also building images, but i guess no way around that | 00:11 |
| *** thingee has quit IRC | 00:11 | |
| *** raunak has joined #openstack-infra | 00:12 | |
| *** markvoelker has joined #openstack-infra | 00:12 | |
| *** amitgandhinz has joined #openstack-infra | 00:15 | |
| fungi | assuming the point of the functional job is to test a change to something in the image | 00:15 |
| *** xarses has quit IRC | 00:18 | |
| clarkb | mostly just thinking we can isolate the slow part to boot thing, load balance traffic through it | 00:19 |
| clarkb | then separately test things that talk to neutron and so on | 00:19 |
| *** piet has joined #openstack-infra | 00:19 | |
| *** amitgandhinz has quit IRC | 00:21 | |
| * clarkb wonders what happens if two metworks with overlapping ip.ranges are attached | 00:25 | |
| amrith | that's easy ... bad things | 00:30 |
| amrith | depends a bit on the kind of network hardware you have | 00:31 |
| amrith | some hardware is smart about it, some others let's just say are not so smart about it. | 00:31 |
| clarkb | well specifically with octavia | 00:31 |
| clarkb | pretty sure you can get packets to wrong backend based on who won arp battle | 00:32 |
| amrith | sorry clarkb, I was thinking the question was a more general one. I have ZERO (0.0000) understanding of octavia. Just from the basic networking perspective if there are arp broadcasts and some devices are dumb about them, you can get very weird things | 00:32 |
| *** piet has quit IRC | 00:32 | |
| clarkb | ya typically depends on who wins arp battle | 00:33 |
| *** kzaitsev_mb has quit IRC | 00:33 | |
| amrith | imagine you are a network driver which receives only some of the packets because the upstream device sent some of them to the other person with the same IP :) | 00:33 |
| amrith | also, overlapping IP ranges are never a problem so long as you don't have IP address collisions | 00:33 |
| amrith | (assuming no vlans) | 00:33 |
| clarkb | thats not entirely true | 00:33 |
| clarkb | right | 00:33 |
| amrith | now, when you get into the world where people spoof mac addresses, life gets to be a lot more fun | 00:34 |
| amrith | or if you have failover protocols which attempt to be 'seamless' by setting the mac on the interface on the fly | 00:34 |
| amrith | life gets very very messy ... | 00:34 |
| amrith | let me read the scrollback and see what you are speaking about specifically | 00:35 |
| clarkb | it gets messy the other way when you arp for io using new mac on failover too | 00:35 |
| clarkb | since many network devices will kill ports in a large arp storm | 00:35 |
| * clarkb looks at f5 | 00:35 | |
| amrith | well, that's where smart network hardware listens for arp broadcasts and won't take advertisements seriously if there is a detectable conflict | 00:36 |
| clarkb | this is independent of that | 00:36 |
| clarkb | cisco for example will shut off an interface if the number of arps exceeds a threshold | 00:37 |
| clarkb | regardless of conflicting arp | 00:37 |
| amrith | but they do that based on a very simplistic protocol | 00:37 |
| amrith | (used to) | 00:37 |
| amrith | getting it wrong ~ easy | 00:38 |
| amrith | from the description above, it seemed more like a server on two networks with identical definitions, no? | 00:38 |
| amrith | the issue being described was more of an issue of which network to send stuff out on? | 00:39 |
| amrith | "some set of servers on subnet A then added to subnet B" | 00:39 |
| amrith | unless A and B are overlapping, should be no problem | 00:39 |
| amrith | but the problem would be confined to machines on one or the other local network | 00:39 |
| amrith | not for data routed through those networks | 00:39 |
| amrith | no? | 00:39 |
| clarkb | I was just wondering if octavia would catch this issue properly | 00:39 |
| clarkb | or if it would be at the mercy of arp | 00:40 |
| amrith | hmm, let me see. | 00:40 |
| amrith | disclaimer: I don't know octavia | 00:40 |
| amrith | but what's the situation you are facing? | 00:40 |
| amrith | assume the networks are | 00:40 |
| clarkb | none was a hypothetical | 00:40 |
| amrith | 10.1.2.3/16 and 10.1.2.3/16 | 00:41 |
| amrith | both are valid | 00:41 |
| amrith | assume that a machine gets attached to both | 00:41 |
| amrith | that machine would be very confused ... | 00:41 |
| *** sarob_ has quit IRC | 00:41 | |
| amrith | but you could specify to route traffic even so ... | 00:41 |
| amrith | by specifying the interface name in the routes | 00:41 |
| amrith | oh. hypothetical ... sorry. | 00:41 |
| amrith | what was the scenario you were picturing? | 00:42 |
| clarkb | user says octavia attach subnet 1.2.3.4/24 and 1.2.3.4/25 | 00:42 |
| clarkb | does octavia say hell no? | 00:42 |
| clarkb | if not is the lb service at the mercy of arp for sending packets to 1.2.3.10? | 00:43 |
| amrith | good point, I don't know. | 00:43 |
| amrith | not necessarily | 00:43 |
| clarkb | ehat about 1.2.3.10 talking to 1.2.3.11? | 00:43 |
| amrith | you can override that behavior with routing table entries | 00:43 |
| clarkb | arps shouldnt transit the octavia interfaces so last thi g should be fine | 00:44 |
| amrith | how many 1.2.3.10's and 1.2.3.11's are there and does this one machine do proxy arp and packet forwarding? | 00:44 |
| clarkb | I have no Idea depends on how octavia configues linux to do networking | 00:44 |
| clarkb | that more what I was trying to figure out | 00:44 |
| * amrith is usless in that regard (like a lot of other things :) ) | 00:45 | |
| amrith | cooking comes to mind, also boiling an egg ... | 00:45 |
| amrith | and tox jsut got its act done, off to see what it found ... ta ta ... | 00:46 |
| amrith | have a good weekend clarkb | 00:46 |
| clarkb | you too | 00:46 |
| *** xyang1 has quit IRC | 00:47 | |
| *** denisra has quit IRC | 00:48 | |
| *** denisra has joined #openstack-infra | 00:48 | |
| rm_work | clarkb: sorry had to disappear for a sec | 00:49 |
| *** bhavik has quit IRC | 00:49 | |
| rm_work | clarkb: we use network namespacing and this is something that we do consider | 00:49 |
| rm_work | though I believe we consider it "a stupid user decision", I think we may actually be handling it | 00:50 |
| rm_work | there was definitely a story for it though | 00:50 |
| rm_work | we spent a lot of time discussing that possibility T_T | 00:50 |
| rm_work | (overlapping subnets) | 00:50 |
| * amrith coughs | 00:51 | |
| clarkb | ya its possible with neutron and totally fine to errornwhen it happens | 00:51 |
| amrith | overlapping subnets are not a "stupid user decision" | 00:51 |
| amrith | case in point, VPN's ... | 00:51 |
| amrith | but maybe that's not germane in octavia | 00:51 |
| *** raunak has quit IRC | 00:51 | |
| clarkb | amrith: it generally is... | 00:51 |
| * amrith shuts up | 00:51 | |
| amrith | really clarkb? | 00:51 |
| clarkb | if a /8 isnt big enough for you seriously consider ipv6 | 00:51 |
| amrith | sorry, I may have no choice ... | 00:52 |
| clarkb | yes because it creates confusing issues that are hard to debug | 00:52 |
| amrith | well, hard to debug isn't the same as stupid user decision | 00:52 |
| amrith | my VPN hands out IP's in the exact same range as some DHCP servers ... | 00:52 |
| amrith | I don't get to control either | 00:52 |
| amrith | (well, I can change my VPN's range, but would i?) | 00:53 |
| clarkb | sure or have it use ipv6 | 00:53 |
| amrith | ipv6 only reduces the odds | 00:53 |
| amrith | doesn't guarantee that it will never happen | 00:53 |
| amrith | it's a MUCH BIGGER space | 00:53 |
| amrith | but in huge spaces, we do see collisions | 00:53 |
| clarkb | it reduces it to the point where if it does happen go buy all the lottery tickets | 00:53 |
| clarkb | like heat death of universe should happen sooner | 00:54 |
| amrith | that's what one dinosaur said to the other | 00:54 |
| *** tphummel has quit IRC | 00:54 | |
| amrith | well, I'm all good because I have my reserved ipv6 space ... | 00:54 |
| amrith | and no one dare use it :) | 00:54 |
| rm_work | we're designing to loadbalance on user networks | 00:55 |
| clarkb | you can also deal.with that situation usimg a proxy typically rather than a vpn | 00:55 |
| rm_work | so by definition they control the rannges | 00:55 |
| clarkb | rm_work: is the load balancer shared among tenants though? | 00:55 |
| rm_work | no | 00:55 |
| rm_work | one tenant creates one LB | 00:56 |
| rm_work | for one task | 00:56 |
| rm_work | these are not like shared-appliance F5s | 00:56 |
| rm_work | if you have two websites, make two loadbalancers | 00:56 |
| rm_work | our tests do not really test the neutron plugging operations BTW | 00:57 |
| rm_work | i was just saying *they need to work* | 00:57 |
| rm_work | so I was concerned about whether containers support that | 00:57 |
| clarkb | rm_work: ya I think you can test that outide of a slower more complicated test | 00:58 |
| rm_work | we don't have tests that are just like "let's try to do a neutron plug" | 00:58 |
| clarkb | use a VM for that if necessary then use a container for testing the other bits | 00:58 |
| rm_work | the problem is they'd use different drivers, so we'd be testing a different branch of our code | 00:58 |
| rm_work | (if the plugging is handled differently) | 00:59 |
| clarkb | I dont think you need different driver sthough | 00:59 |
| rm_work | unless all of the plugging is handled exactly the same | 00:59 |
| clarkb | nova boot will plug an interface for container or VM | 00:59 |
| rm_work | right so we have a compute driver | 00:59 |
| rm_work | and ALSO a networking driver | 00:59 |
| clarkb | use that for simple stuff making it faster | 00:59 |
| rm_work | the compute driver might not change | 00:59 |
| clarkb | then test your fancy networking stuff in another test run | 00:59 |
| rm_work | ... | 00:59 |
| rm_work | but we still need to test it | 00:59 |
| clarkb | yes | 00:59 |
| clarkb | you get the same coverage its just partitioned to allow faster testing | 01:00 |
| rm_work | ok, so we'd have some gates set up to run like... a limited set of tests on VMs | 01:00 |
| rm_work | and then most of our functionality tests would run on the containers driver | 01:00 |
| clarkb | yes | 01:00 |
| rm_work | hmmm | 01:00 |
| rm_work | we can look into that | 01:00 |
| rm_work | i'm still just really concerned that "use containers" isn't as easy as you guys think | 01:01 |
| rm_work | I could be wrong | 01:01 |
| rm_work | but we did a lot of work on this, in fact there are patches up for it | 01:01 |
| clarkb | I honestly dont know, but the theory is that it is because nova api | 01:01 |
| rm_work | but it was never completed because there were so many issues | 01:01 |
| *** zeih has joined #openstack-infra | 01:01 | |
| clarkb | and where it isnt is honestly likely a nova bug | 01:01 |
| rm_work | i'll look into how easy it is to actually get nova-lxc or whatever to run in devstack | 01:02 |
| rm_work | if it's easy enough, i can just flip the flavor in the octavia config and see what happens | 01:02 |
| rm_work | i was lead to believe that nova-lxc was *very* unofficial and not supported | 01:02 |
| rm_work | and possibly no longer maintained | 01:03 |
| clarkb | nova lxc is in tree and they have gates for it now iirc | 01:03 |
| *** xarses has joined #openstack-infra | 01:03 | |
| rm_work | nice | 01:03 |
| rm_work | i'll relay this | 01:03 |
| clarkb | mriedem: likely knows more | 01:03 |
| clarkb | but I recall them setting up testing for it and it uses libvirt like kvm | 01:03 |
| *** bhavik has joined #openstack-infra | 01:04 | |
| greghaynes | rm_work: From the dib standpoint - we should be able to make it as easy as you adding an arg -t <container_format> to dib and itll come out as a container | 01:05 |
| greghaynes | rm_work: that code isnt exercised a lot so there may be issues but I am very interested in making it work that way :) | 01:05 |
| *** zeih has quit IRC | 01:06 | |
| *** thorst has joined #openstack-infra | 01:06 | |
| rm_work | ah so it is a different image, but it's easy enough to do | 01:07 |
| rm_work | ok | 01:07 |
| rm_work | when testing i can change the container format that DIB builds easily enough :) | 01:07 |
| greghaynes | yes, I have no idea how image uploading works for that or what format it would want but its pretty trivial to make dib output various formats | 01:07 |
| greghaynes | yea | 01:08 |
| greghaynes | please add it to dib directly :) | 01:08 |
| rm_work | I mean, hopefully it's just changing the args we pass in our devstack/plugin.py | 01:10 |
| *** rwsu has quit IRC | 01:10 | |
| clarkb | its a different format not a different image | 01:13 |
| clarkb | at least infra uses dib that way | 01:13 |
| clarkb | 3 image formats one image | 01:13 |
| greghaynes | Yea, you should be able to do it just as a separate format | 01:14 |
| *** thorst has quit IRC | 01:14 | |
| greghaynes | I mostly am wondering - what image format does something like nova-lxc want? | 01:14 |
| clarkb | my.guess is raw | 01:14 |
| greghaynes | huh | 01:14 |
| greghaynes | seems like it should be a tarball | 01:14 |
| greghaynes | but also then - what would it run when you boot? | 01:14 |
| clarkb | init | 01:14 |
| greghaynes | which is why formats like aci and such exist | 01:14 |
| clarkb | the kernel and bootloader are useless | 01:15 |
| greghaynes | yea, it could... | 01:15 |
| greghaynes | a lot of container formats have a concept of storing the entrypoint as part of the image | 01:15 |
| *** kzaitsev_mb has joined #openstack-infra | 01:16 | |
| *** amitgandhinz has joined #openstack-infra | 01:18 | |
| *** thorst has joined #openstack-infra | 01:21 | |
| *** thorst has quit IRC | 01:21 | |
| *** thorst has joined #openstack-infra | 01:21 | |
| mriedem | clarkb: the nova lxc job is in the experimental queue, uses lvm for the image backend (ephemeral), | 01:22 |
| mriedem | it's not stable by any means | 01:22 |
| clarkb | darn | 01:22 |
| clarkb | but it is running and hopefully someone is workong on it? | 01:22 |
| mriedem | well, it's experimental queue so not really | 01:23 |
| *** amitgandhinz has quit IRC | 01:23 | |
| mriedem | there was someone from rax trying to get it working part time | 01:23 |
| greghaynes | It totally worked once, right? | 01:23 |
| mriedem | but no real progress lately | 01:23 |
| clarkb | experimental queue is for when people are working on something :P | 01:23 |
| mriedem | greghaynes: not really | 01:23 |
| greghaynes | haha | 01:23 |
| mriedem | greghaynes: i mean you can boot a server, | 01:23 |
| mriedem | but a full tempest run running concurrently tends to tip it over | 01:23 |
| mriedem | and then failures start building up | 01:23 |
| mriedem | plus i think he hit some kernel bug on 14.04 | 01:24 |
| mriedem | so he switched for fedora | 01:24 |
| mtreinish | mriedem: I thought they at least got it to a point where tempest could start | 01:24 |
| mriedem | but it wasn't much better | 01:24 |
| greghaynes | hah | 01:24 |
| mriedem | mtreinish: we can definitely run tempest | 01:24 |
| mriedem | just, lots of things snowball with failures | 01:24 |
| mtreinish | right, but for a while you couldn't even get nova to start with it | 01:24 |
| greghaynes | So yea, that sounds like a good indication of what 'make it work in lxc' will be like - a lot of unrelated fixes | 01:24 |
| mtreinish | getting through devstack took time | 01:25 |
| clarkb | unfortunate because all of these on cloud things could benefit from testing on it | 01:25 |
| clarkb | trove sahara etc | 01:25 |
| greghaynes | even ironic possibly | 01:25 |
| *** yamamoto has joined #openstack-infra | 01:25 | |
| *** thorst has quit IRC | 01:26 | |
| mtreinish | greghaynes: virtual baremetal in a container in a vm on a public cloud | 01:26 |
| greghaynes | mtreinish: septupleo | 01:27 |
| mriedem | i think thomas maddox was the guy working on it from rax | 01:27 |
| *** piet has joined #openstack-infra | 01:27 | |
| mriedem | anyway, i'm single parenting tonight and have to get my kid to bed, so as much as i'd like to talk about the state of lxc in nova on a friday night, i have to run | 01:27 |
| openstackgerrit | YAMAMOTO Takashi proposed openstack-infra/project-config: networking-midonet: Add experimental functional job https://review.openstack.org/333236 | 01:28 |
| greghaynes | mriedem: ty for the infos | 01:30 |
| *** psilvad_ has quit IRC | 01:33 | |
| *** yamamoto has quit IRC | 01:36 | |
| *** yamahata has quit IRC | 01:36 | |
| rm_work | hmm ok | 01:36 |
| rm_work | so maybe i did hear correctly about nova-lxc being kinda broken :P | 01:36 |
| *** tqtran has quit IRC | 01:43 | |
| mriedem | greghaynes: here you go https://review.openstack.org/#/c/274792/ | 01:45 |
| *** kzaitsev_mb has quit IRC | 01:45 | |
| *** sdague has quit IRC | 01:46 | |
| openstackgerrit | Spencer Krum proposed openstack-infra/project-config: Move vinz jobs to javascript from python https://review.openstack.org/333634 | 01:51 |
| *** piet has quit IRC | 01:52 | |
| *** salv-orlando has joined #openstack-infra | 01:56 | |
| *** salv-orlando has quit IRC | 01:59 | |
| *** zeih has joined #openstack-infra | 02:02 | |
| krotscheck | fungi: Nope. Sick with the daycare plague. Can take a tweak though | 02:05 |
| *** zeih has quit IRC | 02:07 | |
| openstackgerrit | Spencer Krum proposed openstack-infra/project-config: Move vinz jobs to javascript from python https://review.openstack.org/333634 | 02:10 |
| *** shashank_hegde has joined #openstack-infra | 02:12 | |
| *** shashank_hegde has quit IRC | 02:12 | |
| *** markvoelker has quit IRC | 02:14 | |
| *** yamahata has joined #openstack-infra | 02:15 | |
| *** amitgandhinz has joined #openstack-infra | 02:19 | |
| openstackgerrit | Spencer Krum proposed openstack-infra/vinz: Add Login button https://review.openstack.org/334124 | 02:20 |
| *** amitgandhinz has quit IRC | 02:23 | |
| *** thorst has joined #openstack-infra | 02:24 | |
| *** thorst has quit IRC | 02:31 | |
| *** rhallisey has quit IRC | 02:32 | |
| *** baoli has joined #openstack-infra | 02:32 | |
| *** baoli_ has joined #openstack-infra | 02:34 | |
| *** yamamoto has joined #openstack-infra | 02:36 | |
| *** baoli has quit IRC | 02:37 | |
| *** eharney has quit IRC | 02:43 | |
| *** yamamoto has quit IRC | 02:45 | |
| *** yamamoto has joined #openstack-infra | 02:45 | |
| *** piet has joined #openstack-infra | 02:45 | |
| *** sdake has joined #openstack-infra | 02:46 | |
| *** sdake has quit IRC | 02:50 | |
| *** yamamoto has quit IRC | 02:55 | |
| *** claudiub|2 has quit IRC | 02:56 | |
| *** zeih has joined #openstack-infra | 03:03 | |
| *** salv-orlando has joined #openstack-infra | 03:06 | |
| *** piet has quit IRC | 03:06 | |
| *** zeih has quit IRC | 03:07 | |
| *** salv-orlando has quit IRC | 03:09 | |
| *** mkoderer__ has quit IRC | 03:11 | |
| *** mkoderer__ has joined #openstack-infra | 03:12 | |
| *** rfolco has joined #openstack-infra | 03:14 | |
| *** markvoelker has joined #openstack-infra | 03:14 | |
| *** rfolco has quit IRC | 03:14 | |
| *** kzaitsev_mb has joined #openstack-infra | 03:16 | |
| *** dims has quit IRC | 03:17 | |
| *** rfolco has joined #openstack-infra | 03:18 | |
| *** rfolco has quit IRC | 03:18 | |
| *** markvoelker has quit IRC | 03:19 | |
| *** amitgandhinz has joined #openstack-infra | 03:20 | |
| *** dims has joined #openstack-infra | 03:23 | |
| *** PollyZ has joined #openstack-infra | 03:23 | |
| *** Sukhdev has joined #openstack-infra | 03:24 | |
| *** amitgandhinz has quit IRC | 03:24 | |
| *** tqtran has joined #openstack-infra | 03:28 | |
| *** thorst has joined #openstack-infra | 03:29 | |
| *** thorst has quit IRC | 03:36 | |
| *** erlon has quit IRC | 03:36 | |
| *** asselin has quit IRC | 03:41 | |
| *** kzaitsev_mb has quit IRC | 03:43 | |
| *** jamesmcarthur has joined #openstack-infra | 03:50 | |
| *** PollyZ has quit IRC | 03:50 | |
| *** jamesmcarthur has quit IRC | 03:54 | |
| *** yamamoto has joined #openstack-infra | 03:56 | |
| *** PollyZ has joined #openstack-infra | 03:59 | |
| *** zeih has joined #openstack-infra | 04:04 | |
| *** yamamoto has quit IRC | 04:05 | |
| *** zeih has quit IRC | 04:08 | |
| *** markvoelker has joined #openstack-infra | 04:15 | |
| *** baoli_ has quit IRC | 04:16 | |
| *** gouthamr_ has quit IRC | 04:18 | |
| *** nwkarsten has joined #openstack-infra | 04:19 | |
| *** markvoelker has quit IRC | 04:20 | |
| *** amitgandhinz has joined #openstack-infra | 04:20 | |
| openstackgerrit | greghaynes proposed openstack/diskimage-builder: Make xenial the ubuntu default https://review.openstack.org/334136 | 04:21 |
| *** Nikolay_St has joined #openstack-infra | 04:23 | |
| *** davidlenwell has joined #openstack-infra | 04:23 | |
| *** amitgandhinz has quit IRC | 04:25 | |
| *** Sukhdev has quit IRC | 04:26 | |
| *** PollyZ has quit IRC | 04:26 | |
| *** thingee has joined #openstack-infra | 04:26 | |
| *** Sukhdev has joined #openstack-infra | 04:28 | |
| *** hdaniel has joined #openstack-infra | 04:29 | |
| *** thorst has joined #openstack-infra | 04:33 | |
| *** raunak has joined #openstack-infra | 04:33 | |
| *** julim has joined #openstack-infra | 04:38 | |
| *** hdaniel has quit IRC | 04:39 | |
| *** kzaitsev_mb has joined #openstack-infra | 04:40 | |
| *** thorst has quit IRC | 04:41 | |
| *** hdaniel has joined #openstack-infra | 04:47 | |
| openstackgerrit | Khai Do proposed openstack-infra/puppet-gerrit: Get java_home from parent modules https://review.openstack.org/331523 | 04:52 |
| *** jcoufal has quit IRC | 04:57 | |
| *** raunak has quit IRC | 04:57 | |
| *** shashank_hegde has joined #openstack-infra | 05:00 | |
| *** zeih has joined #openstack-infra | 05:04 | |
| *** shashank_hegde has quit IRC | 05:06 | |
| *** _nadya_ has joined #openstack-infra | 05:07 | |
| *** tphummel has joined #openstack-infra | 05:08 | |
| *** zeih has quit IRC | 05:09 | |
| *** shashank_hegde has joined #openstack-infra | 05:10 | |
| *** kzaitsev_mb has quit IRC | 05:12 | |
| *** nwkarsten has quit IRC | 05:13 | |
| *** nwkarsten has joined #openstack-infra | 05:14 | |
| *** psachin has joined #openstack-infra | 05:16 | |
| *** markvoelker has joined #openstack-infra | 05:16 | |
| *** nwkarsten has quit IRC | 05:19 | |
| *** markvoelker has quit IRC | 05:21 | |
| *** Sukhdev has quit IRC | 05:21 | |
| *** amitgandhinz has joined #openstack-infra | 05:21 | |
| *** Sukhdev has joined #openstack-infra | 05:23 | |
| *** hdaniel has quit IRC | 05:25 | |
| *** amitgandhinz has quit IRC | 05:26 | |
| *** salv-orlando has joined #openstack-infra | 05:26 | |
| *** salv-orlando has quit IRC | 05:29 | |
| *** kzaitsev_mb has joined #openstack-infra | 05:29 | |
| *** bhavik has quit IRC | 05:33 | |
| *** kzaitsev_mb has quit IRC | 05:34 | |
| *** yamamoto has joined #openstack-infra | 05:34 | |
| *** yamamoto has quit IRC | 05:38 | |
| *** yamamoto has joined #openstack-infra | 05:38 | |
| *** thorst has joined #openstack-infra | 05:39 | |
| *** tqtran has quit IRC | 05:40 | |
| *** snarwade has quit IRC | 05:42 | |
| *** yamamoto has quit IRC | 05:44 | |
| *** thorst has quit IRC | 05:45 | |
| *** _nadya_ has quit IRC | 05:46 | |
| *** e0ne has joined #openstack-infra | 05:46 | |
| *** snarwade has joined #openstack-infra | 05:58 | |
| *** e0ne has quit IRC | 05:58 | |
| *** tphummel has quit IRC | 06:01 | |
| *** kzaitsev_mb has joined #openstack-infra | 06:05 | |
| *** zeih has joined #openstack-infra | 06:05 | |
| *** tphummel has joined #openstack-infra | 06:09 | |
| *** zeih has quit IRC | 06:09 | |
| *** markvoelker has joined #openstack-infra | 06:17 | |
| *** linuxmaniac has joined #openstack-infra | 06:17 | |
| *** linuxmaniac_ has quit IRC | 06:17 | |
| *** amitgandhinz has joined #openstack-infra | 06:22 | |
| *** markvoelker has quit IRC | 06:23 | |
| *** amitgandhinz has quit IRC | 06:26 | |
| *** shashank_hegde has quit IRC | 06:27 | |
| *** tphummel has quit IRC | 06:28 | |
| *** kzaitsev_mb has quit IRC | 06:29 | |
| *** salv-orlando has joined #openstack-infra | 06:38 | |
| *** jyuso1 has joined #openstack-infra | 06:40 | |
| *** julim has quit IRC | 06:41 | |
| *** thorst has joined #openstack-infra | 06:43 | |
| *** salv-orlando has quit IRC | 06:45 | |
| *** thorst has quit IRC | 06:51 | |
| *** chem|week-end has quit IRC | 06:57 | |
| *** zeih has joined #openstack-infra | 07:06 | |
| *** zeih has quit IRC | 07:10 | |
| *** markvoelker has joined #openstack-infra | 07:19 | |
| *** amitgandhinz has joined #openstack-infra | 07:23 | |
| *** markvoelker has quit IRC | 07:23 | |
| *** amitgandhinz has quit IRC | 07:28 | |
| *** rwsu has joined #openstack-infra | 07:36 | |
| *** yamamoto has joined #openstack-infra | 07:40 | |
| *** hashar has joined #openstack-infra | 07:43 | |
| *** shashank_hegde has joined #openstack-infra | 07:43 | |
| *** yamamoto has quit IRC | 07:48 | |
| *** thorst has joined #openstack-infra | 07:48 | |
| *** yamahata has quit IRC | 07:49 | |
| *** salv-orlando has joined #openstack-infra | 07:50 | |
| *** _nadya_ has joined #openstack-infra | 07:51 | |
| *** salv-orlando has quit IRC | 07:52 | |
| *** sdake has joined #openstack-infra | 07:54 | |
| *** sdake has quit IRC | 07:55 | |
| *** thorst has quit IRC | 07:56 | |
| *** amotoki has joined #openstack-infra | 07:56 | |
| *** amotoki has quit IRC | 07:57 | |
| *** hashar has quit IRC | 07:57 | |
| *** _nadya_ has quit IRC | 07:59 | |
| *** zzzeek has quit IRC | 08:00 | |
| *** zzzeek has joined #openstack-infra | 08:00 | |
| *** kzaitsev_mb has joined #openstack-infra | 08:09 | |
| *** jyuso1 has quit IRC | 08:12 | |
| *** kushal has joined #openstack-infra | 08:21 | |
| *** Nikolay_St has quit IRC | 08:22 | |
| *** amitgandhinz has joined #openstack-infra | 08:24 | |
| *** kdas_ has joined #openstack-infra | 08:25 | |
| *** amitgandhinz has quit IRC | 08:28 | |
| *** kushal has quit IRC | 08:29 | |
| *** kdas_ is now known as kushal | 08:32 | |
| *** kushal has joined #openstack-infra | 08:32 | |
| *** shashank_hegde has quit IRC | 08:36 | |
| *** Sukhdev has quit IRC | 08:37 | |
| *** thorst has joined #openstack-infra | 08:53 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/puppet-httpd: Update context on vhost for selinux https://review.openstack.org/333873 | 08:55 |
| *** snarwade has quit IRC | 08:59 | |
| *** thorst has quit IRC | 09:01 | |
| *** binbincong has joined #openstack-infra | 09:01 | |
| *** liusheng has quit IRC | 09:02 | |
| *** liusheng has joined #openstack-infra | 09:02 | |
| *** pilgrimstack has joined #openstack-infra | 09:04 | |
| *** flepied1 has quit IRC | 09:07 | |
| *** flepied has joined #openstack-infra | 09:07 | |
| *** kzaitsev_mb has quit IRC | 09:13 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/puppet-httpd: Update context on vhost for selinux https://review.openstack.org/333873 | 09:14 |
| *** snarwade has joined #openstack-infra | 09:15 | |
| *** markvoelker has joined #openstack-infra | 09:21 | |
| *** pilgrimstack has quit IRC | 09:21 | |
| *** kushal has quit IRC | 09:23 | |
| *** amitgandhinz has joined #openstack-infra | 09:24 | |
| *** markvoelker has quit IRC | 09:25 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/puppet-httpd: Update context on vhost for selinux https://review.openstack.org/333873 | 09:28 |
| *** amitgandhinz has quit IRC | 09:29 | |
| *** pilgrimstack has joined #openstack-infra | 09:31 | |
| *** YorikSar_ has joined #openstack-infra | 09:44 | |
| *** xwizard_ has joined #openstack-infra | 09:45 | |
| *** YorikSar has quit IRC | 09:47 | |
| *** xwizard___ has quit IRC | 09:48 | |
| *** pilgrimstack has quit IRC | 09:51 | |
| *** esikachev has joined #openstack-infra | 09:56 | |
| *** thorst has joined #openstack-infra | 09:59 | |
| *** thorst has quit IRC | 10:05 | |
| *** salv-orlando has joined #openstack-infra | 10:10 | |
| *** salv-orlando has quit IRC | 10:13 | |
| *** markvoelker has joined #openstack-infra | 10:21 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/puppet-httpd: Update context on vhost for selinux https://review.openstack.org/333873 | 10:25 |
| *** m3m0_ has quit IRC | 10:25 | |
| *** amitgandhinz has joined #openstack-infra | 10:25 | |
| *** mtreinish has quit IRC | 10:25 | |
| *** markvoelker has quit IRC | 10:26 | |
| *** zhurong has joined #openstack-infra | 10:27 | |
| *** m3m0 has joined #openstack-infra | 10:27 | |
| *** mtreinish has joined #openstack-infra | 10:29 | |
| *** rvba has joined #openstack-infra | 10:29 | |
| *** rvba has quit IRC | 10:29 | |
| *** rvba has joined #openstack-infra | 10:29 | |
| *** amitgandhinz has quit IRC | 10:30 | |
| *** kzaitsev_mb has joined #openstack-infra | 10:31 | |
| *** kzaitsev_mb has quit IRC | 10:41 | |
| *** e0ne has joined #openstack-infra | 10:46 | |
| *** arif-ali has quit IRC | 10:47 | |
| *** jamielennox is now known as jamielennox|away | 10:50 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/puppet-httpd: Update context on vhost for selinux https://review.openstack.org/333873 | 10:50 |
| *** kzaitsev_mb has joined #openstack-infra | 10:50 | |
| *** esikachev has quit IRC | 11:01 | |
| *** thorst has joined #openstack-infra | 11:05 | |
| *** kzaitsev_mb has quit IRC | 11:05 | |
| *** claudiub|2 has joined #openstack-infra | 11:09 | |
| *** thorst has quit IRC | 11:11 | |
| *** esikachev has joined #openstack-infra | 11:12 | |
| *** salv-orlando has joined #openstack-infra | 11:20 | |
| *** markvoelker has joined #openstack-infra | 11:22 | |
| *** esikache1 has joined #openstack-infra | 11:24 | |
| *** YorikSar has joined #openstack-infra | 11:24 | |
| *** xwizard___ has joined #openstack-infra | 11:25 | |
| *** amitgandhinz has joined #openstack-infra | 11:26 | |
| *** markvoelker has quit IRC | 11:26 | |
| *** esikachev has quit IRC | 11:27 | |
| *** salv-orlando has quit IRC | 11:27 | |
| *** YorikSar_ has quit IRC | 11:28 | |
| *** xwizard_ has quit IRC | 11:28 | |
| *** amitgandhinz has quit IRC | 11:30 | |
| *** arif-ali has joined #openstack-infra | 11:32 | |
| *** YorikSar_ has joined #openstack-infra | 11:33 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/puppet-httpd: Update context on vhost for selinux https://review.openstack.org/333873 | 11:33 |
| *** esikachev has joined #openstack-infra | 11:33 | |
| *** xwizard_ has joined #openstack-infra | 11:33 | |
| *** YorikSar has quit IRC | 11:36 | |
| *** esikache1 has quit IRC | 11:36 | |
| *** xwizard___ has quit IRC | 11:37 | |
| *** mrunge has joined #openstack-infra | 11:53 | |
| *** arif-ali has quit IRC | 11:53 | |
| Jeffrey4l_ | question: what the method in noodpool to set up the repo mirror? do we have any project/scripts related to this? | 12:01 |
| *** arif-ali has joined #openstack-infra | 12:02 | |
| *** esikachev has quit IRC | 12:04 | |
| *** rhallisey has joined #openstack-infra | 12:09 | |
| *** thorst has joined #openstack-infra | 12:09 | |
| openstackgerrit | Merged openstack-infra/elastic-recheck: Update query for ceph scheduling bug 1475333 https://review.openstack.org/333963 | 12:12 |
| openstack | bug 1475333 in devstack "gate-tempest-dsvm-full-ceph fails with Insufficient free space for volume creation on host ubuntu@ceph#ceph (requested / avail): 1/0.0" [High,Confirmed] https://launchpad.net/bugs/1475333 | 12:12 |
| *** ericksonsantos has quit IRC | 12:13 | |
| *** ericksonsantos has joined #openstack-infra | 12:14 | |
| *** thorst has quit IRC | 12:16 | |
| *** baoli has joined #openstack-infra | 12:22 | |
| *** markvoelker has joined #openstack-infra | 12:23 | |
| *** HeOS has quit IRC | 12:25 | |
| *** amitgandhinz has joined #openstack-infra | 12:27 | |
| *** markvoelker has quit IRC | 12:28 | |
| *** Nikolay_St has joined #openstack-infra | 12:28 | |
| *** salv-orlando has joined #openstack-infra | 12:29 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/puppet-httpd: Update context on vhost for selinux https://review.openstack.org/333873 | 12:31 |
| *** amitgandhinz has quit IRC | 12:31 | |
| *** thorst has joined #openstack-infra | 12:32 | |
| *** esikachev has joined #openstack-infra | 12:34 | |
| *** thorst_ has joined #openstack-infra | 12:35 | |
| *** salv-orlando has quit IRC | 12:37 | |
| *** thorst has quit IRC | 12:38 | |
| *** baoli has quit IRC | 12:39 | |
| *** thorst_ has quit IRC | 12:40 | |
| *** e0ne has quit IRC | 12:46 | |
| *** dkaiharodsev has joined #openstack-infra | 12:51 | |
| *** dkaiharodsev has left #openstack-infra | 12:55 | |
| *** dkaiharodsev has joined #openstack-infra | 12:56 | |
| *** nwkarsten has joined #openstack-infra | 12:58 | |
| *** dkaiharodsev has quit IRC | 13:10 | |
| *** nwkarsten has quit IRC | 13:15 | |
| *** nwkarsten has joined #openstack-infra | 13:15 | |
| *** nwkarsten has quit IRC | 13:20 | |
| *** markvoelker has joined #openstack-infra | 13:23 | |
| *** binbincong has quit IRC | 13:27 | |
| *** amitgandhinz has joined #openstack-infra | 13:27 | |
| *** binbincong has joined #openstack-infra | 13:27 | |
| *** markvoelker has quit IRC | 13:28 | |
| *** amitgandhinz has quit IRC | 13:32 | |
| *** piet has joined #openstack-infra | 13:34 | |
| *** thorst has joined #openstack-infra | 13:36 | |
| *** salv-orlando has joined #openstack-infra | 13:39 | |
| *** sdake has joined #openstack-infra | 13:39 | |
| *** thorst has quit IRC | 13:44 | |
| *** binbincong has quit IRC | 13:44 | |
| *** esikache1 has joined #openstack-infra | 13:44 | |
| *** rdx_matt_ has joined #openstack-infra | 13:44 | |
| *** sdake has quit IRC | 13:45 | |
| *** madhuvishy_ has joined #openstack-infra | 13:45 | |
| *** jaranovich_ has joined #openstack-infra | 13:45 | |
| *** crinkle_ has joined #openstack-infra | 13:45 | |
| *** mattfarina_ has joined #openstack-infra | 13:46 | |
| *** tdurakov_ has joined #openstack-infra | 13:46 | |
| *** vdrok_ has joined #openstack-infra | 13:46 | |
| *** ashtokolov_ has joined #openstack-infra | 13:46 | |
| *** rpodolyaka_pto_ has joined #openstack-infra | 13:46 | |
| *** larainema_ has joined #openstack-infra | 13:46 | |
| *** isq_ has joined #openstack-infra | 13:46 | |
| *** yonglihe_ has joined #openstack-infra | 13:47 | |
| *** zhiyan_ has joined #openstack-infra | 13:47 | |
| *** ameade_ has joined #openstack-infra | 13:47 | |
| *** xgerman_ has joined #openstack-infra | 13:47 | |
| *** dirk_ has joined #openstack-infra | 13:47 | |
| *** trown has joined #openstack-infra | 13:47 | |
| *** r-mibu_ has joined #openstack-infra | 13:48 | |
| *** sweston_ has joined #openstack-infra | 13:48 | |
| *** irdkehn_ has joined #openstack-infra | 13:48 | |
| *** tristanC_ has joined #openstack-infra | 13:48 | |
| *** pleia2_ has joined #openstack-infra | 13:48 | |
| mriedem | i'm having a hard time figuring out why the ansible inventory script here fails http://logs.openstack.org/62/327762/5/check/gate-tempest-dsvm-multinode-live-migration/5175850/console.html#_2016-06-24_14_54_22_594577 | 13:48 |
| *** TheJulia_ has joined #openstack-infra | 13:48 | |
| *** strigazi_ has joined #openstack-infra | 13:49 | |
| *** kragniz_ has joined #openstack-infra | 13:49 | |
| *** lifeless_ has joined #openstack-infra | 13:49 | |
| mriedem | assuming it's: "fatal: [node]: FAILED! => {"ansible_job_id": "693159264605.1403", "changed": false, "failed": true, "finished": 1, "msg": "Exit code 2", "rc": 2}" | 13:49 |
| *** kozhukalov_ has joined #openstack-infra | 13:49 | |
| mriedem | in | 13:50 |
| mriedem | http://logs.openstack.org/62/327762/5/check/gate-tempest-dsvm-multinode-live-migration/5175850/_zuul_ansible/ansible_log.txt | 13:50 |
| *** dstufft_ has joined #openstack-infra | 13:50 | |
| *** _d34dh0r53_ has joined #openstack-infra | 13:50 | |
| *** ianw_ has joined #openstack-infra | 13:50 | |
| *** jroll|dupe has joined #openstack-infra | 13:50 | |
| *** jroll|dupe has quit IRC | 13:50 | |
| *** jroll|dupe has joined #openstack-infra | 13:50 | |
| *** raildo-a` has joined #openstack-infra | 13:50 | |
| *** NTolerance has joined #openstack-infra | 13:50 | |
| *** kong_ has joined #openstack-infra | 13:50 | |
| *** reed_ has joined #openstack-infra | 13:51 | |
| *** f1ller- has joined #openstack-infra | 13:51 | |
| *** manfred has joined #openstack-infra | 13:51 | |
| *** scottynomad_ has joined #openstack-infra | 13:51 | |
| *** Shalmazar has joined #openstack-infra | 13:51 | |
| *** teran has quit IRC | 13:51 | |
| *** timsim has quit IRC | 13:51 | |
| *** monester has quit IRC | 13:51 | |
| *** cargonza-holiday has quit IRC | 13:51 | |
| *** edwarnicke has quit IRC | 13:51 | |
| *** tdurakov has quit IRC | 13:51 | |
| *** mrunge has quit IRC | 13:51 | |
| *** larainema has quit IRC | 13:51 | |
| *** timello has quit IRC | 13:51 | |
| *** kong has quit IRC | 13:51 | |
| *** r-mibu has quit IRC | 13:51 | |
| *** docaedo has quit IRC | 13:51 | |
| *** mattfarina has quit IRC | 13:51 | |
| *** mhu has quit IRC | 13:51 | |
| *** f1ller has quit IRC | 13:51 | |
| *** madhuvishy has quit IRC | 13:51 | |
| *** fmccrthy has quit IRC | 13:51 | |
| *** sweston has quit IRC | 13:51 | |
| *** esikachev has quit IRC | 13:51 | |
| *** rdx_matt has quit IRC | 13:51 | |
| *** irdkehn has quit IRC | 13:51 | |
| *** jbryce has quit IRC | 13:51 | |
| *** lewo has quit IRC | 13:51 | |
| *** zz_dimtruck has quit IRC | 13:51 | |
| *** trash has quit IRC | 13:51 | |
| *** tristanC has quit IRC | 13:51 | |
| *** raildo-afk has quit IRC | 13:51 | |
| *** pcarver has quit IRC | 13:51 | |
| *** pleia2 has quit IRC | 13:51 | |
| *** sileht has quit IRC | 13:51 | |
| *** jaranovich has quit IRC | 13:51 | |
| *** krotscheck has quit IRC | 13:51 | |
| *** Madasi has quit IRC | 13:51 | |
| *** hughhalf has quit IRC | 13:51 | |
| *** rackertom has quit IRC | 13:51 | |
| *** aspiers has quit IRC | 13:51 | |
| *** d34dh0r53 has quit IRC | 13:51 | |
| *** snarwade has quit IRC | 13:51 | |
| *** odyssey4me has quit IRC | 13:51 | |
| *** yarkot1 has quit IRC | 13:51 | |
| *** scottynomad has quit IRC | 13:51 | |
| *** htruta has quit IRC | 13:51 | |
| *** jroll has quit IRC | 13:51 | |
| *** TheJulia has quit IRC | 13:51 | |
| *** dhellmann has quit IRC | 13:51 | |
| *** lunarlamp has quit IRC | 13:51 | |
| *** gtmanfred has quit IRC | 13:51 | |
| *** zhiyan has quit IRC | 13:51 | |
| *** kozhukalov has quit IRC | 13:51 | |
| *** vdrok has quit IRC | 13:51 | |
| *** rpodolyaka_pto has quit IRC | 13:51 | |
| *** ashtokolov has quit IRC | 13:51 | |
| *** beisner has quit IRC | 13:51 | |
| *** madorn has quit IRC | 13:51 | |
| *** strigazi has quit IRC | 13:51 | |
| *** isq has quit IRC | 13:51 | |
| *** crinkle has quit IRC | 13:51 | |
| *** dtroyer has quit IRC | 13:51 | |
| *** comstud has quit IRC | 13:51 | |
| *** trown|outtypewww has quit IRC | 13:51 | |
| *** reed has quit IRC | 13:51 | |
| *** dirk has quit IRC | 13:51 | |
| *** ameade has quit IRC | 13:51 | |
| *** sneti has quit IRC | 13:51 | |
| *** peterstac has quit IRC | 13:51 | |
| *** kragniz has quit IRC | 13:51 | |
| *** lifeless has quit IRC | 13:51 | |
| *** lennyb has quit IRC | 13:51 | |
| *** nt has quit IRC | 13:51 | |
| *** dstufft has quit IRC | 13:51 | |
| *** yonglihe has quit IRC | 13:51 | |
| *** xgerman has quit IRC | 13:51 | |
| *** cgross has quit IRC | 13:51 | |
| *** afazekas has quit IRC | 13:51 | |
| *** akscram has quit IRC | 13:51 | |
| *** ianw has quit IRC | 13:51 | |
| *** dhellmann has joined #openstack-infra | 13:51 | |
| *** monester has joined #openstack-infra | 13:51 | |
| *** odyssey4me has joined #openstack-infra | 13:51 | |
| *** rackertom has joined #openstack-infra | 13:51 | |
| *** dtroyer has joined #openstack-infra | 13:51 | |
| *** madorn has joined #openstack-infra | 13:51 | |
| *** manfred is now known as gtmanfred | 13:51 | |
| *** NTolerance is now known as nt | 13:51 | |
| *** jbryce_ has joined #openstack-infra | 13:51 | |
| *** dimtruckl has joined #openstack-infra | 13:51 | |
| *** piet has quit IRC | 13:51 | |
| *** ianw_ is now known as ianw | 13:51 | |
| *** larainema_ is now known as larainema | 13:51 | |
| *** scottynomad_ is now known as scottynomad | 13:51 | |
| *** rackertom has quit IRC | 13:51 | |
| *** rackertom has joined #openstack-infra | 13:51 | |
| *** monester has quit IRC | 13:51 | |
| *** monester has joined #openstack-infra | 13:51 | |
| *** krotscheck has joined #openstack-infra | 13:51 | |
| *** lewo has joined #openstack-infra | 13:51 | |
| *** Shalmazar is now known as Madasi | 13:51 | |
| *** tdurakov_ is now known as tdurakov | 13:51 | |
| *** comstud has joined #openstack-infra | 13:51 | |
| *** trash has joined #openstack-infra | 13:51 | |
| *** sileht has joined #openstack-infra | 13:51 | |
| *** salv-orlando has quit IRC | 13:51 | |
| *** dimtruckl is now known as dimtruck | 13:51 | |
| *** jbryce_ is now known as jbryce | 13:51 | |
| *** snarwade has joined #openstack-infra | 13:51 | |
| *** jaranovich_ is now known as jaranovich | 13:51 | |
| *** dstufft_ is now known as dstufft | 13:51 | |
| *** lunarlamp has joined #openstack-infra | 13:51 | |
| *** peterstac has joined #openstack-infra | 13:52 | |
| *** cargonza-holiday has joined #openstack-infra | 13:52 | |
| *** timsim has joined #openstack-infra | 13:52 | |
| *** hughhalf has joined #openstack-infra | 13:52 | |
| *** mrunge has joined #openstack-infra | 13:52 | |
| *** afazekas has joined #openstack-infra | 13:52 | |
| *** jroll|dupe is now known as jroll | 13:52 | |
| *** kong_ is now known as kong | 13:52 | |
| *** sweston_ is now known as sweston | 13:52 | |
| *** docaedo has joined #openstack-infra | 13:52 | |
| *** reed_ is now known as reed | 13:52 | |
| *** mattfarina_ is now known as mattfarina | 13:52 | |
| *** lennyb has joined #openstack-infra | 13:52 | |
| *** aspiers has joined #openstack-infra | 13:52 | |
| *** mhu has joined #openstack-infra | 13:53 | |
| *** yonglihe_ is now known as yonglihe | 13:53 | |
| *** rpodolyaka_pto_ is now known as rpodolyaka_pto | 13:53 | |
| *** vdrok_ is now known as vdrok | 13:53 | |
| *** cgross has joined #openstack-infra | 13:53 | |
| *** madhuvishy_ is now known as madhuvishy | 13:53 | |
| *** kozhukalov_ is now known as kozhukalov | 13:53 | |
| *** akscram has joined #openstack-infra | 13:53 | |
| *** ashtokolov_ is now known as ashtokolov | 13:53 | |
| *** xgerman_ is now known as xgerman | 13:54 | |
| *** dirk_ is now known as dirk | 13:54 | |
| *** tristanC_ is now known as tristanC | 13:54 | |
| *** htruta has joined #openstack-infra | 13:54 | |
| *** edwarnicke has joined #openstack-infra | 13:54 | |
| *** pcarver has joined #openstack-infra | 13:54 | |
| *** beisner has joined #openstack-infra | 13:55 | |
| *** fmccrthy has joined #openstack-infra | 13:55 | |
| *** teran has joined #openstack-infra | 13:56 | |
| openstackgerrit | Khai Do proposed openstack-infra/puppet-gerrit: Allow puppet to configure gerrit commitmessage parameters https://review.openstack.org/316175 | 13:56 |
| *** ameade_ is now known as ameade | 13:56 | |
| *** zhiyan_ is now known as zhiyan | 13:57 | |
| *** timello has joined #openstack-infra | 13:58 | |
| *** amitgandhinz has joined #openstack-infra | 13:58 | |
| *** amitgandhinz has quit IRC | 13:58 | |
| pabelanger | mriedem: no, that is the ansible logging from our zuul-launchers. you'll need to look in your job for the version of ansible you ran. you'll likely need to enable logging in ansible.cfg to dump it to /var/logs on the node, if not already done so. | 13:59 |
| *** rfolco has joined #openstack-infra | 14:00 | |
| *** rfolco has quit IRC | 14:00 | |
| mriedem | pabelanger: i'm not really following | 14:01 |
| mriedem | you mean to get the reason why the inventory thing failed? | 14:01 |
| mriedem | because devstack on both nodes looked fine | 14:01 |
| mriedem | this isn't probably something specific to this job, re: logging, it should probably be done at some basic level for all jobs | 14:02 |
| *** kragniz_ is now known as kragniz | 14:03 | |
| *** yarkot1 has joined #openstack-infra | 14:03 | |
| pabelanger | mriedem: if I understand, you are trying to figure out why ansible is returns exit_code 2, in the first log you linked | 14:04 |
| mriedem | yeah | 14:04 |
| pabelanger | the ansible_log.txt file you then linked, if not from that instances of ansible. It is actually from zl04, ansible-playbook we used to actually start the job. | 14:05 |
| pabelanger | if you are wanting more details about the ansible task in your devstack process, you'll need to enable log_path in ansible.cfg so you can get a debug log from that process: http://docs.ansible.com/ansible/intro_configuration.html#log-path | 14:06 |
| mriedem | hm, and log_path here isn't what i want b/c it's the wrong ansible instance, if i'm following you | 14:07 |
| mriedem | http://logs.openstack.org/62/327762/5/check/gate-tempest-dsvm-multinode-live-migration/5175850/_zuul_ansible/ansible.cfg | 14:07 |
| pabelanger | no | 14:07 |
| pabelanger | anything in _zuul_ansible, comes from our zuul-launcher servers, not the jobs that run in the worker | 14:08 |
| pabelanger | so you are seeing output of how to actually run the job | 14:08 |
| pabelanger | no the data from inside the jog | 14:08 |
| pabelanger | job* | 14:08 |
| openstackgerrit | Khai Do proposed openstack-infra/system-config: Override commit message length validation to 72 chars https://review.openstack.org/316197 | 14:09 |
| pabelanger | so, in your case, anything related to job output will only exist in the logs folder | 14:09 |
| mriedem | yeah, so i guess i have no idea where project-config configures ansible for the jobs themselves | 14:09 |
| mriedem | is that in project-config? | 14:09 |
| pabelanger | in zuul directly | 14:09 |
| pabelanger | let me get some code | 14:10 |
| openstackgerrit | yolanda.robla proposed openstack-infra/puppet-httpd: Update context on vhost for selinux https://review.openstack.org/333873 | 14:10 |
| pabelanger | mriedem: for example, here is how we build up our playbook and post_playbook files: http://git.openstack.org/cgit/openstack-infra/zuul/tree/zuul/launcher/ansiblelaunchserver.py#n1136 | 14:11 |
| pabelanger | you'll also see line 1240 is the setup of ansible.cfg | 14:11 |
| openstackgerrit | Khai Do proposed openstack-infra/system-config: Override commit message length validation to 72 chars https://review.openstack.org/316197 | 14:11 |
| pabelanger | this code, has no affect on devstack version of ansible | 14:11 |
| pabelanger | since zuul-launcher, SSh into the node, they run bash scripts. | 14:12 |
| pabelanger | then* | 14:12 |
| openstackgerrit | Khai Do proposed openstack-infra/system-config: Override commit message length validation to 72 chars https://review.openstack.org/316197 | 14:13 |
| *** antonym has quit IRC | 14:16 | |
| mriedem | pabelanger: ok...so where/what ansible.cfg do i need to modify for ansible that runs on the node (if that's what it does, i've never hacked on ansible) | 14:17 |
| mriedem | i'm assuming zuul-launcher is ssh'ing into the node to run devstack? | 14:17 |
| pabelanger | mriedem: you'll need to add the logic to devstack-vm-gate-wrap.sh along with ansible.cfg | 14:18 |
| mriedem | ok so i'm looking in the wrong repo | 14:19 |
| * mriedem checks d-g | 14:19 | |
| openstackgerrit | Khai Do proposed openstack-infra/puppet-gerrit: Allow puppet to configure gerrit commitmessage parameters https://review.openstack.org/316175 | 14:19 |
| pabelanger | mriedem: right, http://logs.openstack.org/62/327762/5/check/gate-tempest-dsvm-multinode-live-migration/5175850/_zuul_ansible/ansible_log.txt is the output of that process | 14:19 |
| pabelanger | the _zuul_ansible will contain the scripts and playbooks needed to do that too | 14:19 |
| *** antonym has joined #openstack-infra | 14:23 | |
| *** antonym has quit IRC | 14:23 | |
| *** antonym has joined #openstack-infra | 14:23 | |
| *** esikache1 has quit IRC | 14:24 | |
| *** nwkarsten has joined #openstack-infra | 14:24 | |
| *** markvoelker has joined #openstack-infra | 14:24 | |
| mriedem | pabelanger: so i'm guessing i have to create/modify /etc/ansible/ansible.cfg and add/uncomment the log_path option in there? and set that to /var/logs/ansible.log, which is also the default, but not enabled by default it looks like from the docs | 14:26 |
| *** esikache1 has joined #openstack-infra | 14:26 | |
| mriedem | and then just collect that when d-g ends and collects the logs | 14:26 |
| mriedem | i know how to do the log collection in d-g, just not sure about the ansible config | 14:26 |
| mriedem | i see where d-g is installing ansible in a venv though | 14:26 |
| pabelanger | mriedem: Ya, that's basically the steps | 14:27 |
| *** robot is now known as {8 | 14:27 | |
| mriedem | ok, progress! :) | 14:27 |
| mriedem | looks like ansible also logs by default to syslog | 14:28 |
| mriedem | and i see that here http://logs.openstack.org/62/327762/5/check/gate-tempest-dsvm-multinode-live-migration/5175850/logs/syslog.txt.gz | 14:28 |
| mriedem | directing to a specific log file just makes it easier to read | 14:28 |
| *** markvoelker has quit IRC | 14:28 | |
| *** esikache1 has quit IRC | 14:30 | |
| pabelanger | ya, not sure of the top of my head how to use syslog and ansible | 14:30 |
| pabelanger | sure it is possible | 14:30 |
| openstackgerrit | yolanda.robla proposed openstack-infra/puppet-httpd: Update context on vhost for selinux https://review.openstack.org/333873 | 14:32 |
| mriedem | pabelanger: this has been helpful though, thanks for your time | 14:36 |
| pabelanger | mriedem: np | 14:37 |
| *** thorst has joined #openstack-infra | 14:41 | |
| *** HeOS has joined #openstack-infra | 14:48 | |
| *** savihou has joined #openstack-infra | 14:49 | |
| *** thorst has quit IRC | 14:50 | |
| *** gouthamr has joined #openstack-infra | 14:52 | |
| *** gouthamr_ has joined #openstack-infra | 14:54 | |
| *** mhu has quit IRC | 14:56 | |
| *** gouthamr has quit IRC | 14:56 | |
| *** nwkarsten has quit IRC | 15:03 | |
| *** nwkarsten has joined #openstack-infra | 15:03 | |
| *** thorst has joined #openstack-infra | 15:06 | |
| *** mhu has joined #openstack-infra | 15:06 | |
| openstackgerrit | Pedro Alvarez Piedehierro proposed openstack-infra/storyboard: Add support for email threading by story_id https://review.openstack.org/334179 | 15:07 |
| *** zhurong has quit IRC | 15:08 | |
| *** nwkarsten has quit IRC | 15:08 | |
| *** yamahata has joined #openstack-infra | 15:09 | |
| *** nwkarsten has joined #openstack-infra | 15:10 | |
| *** dimtruck is now known as zz_dimtruck | 15:19 | |
| *** nwkarsten has quit IRC | 15:20 | |
| *** nwkarsten has joined #openstack-infra | 15:21 | |
| *** sdague has joined #openstack-infra | 15:21 | |
| *** claudiub|2 has quit IRC | 15:22 | |
| *** esikache1 has joined #openstack-infra | 15:24 | |
| *** markvoelker has joined #openstack-infra | 15:25 | |
| *** jcoufal has joined #openstack-infra | 15:25 | |
| *** nwkarsten has quit IRC | 15:26 | |
| *** jcoufal has quit IRC | 15:28 | |
| *** esikache1 has quit IRC | 15:28 | |
| *** markvoelker has quit IRC | 15:30 | |
| *** thorst has quit IRC | 15:35 | |
| *** thorst has joined #openstack-infra | 15:35 | |
| *** arxcruz has joined #openstack-infra | 15:36 | |
| *** arxcruz has quit IRC | 15:40 | |
| *** sigmavirus24 is now known as sigmavirus24_awa | 15:41 | |
| *** thorst has quit IRC | 15:44 | |
| *** esikache1 has joined #openstack-infra | 15:49 | |
| *** esikache1 has quit IRC | 15:57 | |
| *** esikache1 has joined #openstack-infra | 15:57 | |
| *** salv-orlando has joined #openstack-infra | 15:57 | |
| *** psachin has quit IRC | 16:01 | |
| *** zqfan has quit IRC | 16:03 | |
| *** salv-orlando has quit IRC | 16:04 | |
| *** yamahata has quit IRC | 16:05 | |
| *** esikache1 has quit IRC | 16:12 | |
| *** HeOS has quit IRC | 16:15 | |
| *** zz_dimtruck is now known as dimtruck | 16:18 | |
| *** _nadya_ has joined #openstack-infra | 16:18 | |
| *** markvoelker has joined #openstack-infra | 16:26 | |
| *** markvoelker has quit IRC | 16:30 | |
| *** TheJulia_ has quit IRC | 16:35 | |
| *** thorst has joined #openstack-infra | 16:41 | |
| *** raunak_ has joined #openstack-infra | 16:43 | |
| *** ffrank has joined #openstack-infra | 16:44 | |
| *** thorst has quit IRC | 16:49 | |
| *** thiagop has quit IRC | 16:50 | |
| *** thiagop has joined #openstack-infra | 16:50 | |
| *** raunak_ has quit IRC | 16:50 | |
| *** piet has joined #openstack-infra | 16:55 | |
| *** devanand1 has joined #openstack-infra | 16:58 | |
| *** dimtruck is now known as zz_dimtruck | 17:00 | |
| *** devananda has quit IRC | 17:04 | |
| *** devanand1 is now known as devananda | 17:04 | |
| *** devananda has joined #openstack-infra | 17:04 | |
| *** ffrank has quit IRC | 17:08 | |
| *** denisra_ has joined #openstack-infra | 17:09 | |
| *** piet has quit IRC | 17:10 | |
| *** denisra has quit IRC | 17:10 | |
| *** denisra_ is now known as denisra | 17:11 | |
| *** salv-orlando has joined #openstack-infra | 17:11 | |
| *** raunak_ has joined #openstack-infra | 17:14 | |
| *** claudiub|2 has joined #openstack-infra | 17:17 | |
| *** shashank_hegde has joined #openstack-infra | 17:21 | |
| *** claudiub|2 has quit IRC | 17:22 | |
| *** claudiub|2 has joined #openstack-infra | 17:25 | |
| *** markvoelker has joined #openstack-infra | 17:26 | |
| *** markvoelker has quit IRC | 17:31 | |
| openstackgerrit | Nattaphoom Chaipreecha proposed openstack-dev/hacking: Fix Google Python Style Guide link. https://review.openstack.org/334198 | 17:32 |
| *** piet has joined #openstack-infra | 17:38 | |
| *** lvdongbing has joined #openstack-infra | 17:42 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/puppet-openafs: Enable the openafs-client service https://review.openstack.org/333496 | 17:43 |
| *** Nikolay_St has quit IRC | 17:45 | |
| *** thorst has joined #openstack-infra | 17:47 | |
| *** piet has quit IRC | 17:52 | |
| *** thorst has quit IRC | 17:54 | |
| *** amitgandhinz has joined #openstack-infra | 18:02 | |
| *** amitgandhinz has quit IRC | 18:07 | |
| *** sputnik13 has joined #openstack-infra | 18:14 | |
| *** aviau has quit IRC | 18:18 | |
| *** aviau has joined #openstack-infra | 18:18 | |
| *** yamahata has joined #openstack-infra | 18:31 | |
| *** sputnik13 has quit IRC | 18:36 | |
| *** hongbin has joined #openstack-infra | 18:40 | |
| *** shashank_hegde has quit IRC | 18:41 | |
| *** claudiub|2 has quit IRC | 18:41 | |
| *** _nadya_ has quit IRC | 18:43 | |
| *** thorst has joined #openstack-infra | 18:52 | |
| *** thorst has quit IRC | 18:56 | |
| *** thorst has joined #openstack-infra | 18:56 | |
| *** thorst has quit IRC | 18:56 | |
| *** sputnik13 has joined #openstack-infra | 18:59 | |
| *** pilgrimstack has joined #openstack-infra | 19:00 | |
| *** amitgandhinz has joined #openstack-infra | 19:03 | |
| *** pilgrimstack has quit IRC | 19:08 | |
| *** amitgandhinz has quit IRC | 19:08 | |
| *** tphummel has joined #openstack-infra | 19:09 | |
| *** sdague has quit IRC | 19:09 | |
| *** sputnik13 has quit IRC | 19:21 | |
| *** Jeffrey4l__ has joined #openstack-infra | 19:31 | |
| *** Jeffrey4l_ has quit IRC | 19:35 | |
| *** lvdongbing has quit IRC | 19:37 | |
| *** Nikolay_St has joined #openstack-infra | 19:49 | |
| *** e0ne has joined #openstack-infra | 19:54 | |
| *** njohnston is now known as njohnston|weeken | 19:55 | |
| *** raunak_ has quit IRC | 19:55 | |
| *** njohnston|weeken is now known as njohnston|afk | 19:55 | |
| *** raunak has joined #openstack-infra | 19:55 | |
| *** thorst has joined #openstack-infra | 19:57 | |
| *** tphummel has quit IRC | 19:57 | |
| *** thorst has quit IRC | 20:05 | |
| *** piet has joined #openstack-infra | 20:09 | |
| *** hughhalf has quit IRC | 20:10 | |
| *** yamahata has quit IRC | 20:17 | |
| *** piet has quit IRC | 20:21 | |
| *** jtomasek has quit IRC | 20:21 | |
| *** piet has joined #openstack-infra | 20:26 | |
| *** hughhalf has joined #openstack-infra | 20:30 | |
| *** hongbin has quit IRC | 20:37 | |
| *** hongbin has joined #openstack-infra | 20:38 | |
| *** raunak has quit IRC | 20:40 | |
| *** raunak has joined #openstack-infra | 20:40 | |
| *** sdake has joined #openstack-infra | 20:42 | |
| *** sdake has quit IRC | 20:45 | |
| *** amitgandhinz has joined #openstack-infra | 20:52 | |
| *** amitgandhinz has quit IRC | 20:57 | |
| *** piet has quit IRC | 21:00 | |
| *** shashank_hegde has joined #openstack-infra | 21:02 | |
| *** thorst has joined #openstack-infra | 21:03 | |
| *** piet has joined #openstack-infra | 21:08 | |
| *** thorst has quit IRC | 21:11 | |
| *** HeOS has joined #openstack-infra | 21:13 | |
| *** tphummel has joined #openstack-infra | 21:19 | |
| *** piet has quit IRC | 21:20 | |
| *** Nikolay_St has quit IRC | 21:26 | |
| *** jianghuaw has quit IRC | 21:31 | |
| *** kzaitsev_mb has joined #openstack-infra | 21:52 | |
| *** esikache1 has joined #openstack-infra | 21:54 | |
| *** esikache1 has quit IRC | 21:55 | |
| *** raunak has quit IRC | 21:59 | |
| *** e0ne has quit IRC | 22:03 | |
| *** tphummel has quit IRC | 22:08 | |
| *** thorst has joined #openstack-infra | 22:10 | |
| *** amitgandhinz has joined #openstack-infra | 22:12 | |
| *** thorst has quit IRC | 22:16 | |
| *** amitgandhinz has quit IRC | 22:17 | |
| *** gouthamr_ has quit IRC | 22:22 | |
| *** gouthamr has joined #openstack-infra | 22:24 | |
| *** liusheng has quit IRC | 22:25 | |
| *** liusheng has joined #openstack-infra | 22:25 | |
| *** abregman has joined #openstack-infra | 22:29 | |
| *** yamahata has joined #openstack-infra | 22:30 | |
| *** dkehn_ has quit IRC | 22:42 | |
| *** dkehn__ has joined #openstack-infra | 22:42 | |
| *** kzaitsev_mb has quit IRC | 22:44 | |
| openstackgerrit | Merged openstack-infra/elastic-recheck: Revert "Add e-r query for 1590139" https://review.openstack.org/333527 | 22:46 |
| *** lifeless_ is now known as lifeless | 22:53 | |
| *** psilvad_ has joined #openstack-infra | 22:55 | |
| *** salv-orl_ has joined #openstack-infra | 22:59 | |
| *** yamahata has quit IRC | 22:59 | |
| *** salv-orlando has quit IRC | 23:02 | |
| *** yamahata has joined #openstack-infra | 23:02 | |
| *** psilvad_ has quit IRC | 23:06 | |
| *** andreaf has quit IRC | 23:09 | |
| *** hongbin has quit IRC | 23:10 | |
| *** psilvad_ has joined #openstack-infra | 23:12 | |
| *** thorst has joined #openstack-infra | 23:13 | |
| *** amitgandhinz has joined #openstack-infra | 23:13 | |
| *** andreaf has joined #openstack-infra | 23:13 | |
| *** yamahata has quit IRC | 23:14 | |
| *** amitgandhinz has quit IRC | 23:18 | |
| *** thorst has quit IRC | 23:21 | |
| *** markvoelker has joined #openstack-infra | 23:31 | |
| *** raunak has joined #openstack-infra | 23:34 | |
| *** markvoelker has quit IRC | 23:35 | |
| *** raunak has quit IRC | 23:42 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!