| *** azbiswas has joined #openstack-neutron-ovn | 00:01 | |
| *** azbiswas has quit IRC | 00:07 | |
| *** azbiswas has joined #openstack-neutron-ovn | 01:39 | |
| *** azbiswas has quit IRC | 01:42 | |
| *** azbiswas has joined #openstack-neutron-ovn | 01:51 | |
| *** azbiswas has quit IRC | 01:51 | |
| *** azbiswas has joined #openstack-neutron-ovn | 01:54 | |
| *** azbiswas has quit IRC | 01:54 | |
| *** armax has joined #openstack-neutron-ovn | 02:32 | |
| *** salv-orlando has quit IRC | 02:52 | |
| *** salv-orlando has joined #openstack-neutron-ovn | 02:53 | |
| *** azbiswas has joined #openstack-neutron-ovn | 03:43 | |
| *** salv-orl_ has joined #openstack-neutron-ovn | 04:09 | |
| *** salv-orlando has quit IRC | 04:11 | |
| *** zhouhan has joined #openstack-neutron-ovn | 04:38 | |
| *** armax has quit IRC | 04:58 | |
| *** yamamoto has joined #openstack-neutron-ovn | 05:04 | |
| *** yamamoto has quit IRC | 05:04 | |
| *** yamamoto has joined #openstack-neutron-ovn | 05:05 | |
| *** yamamoto has quit IRC | 05:05 | |
| *** azbiswas has quit IRC | 05:10 | |
| *** yamamoto has joined #openstack-neutron-ovn | 05:18 | |
| *** yamamoto has quit IRC | 05:22 | |
| *** yamamoto has joined #openstack-neutron-ovn | 05:22 | |
| *** yamamoto has quit IRC | 05:25 | |
| *** yamamoto has joined #openstack-neutron-ovn | 06:22 | |
| *** salv-orl_ has quit IRC | 06:31 | |
| *** salv-orlando has joined #openstack-neutron-ovn | 06:31 | |
| *** azbiswas has joined #openstack-neutron-ovn | 07:10 | |
| *** azbiswas has quit IRC | 07:15 | |
| *** yamamoto has quit IRC | 07:37 | |
| *** zhouhan has quit IRC | 07:44 | |
| *** fzdarsky has joined #openstack-neutron-ovn | 08:18 | |
| *** palexster1 is now known as palexster | 08:26 | |
| *** openstackgerrit has quit IRC | 08:32 | |
| *** openstackgerrit has joined #openstack-neutron-ovn | 08:33 | |
| *** yamamoto has joined #openstack-neutron-ovn | 08:37 | |
| *** yamamoto has quit IRC | 08:44 | |
| *** salv-orlando has quit IRC | 08:44 | |
| *** salv-orlando has joined #openstack-neutron-ovn | 08:45 | |
| *** palexster has quit IRC | 09:29 | |
| *** palexster has joined #openstack-neutron-ovn | 09:34 | |
| *** yamamoto has joined #openstack-neutron-ovn | 09:36 | |
| *** yamamoto has quit IRC | 09:38 | |
| *** yamamoto has joined #openstack-neutron-ovn | 09:39 | |
| *** salv-orlando has quit IRC | 09:40 | |
| *** salv-orlando has joined #openstack-neutron-ovn | 09:41 | |
| *** yamamoto has quit IRC | 09:57 | |
| *** yamamoto has joined #openstack-neutron-ovn | 09:57 | |
| openstackgerrit | Numan Siddique proposed openstack/networking-ovn: Add port 'up' and 'down' notification https://review.openstack.org/178826 | 10:05 |
|---|---|---|
| *** yamamoto has quit IRC | 10:10 | |
| *** salv-orl_ has joined #openstack-neutron-ovn | 10:10 | |
| *** yamamoto has joined #openstack-neutron-ovn | 10:11 | |
| *** salv-orlando has quit IRC | 10:12 | |
| *** yamamoto has quit IRC | 11:00 | |
| *** yamamoto has joined #openstack-neutron-ovn | 11:00 | |
| *** yamamoto has quit IRC | 11:03 | |
| *** salv-orl_ has quit IRC | 11:03 | |
| *** yamamoto has joined #openstack-neutron-ovn | 11:03 | |
| *** salv-orlando has joined #openstack-neutron-ovn | 11:04 | |
| *** salv-orlando has quit IRC | 11:05 | |
| *** salv-orlando has joined #openstack-neutron-ovn | 11:05 | |
| *** azbiswas has joined #openstack-neutron-ovn | 11:14 | |
| *** azbiswas has quit IRC | 11:18 | |
| *** palexster has quit IRC | 11:19 | |
| *** yamamoto has quit IRC | 11:26 | |
| *** palexster has joined #openstack-neutron-ovn | 12:02 | |
| *** rtheis has joined #openstack-neutron-ovn | 12:09 | |
| *** rtheis_ has joined #openstack-neutron-ovn | 12:20 | |
| *** rtheis has quit IRC | 12:22 | |
| *** yamamoto has joined #openstack-neutron-ovn | 12:26 | |
| *** HenryG has quit IRC | 12:27 | |
| *** HenryG has joined #openstack-neutron-ovn | 12:27 | |
| *** palexster has quit IRC | 12:28 | |
| openstackgerrit | Babu Shanmugam proposed openstack/networking-ovn: Note: ** WIP ** Enabling qos support through Logical_Port.options https://review.openstack.org/265798 | 12:29 |
| *** palexster has joined #openstack-neutron-ovn | 12:29 | |
| *** yamamoto has quit IRC | 12:30 | |
| *** salv-orlando has quit IRC | 12:55 | |
| *** salv-orlando has joined #openstack-neutron-ovn | 12:56 | |
| *** yamamoto has joined #openstack-neutron-ovn | 13:02 | |
| *** salv-orlando has quit IRC | 13:11 | |
| *** salv-orlando has joined #openstack-neutron-ovn | 13:12 | |
| *** palexster1 has joined #openstack-neutron-ovn | 13:31 | |
| *** palexster has quit IRC | 13:32 | |
| *** palexster1 is now known as palexster | 13:34 | |
| *** yamamoto has quit IRC | 13:42 | |
| *** yamamoto has joined #openstack-neutron-ovn | 14:02 | |
| *** dslev has quit IRC | 14:14 | |
| *** dslevin has quit IRC | 14:15 | |
| openstackgerrit | Numan Siddique proposed openstack/networking-ovn: Add port 'up' and 'down' notification https://review.openstack.org/178826 | 14:19 |
| mestery | Morning folks | 14:26 |
| Sam-I-Am | mornings | 14:26 |
| *** dslevin has joined #openstack-neutron-ovn | 14:30 | |
| russellb | morning | 14:41 |
| Sam-I-Am | russellb: moo. | 14:42 |
| russellb | quack | 14:42 |
| Sam-I-Am | russellb: tested out some ovn on friday | 14:46 |
| mestery | russellb: Hit an issue with ovn-controller spinning on one of the nodes on my 5-node scale testing cluster over the weekend, sent email to dev@openvswitch.org | 14:47 |
| russellb | Sam-I-Am: yeah, i saw you digging into figuring out some of how things accidentally work | 14:47 |
| russellb | mestery: saw your mail, not sure what the issue is | 14:47 |
| russellb | mestery: not sure if FDs thing is a symptom or cause | 14:48 |
| mestery | russellb: Me either to be honest. I have the nodes in that state now in case we want to debug more | 14:48 |
| Sam-I-Am | russellb: yeah, trying to gather my thoughts on what i found | 14:48 |
| Sam-I-Am | where gathering thoughts also involves caffeinating | 14:49 |
| russellb | Sam-I-Am: i'm still on coffee #1 here | 14:49 |
| Sam-I-Am | i'm on coffee None now | 14:49 |
| russellb | Sam-I-Am: l3 agent isn't something we really care about though, so i wouldn't worry *too* much about it | 14:49 |
| russellb | it's just there to help bootstrap the project | 14:49 |
| Sam-I-Am | and somehow trying to install devstack before coffee | 14:49 |
| russellb | mestery: i'm tempted to just increase system-wide fd limit and reset things and see if it happens again | 14:49 |
| mestery | russellb: Yeah, could try that. What's interesting is it only happens on the all-in-one control node, the 4 compute only nodes are fine. | 14:50 |
| mestery | The all-in-one also is running nova compute | 14:50 |
| russellb | actually, that reminds me | 14:51 |
| russellb | those machines are quite large. | 14:51 |
| russellb | and the default devstack config has workers based on the # of CPUs | 14:51 |
| russellb | and each worker pools DB connections | 14:51 |
| mestery | Oh wow | 14:51 |
| mestery | These machines have 56 CPUs ... | 14:51 |
| russellb | i'm not sure what the fd limit is, but just the openstack stuff is going to eat a lot | 14:52 |
| mestery | Yeah, that's right | 14:52 |
| russellb | cat /proc/sys/fs/file-max | 14:53 |
| mestery | 26227039 | 14:53 |
| russellb | sounds like a bug then | 14:54 |
| mestery | :) | 14:54 |
| russellb | lsof | wc -l | 14:55 |
| mestery | [stack@oc-syd01-prod-compute-191 devstack]$ lsof | wc -l | 14:55 |
| mestery | 21467 | 14:55 |
| mestery | [stack@oc-syd01-prod-compute-191 devstack]$ | 14:55 |
| russellb | err, try sudo | 14:56 |
| *** regXboi has joined #openstack-neutron-ovn | 14:57 | |
| mestery | Yikes ... that command isn't returning back | 14:58 |
| mestery | [stack@oc-syd01-prod-compute-191 devstack]$ sudo lsof | wc -l | 14:59 |
| mestery | 134209 | 14:59 |
| mestery | [stack@oc-syd01-prod-compute-191 devstack]$ | 14:59 |
| mestery | Took a bit | 14:59 |
| russellb | this is the all-in-one controller right? | 14:59 |
| mestery | Yup | 14:59 |
| russellb | that number isn't absurd or anything | 14:59 |
| mestery | Yeah | 15:00 |
| russellb | my laptop has more than that | 15:00 |
| mestery | lol | 15:00 |
| russellb | i assume this is on a private network i can't reach? :) | 15:01 |
| mestery | Well ... the machine is publicaly accessible actually | 15:05 |
| mestery | Let me see if I can get you some access ... | 15:05 |
| *** azbiswas has joined #openstack-neutron-ovn | 15:16 | |
| mestery | russellb: Looking at getting you access now, buit if not, anything else you want me to look at? | 15:18 |
| russellb | don't think so ... i was just going to poke around with gdb, maybe enable some more logging, not sure yet | 15:19 |
| mestery | OK | 15:19 |
| russellb | basically i have no idea :) | 15:20 |
| *** azbiswas has quit IRC | 15:20 | |
| mestery | lol, well, we'll figure this out | 15:22 |
| Sam-I-Am | russellb: if i create a provider network via ovn, the instance i boot on it should be able to get dhcp, right? | 15:24 |
| russellb | if you have a DHCP server on the provider network, sure | 15:25 |
| Sam-I-Am | i do | 15:25 |
| Sam-I-Am | it doesnt work | 15:25 |
| russellb | actually | 15:25 |
| Sam-I-Am | dhcp reqs dont make it to the dhcp namespace | 15:25 |
| Sam-I-Am | and for some reason, i cant get anything out of a mirror on br-int | 15:25 |
| Sam-I-Am | not sure if thats a side-effect of ovnbits or not | 15:25 |
| Sam-I-Am | russellb: i found out that the devstack config sets 'external_network_bridge' in the l3 agent config, which effectively drops qg-* on br-ex, therefore making network connectivity into VMs from the host via FIP appear to work, but its not actually using OVN. | 15:38 |
| Sam-I-Am | if i make that option value empty, which is the recommended value now (and option being removed soon), connectivity to the VM doesnt work because qg-* is on br-int, and not really part of OVN | 15:39 |
| Sam-I-Am | also interesting that the neutron net-create for the ext net does not specify any kind of network type... it should be a flat provider network | 15:39 |
| russellb | Sam-I-Am: yeah, i knew that the "external" connectivity was a hack that devstack seemed to make work, but i hadn't looked that deeply | 15:40 |
| Sam-I-Am | i added the provider network according to the ovn docs and 'conventional' cloud networking works - i can access my VM on the privnet via FIP. however, my VM on the provider/pubnet does not get dhcp. | 15:40 |
| russellb | the FIP works right? just a hack | 15:41 |
| Sam-I-Am | yeah, the fip works the "ovn way" if you change the L3 agent config and add br-ex as a provider network in OVN | 15:41 |
| russellb | Sam-I-Am: ok, well that's good progress ... | 15:41 |
| russellb | re: DHCP on the provider net, i'm not sure why it's not working, i guess i hadn't tried that | 15:42 |
| Sam-I-Am | i think the gate should do these things | 15:42 |
| Sam-I-Am | russellb: do you want to poke at this box? | 15:42 |
| russellb | sure | 15:42 |
| Sam-I-Am | i stopped friday after i couldnt get a mirror to work | 15:42 |
| Sam-I-Am | there's a mirror on br-int now, but nothing shows up | 15:42 |
| russellb | i want to try to see where the dhcp request is going | 15:43 |
| russellb | actually...... | 15:43 |
| russellb | is the dhcp server on the same host? | 15:43 |
| Sam-I-Am | yes | 15:43 |
| russellb | nm, that should be fine | 15:43 |
| Sam-I-Am | there's 2 compute nodes (the AIO and a compute node), but the vm doesnt get dhcp regardless of where it boots | 15:43 |
| russellb | i found out late last week that provider networks have a very bad flaw, where 2 ports on the same provider net on the same host can't talk to each other | 15:43 |
| russellb | i need to work on that asap... | 15:44 |
| Sam-I-Am | also curious... the docs mention a kernel version limitation for geneve. the stock ubuntu 14.04 kernel is supposedly too old, yet it seems to work? | 15:44 |
| russellb | Sam-I-Am: because we're loading custom modules from ovs master | 15:44 |
| Sam-I-Am | russellb: hmm, that could be this problem. dhcp and the vm would have ports on the same prov net. | 15:44 |
| russellb | but the dhcp server isn't a part of OVN, right? | 15:45 |
| Sam-I-Am | no, but it has a port somewhere, no? | 15:45 |
| russellb | how is dhcp set up | 15:45 |
| russellb | 2 OVN ports | 15:45 |
| russellb | is what's broken | 15:45 |
| Sam-I-Am | shoot me your pubkey to mkassawara at gmail.com and i'll send you instructions for access | 15:45 |
| *** shettyg has joined #openstack-neutron-ovn | 15:51 | |
| *** salv-orl_ has joined #openstack-neutron-ovn | 16:09 | |
| *** salv-orlando has quit IRC | 16:12 | |
| *** azbiswas has joined #openstack-neutron-ovn | 16:24 | |
| *** tongli has joined #openstack-neutron-ovn | 16:31 | |
| tongli | @russelib, good morning, | 16:32 |
| *** azbiswas_ has joined #openstack-neutron-ovn | 16:34 | |
| tongli | @russelib, I am trying to setup 10 nodes OS cloud using OVN, wonder if there is some kind deployment doc ? | 16:34 |
| tongli | @russelib, also for security, should iptables be used or there is a component working with OVN which does not use iptables? | 16:35 |
| *** azbiswas has quit IRC | 16:37 | |
| Sam-I-Am | tongli: sounds like what i'm doing | 16:38 |
| tongli | @Sam-I-Am, matt, hi, good morning. | 16:39 |
| Sam-I-Am | ovn does not use iptables like the ovs agent does | 16:39 |
| tongli | @Sam-I-Am, I was able to get everything working using VLAN in my own env. in the lab env, since the second nic is on a switch which does not allow VLAN, | 16:40 |
| Sam-I-Am | you can deploy ovn in devstack here - http://docs.openstack.org/developer/networking-ovn/testing.html | 16:40 |
| tongli | @Sam-I-Am, yes, I actually did all that in my own env. wonder if there is more generic doc on a more real production deployment. | 16:41 |
| tongli | like the install guide for the liberty release. | 16:41 |
| Sam-I-Am | no, there isnt | 16:42 |
| Sam-I-Am | because there is no distro support | 16:42 |
| Sam-I-Am | i'm planning to make operator deployment docs when ovn can be installed using regular packages, or at least close to it | 16:42 |
| tongli | @Sam-I-Am, I wonder if anyone actually put the OVN in a more production like env and made sure everything actually works. | 16:43 |
| tongli | I mean using OS liberty release. | 16:43 |
| Sam-I-Am | iirc, it is not available for liberty | 16:43 |
| Sam-I-Am | there are no distro packages for the ovn repo | 16:44 |
| tongli | @Sam-I-Am, so if I am trying to deploy Liberty on my 10 nodes cloud, using OVN may not be a smart thing to do? | 16:45 |
| Sam-I-Am | i dont think it'll work, at least not easily | 16:45 |
| tongli | @Sam-I-Am, thanks for the info, curious how does it work in a devstack env? | 16:47 |
| Sam-I-Am | it uses openstack source, ovn source, and openvswitch source | 16:47 |
| Sam-I-Am | key is being able to use source | 16:47 |
| Sam-I-Am | mixing packages + deb packages of python libs + python libs via pip usually gets messy | 16:48 |
| tongli | @Sam-I-Am, oh, I was not aware of that. should look at it more closely when I run Kyle's vagrant scripts. | 16:48 |
| Sam-I-Am | take a look at the devstack plugin.sh in the ovn repo | 16:48 |
| tongli | @Sam-I-Am, thanks. | 16:50 |
| tongli | @Sam-I-Am, you mentioned earlier that it is like something you are doing. so you are creating a doc for doing that or you are trying to get OVN deployed? | 16:53 |
| *** yamamoto has quit IRC | 16:53 | |
| Sam-I-Am | tongli: i have ovn deployed on 2 nodes and ran into problems | 16:54 |
| Sam-I-Am | waiting on russellb to determine whats going on and then discuss if we can gate this more accurately | 16:54 |
| tongli | @Sam-I-Am, i c. thanks again for the info. | 16:54 |
| Sam-I-Am | if you have scrollback in here, you can see some of what i found | 16:55 |
| *** armax has joined #openstack-neutron-ovn | 16:55 | |
| *** tongli_ has joined #openstack-neutron-ovn | 16:59 | |
| *** tongli has quit IRC | 17:02 | |
| *** tongli_ has quit IRC | 17:06 | |
| *** yamamoto has joined #openstack-neutron-ovn | 17:13 | |
| *** yamamoto has quit IRC | 17:13 | |
| *** yamamoto has joined #openstack-neutron-ovn | 18:13 | |
| *** azbiswas_ has quit IRC | 18:13 | |
| *** yamamoto has quit IRC | 18:21 | |
| *** chandrav has joined #openstack-neutron-ovn | 18:37 | |
| *** flaviof has joined #openstack-neutron-ovn | 18:40 | |
| *** tongli has joined #openstack-neutron-ovn | 18:57 | |
| *** gangil has joined #openstack-neutron-ovn | 19:02 | |
| *** gangil has joined #openstack-neutron-ovn | 19:02 | |
| *** azbiswas has joined #openstack-neutron-ovn | 19:39 | |
| openstackgerrit | Han Zhou proposed openstack/networking-ovn: Use elevated context when updating ACLs for remote-group-id https://review.openstack.org/266009 | 19:49 |
| *** chandrav has quit IRC | 19:53 | |
| *** gangil has quit IRC | 20:04 | |
| *** markmcclain has quit IRC | 20:12 | |
| *** markmcclain has joined #openstack-neutron-ovn | 20:12 | |
| *** aginwala has joined #openstack-neutron-ovn | 20:23 | |
| *** dslevin has quit IRC | 20:25 | |
| *** chandrav has joined #openstack-neutron-ovn | 20:28 | |
| Sam-I-Am | russellb: whats your thought on making the ovn gate more realistic? | 20:29 |
| russellb | yes. | 20:29 |
| Sam-I-Am | as in, get rid of the br-int external connectivity bypass | 20:30 |
| Sam-I-Am | probably not an issue once the conv l3 agent goes away, but not sure when that will be done. | 20:30 |
| * russellb nods | 20:30 | |
| russellb | i'm all for it | 20:30 |
| russellb | anything to test more of OVN, more realistic | 20:30 |
| russellb | etc .. | 20:30 |
| Sam-I-Am | whats scary about neutron gating in general... the hack is everywhere :/ | 20:31 |
| Sam-I-Am | unless you call q_use_providernet* | 20:31 |
| Sam-I-Am | or set... | 20:31 |
| Sam-I-Am | if you take a look at the public network in a non-ovn gate job, its marked as vxlan... which is unpossible. | 20:31 |
| Sam-I-Am | meh, dozens of coffee shops near me and i pick the one with wi-fail | 20:35 |
| Sam-I-Am | also now rolling into my 18th hour of listening to david bowie | 20:37 |
| Sam-I-Am | so life is good :) | 20:37 |
| *** aginwala has quit IRC | 20:41 | |
| *** fzdarsky has quit IRC | 20:42 | |
| *** azbiswas has quit IRC | 20:48 | |
| *** chandrav has quit IRC | 20:48 | |
| *** markmcclain has quit IRC | 20:50 | |
| *** chandrav has joined #openstack-neutron-ovn | 20:51 | |
| *** azbiswas has joined #openstack-neutron-ovn | 20:51 | |
| *** markmcclain has joined #openstack-neutron-ovn | 20:52 | |
| *** aginwala has joined #openstack-neutron-ovn | 20:57 | |
| *** gangil has joined #openstack-neutron-ovn | 21:03 | |
| *** gangil has joined #openstack-neutron-ovn | 21:03 | |
| *** aginwala has quit IRC | 21:17 | |
| *** dslev has joined #openstack-neutron-ovn | 21:28 | |
| *** aginwala has joined #openstack-neutron-ovn | 21:42 | |
| -openstackstatus- NOTICE: Gerrit is restarting to resolve java memory issues | 22:05 | |
| *** salv-orlando has joined #openstack-neutron-ovn | 22:09 | |
| *** salv-orl_ has quit IRC | 22:12 | |
| *** rtheis_ has quit IRC | 22:17 | |
| *** rtheis has joined #openstack-neutron-ovn | 22:17 | |
| *** rtheis has quit IRC | 22:18 | |
| *** flaviof has quit IRC | 22:24 | |
| *** aginwala has quit IRC | 22:25 | |
| *** aginwala has joined #openstack-neutron-ovn | 22:28 | |
| *** regXboi has quit IRC | 22:28 | |
| *** markmcclain has quit IRC | 22:29 | |
| *** tongli has quit IRC | 22:31 | |
| *** tongli has joined #openstack-neutron-ovn | 22:31 | |
| *** tongli has quit IRC | 22:35 | |
| *** markmcclain has joined #openstack-neutron-ovn | 22:41 | |
| *** flaviof has joined #openstack-neutron-ovn | 22:50 | |
| *** shettyg has quit IRC | 22:54 | |
| *** dslev has quit IRC | 23:00 | |
| *** gangil has quit IRC | 23:04 | |
| *** gangil has joined #openstack-neutron-ovn | 23:07 | |
| *** gangil has joined #openstack-neutron-ovn | 23:07 | |
| *** markmcclain has quit IRC | 23:26 | |
| *** markmcclain has joined #openstack-neutron-ovn | 23:26 | |
| *** aginwala has quit IRC | 23:31 | |
| *** aginwala has joined #openstack-neutron-ovn | 23:35 | |
| *** chandrav has quit IRC | 23:49 | |
| *** chandrav has joined #openstack-neutron-ovn | 23:51 | |
| *** azbiswas has quit IRC | 23:51 | |
| *** chandrav has quit IRC | 23:52 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!