Thursday, 2015-04-30

*** blp has quit IRC00:16
*** gsagie_ has joined #openstack-neutron-ovn04:42
gsagie_russellb, otherwiseguy: here by any chance?04:43
otherwiseguygsagie_: I am.04:43
gsagie_otherwiseguy : i commented on the transaction reducing task, i first did it the same way you mentioned but russell raised a concern about the fact it does un-needed searches for rows04:44
gsagie_for each command04:44
otherwiseguygsagie_: Hmm.04:45
gsagie_I am usually against doing "performance optimizations" before actually seeing problems, but since we almost always set all fields and since there aren't that many fields i thought it wouldn't look so bad that way04:47
otherwiseguygsagie_: In this case, I'd probably do the optimization when we find its the bottleneck. I remember actually getting rid of the lookups in general by adding indexed lookups, and even with n=4000 it didn't end up saving a noticeable amount of time.04:47
otherwiseguySo I just removed the optimization since it made things horribly complex. In this case, I could go either way.04:48
gsagie_it does reduce a lot of code as well04:49
gsagie_not a lot, but some code :)04:49
*** ajo has joined #openstack-neutron-ovn04:49
otherwiseguyJust don't want to get into a situation where we're scattering arg=None, arg=Something, arg=None, etc. to set just one thing, but not another. I've got to go to bed now, but I'll look again in the morning. Might be some generalized Set()/Get()/Find() kinds of functions we could add like the neutron ovsdb code has as well.04:54
gsagie_ok, good night04:56
otherwiseguygsagie_: good night. sorry for the back and forth suggestions on the patch. i hate it when that happens.04:58
gsagie_its ok np, i actually liked the approach you suggested first, but i think that with python this actually doesn't look so bad, because what you just said will translate to just this =>   set_port(arg=something)05:01
gsagie_we have the default values, so anything that is not set will not be05:01
gsagie_i will just convert the None to something like "UNSET" (with value of -1 for example)05:03
gsagie_but np, doing with transactions is fine too as far as i am concerned05:03
*** openstackgerrit has quit IRC05:22
*** openstackgerrit has joined #openstack-neutron-ovn05:22
*** gsagie_ has quit IRC05:37
openstackgerritGal Sagie proposed stackforge/networking-ovn: Reduce the number of OVSDB transactions
openstackgerritGal Sagie proposed stackforge/networking-ovn: Reduce the number of OVSDB transactions
*** ajo has quit IRC08:34
*** ajo_ has joined #openstack-neutron-ovn08:34
*** openstackgerrit_ has joined #openstack-neutron-ovn10:16
*** openstackgerrit_ has quit IRC10:31
russellbotherwiseguy: gsagie i'm not sure i understand why all of the lookups would be needed.  in the create case, we just need to keep the created row reference around and pass it to each command12:34
openstackgerritGal Sagie proposed stackforge/networking-ovn: Reduce Transactions Usage in Mech driver
openstackgerritGal Sagie proposed stackforge/networking-ovn: Reduce the number of OVSDB transactions
*** ajo_ is now known as ajo13:40
otherwiseguyrussellb: gsagie: one thing I was thinking is do it like gsagie is doing it, but instead of enumerating the arguments (which would have to be updated if the schema changed), instead have Add/Update Commands that took **columns and just loop through columns.items doing setarg on them. You'd probably still need special commands if you just wanted to modify a column that was a set or a dict instead of completely13:54
otherwiseguy overwriting it.13:54
otherwiseguyMagic value of UNSET = -1 thing might be dangerous. If we were going with the magic value thing it should just be an object() using 'is' as a comparison.13:56
otherwiseguyor a tuple13:56
otherwiseguysame thing really, just have 'is' match on object id.13:57
russellbotherwiseguy: did you see the alternative patch?13:57
otherwiseguyrussellb: did that address your concern about redundant lookups?14:00
russellbbut it doesn't make it any worse than it already is today14:06
russellbdoing 1 lookup seems pretty easy to fix14:06
otherwiseguyI like the general style of using multi-op transactions. The reason there were so many commands in the neutron version was trying to support ovs-vsctl and ovs_lib and IDL all at the same time. Since ovs-vsctl did add-port blah -- set Port blah col=val, etc. And ovs_lib kind of did all kinds of weird things, it ended up pretty piecemeal. Being able to create objects with their columns filled seems like a decent14:07
otherwiseguy abstraction to be able to use.14:07
otherwiseguyAnd with **columns it doesn't get ugly with all of the if:if:if stuff.14:08
otherwiseguyNot sure if passing around rows in Commands would work very well with transactions that can be retried, etc. Might have a reference to a row that was deleted or something?14:09
otherwiseguyCommands generally have to be able to hold all state they need to be able to be run, and re-run if something changes. That generally means re-doing the lookup when retrying.14:10
otherwiseguyI'm certainly ok with the multi-op transaction patch as it is--it's what I would have done. Just got to thinking a bit about why I would have done it and started thinking about gsagie's implementation as I was falling asleep and ideas kindof swirled around as I drifted off to sleep. :p14:11
openstackgerritRussell Bryant proposed stackforge/networking-ovn: local.conf.sample: Enable the rest of OpenStack
* russellb going to stand up some VMs and see what breaks ...14:26
russellbif anyone else wants to try, you just need to update your ovs tree to ...14:30
russellb$ git add remote
russellb$ git fetch blp14:30
russellb$ git checkout blp/ovn14:30
russellbwell, it doesn't compile.14:33
russellbor he didn't push his latest patches to the branch14:34
* russellb looks into it14:34
*** gsagie_ has joined #openstack-neutron-ovn14:44
gsagie_so, i guess we going with the transactions :)14:47
otherwiseguygsagie: russellb: was thinking something like might be a solution to unnecessary lookups while somewhat sane API.14:47
russellbotherwiseguy: yeah, that's one way ... I was thinking just return the row from the first command that creates it, and pass it in to the rest of the commands14:49
russellbinstead of the neutron port id14:49
otherwiseguyrussellb: see above, I don't think that works if we have to retry the transaction due to a TRY_AGAIN. The row may be invalid, right?14:49
russellbotherwiseguy: oh, right14:50
russellbcarry on14:50
russellbsorry missed that14:50
otherwiseguyrussellb: it's complicated enough, I'm not 100% sure that's right, but it *feels* right so I'm going with it. :p14:50
otherwiseguyAlso not sure what I gist'd actually works either. :p14:51
otherwiseguyBut it was an idea and I had to push it out of my head somewhere. :p14:51
gsagie_markmcclain: don't have any percentage yet..;) but you might want to add Akanda here :
russellbotherwiseguy: what you propose is closer to gsagie's first patch14:52
russellbat this point i'm not sure i care much :)14:52
russellbwhatever you guys think14:52
russellbas long as we do things in 1 transaction14:52
russellband cutting down on lookups would be ideal14:52
russellbso maybe the 2nd approach isn't so good14:52
russellbi say i don't care, but i clearly do14:53
russellbi just don't feel good :(14:53
russellbfinally going to dr today14:53
otherwiseguyrussellb: yeah, just with some syntactic differences. Basically a "don't have to modify stuff just because the schema added a column"14:53
gsagie_russellb : again?14:53
russellbbeen sick for almost 2 weeks now14:53
gsagie_otherwiseguy : i like the generic of your solution, but now the user needs to know what options he can set14:54
otherwiseguyrussellb: perhaps you have been working too hard. maybe you should add a third earlier trip to the caribbean.14:54
gsagie_so it looks good, but i think complicate a bit14:54
otherwiseguygsagie_: docstrings! :)14:55
openstackgerritRussell Bryant proposed stackforge/networking-ovn: devstack: Change tunnel encap to geneve.
otherwiseguygsagie_: yeah, that was always something I found a little annoying about the 'load a schema from a file" thing is that it makes one have to know what was in the schema.14:56
russellblatest ovn patches here if anyone wants to try them
gsagie_cool, lots of progress14:57
otherwiseguygsagie_: I actually made a toy version of a lib that actually automatically created classes for each of the Table types at runtime when pulling a schema from the server. :p So even the class names you'd never see in code. :p14:57
russellbgsagie_: yeah, about to try to start some VMs and see what happens14:57
otherwiseguyThe lib code was like 50 lines. :p14:58
gsagie_otherwiseguy: i know these kind of solutions they look nice but pain to debug :) i did a same thing with Netconf some time ago btw..14:58
otherwiseguyYeah, it was more of a "do it because I can" thing that I immediately threw away. :)14:59
otherwiseguyIDL itself is that way as far as column names go, though.14:59
otherwiseguyIt's not too bad in that if you mess up it just throws an exception at you. :p15:00
gsagie_as far as the patch goes, i don't mind to go with your proposal if you think it looks better, i prefer to listen to the more experienced python people ;)15:01
gsagie_you want to edit it ?15:01
gsagie_i will only get to it on sunday15:01
russellb/home/rbryant/devstack/functions-common: fork: Cannot allocate memory15:02
otherwiseguygsagie_: Only if you don't dislike the generalized stuff too much. Your opinion is as valid as mine there. It's your fault for convincing me that lots of little functions were unnecessary. :p I would have just gone with the first multi-op transaction thing otherwise.15:04
otherwiseguyrussellb: what are you doing to that poor machine?15:05
russellbotherwiseguy: ./ ?15:05
russellbadded API_WORKERS=1 to local.conf, hopefully that helps15:05
russellbi miss my $500/month rackspace credit hehe15:06
gsagie_i say its time that RedHat buys you a new laptop ;)15:06
russellbgsagie: this laptop is literally less than a week old15:06
russellbbut i opted for the ultra lightweight model that has a max of 8 GB RAM15:07
russellb2 GB RAM in this VM15:07
russellbthat should be able to run devstack ...15:07
gsagie_its pretty slow on my VM when i assign it only 2GB15:09
russellbback in my day i could run devstack with 1 GB of RAM just fine!15:10
gsagie_otherwiseguy: i dont dislike it..15:10
russellbImportError: /usr/lib64/python2.7/lib-dynload/ failed to map segment from shared object15:10
russellbc-api failed to start15:10
* otherwiseguy remembers having to load emm386.exe to be able to run games that needed more than 640k of memory15:12
otherwiseguysomeone needs to +2 russellb's ovs in global-requirements patch: Though I'm not sure who the requirements cores are.15:14
russellband to see who's doing reviews actively ...
otherwiseguyrussellb: thanks. pinged sdague/dhellmann.15:25
gsagie_otherwiseguy: if you want to edit my patch before sunday feel free with it, i am ok with the approach you presented :)15:30
otherwiseguygsagie_: Ok, I'll see if I can get around to it. I've also got this ipv6-only cloudinit thing I keep putting off because ovn is more fun. :p15:31
russellbotherwiseguy: i've been avoiding several other things for OVN heh15:35
russellbI *think* I just booted 2 VMs with Nova and pinged between them via OVN15:46
gsagie_cool :)15:46
russellbit just ..... worked15:47
russellbeven DHCP worked15:47
mesteryrussellb: COOL!15:53
russellbi'm going through some checks to make sure it's doing what i think it's doing15:54
gsagie_heh yeah, things that work the first time are scary15:54
openstackotherwiseguy: Error: "!" is not a valid command.16:10
* otherwiseguy glares at openstack 16:11
russellbotherwiseguy: do you know if cloud-init has a log file or something?16:11
russellbtrying to double check what it did16:11
otherwiseguyrussellb: I should, but I don't. I know horizon has a 'log' tab. :p16:12
russellbanyone want to try to replicate my test?16:15
markmcclainrussellb: thanks will do16:16
russellbmarkmcclain: :)16:16
russellbmarkmcclain: use ovn branch16:16
russellbalso using these changes
russellbmarkmcclain: don't have L3 working yet, fix that for me :-p16:19
markmcclainhaha… on it16:20
* russellb steps out for a quick lunch16:22
russellbOVN meeting in #openvswitch in 38 minutes16:22
*** dougwig has joined #openstack-neutron-ovn16:30
russellbdougwig: o/16:37
dougwigrussellb: hiya.  :)16:37
*** blp has joined #openstack-neutron-ovn16:47
*** armax has joined #openstack-neutron-ovn16:49
blprussellb: Congratulations!16:51
blpIt sounds like you're the first to get the system up.16:51
russellbblp: \o/16:52
russellbblp: i'm a little suspicious :-)16:52
russellbbut it appears to have worked16:52
russellbblp: i was just looking at git ... looks like it was 2 months from first commit to the ovn branch to now.  not bad!16:52
russellband that first commit was the design stuff ... code started mid-march16:52
blprussellb: I'm about to try it out myself this morning.  I'll be using ovn-nbctl though instead of Neutron so it's not quite as end-to-end.16:54
blpWow, that's moving pretty fast then. I hadn't thought about the calendar.16:55
blpThat includes a lot of distractions too--I took off a week for vacation and Justin took off some time too I think.16:55
russellbblp: sounds good, i'm eager to see if someone else can make it work too16:55
markmcclainrussellb: my instances are going into error state, so looking now to see why17:07
openstackgerritMerged stackforge/networking-ovn: local.conf.sample: Enable the rest of OpenStack
openstackgerritMerged stackforge/networking-ovn: devstack: Change tunnel encap to geneve.
*** blp has quit IRC17:47
*** gsagie_ has quit IRC17:48
mesteryrussellb: I'm trying to replicate yoru stuff, and my VMs are going into ERROR too. But I get this error:17:52
mesterynova.compute.manager (No compute node record for host ovn)17:52
mesteryThat's the error17:52
mesteryI think it's a DNS issue of some sort is my guess with my host17:52
*** blp has joined #openstack-neutron-ovn17:54
russellbmestery: could be17:56
russellbmestery: make sure nova-compute is still running17:56
mesteryrussellb: It's there, I see it17:56
russellbwell, of course it is based on that error message17:56
mesterynova host-list shows compute there, with a host_name of "ovn" as well17:56
russellbcheck "nova service-list" as well17:57
mesterynova-compute shows up ok there too ...17:57
mesterywith hostname of "ovn" as well17:57
russellbso nova is full of lies17:57
mesteryThis just feels like a hostname/DNS issue ...17:58
russellbovn in /etc/hosts ?17:58
* russellb shrugs17:58
russellbthat seems to be my problem far too often :)17:58
mesteryMe too, let me check17:58
mesteryIt was, but pointing to localhost.17:59
mesteryLet me try again17:59
russellbsame here17:59
russellbi'm on fedora, but that shouldn't make a difference18:00
mesteryright, I don't think so18:00
mesterydamn, same problem18:00
russellbmestery: connect to nova db via mysql, check compute_nodes table?18:02
mesteryrussellb: That's likely the next step.18:02
russellbMariaDB [nova]> select hypervisor_hostname from compute_nodes;18:03
russellb| hypervisor_hostname |18:03
russellb| devstack            |18:03
russellb(devstack is my hostname)18:03
mesterymysql> select hypervisor_hostname from compute_nodes;18:03
mestery| hypervisor_hostname |18:03
mestery| ovn                 |18:03
mestery1 row in set (0.00 sec)18:03
russellbi've got to run ... dr appt for me then 1 yr checkup for the kid18:04
mesteryNo worries, we'll chat later18:04
mesteryI'll keep digging18:04
mesterymarkmcclain: My errors were nova related, may want to check your n-cpu logs for something similar, see above ^^^^18:05
markmcclainmestery: good call… I'll dig there18:06
mesterymarkmcclain: I cant't quite figure out why it's not booting, so looking. Let me know if you see the same thing.18:06
*** gsagie_ has joined #openstack-neutron-ovn18:17
openstackgerritGal Sagie proposed stackforge/networking-ovn: Reduce the number of OVSDB transactions
gsagie_russellb, otherwiseguy ^^ this is according to otherwiseguy suggestion18:24
otherwiseguygsagie_ is going to need a vacation soon working all of these hours. I talked to him 13 hours ago on IRC...18:26
*** ajo has quit IRC18:26
otherwiseguymake that 14.18:27
gsagie_otherwiseguy: :) i will need you to send it to my boss18:27
otherwiseguygsagie_: Send me their email and I'll tell them all about how you work too hard. :)18:28
openstackgerritGal Sagie proposed stackforge/networking-ovn: Reduce the number of OVSDB transactions
otherwiseguygsagie: looks really nice. Just left a little question/comment.18:53
*** gsagie_ has quit IRC18:54
*** ajo has joined #openstack-neutron-ovn19:33
openstackgerritTerry Wilson proposed stackforge/networking-ovn: Add port 'up' notification
*** ajo has quit IRC20:28
russellbmestery: did you figure it out?20:31
* russellb just got back20:31
russellbor markmcclain ?20:34
*** ajo has joined #openstack-neutron-ovn20:34
markmcclainrussellb: not yet.. I pulled into internal stuff this afternoon20:34
russellbno worries20:35
russellbjust seeing if i can help20:35
markmcclainrussellb: thanks for checking in20:35
russellbi should kill my devstack VM and start over20:35
russellbsee if I can do it again20:35
mesteryrussellb: Yeah, give that a try20:43
mesteryrussellb: I never figured out the nova issue, I've some time now for about 1.5 hours or so to debug a bit more20:43
russellbon an unrelated note, i added a job to publish networking-ovn docs20:44
mesteryrussellb: nice!20:47
russellbgoing to write a more detailed "how to test" document as I rebuild a new devstack VM20:48
mesteryrussellb: ++20:48
russellbwhat ubuntu version do you guys use21:07
russellb14.04 ?21:07
russellbwhatever the latest LTS was i suspect21:07
blpI'm thinking about building a sort of "super-sandbox" that contains multiple simulated hypervisors.21:10
russellbblp: ooh21:11
russellbblp: <3 ovs-sandbox21:11
mesteryrussellb: 14.0421:11
russellbmestery: thanks21:11
* russellb documenting away!21:11
mesteryrussellb: I tore mine down and restarted it, lets see what happens21:11
blpIt's possible to connect multiple ovs-vswitchd processes via simulated "network cables" that operate over unix domain sockets.  I think we could make that an alternate "tunnel" type.21:12
russellbsounds super cool :)21:12
mesteryblp: awesome!21:12
blpAnd then it becomes possible to do end-to-end testing of OVN within the automated testsuite run by "make check".21:12
russellbblp: yeah, i never got around to writing any tests for the stuff i was working on :(21:13
blpAnyway, it's not going to happen today, but it's a direction for the future.21:13
russellbi was so focused on "make stuff work as quick as possible"21:13
russellbblp: we'll also have all this running in OpenStack CI with real VMs and such21:13
russellbwell, it already is in OpenStack CI21:14
mesteryrussellb: Indeed, and once we get pings working, we can move to running more tests than just deployment :)21:14
russellbbut not running any of the functional test suite yet21:14
russellbmestery: yes!21:14
blprussellb: Yes, that's great. For this afternoon I'm getting working in my own VM environment.21:16
russellbblp: cool, report back when it works :)21:17
russellbso we can celebrate, heh21:17
blpMaybe in Vancouver I can get a cake with the OVN logo on it.21:19
russellbbe sure to run "$ ovn-ctl preheat" first21:19
mesteryrussellb: Rats, same error with a fresh devstack install :(21:20
russellbmestery: i blame ubuntu :-p21:21
russellbmestery: will be testing in a bit, just going slow because i'm writing a bunch as i go21:21
mesteryrussellb: Let me try fedora too! :)21:21
mesterybenefits of having access to a large cloud ;)21:21
russellbwho knows what i had done to mine ... i had been using the same devstack VM for a few weeks21:21
blprussellb: haha21:24
blpAt some point I do need to install OpenStack for testing too. Never done it.21:25
blpI suspect that the 128 MB RAM VMs I use for testing are ridiculously undersized for OpenStack ;-)21:25
russellbblp: I'm hoping the doc I'm writing right now will make that easy for you, justin, or whoever else21:25
blprussellb: Oh, that's great! Didn't realize you were going into that kind of depth, but it will make it easy for me.21:26
russellbblp: i used to be able to test with 2 GB of RAM, but it kept breaking today, so now I have to do 4 GB :(21:26
russellbblp: yeah, trying to lay it out with copy/paste commands21:26
russellbcan't all be copy/paste once you get into crap that has a bunch of UUIDs, but anyway, clear directions :)21:27
blpI started out with 32 MB RAM per VM.  Most stuff was OK but valgrind exploded immediately.21:28
mesteryrussellb: This is embarrassing, but what is the default username for fedora cloud images? o_O21:28
russellbmestery: this is also embarassing, i don't know21:28
mesteryMan, my google fu is failing me!21:28
russellbi don't know.21:29
* mestery tries again21:29
russellbyou think i actually use any of this stuff?21:29
mesteryMe either. Guilty! :)21:29
mesteryfedora: No. cloud: No.21:30
* mestery works his way down the list21:30
blpmestery: It wasn't "lol"?21:31
mesteryblp: Ha!21:31
russellbyeah try lol21:31
russellbor hax21:32
mesteryOooooo ... maybe it's "ubuntusucks" :P21:32
*** ajo has quit IRC21:32
russellbworth a shot21:32
* russellb runs stack.sh21:33
russellbcome onnnnnn ...21:33
russellbi feel like i'm rolling dice every time i run this thing21:33
* mestery kicks off on a fedora VM21:37
russellbmestery: what was the user?21:38
mesteryrussellb: Embarrassingly, it was fedora, but for some reason vagrant was deceiving me.21:39
* mestery shakes his fist at this new-fangled software21:39
russellbyou use vagrant with an openstack cloud?21:39
russellbi'm not hip enough for vagrant21:39
mesteryLove it!21:39
mesteryI never have to remember IPs! :)21:39
* mestery courses IP addresses21:39
russellbcourses?  you eat them, but in between main and dessert?21:40
mesteryMOAR IPv6!21:40
* russellb naps during stack.sh21:41
* mestery has to take off soon, kid duty begins21:43
mesteryI'll be back at this later tonight though21:43
russellbcool ... wife and kid are watching sesame street right by me heh21:44
russellbreally just to get through a run before closing up though21:45
russellbConfiguring OVN21:48
russellbCreating OVS, OVN-Southbound and OVN-Northbound Databases21:48
russellbovsdb-tool: failed to read schema: "/opt/stack/ovs/ovn/ovn-sb.ovsschema" could not be read as JSON (error opening "/opt/stack/ovs/ovn/ovn-sb.ovsschema": No such file or directory)21:48
* russellb glares21:48
russellbi love stupid errors that occur because i used to have old files sitting around21:49
russellboh nevermind, this is because the git repo isn't called "ovs" it's "ovs-reviews"21:50
russellbblp: hey can you rename your git repo so I don't have to fix my shell script?21:50
blprussellb: haha21:55
blpI once ran into a problem where all the OVS tests that used SSL started failing one day.21:56
blpIt turned out that I'd kept around the same build directory for so long that the SSL certificates that the testsuite generates (just once) had expired.21:56
blpI think the expiration time was set to 2 years.22:00
blpAfter t22:01
blpAfter that I think I submitted a patch to extend it.22:01
russellbha, because obviously having to recreate your build dir sooner than 2 years is really inconvenient22:02
russellbmestery: well, devstack ran.22:03
openstackgerritRussell Bryant proposed stackforge/networking-ovn: docs: Remove content from usage.rst
openstackgerritRussell Bryant proposed stackforge/networking-ovn: devstack: Fix git repo name assumptions.
openstackgerritRussell Bryant proposed stackforge/networking-ovn: devstack: Give example of custom ovs git repo.
openstackgerritRussell Bryant proposed stackforge/networking-ovn: WIP: Add testing howto doc.
*** armax has quit IRC22:15
russellbi'm out for the night ... will finish up that doc tomorrow22:17
*** blp has left #openstack-neutron-ovn23:37
*** blp has joined #openstack-neutron-ovn23:37
blpI got OVN working in my test setup too.23:37

Generated by 2.14.0 by Marius Gedminas - find it at!