19:00:00 <oanson> #startmeeting Dragonflow
19:00:00 <openstack> Meeting started Mon Aug 28 19:00:00 2017 UTC and is due to finish in 60 minutes.  The chair is oanson. Information about MeetBot at http://wiki.debian.org/MeetBot.
19:00:02 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
19:00:05 <openstack> The meeting name has been set to 'dragonflow'
19:00:13 <oanson> Hi all.
19:00:19 <oanson> Who is here for the Dragonflow weekly?
19:00:53 <dimak> hello
19:01:00 <dimak> o/
19:01:04 <oanson> Hi
19:01:12 <oanson> I really hope it's not just the two of us
19:01:40 <dimak> mlavalle just joined the channel :)
19:01:46 <mlavalle> o/
19:01:54 <oanson> mlavalle, hi. Welcome
19:02:01 <mlavalle> thaks1
19:02:04 <mlavalle> thanks1
19:02:06 <mlavalle> 1
19:02:09 <mlavalle> !
19:02:09 <oanson> Let's wait another half-minute. Maybe leyal or irenab will join too
19:02:34 <dimak> mlavalle, holding up against the floods and the tornadoes?
19:02:36 <oanson> mlavalle, post-lunch drowsiness?
19:03:02 <mlavalle> oanson: no, just clumsiness
19:03:16 <oanson> I see.
19:03:20 <oanson> All right. Let's begin!
19:03:22 <oanson> #topic roadmap
19:03:32 <oanson> Reminder, the agenda is here: https://wiki.openstack.org/wiki/Meetings/Dragonflow#Meeting_28th_August.2C_2017_.281900_UTC.29
19:03:48 <oanson> LBaaS: Updated the spec according to irenab's and pino's notes
19:03:54 <oanson> It passes gate :)
19:04:19 <dimak> Cool, I promise to give it another go tomorrow
19:04:20 <oanson> L3 flavour - dimak you were busy with bugs this week. Any progress on this?
19:04:37 <oanson> dimak, no rush on the LBaaS. It's a Queens feature.
19:04:39 <dimak> Nope, it was mainly bugs
19:04:48 <dimak> Trying to get tempest happy
19:04:53 <oanson> Sure, no worries.
19:05:03 <oanson> I saw it *was* happy today, but we'll get to that
19:05:18 <oanson> etcd publisher - lihi got the etcd3gw-based db driver in.
19:05:56 <oanson> There is a small holdback with the publisher - the code assumes utf-8, but we want it to be raw binary data (we use msgpack, which is very much not utf-8 compatible)
19:05:58 <dimak> oanson, I opened a bug earlier on the unique key allocation race
19:06:10 <oanson> dimak, linky?
19:06:22 <dimak> https://bugs.launchpad.net/dragonflow/+bug/1713496
19:06:22 <openstack> Launchpad bug 1713496 in DragonFlow "etcd driver implement allocate_unique_key in a raceful manner" [Undecided,New]
19:06:40 <oanson> I don't think it's just etcd. I think it's all drivers.
19:06:55 <oanson> I recall redis having similar behaviour (from reading the code)
19:07:00 <dimak> I looked at redis and it should be ok
19:07:29 <oanson> Then the solution shouldn't be too complex.
19:07:34 <oanson> I'll send something in tomorrow
19:07:40 <dimak> INCR key  - Increments the number stored at key by one. If the key does not exist, it is set to 0 before performing the operation
19:08:13 <oanson> Yep. Looks all right.
19:08:15 <dimak> I think we should move it to df-db init anyway
19:08:55 <dimak> there we can iterate all models and create the keys for all models that have unique_key
19:08:56 <oanson> We could do both. Putting it in df-db init means scanning all models and identifying the ones having unique_key.
19:09:06 <dimak> ^
19:09:09 <oanson> Glad we think alike :)
19:09:39 <oanson> All right. I'll consult lihi and see who does what, since she touched that code last
19:09:48 <dimak> Sure
19:10:00 <oanson> RPM packaging - sadly I've ignored this. I apologize.
19:10:25 <oanson> I've added a new item to the roadmap - grenade gate. I want us to start behaving properly with past versions
19:10:30 <oanson> That means having an upgrade path.
19:10:48 <oanson> Now with the NB model framework, this shouldn't be too difficult to maintain.
19:11:08 <dimak> yes
19:11:19 <oanson> I've uploaded to relevant patches. Once gerrit cooperates, I'll also have the links
19:11:46 <oanson> https://review.openstack.org/496837 and https://review.openstack.org/401210 . Actually the second one isn't mine
19:11:55 <oanson> I'm renewing the work xiaohhui did in the past
19:12:16 <dimak> I remember that
19:12:27 <dimak> we should consider controllers with old versions
19:12:39 <dimak> so we can allow gradual upgrade
19:12:45 <oanson> That's a good idea.
19:12:50 <oanson> Any thoughts on how to do that?
19:13:11 <oanson> Or just have a deprecation period on destructive DB changes?
19:13:15 <dimak> We can do what nova does
19:13:26 <oanson> What does nova do?
19:13:58 <dimak> If an agent receives an object it does not support (by version, they use OVO)
19:14:11 <dimak> then the bounce it off nova-scheduler
19:14:19 <dimak> and it translates it to an older version
19:14:44 <oanson> Hmm... Give me a day to think if we can do this without OVO
19:15:04 <oanson> We should be able to. We have the database version, and the nb_api can hide all this logic
19:15:16 <dimak> OVO is there to handle object metadata, its type and version
19:15:31 <dimak> and some other fields
19:15:53 <oanson> I'll read more about it too. I saw Neutron are working for almost two cycles (I think) to make the transfer.
19:16:26 <oanson> #action oanson read about OVO. Add migration and upgrade spec
19:16:44 <oanson> Anything else for roadmap?
19:16:47 <dimak> the only issue with that approach is that all conductor instances have to be upgraded first (and maybe at once?)
19:17:03 <dimak> Nope
19:17:31 <oanson> dimak, let's defer this conversation to the spec. If it's just us, I suspect we will have to repeat this conversation when the others join :)
19:17:49 <dimak> yeah
19:17:53 <oanson> Cool
19:17:53 <oanson> #topic Bugs
19:18:06 <oanson> I'm please to report that we have only two high bugs left
19:18:21 <oanson> Bug #1707496
19:18:22 <openstack> bug 1707496 in DragonFlow "Stabilize tempest job" [High,In progress] https://launchpad.net/bugs/1707496 - Assigned to Dima Kuznetsov (dimakuz)
19:18:27 <dimak> there's another one I caught today actually
19:18:40 <oanson> Shame on us... :(
19:18:43 <dimak> well, not excatly today
19:18:59 <oanson> One bug at a time. Let's finish with these two. They'll be quick
19:19:05 <dimak> but dnat doesn't delete flows properly when fip is deleted before it is disassociated
19:19:29 <oanson> dimak, I saw that the last item on this bug is a devstack patch.
19:19:52 <dimak> Its the only one at the moment
19:20:03 <oanson> I suggest that if there's no response by, say, Wednesday, we do a workaround locally. We can remove it in Queens once the devstack patch is merged.
19:20:04 <dimak> but we should look into bgp tests
19:20:17 <dimak> they still fail occasionally
19:20:23 <oanson> Yes.
19:20:33 <oanson> I'll set up an environment and see if I can reproduce locally
19:20:50 <dimak> I finally managed to get an env up with DR so I can run them a few times
19:20:58 <oanson> Otherwise, I will contact the Neutron DR guys. I think I saw these gates are non-voting on their end. I can ask why.
19:21:02 <dimak> lets discuss this tomorrow
19:21:04 <oanson> Sure
19:21:27 <oanson> But I think the BGP issues do not block the bug. If that's the only issue, it can be closed.
19:21:36 <oanson> Bug #1712503
19:21:38 <openstack> bug 1712503 in DragonFlow "L3 application does not take care of allowed address pairs" [High,In progress] https://launchpad.net/bugs/1712503 - Assigned to Dima Kuznetsov (dimakuz)
19:22:13 <oanson> I saw there is a patch up for this one. The gate fails for some reason on an ARP request
19:22:14 <dimak> I have a fix up for that as well
19:22:40 <dimak> I'm still debugging
19:23:15 <oanson> dimak, sorry?
19:23:38 <oanson> I didn't understand - you have a fix, or it's not 100% yet?
19:23:39 <dimak> Still debugging the failing fullstack test
19:24:03 <oanson> Sure. Got it.
19:24:16 <dimak> Not sure yet whether the fault is with the test or the fix :)
19:25:03 <oanson> It looks like it's failing on an ARP packet. But I haven't dug any deeper
19:25:33 <oanson> If it's taking too long, we could consider a tempest test for now. Add a fullstack test in another patch
19:25:49 <oanson> If there is a tempest test ready-made for it, that is
19:25:57 <dimak> Sure
19:26:20 <oanson> Now what's this new bug you've found? <dimak> but dnat doesn't delete flows properly when fip is deleted before it is disassociated
19:27:01 <dimak> Deleting associated FIP will leave its flows in place
19:27:09 <oanson> Except flow pollution (which I agree is bad), what else does it cause?
19:27:38 <dimak> Stale NAT rule in the network
19:28:01 <oanson> That means dNAT still exists after it's been removed?
19:28:03 <dimak> I haven't tried it but I assume you'd still be able to access the port by FIP's address
19:28:13 <dimak> I'll confirm it tomorrow
19:28:32 <oanson> Mostly I want to see if we can say it's Medium, and bump it to next week
19:29:05 <dimak> Ok
19:29:08 <dimak> I'll report back
19:29:15 <oanson> Cheers.
19:29:18 <oanson> Anything else for bugs?
19:29:25 <dimak> Not on my end
19:29:42 <oanson> mlavalle, I saw you uploaded a patch to fix our vagrant deployment?
19:29:50 <mlavalle> correct
19:30:05 <mlavalle> and the problem is excatly the line you commented
19:30:21 <mlavalle> I'll remove it and try again
19:30:53 <oanson> I ran into it in the all-in-one configuration. We need to go over them and make sure they're all working. I did etcd and redis, but there are many more.
19:30:57 <oanson> mlavalle, great. Thanks!
19:31:28 <oanson> #topic Open Discussion
19:31:33 <mlavalle> yeah, for the time being, I want to get it working with etcd and once that works, I'll try others
19:31:45 <oanson> mlavalle, sounds like a plan!
19:32:18 <oanson> I added two items for open discussion today: vPTG and Tagging
19:32:26 <oanson> Let's start with tagging: That happens on Friday
19:32:30 <mlavalle> LOL, vPTG
19:32:51 <_pino> Is it vPTG or VTG?
19:33:14 <mlavalle> I like vPTG
19:33:15 <oanson> I think it's Virtual Project Team Gathering. But then again, vPTG is not a TLA.
19:33:15 <_pino> Eshed and I will be in California, but I will try to join remotely.
19:34:17 <_pino> How many sessions/hours were you thinking of doing for DF? Is there already an etherpad with the topics?
19:34:24 <oanson> The PTG itself has a remote option? I recall last PTG they were playing around with the option, but it wasn't 100% successful
19:34:44 <mlavalle> I think it really depends on each team
19:34:56 <mlavalle> I can raise the idea with the Neutron team
19:35:15 <oanson> _pino, we haven't built an agenda yet. In any case, it will probably be a week after the pPTG.
19:35:39 <oanson> mlavalle, that would be great for us. Then we can listen in from here.
19:36:03 <mlavalle> ok, the weekly meeting is tomorrow morning. I'll bring it up
19:36:11 <oanson> Cool. Thanks.
19:36:41 <oanson> I suspect we won't be the only project doing a vPTG. It might be worthwhile to collaborate and work out a framework where projects can inter-connect
19:36:46 <mlavalle> as far as the vPTG, I will be attending New Employee orientation the 19 - 20
19:37:16 <mlavalle> and flying back from Santa Clara on the 21st
19:37:30 <mlavalle> please take that into consideration
19:37:39 <oanson> I think 18th-20th are the only dates we have, due to local holidays. It's either that or end of September (I haven't checked the dates).
19:37:41 <dimak> 20th is holiday here
19:38:01 <oanson> Let me review the calendar and see what we can do.
19:38:55 <oanson> I would have said I don't mind working the holidays here (I don't like holidays), but some toddler might object :)
19:39:22 <mlavalle> is it Yom Kipur?
19:39:41 <oanson> No, New Years (I think).
19:40:18 <oanson> Yep, New Years. Yom Kipur is the 30th.
19:40:36 <mlavalle> 5778?
19:41:08 <oanson> No, you got me. I stopped counting when I finished middle school, and I don't remember that far back. :)
19:41:38 <mlavalle> LOL
19:41:59 <dimak> mlavalle, I think you're a few billion years off :P
19:42:13 <dimak> about a dozen
19:42:40 <mlavalle> well, yeah, it depends on your frame of reference
19:42:57 <_pino> It just occurred to me - so you're not going to do the vPTG during the week of Sept. 11 - that would make attendance very low? So you can do it any other time of our choosing?
19:43:13 <oanson> I like the frame of reference where the universe is 47 years, 7 months, and about 28 days old
19:43:21 <mlavalle> ++
19:43:42 <oanson> _pino, yeah. But we want to keep it near the original pPTG. Since that's the start of the cycle.
19:43:57 <mlavalle> I think that makes sense
19:44:42 <oanson> There are a couple of items we want to discuss in the vPTG, which might take the cycle: Selective-proactive, upgrades, deployment, etc.
19:45:49 <oanson> Let's continue this offline. I also want to talk to the other projects which do this to see how they handle things like that.
19:46:10 <oanson> This will also give mlavalle a chance to inquire about adding virtual presence to the Neutron PTG.
19:46:19 <mlavalle> yes
19:46:55 <oanson> Next item I want to discuss is tagging. It happens this Friday, or any split-second when there are no High/Critical bugs.
19:47:19 <oanson> Once we tag, we should also inform OSA. They need to know when the tag is ready.
19:47:52 <oanson> Any new bugs we discover can be back-ported, but within reason.
19:48:15 <oanson> Anything else for Open Discussion?
19:48:41 <oanson> Anything else in general?
19:48:59 <dimak> Have a great evening
19:49:23 <oanson> Yep. Thanks everyone for coming.
19:49:30 <mlavalle> dimak: did you watch the season finale of GoT?
19:49:36 <dimak> Not yet
19:49:45 <dimak> The meeting is holding it up
19:49:47 <dimak> :)
19:49:48 <mlavalle> enjoy!
19:49:53 <dimak> Thanks
19:50:21 <oanson> #endmeeting