18:06:33 <SumitNaiksatam> #startmeeting networking_policy
18:06:34 <openstack> Meeting started Thu Jan 25 18:06:33 2018 UTC and is due to finish in 60 minutes.  The chair is SumitNaiksatam. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:06:35 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
18:06:37 <openstack> The meeting name has been set to 'networking_policy'
18:06:45 <SumitNaiksatam> #topic Vancouver Summit
18:07:03 <SumitNaiksatam> we had a short discussion here in SJ yesterday
18:07:29 <SumitNaiksatam> annakk: i believe you chatted with gary as well
18:07:46 <annakk> yes, he is in line with what we said
18:07:56 <SumitNaiksatam> (in the context of a submission for the summit)
18:08:01 <SumitNaiksatam> annakk: okay great
18:08:04 <annakk> and he said he'd like to join the talk
18:08:22 <SumitNaiksatam> annakk: do you want to take a stab are writing the abstract?
18:08:42 <annakk> yes
18:08:47 <SumitNaiksatam> ok great
18:09:15 <annakk> so are we planning a demo of both drivers with focus on service chaining?
18:09:31 <SumitNaiksatam> annakk: i am not so sure about the part on service chaining
18:09:46 <SumitNaiksatam> demo of the two drivers is something we can certainly attempt
18:10:01 <SumitNaiksatam> we can start with your work since its something new, at least as a driver for GBP
18:10:11 <SumitNaiksatam> rkukura: tbachman: thoughts?
18:10:29 <annakk> without service chaining its weird to put this in NFV track, no?
18:10:56 <SumitNaiksatam> annakk: ah okay, i would say lets first see how the abstract shapes up
18:11:03 <SumitNaiksatam> we can accordingly decided where to put it
18:11:21 <SumitNaiksatam> i didnt necessarily have the NFV theme central in my mind
18:11:21 <annakk> ok
18:11:22 <tbachman> SumitNaiksatam: were we looking at one talk, or multiple?
18:11:45 <rkukura> is the idea to provide an update on the project status and maybe discuss what’s next?
18:11:46 <SumitNaiksatam> tbachman: would be great if we had multiple, but i guess start with the outline of the first one :-)
18:11:53 <tbachman> sounds good
18:12:10 <SumitNaiksatam> but it very often happens that we start with one and then as we go along we realize that we have a number of things to talk about
18:12:15 <SumitNaiksatam> tbachman: so agree
18:12:58 <SumitNaiksatam> annakk: the NFV theme connection would have been strong if your driver was doing something in that domain
18:13:53 <annakk> there is such plan, not sure it will materialize till May
18:14:09 <SumitNaiksatam> annakk: its okay to put forward looking things
18:14:17 <SumitNaiksatam> it doesnt have to be absolutely ready
18:14:42 <SumitNaiksatam> i think even a conceptual design is fine (as long as its coherent)
18:14:52 <SumitNaiksatam> annakk: but up to you, whatever you feel comfortable
18:14:54 <annakk> but on aim side is there some service chaining to show?
18:15:25 <SumitNaiksatam> annakk: much of what we did with the apic drivers has already been shown before
18:15:42 <SumitNaiksatam> so if you were to show something with NSX, we could show those things again
18:15:45 <annakk> ok
18:15:50 <SumitNaiksatam> but by itself, it might not make much sense
18:16:05 <annakk> understood
18:16:39 <SumitNaiksatam> rkukura: sorry i missed your earlier comment
18:16:47 <annakk> i might reach out for help to you/gary about the abstract since i've never done an openstack proposal before
18:17:28 <SumitNaiksatam> rkukura: i agree, we should discuss that
18:17:50 <SumitNaiksatam> rkukura: but i am thinking that by itself it might not make a strong case for us being accepted for the talk
18:18:22 <SumitNaiksatam> rkukura: by “that” i meant “project update"
18:18:41 <SumitNaiksatam> if we dont have anything else, we can still use that though :-)
18:19:22 <SumitNaiksatam> annakk: sure, i dont think there is a set pattern, feel free to express yourself :-)
18:19:30 <SumitNaiksatam> (i meant in the abstract)
18:19:31 <rkukura> agreed, but we need  some way to be relevant to the OpenStack community, since we aren’t an official project (at the moment)
18:19:43 <SumitNaiksatam> rkukura: completely agree
18:20:21 <SumitNaiksatam> annakk: we were also discussing about how the larger goal of this project was to tackly the notion of policy in general
18:20:39 <SumitNaiksatam> not limited to just networking, but across compute and storage domain also
18:20:46 <rkukura> I’d hope that if we do get the opportunity to present, we can use it to attract  interest/participation from  the community
18:20:57 <SumitNaiksatam> and also across technologies, virtual, plysical, container, etc
18:21:40 <SumitNaiksatam> rkukura: i agree, very important point, and should definitely be our goal, to promote the project so as to attract more people to work with us
18:22:39 <SumitNaiksatam> okay, if nothing more on this, we can move to the next topic
18:22:54 <SumitNaiksatam> #topic DB writer pattern quirks
18:23:23 <SumitNaiksatam> so last week i made an attempt to explain this
18:23:30 <SumitNaiksatam> here is a specific example:
18:23:32 <SumitNaiksatam> #link https://review.openstack.org/#/c/532277/1..4/gbpservice/neutron/tests/unit/services/sfc/test_aim_sfc_driver.py
18:23:49 <SumitNaiksatam> search for: test_vrf_update
18:24:07 <SumitNaiksatam> L661 on left side, L763 or right side
18:24:47 <SumitNaiksatam> so firstly, if you see L670 on the left side
18:25:14 <SumitNaiksatam> we are getting a reference to a DB object, and then using it in L674
18:25:48 <SumitNaiksatam> this created issues, since this was not wrapped in a transaction
18:26:21 <SumitNaiksatam> so if you see on L774 on the RHS, we added the transaction block
18:27:58 <SumitNaiksatam> the processing now proceeds to L777 which in turn calls _add_network_mapping_and_notify L3387 in #link https://review.openstack.org/#/c/532277/4/gbpservice/neutron/plugins/ml2plus/drivers/apic_aim/mechanism_driver.py
18:29:15 <SumitNaiksatam> out here on L3389, a writer pattern is used
18:29:31 <SumitNaiksatam> but it does not create a new sub-transaction
18:29:57 <SumitNaiksatam> so the test case which was originally calling this, actually mocks the notify on L3392, and raises an exception
18:31:29 <SumitNaiksatam> if instead of using the writer pattern on L3389, we had use the older session.begin(subtransactions=True) pattern, the netwrok_mapping which is happening in the L3390, would have never gotten persisted
18:31:52 <SumitNaiksatam> however, going back to #link https://review.openstack.org/#/c/532277/1..4/gbpservice/neutron/tests/unit/services/sfc/test_aim_sfc_driver.py
18:32:24 <SumitNaiksatam> on LHS if you see L674, it asserts that the exception is raised, but it actually eats up the exception
18:32:41 <SumitNaiksatam> hence the network_mapping done earlier gets persisted
18:33:07 <SumitNaiksatam> and the test fails
18:33:25 <SumitNaiksatam> i am sure now i have managed to confuse you way more than what i had last week :-)
18:33:32 <SumitNaiksatam> applause please!
18:34:20 <tbachman> lol!
18:34:27 * tbachman does his best golf clap
18:34:47 <SumitNaiksatam> tbachman: thanks :-)
18:34:50 <tbachman> heh
18:34:51 <SumitNaiksatam> happy to discuss this particular case further, but if i had to drive a couple of takeways from this
18:35:35 <SumitNaiksatam> 1. be careful when you assert and exception, because that call will eat the exception, so you can no longer except the original rollback to happen
18:36:09 <SumitNaiksatam> 2. in places like L3389 in #link https://review.openstack.org/#/c/532277/4/gbpservice/neutron/plugins/ml2plus/drivers/apic_aim/mechanism_driver.py
18:36:29 <SumitNaiksatam> the use of: with context.session.begin(subtransactions=True)
18:37:39 <SumitNaiksatam> would be a better choice, assumign we always know that its going to be nested within an outside: with context_manager.writer(context)
18:38:33 <SumitNaiksatam> i meant to say assert *an* exception in (1), not “assert and exception”
18:39:22 <SumitNaiksatam> to extend (2) further, the “with context.session.begin(subtransactions=True)” can be nested within a writer or reader block
18:39:34 <annakk> i wonder if this can be relevant? https://review.openstack.org/#/c/433758/
18:39:47 <annakk> i vaguely remember similar problems
18:40:08 <SumitNaiksatam> this is helpful in the cases where you do want to create a new subtranction (the new subtransaction will immediately rollback when the context exits, if there is a failure)
18:40:59 <SumitNaiksatam> annakk: could be
18:41:20 <SumitNaiksatam> but basically, i would like to avoid using any flush
18:41:32 <SumitNaiksatam> it should not be needed, at least in our code
18:41:34 <rkukura> every  query does a flush, doesn’t it?
18:41:41 <rkukura> you mean explicit flush
18:41:48 <SumitNaiksatam> rkukura: yes explicit flush in our code
18:42:03 <SumitNaiksatam> flush or even refresh
18:42:21 <SumitNaiksatam> in the past we have reluctantly done that
18:42:48 <SumitNaiksatam> but ideally we should not need to do it (assume there are no bugs in the underlying oslo_db code)
18:42:58 <SumitNaiksatam> and we are undertanding its usage correctly :-)
18:44:05 <SumitNaiksatam> happy to discuss this particular case further if there are questions/comments. or we can review the patch and come back to it next week
18:44:32 <SumitNaiksatam> annakk: thanks for the pointer, at least i hadnt noticed it
18:44:55 <SumitNaiksatam> need to check if its still in place or if they have reverted it
18:45:21 <SumitNaiksatam> #topic Open Discussion
18:45:25 <annakk> SumitNaiksatam: yes I think they added default behavior later on but I can't find it
18:45:31 <SumitNaiksatam> annakk: okay
18:45:47 <SumitNaiksatam> anything else we need to discuss today?
18:46:15 <rkukura> SumitNaiksatam: Do you have guidance on whether all new code should be using the new API, or if there are cases where context.session.begin() really should be used?
18:46:28 <annakk> I just want to mention I started to test pike, but stack on some infra issues for now, unrelated to the pike changes
18:46:49 <SumitNaiksatam> annakk: okay good to know, keep us update on how it goes
18:46:55 <annakk> sure
18:46:56 <rkukura> I admit I am not familar  enough with Ivar’s patch yet to have followed the above discussion in realtime
18:47:23 <SumitNaiksatam> rkukura: yes, basically learnt by experience here
18:47:48 <SumitNaiksatam> so my understanding is that the writer pattern can be nested
18:47:55 <SumitNaiksatam> and it all works fine
18:48:14 <SumitNaiksatam> the big difference is that, a new subtransaction is not created when nesting happens
18:48:59 <SumitNaiksatam> so if you expected something as a result of the (previously used) subtransaction to commit or rollback, it may not behave the same way
18:50:03 <SumitNaiksatam> for new code, i would prefer to use the older pattern (i.e. subtransactions=True) if you are absolutely sure that the code would not be called at the top level of the transaction
18:50:27 <SumitNaiksatam> but when in doubt, use the newer writer pattern
18:51:34 <SumitNaiksatam> there are almost 4 levels of DB abstractions that we are navigating here
18:51:52 <SumitNaiksatam> GBP -> neutron -> oslo_db -> sqlalchemy -> mysql !
18:51:54 <rkukura> it seems we should be using readers where possible, and that they should act like writers when nested inside a writer transaction. Is that the case?
18:51:57 <annakk> almost 4 :)
18:52:09 <SumitNaiksatam> rkukura: that is correct
18:52:15 <SumitNaiksatam> reader will get elevated
18:52:20 <SumitNaiksatam> when nested
18:54:07 <SumitNaiksatam> if you nest a writer inside a reader, the session that was used for the reader will now change
18:54:26 <SumitNaiksatam> and so will the transaction
18:54:58 <SumitNaiksatam> the transaction is gauranteed to be preserved only when either a reader or writer is nested within a top level writer
18:56:11 <SumitNaiksatam> time check, 5 minutes
18:56:18 <rkukura> so what is the advantage of using the old session.begin?
18:57:26 <SumitNaiksatam> if your code is written with complete understanding of the newer approach, there is no advantage
18:58:06 <annakk> do we know of areas with missing code coverage where such problems can be revealed, if exist?
18:58:21 <annakk> *test coverage*
18:58:51 <SumitNaiksatam> annakk: tough question :-)
18:59:25 <annakk> :)
19:00:04 <SumitNaiksatam> alrighty, lets wrap up for today
19:00:11 <SumitNaiksatam> thanks rkukura tbachman annakk for joining
19:00:15 <annakk> thanks! bye
19:00:18 <rkukura> thank you SumitNaiksatam
19:00:19 <SumitNaiksatam> and apologies again for the delay in starting
19:00:20 <SumitNaiksatam> bye
19:00:21 <rkukura> bye
19:00:24 <SumitNaiksatam> #endmeeting