18:02:46 <SumitNaiksatam> #startmeeting networking_policy 18:02:46 <openstack> Meeting started Thu Nov 2 18:02:46 2017 UTC and is due to finish in 60 minutes. The chair is SumitNaiksatam. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:02:47 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 18:02:49 <openstack> The meeting name has been set to 'networking_policy' 18:02:53 <annakk> hello! 18:02:57 <SumitNaiksatam> annakk: hi 18:03:08 <rkukura> hi annakk! 18:03:09 <SumitNaiksatam> not much of an agenda from me today 18:03:22 <SumitNaiksatam> i was hoping to have made more progress with my pike-prep patches 18:03:38 <SumitNaiksatam> unfortunately there are still some UTs failing with the 2nd patch 18:04:00 <SumitNaiksatam> and then there is a timeout issue with the newton branch 18:04:15 <SumitNaiksatam> (ideally we shouldnt be dealing with the newton branch any more!) 18:04:46 <SumitNaiksatam> annakk: wanted to check with you 18:05:12 <SumitNaiksatam> but update you first about our meeting in SJ (involving rkukura and tbachman while they visited here last week) 18:05:37 <SumitNaiksatam> we tried to scope out the work required to sync with Pike 18:05:48 <SumitNaiksatam> some of that already started 18:06:06 <SumitNaiksatam> tbachman would be looking at some of the apic repos is some work is needed 18:06:20 <annakk> sounds good 18:06:31 <SumitNaiksatam> annakk: you had mentioned that you had identified some potentially tricky item for Pike sync? 18:07:14 <annakk> i remember there were more OVO in pike, but I don't recall anything particularly tricky 18:07:33 <SumitNaiksatam> annakk: ah okay, that sounds encouraging 18:07:45 <SumitNaiksatam> i was hoping that the OVO work was transparent to us 18:07:59 <SumitNaiksatam> but we might need to do some work in the DB plugin, i havent looked into it 18:08:29 <annakk> we hit some ovo-related issues in ocata 18:08:56 <SumitNaiksatam> annakk: okay, can you summarize those if you recall them? 18:09:07 <annakk> i think it was related to partial ovo in neutron 18:09:34 <SumitNaiksatam> annakk: okay, i believe the OVO work was completed in Pike? 18:09:51 <annakk> I'm not sure 18:10:02 <annakk> I think all core objects are done 18:10:18 <annakk> hopefully we'll be better off this time 18:10:23 <SumitNaiksatam> annakk: okay 18:10:56 <SumitNaiksatam> the current set of failures i am seeing in my patch are a bit wierd, something seems to be failing in the API layer (response path) when creating address scopes 18:11:37 <SumitNaiksatam> will reach out to you guys if i cant make more progress, but at this point i need to give it a little more time 18:12:05 <rkukura> ok 18:12:06 <SumitNaiksatam> any suggestions on how to make progress on getting past the timeout issue for py27 in the stable/newton branch? 18:12:21 <annakk> ok, i think address scopes and subnets were the problematic ovo areas last time 18:12:24 <SumitNaiksatam> this is the patch: #link https://review.openstack.org/#/c/516766/ 18:12:38 <SumitNaiksatam> i tried disabling some tests, still failing 18:12:56 <SumitNaiksatam> the root cause is that the job is not getting resources, i.e. worker threads 18:13:21 <SumitNaiksatam> but we cant control that 18:13:51 <SumitNaiksatam> i has made this observation before that EoL’ed branches seem to get fewer resources in the gate jobs 18:13:59 <SumitNaiksatam> (which sounds logical to me) 18:14:03 <rkukura> is this a case where certain (or random) UTs are timing out, or some timeout in the setup or something? 18:14:11 <SumitNaiksatam> we are not seeing the timeout issue in the ocata or master branchbes 18:14:15 <annakk> no ideas from me.. (other than increase the timeout even more :)) 18:14:19 <tbachman> lol 18:14:26 <SumitNaiksatam> annakk: increasing the time out does not help 18:14:35 <SumitNaiksatam> since its not one particular test that times out 18:14:43 <SumitNaiksatam> i already tried increasing the timeout 18:14:49 <SumitNaiksatam> rkukura: thats the tought part 18:14:51 <rkukura> The tesr.conf timeout is only for individual UTs, right? 18:15:02 <SumitNaiksatam> rkukura: yes 18:15:15 <SumitNaiksatam> rkukura: the logs dont get persisted at the time the timeout happens 18:15:23 <SumitNaiksatam> so you cant tell which test was executing at that time 18:16:06 <SumitNaiksatam> i do see that the tests which regularly take the longest to execute, are actually completing in the timed out runs 18:16:55 <SumitNaiksatam> i think the timeout just happens on the entire job, meaning if the job only gets 4 threads (or lesser), the job is not going to complete in 35 mins or so 18:17:10 <SumitNaiksatam> if you see the master or ocata the jobs get 8 worker threads 18:17:36 <rkukura> SumitNaiksatam: that makes sense, I guess 18:17:47 <SumitNaiksatam> so i guess the obvious thing to do would be to disable a few more tests :-( 18:17:51 <rkukura> Is there any easy way to try with most tests disabled? 18:17:59 <SumitNaiksatam> rkukura: yeah just saying 18:18:12 <rkukura> Maybe just leave one module enabled? 18:18:19 <SumitNaiksatam> i disabled the most obvious big chunk of tests 18:18:23 <SumitNaiksatam> rkukura: okay 18:18:38 <SumitNaiksatam> i really hope we dont have to keep going back to newton 18:18:46 <SumitNaiksatam> but i guess people are still using newton 18:18:53 * tbachman nods 18:19:09 <SumitNaiksatam> the patch in question here is #link https://review.openstack.org/#/c/515203/ 18:19:25 <SumitNaiksatam> which i believe is in response to some customer issue 18:19:48 <SumitNaiksatam> anyway, i guess we have discussed this enough 18:20:05 <SumitNaiksatam> oh just remembered, annakk weren’t you going to be in Sydney? 18:20:12 <annakk> yes 18:20:28 * tbachman wonders if/when annakk is getting on a plane 18:20:42 <tbachman> should be pretty soon, if you don’t want to knackered when you start the summit 18:20:50 <tbachman> s/to/to be/ 18:20:50 <annakk> tomorrow evening 18:20:56 <SumitNaiksatam> annakk: awesome 18:21:11 <tbachman> annakk: would love to hear about it when you return! 18:21:14 <SumitNaiksatam> i am sure you are looking forward to it, will be fun! 18:21:21 <SumitNaiksatam> tbachman: +1 18:21:51 <SumitNaiksatam> annakk: have you attended the summit before, or is this your first one? 18:22:19 <annakk> I've been once in vancouver, but I wasn't involved with openstack back then 18:22:33 <SumitNaiksatam> annakk: right, i was guessing you might have attended that 18:22:48 <SumitNaiksatam> vancouver was really nice, loved it! 18:22:54 <annakk> i hope to see you all here in may 18:23:05 <SumitNaiksatam> annakk: oh sure, hopefully 18:23:16 <SumitNaiksatam> we might need to use this template #link https://www.openstack.org/blog/2017/10/dear-boss-i-want-to-attend-openstack-summit-sydney/ 18:23:21 <SumitNaiksatam> :-P 18:23:40 <SumitNaiksatam> rkukura you are not going to Sydney, are you? 18:23:50 <rkukura> no 18:24:04 <rkukura> would love to go diving 18:24:15 <SumitNaiksatam> rkukura: yeah, absolutely! 18:24:27 <SumitNaiksatam> before it disappears 18:24:31 <rkukura> right 18:24:43 <SumitNaiksatam> you made it to the carribean and napa in time i guess! 18:24:59 <rkukura> maybe I shouldn’t go anywere 18:25:02 <rkukura> anywhere 18:25:04 * tbachman will have to leave in about 5-10 minutes 18:25:13 <SumitNaiksatam> tbachman: sure 18:25:19 <SumitNaiksatam> anything else we need to discuss today? 18:25:31 <rkukura> not from me 18:25:42 <annakk> i was wondering about the failing jobs? 18:26:02 <SumitNaiksatam> annakk: yes its on my todo list to fix those 18:26:06 <SumitNaiksatam> sorry about the delay 18:26:16 <SumitNaiksatam> the best thing would have been to migrate them to the devstack plugin 18:26:30 <annakk> np 18:26:50 <SumitNaiksatam> good point, i should have brought it up 18:26:58 <SumitNaiksatam> those jobs are running with patched devstack 18:27:07 <annakk> i just wanted to ask if we know what's wrong with then 18:27:08 <SumitNaiksatam> the aim job uses the devstack plugin (and passes) 18:27:28 <SumitNaiksatam> annakk: i do some patching to checkout the appropriate branches 18:27:33 <SumitNaiksatam> that is somehow failing 18:27:49 <SumitNaiksatam> its not even completing the devstack install 18:28:01 <SumitNaiksatam> so its not the case that the tests are failing 18:28:16 <SumitNaiksatam> annakk: you use the devstack plugin for your vmware job, right? 18:28:21 <annakk> yes 18:28:43 <SumitNaiksatam> yeah, so the ideal thing would be for us to migrate the setup for these jobs to use the devstack plugin 18:28:50 <SumitNaiksatam> but thats a little more work 18:28:58 <SumitNaiksatam> i will see how i can get this fixed at the earliest 18:29:18 <SumitNaiksatam> annakk: what is time sensitivity on getting the pike branch ready? 18:30:06 <SumitNaiksatam> (i do realize that as a matter of discipline we should be getting it ready asap) 18:30:32 <annakk> I need to check, I think it's not urgent since it won't make it to next release anyway 18:30:50 <SumitNaiksatam> annakk: okay, good to know 18:31:10 <SumitNaiksatam> but we will try to sync this up at the earliest 18:31:15 <SumitNaiksatam> thanks all for joining today 18:31:26 <annakk> thanks! 18:31:26 <SumitNaiksatam> annakk: enjoy your time in Sydney! 18:31:30 <annakk> thanks! 18:31:31 <tbachman> SumitNaiksatam: thanks! 18:31:38 <SumitNaiksatam> rkukura: tbachman: thanks 18:31:40 <SumitNaiksatam> bye 18:31:46 <SumitNaiksatam> #endmeeting