00:05:18 <thinrichs> #startmeeting CongressTeamMeeting
00:05:19 <openstack> Meeting started Thu May 19 00:05:18 2016 UTC and is due to finish in 60 minutes.  The chair is thinrichs. Information about MeetBot at http://wiki.debian.org/MeetBot.
00:05:20 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
00:05:23 <openstack> The meeting name has been set to 'congressteammeeting'
00:06:16 <ekcs> hi all okay finally got in. weird IRC client issue. sorry about that.
00:06:41 <thinrichs> ekcs: no problem.  I had to restart my machine to get in—my IRC client was not working at all
00:07:00 <thinrichs> To start off, quick congratulations are in order.
00:07:11 <thinrichs> ramineni, ekcs: you're now officially core reviewers!
00:07:14 <thinrichs> Congratulations!
00:07:32 <ramineni_> thinrichs: thanks :)
00:07:35 <ekcs> awesomeness! congrats ramineni.
00:07:59 <masahito> congratulations!
00:08:07 <ramineni_> ekcs: you too :)
00:08:34 <ekcs> thanks!
00:08:46 <thinrichs> You should have +2 rights now.  I updated gerrit just before the meeting.
00:09:11 <tsandall> congrats!
00:10:21 <ekcs> now I can get free PyCharm
00:11:22 <thinrichs> ekcs: I haven't used PyCharm before, but it looks interesting.
00:11:30 <thinrichs> ekcs: they give free versions to OpenStack core reviewers?
00:12:20 <ekcs> thinrichs: yup, with conditions.
00:12:25 <ekcs> #link https://www.jetbrains.com/buy/opensource/?product=pycharm
00:12:59 <thinrichs> Interesting.  Never heard of a vendor giving stuff to open source devs before.
00:13:15 <thinrichs> Ok, on to business as usual.
00:13:19 <thinrichs> #topic status updates
00:13:35 <thinrichs> Last week we were all going to look at ekcs's HA design and give feedback.
00:13:45 <thinrichs> ekcs: did you get what you needed?
00:14:28 <ekcs> I think enough to start the spec. I have a very very rough and incomplete spec draft up.
00:15:51 <ekcs> I do want to get people’s take on whether occasionally duplicated action executions are tolerable.
00:17:01 <thinrichs> I suppose that what we'll end up doing is choosing an architecture and then documenting the semantics of that arch.
00:17:11 <thinrichs> Or perhaps having a couple arch and let the user choose the one they want.
00:17:45 <ekcs> ok.
00:18:06 <thinrichs> I could imagine an "at-most" once semantics, and "at least once" semantics, and/or an "exactly once" semantics.  (Though not sure the last is achievable.)
00:19:15 <thinrichs> Maybe it'd be good to have these different options in the spec so we can think through how we'd accomplish each and what the tradeoffs are.
00:19:30 <ekcs> yup.
00:20:26 <ekcs> #link https://review.openstack.org/#/c/318383/1
00:20:31 <thinrichs> That said, we don't want to get bogged down in worrying about action-execution semantics if the rest is straightforward.
00:20:49 <ekcs> At this point though, it may still be easier to discuss over etherpad rather than gerrit.
00:21:17 <ekcs> ok.
00:21:20 <thinrichs> ekcs: etherpad sounds fine to me.  Has it changed enough that we should all take another look?
00:22:10 <ekcs> yea I added an overview comparison of different approaches. And recommended a different approach from what we discussed before.
00:22:36 <ekcs> replicate PEs (including react functionaliy), implement deduplication in ExecutionDriver (DS driver)
00:22:56 <thinrichs> #action Everyone takes another look at the etherpad
00:23:05 <thinrichs> ekcs: link?
00:23:10 <ekcs> #link https://etherpad.openstack.org/p/newton-congress-availability
00:24:20 <thinrichs> Anything else on the HA front?
00:25:01 <ekcs> The spec on gerrit is cleaner. So it may be easier to look at that. etherpad carries past discussions and everything, but because things are still quite fluid adding new discussion to etherpad may be easier than gerrit. up to you guys.
00:25:09 <ekcs> nothing else from me.
00:26:10 <thinrichs> Gerrit is the traditional place to discuss specs and keeps the history around as you push new versions of the spec (along with all the comments).
00:26:30 <thinrichs> So maybe we should use that, since we now have a proposal.
00:26:37 <ekcs> ok.
00:26:49 <masahito> I forgot looking the link last week. I'll check etherpad and the spec.
00:27:11 <thinrichs> On to the distributed architecture.
00:27:33 <thinrichs> I haven't kept up with the status this week.  How does everything look?
00:29:25 <thinrichs> Looks like there's 1 crucial patch from masahito for reloading policy rules.
00:29:42 <thinrichs> masahito: I'm guessing you're testing it out with the distributed_architecture flag set to True.
00:29:45 <thinrichs> how is it going?
00:30:51 <masahito> I think we need to address 2 things to pass jenkins test with the architecture.
00:31:21 <masahito> one is loading persisted rule and add default policy, which is addressed by my patch.
00:32:14 <masahito> another is enabling execute action in the architecture, which could be addressed by ramineni_'s patch.
00:32:44 <masahito> my patch: https://review.openstack.org/#/c/317008/
00:32:58 <masahito> ramineni_'s patch: https://review.openstack.org/#/c/316028/
00:33:08 <ramineni_> masahito: i  have added the policy execution in https://review.openstack.org/#/c/314941/
00:33:37 <ramineni_> masahito: rebased on your patch , so jenkins pass now
00:34:21 <thinrichs> Awesome!!  Unit and tempest tests all passing.
00:35:50 <thinrichs> That's a giant step forward.
00:36:05 <thinrichs> So once we get those 2 patches in, we can make the tests gating.
00:36:18 <ekcs> awesomeness
00:36:22 <thinrichs> And start removing the old tests
00:36:55 <thinrichs> and start removing logic for the old dse
00:37:21 <thinrichs> Let's all take a look at those patches and get them through review.
00:37:39 <thinrichs> ramineni: I also see you ported test_congress to dse2
00:37:44 <thinrichs> How did that go?
00:37:49 <thinrichs> Anything interesting show up?
00:38:02 <ramineni_> thinrichs: yes, migrated all the tests ..
00:38:21 <ramineni_> thinrichs: only policy execution was not working , so added some code for that
00:39:36 <thinrichs> Cool.  Those tests have always been a little odd, but good sanity checks that get run during unit testing
00:40:12 <ramineni_> ya
00:40:51 <thinrichs> Anyone else have any other status updates for the group?
00:42:53 <thinrichs> I managed to resurrect the code to deal with converting datasource IDs to names, and I got the VM-placement code turned into a datasource for the person asking for it.
00:43:15 <thinrichs> Nothing else technical from me.
00:43:40 <thinrichs> #topic newton1 planning
00:43:46 <thinrichs> newton1 is June 3
00:44:17 <thinrichs> Do we think it's reasonable to make the distributed_architecture flag default to True by then?
00:44:41 <thinrichs> masahito, ramineni, ekcs: thoughts?
00:45:09 <ekcs> might be able to do it with minimal testing.
00:45:59 <ramineni_> should be possible
00:46:17 <masahito> I think we can.
00:46:28 <thinrichs> That sounds like a good goal then.  Let's make it happen.
00:46:34 <thinrichs> Anything else for tonight?
00:46:44 <ekcs> is there any advantage to do it other than as a motivater for ourselves? I’m guessing some people might try it and help us identify problems.
00:48:07 <thinrichs> Just motivation.  Once distributed_arch is True all the time, we've effectively moved to the distributed_arch and can start removing the old code.
00:49:05 <ekcs> Okay got it. Yea it’d be great to get to dist arch true by default.
00:49:24 <thinrichs> Agreed
00:49:26 <thinrichs> Anything else for today?
00:49:35 <ekcs> but actually removing old code maybe should leave till last step like newton 3
00:49:47 <ekcs> about vmplacement and other specialty policy engines. I want to understand the vision. Are they going to be “equal” with agnostic? or agnostic is always going to be main PE.
00:50:53 <thinrichs> ekcs: I'm not sure when removing the old code is right.  I imagine we'll know it when we get there.
00:50:54 <masahito> should we move in-memory MQ to rabbit MQ?
00:51:31 <masahito> IIRC, ramineni_ suggests it at last summit.
00:52:04 <thinrichs> ekcs: we've never had a super-clear vision for the specialized policy engine thing
00:52:26 <thinrichs> ekcs: Right now I think of it more as a way to experiment
00:52:53 <ekcs> thinrichs: ok. just reminded of it when you mentioned vmplacement. Something I kind of thought about in HA deployment design. Not a blocking issue.
00:53:31 <thinrichs> masahito: move to rabbit MQ always, or just when we do an HA deploy?
00:54:00 <masahito> I mean moving to rabbit MQ always.
00:54:25 <ekcs> masahito: probably not rabbitMQ always. cuz then it requires everyone to deploy rabbitMQ. even single node.
00:54:52 <masahito> before we change the flag set to True.
00:55:24 <thinrichs> Is there a downside to using the in-memory version if someone is deploying on 1 node?
00:55:46 <thinrichs> As ekcs says, it would be nice if someone is just testing it out to not need to install and run rabbitmq
00:57:28 <masahito> if we can use rabbitmq the messages are stored in it. so HA stuff for messages is done.
00:57:29 <ekcs> thinrichs: one potential downside. I’m not sure how solid support for in-mem driver is. like whether it’s just something intended to kind of work for testing. and maybe features get messed up and removed without much consideration.
00:59:18 <thinrichs> masahito: rabbitmq is definitely necessary for HA.  I'm just wondering about people spinning up Congress for some tests who don't really care about HA.
00:59:45 <thinrichs> ekcs: makes sense
01:00:03 <masahito> thinrichs: oh. ok
01:00:06 <ekcs> thinrichs: I’ll take a closer look at that. I don’t expect it to be a problem though.
01:00:22 <thinrichs> So if the in-memory support may not be so solid, it makes sense to make sure everything is running on RabbitMQ before switching the flag to True by default.
01:00:25 <ekcs> it’s kombu project I think they’re going to support memory transport.
01:00:47 <thinrichs> Oops—out of time.
01:00:49 <ekcs> but i’ll verify.
01:01:18 <thinrichs> Let's follow up with that, and do some testing with RabbitMQ so we have a fallback if the in-memory stuff is flakey.
01:01:22 <thinrichs> Thanks all!
01:01:25 <thinrichs> #endmeeting