00:02:55 <thinrichs> #startmeeting CongressTeamMeeting
00:02:56 <openstack> Meeting started Thu Aug 18 00:02:55 2016 UTC and is due to finish in 60 minutes.  The chair is thinrichs. Information about MeetBot at http://wiki.debian.org/MeetBot.
00:02:57 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
00:03:00 <openstack> The meeting name has been set to 'congressteammeeting'
00:03:27 <ramineni_> hi
00:03:29 <masahito> hi
00:03:54 <ekcs> hi
00:04:04 <thinrichs> aimeeu: courtesy ping (she maybe on vacation)
00:04:05 <ekcs> welcome back masahito !
00:04:26 <ekcs> Yes, I think aimeeu is away.
00:04:46 <thinrichs> Agenda for today..
00:04:50 <masahito> ekcs: thanks
00:04:57 <thinrichs> 1. milestones for newton
00:04:59 <thinrichs> 2. status
00:05:01 <thinrichs> Anything else?
00:05:47 <thinrichs> #topic Newton milestones
00:05:53 <thinrichs> Here's the schedule...
00:05:58 <thinrichs> #link http://releases.openstack.org/newton/schedule.html
00:06:04 <thinrichs> Few noteworthy items..
00:06:33 <thinrichs> Aug 29-Sep 02: newton-3 and final release for client libraries
00:06:51 <thinrichs> So that's about 2 weeks away
00:07:29 <thinrichs> masahito: are you still planning to expose lazy polling for datasources thru the aPI?  We'll want client code in place for the same.
00:07:48 <masahito> Yes.
00:08:11 <masahito> I notice that we don't need to modify client codes.
00:08:34 <masahito> because the flag for lazy is in config option of datasource.
00:09:01 <thinrichs> That's good.  So it's set at the time of creating the datasource, and if you want to change it you delete the datasource and create a new one.
00:09:10 <thinrichs> Is that what you had in mind?
00:09:19 <masahito> yes.
00:09:35 <thinrichs> Cool.  I love getting functionality in without changing the API!
00:09:53 <thinrichs> Means we have a solid API.
00:10:32 <masahito> If we want to change config value of datasource on the fly, I think it's different API.
00:10:46 <masahito> but now, this is out of scope.
00:11:04 <thinrichs> masahito: makes sense to me
00:11:21 <thinrichs> Other item on the calendar is that PTL nominations are Sep 12-16.
00:11:33 <thinrichs> If anyone is interested and wants to chat about what's involved, let me know.
00:12:22 <ekcs> got it.
00:12:28 <thinrichs> One thing we should discuss is what we hope to finish by newton-3
00:12:40 <thinrichs> I think we can save that til after we go through status updates
00:12:58 <thinrichs> Moving on.
00:13:01 <thinrichs> #topic status
00:13:13 <thinrichs> ramineni_: want to give your status update?
00:13:27 <ramineni_> thinrichs: ok
00:15:30 <ramineni_> thinrichs: im trying to add a new job in gate for HA , to run existing tempest tests in HA, need to test locally first
00:16:49 <ramineni_> thinrichs: but im thinking of starting only one more PE process not the API one
00:16:58 <thinrichs> Remind me what deployments we're testing now…
00:17:00 <thinrichs> PE+DSD+API in 1 process,
00:17:08 <thinrichs> PE, DSD, API in 3 separate processes
00:17:37 <thinrichs> What's the HA version?
00:18:00 <ramineni_> thinking of API+PE+PE+DSD
00:18:14 <ramineni_> all seperate processes
00:18:52 <ramineni_> because we cant start myultiple API´s on the same node, it will conflict with port
00:19:19 <ramineni_> i think we should be able to test with multiple PE´s
00:19:53 <thinrichs> So for policy API calls we'll let oslo-messaging pick the PE to send the message to
00:20:02 <ramineni_> yes
00:20:13 <thinrichs> As long as we think oslo-messaging won't always pick the same PE, that seems fine
00:20:20 <thinrichs> ekcs: what do you think?
00:20:58 <thinrichs> Another option: is it possible/easy to (a) startup multiple API servers on different ports and (b) write tempest tests to send requests to different ports?
00:21:09 <ekcs> we discussed it last time. That’s what I also thought is best, but we should check to make sure oslo won’t always pick same.
00:21:53 <ramineni_> thinrichs: thats what current HA tempest tests does
00:22:06 <ramineni_> it starts on different port
00:22:21 <ramineni_> but isn´t it the same
00:22:43 <ramineni_> anyway we are sending it to one API only
00:23:05 <thinrichs> ramineni_: Ok.  Sounds like you're doing all the right things then.
00:23:13 <thinrichs> ramineni_: anything you need from us?
00:23:30 <ekcs> if it’s easy to start on diff ports, then I think that would be best.
00:23:52 <thinrichs> and if it's easy to send requests to different ports.  We don't do that today
00:24:12 <ramineni_> ekcs: y?
00:24:42 <ekcs> two reasons: 1. that way we have more control over the test. send to A then to B. more reproducible if there is failure.
00:25:01 <ekcs> 2. that way we test in the recommended deployment config of API+PE in one node.
00:25:28 <ekcs> but I didn’t know whether it was easy or hard to start on diff ports and send requests to diff ports.
00:25:31 <ekcs> just my thoughts.
00:25:33 <thinrichs> ekcs: +1 Lots of nondeterminism in letting oslo-messaging pick the PE
00:26:01 <thinrichs> Seems worth investigating whether we can send API requests to different ports in tempest
00:26:18 <ramineni_> ekcs: thinrichs: i still dont get it, even if we start on different ports PE would be automatically chosen
00:26:45 <ramineni_> may be we could discuss this later
00:27:12 <ekcs> yea maybe I’m confused about whether we can send a request to a specific port.
00:27:21 <thinrichs> If we deployed 3 processes: (i) API+PE, (ii) API + PE, (iii) DSD
00:27:34 <thinrichs> then we can write tests that send API calls to (i) or (ii) explicitly.
00:27:49 <thinrichs> (Assuming we can send API calls to different ports)
00:28:57 <thinrichs> Let's come back to this later if there's time.
00:29:05 <thinrichs> masahito: want to give a status report?
00:29:50 <masahito> thinrichs: sure
00:30:31 <masahito> I'm updating a patch for lazy datasource. I'll push it today.
00:30:48 <masahito> that's from my side. not a lot progress.
00:31:41 <thinrichs> masahito: thanks
00:31:59 <thinrichs> ekcs: status update?
00:32:13 <ekcs> Submitted patches for:
00:32:14 <ekcs> 1. sync datasources by UUID to avoid mix-up due to datasoure name reuse. #link https://review.openstack.org/#/c/356157/
00:32:14 <patchbot> ekcs: patch 356157 - congress - HAHT - datasource synchronizer use UUID not name
00:32:15 <ekcs> 2. resubscribe if sequenced data missing for too long. #link https://review.openstack.org/#/c/356806/
00:32:15 <patchbot> ekcs: patch 356806 - congress - DSE2 - resubscribe if update missing for too long
00:32:16 <ekcs> 3. quick fix for ironic_driver that prevented congress from starting.
00:32:19 <ekcs> That’s all.
00:33:03 <thinrichs> Anything to discuss on those patches here?
00:34:01 <ekcs> not really.
00:34:27 <thinrichs> On the resubscribe if update missing patch, did you run into that case?
00:34:56 <ekcs> no I didn’t, but it was always planned (in my head =p )
00:35:20 <ekcs> without it, if a single update gets lost, no more updates can be processed from that table forever.
00:35:50 <thinrichs> Just wondered if you were doing some serious testing.  Seemed like a tough one to hit by accident.
00:36:25 <thinrichs> I'll take a look tomorrow probably
00:36:30 <thinrichs> I'll do a quick update
00:36:33 <ekcs> no haven’t done heavy testing. the version without resubscribe was basically meant for single-process dse2 use.
00:36:54 <thinrichs> I've been underwater all week.  No real progress.  Just tried to keep up with reviews.
00:37:19 <thinrichs> Let's move to the newton-3 discussion
00:37:22 <thinrichs> #topic newton-3
00:37:39 <thinrichs> We should figure out what we want to finish up in the next 2 weeks.
00:38:09 <ekcs> I think HAHT is on track for newton-3. All major required code changes are merged or in review. Remaining are tests and docs and optional features and I'm sure bug fixes. Still a fair amount of work for tests and docs, but I assume it's not the end of the world if those slip a couple weeks.
00:38:10 <ekcs> Here are the remaining bugs targeting newton-3 #link https://launchpad.net/congress/+milestone/newton-3
00:39:05 <ekcs> slip into RC1 that is.
00:40:03 <ekcs> actually here’s a question about newton-3.
00:40:06 <ekcs> Soft StringFreeze¶
00:40:06 <ekcs> You are no longer allowed to accept proposed changes containing modifications in user-facing strings. Such changes should be rejected by the review team and postponed until the next series development opens (which should happen when RC1 is published).
00:40:28 <ekcs> does that mean no changes to docs or just no changes to program output?
00:40:49 <thinrichs> Pretty sure that's program output
00:41:01 <ekcs> ok good =)
00:41:12 <thinrichs> That's for translation, I'm pretty sure.  It gives the translation team time to internationalize the messages.
00:41:13 <ekcs> “user-facing strings” is a bit ambiguous.
00:41:39 <thinrichs> Docs always lag behind code, so freezing the docs before the code doesn't make sense
00:42:07 <thinrichs> We haven't gotten around to internationalizing, so the string freeze doesn't apply
00:42:33 <thinrichs> Here's my first question: are we stable without HAHT deployments?  I.e. if we deploy in 1 process?
00:43:08 <ekcs> i’ve only done very light testing so far.
00:43:20 <thinrichs> We have tempest tests running, so that gives us some confidence
00:43:40 <thinrichs> I'd say that once we hit newton-3, we should do some of that testing immediately.
00:44:27 <thinrichs> The goal then is to get the HAHT code in by newton-3 and use the remainder of the cycle to find/fix bugs.
00:44:50 <thinrichs> It looks like all the code is underway.
00:45:00 <thinrichs> Any worries about landing the ones in progress?
00:45:24 <thinrichs> The synchronizer update for distributed deployment (ramineni_)?
00:45:37 <thinrichs> datasource add/delete (ekcs)?
00:46:29 <ekcs> no notable concern on my end.
00:46:37 <ramineni_> i have patch for that already
00:47:14 <thinrichs> Awesome!
00:47:37 <ekcs> I hope we can get all tests in by newton-3 too, to expose bugs as early as possible. should be doable.
00:47:55 <thinrichs> ekcs: +1
00:48:07 <thinrichs> Maybe what I can do is save up some Congress time and once those two land start manual testing
00:48:40 <ekcs> new devstack will be interesting.
00:48:57 <ekcs> s/new/fresh
00:49:18 <thinrichs> That's true.  That'll take some time to get set up.
00:49:40 <thinrichs> Is our devstack plugin stable at this point?
00:50:02 <thinrichs> I could start getting that running now and then just upgrade the Congress code once it comes in
00:50:53 <ramineni_> thinrichs: sorry, I have to leave now, will catchup later
00:51:05 <thinrichs> masahito: are you expecting your patches to land in time?
00:51:14 <thinrichs> ramineni_: sure.  Thanks.  See you later
00:51:26 <masahito> thinrichs: yap.
00:52:10 <thinrichs> Great!
00:52:18 <thinrichs> That's all I have for today.
00:52:25 <thinrichs> Anything else to discuss?
00:52:28 <thinrichs> #topic open discussion
00:52:33 <ekcs> i think I realized that ramineni_ meant on ha tempest test
00:53:08 <ekcs> we’re trying to reuse existing tempest tests on new deployment configuration. so we’re not targeting specific nodes anyway.
00:53:34 <thinrichs> ekcs: that makes sense.  Reusing existing tests
00:54:03 <thinrichs> So I guess we'll need new variants of tests for the HA stuff
00:54:28 <thinrichs> I guess those should be focused on testing just the HAHT features.  We can leave the more basic functionality to the existing tests
00:54:35 <ekcs> so the tests themselves won’t target specific nodes. but we could still write a new layer that sends requests to diff ports in a deterministic fashion.
00:55:20 <ekcs> right. and from our discussion last time, i think the conclusion is that most of the detailed HA feature testing should be done in test_congress style unit tests.
00:55:39 <thinrichs> ekcs: makes sense
00:55:56 <ekcs> so it may or may not be worth doing extra work to adapt the existing tempest tests to distribute requests to diff ports.
00:56:34 <thinrichs> Agreed
00:56:39 <ekcs> determinism still nice. but seems like we can go either way.
00:57:12 <thinrichs> I'd say start with something that tests multiple PEs and if nondeterminism becomes a problem address it then
00:57:15 <masahito> agreed.
00:57:21 <ekcs> agreed.
00:57:49 <thinrichs> So it seems we're agreed about the API + PE + PE + DSD setup for running tempest tests
00:58:06 <ekcs> makes sense to me.
00:58:08 <thinrichs> ekcs: how thorough would you say our test-congress tests are for HAHT?
00:58:33 <thinrichs> 2 min left
00:58:34 <ekcs> not thorough at all at the moment.
00:59:02 <ekcs> I was thinking specific failover tests would be in tempest. but now targeting unittests after discussion.
00:59:27 <thinrichs> So one of those bugs is actually targeting test-congress tests, instead of tempest
00:59:29 <ekcs> not thorough in terms of simulating actual failvover scenario.
00:59:48 <ekcs> pretty good for testing specific behavior.
00:59:56 <ekcs> basically their more unit tests than integrationtests.
01:00:01 <thinrichs> ekcs: got it.  Good to know
01:00:15 <ekcs> yes. i’ll change the bug.
01:01:03 <thinrichs> Out of time.  Thanks all!
01:01:05 <thinrichs> #endmeeting