00:02:55 #startmeeting CongressTeamMeeting 00:02:56 Meeting started Thu Aug 18 00:02:55 2016 UTC and is due to finish in 60 minutes. The chair is thinrichs. Information about MeetBot at http://wiki.debian.org/MeetBot. 00:02:57 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 00:03:00 The meeting name has been set to 'congressteammeeting' 00:03:27 hi 00:03:29 hi 00:03:54 hi 00:04:04 aimeeu: courtesy ping (she maybe on vacation) 00:04:05 welcome back masahito ! 00:04:26 Yes, I think aimeeu is away. 00:04:46 Agenda for today.. 00:04:50 ekcs: thanks 00:04:57 1. milestones for newton 00:04:59 2. status 00:05:01 Anything else? 00:05:47 #topic Newton milestones 00:05:53 Here's the schedule... 00:05:58 #link http://releases.openstack.org/newton/schedule.html 00:06:04 Few noteworthy items.. 00:06:33 Aug 29-Sep 02: newton-3 and final release for client libraries 00:06:51 So that's about 2 weeks away 00:07:29 masahito: are you still planning to expose lazy polling for datasources thru the aPI? We'll want client code in place for the same. 00:07:48 Yes. 00:08:11 I notice that we don't need to modify client codes. 00:08:34 because the flag for lazy is in config option of datasource. 00:09:01 That's good. So it's set at the time of creating the datasource, and if you want to change it you delete the datasource and create a new one. 00:09:10 Is that what you had in mind? 00:09:19 yes. 00:09:35 Cool. I love getting functionality in without changing the API! 00:09:53 Means we have a solid API. 00:10:32 If we want to change config value of datasource on the fly, I think it's different API. 00:10:46 but now, this is out of scope. 00:11:04 masahito: makes sense to me 00:11:21 Other item on the calendar is that PTL nominations are Sep 12-16. 00:11:33 If anyone is interested and wants to chat about what's involved, let me know. 00:12:22 got it. 00:12:28 One thing we should discuss is what we hope to finish by newton-3 00:12:40 I think we can save that til after we go through status updates 00:12:58 Moving on. 00:13:01 #topic status 00:13:13 ramineni_: want to give your status update? 00:13:27 thinrichs: ok 00:15:30 thinrichs: im trying to add a new job in gate for HA , to run existing tempest tests in HA, need to test locally first 00:16:49 thinrichs: but im thinking of starting only one more PE process not the API one 00:16:58 Remind me what deployments we're testing now… 00:17:00 PE+DSD+API in 1 process, 00:17:08 PE, DSD, API in 3 separate processes 00:17:37 What's the HA version? 00:18:00 thinking of API+PE+PE+DSD 00:18:14 all seperate processes 00:18:52 because we cant start myultiple API´s on the same node, it will conflict with port 00:19:19 i think we should be able to test with multiple PE´s 00:19:53 So for policy API calls we'll let oslo-messaging pick the PE to send the message to 00:20:02 yes 00:20:13 As long as we think oslo-messaging won't always pick the same PE, that seems fine 00:20:20 ekcs: what do you think? 00:20:58 Another option: is it possible/easy to (a) startup multiple API servers on different ports and (b) write tempest tests to send requests to different ports? 00:21:09 we discussed it last time. That’s what I also thought is best, but we should check to make sure oslo won’t always pick same. 00:21:53 thinrichs: thats what current HA tempest tests does 00:22:06 it starts on different port 00:22:21 but isn´t it the same 00:22:43 anyway we are sending it to one API only 00:23:05 ramineni_: Ok. Sounds like you're doing all the right things then. 00:23:13 ramineni_: anything you need from us? 00:23:30 if it’s easy to start on diff ports, then I think that would be best. 00:23:52 and if it's easy to send requests to different ports. We don't do that today 00:24:12 ekcs: y? 00:24:42 two reasons: 1. that way we have more control over the test. send to A then to B. more reproducible if there is failure. 00:25:01 2. that way we test in the recommended deployment config of API+PE in one node. 00:25:28 but I didn’t know whether it was easy or hard to start on diff ports and send requests to diff ports. 00:25:31 just my thoughts. 00:25:33 ekcs: +1 Lots of nondeterminism in letting oslo-messaging pick the PE 00:26:01 Seems worth investigating whether we can send API requests to different ports in tempest 00:26:18 ekcs: thinrichs: i still dont get it, even if we start on different ports PE would be automatically chosen 00:26:45 may be we could discuss this later 00:27:12 yea maybe I’m confused about whether we can send a request to a specific port. 00:27:21 If we deployed 3 processes: (i) API+PE, (ii) API + PE, (iii) DSD 00:27:34 then we can write tests that send API calls to (i) or (ii) explicitly. 00:27:49 (Assuming we can send API calls to different ports) 00:28:57 Let's come back to this later if there's time. 00:29:05 masahito: want to give a status report? 00:29:50 thinrichs: sure 00:30:31 I'm updating a patch for lazy datasource. I'll push it today. 00:30:48 that's from my side. not a lot progress. 00:31:41 masahito: thanks 00:31:59 ekcs: status update? 00:32:13 Submitted patches for: 00:32:14 1. sync datasources by UUID to avoid mix-up due to datasoure name reuse. #link https://review.openstack.org/#/c/356157/ 00:32:14 ekcs: patch 356157 - congress - HAHT - datasource synchronizer use UUID not name 00:32:15 2. resubscribe if sequenced data missing for too long. #link https://review.openstack.org/#/c/356806/ 00:32:15 ekcs: patch 356806 - congress - DSE2 - resubscribe if update missing for too long 00:32:16 3. quick fix for ironic_driver that prevented congress from starting. 00:32:19 That’s all. 00:33:03 Anything to discuss on those patches here? 00:34:01 not really. 00:34:27 On the resubscribe if update missing patch, did you run into that case? 00:34:56 no I didn’t, but it was always planned (in my head =p ) 00:35:20 without it, if a single update gets lost, no more updates can be processed from that table forever. 00:35:50 Just wondered if you were doing some serious testing. Seemed like a tough one to hit by accident. 00:36:25 I'll take a look tomorrow probably 00:36:30 I'll do a quick update 00:36:33 no haven’t done heavy testing. the version without resubscribe was basically meant for single-process dse2 use. 00:36:54 I've been underwater all week. No real progress. Just tried to keep up with reviews. 00:37:19 Let's move to the newton-3 discussion 00:37:22 #topic newton-3 00:37:39 We should figure out what we want to finish up in the next 2 weeks. 00:38:09 I think HAHT is on track for newton-3. All major required code changes are merged or in review. Remaining are tests and docs and optional features and I'm sure bug fixes. Still a fair amount of work for tests and docs, but I assume it's not the end of the world if those slip a couple weeks. 00:38:10 Here are the remaining bugs targeting newton-3 #link https://launchpad.net/congress/+milestone/newton-3 00:39:05 slip into RC1 that is. 00:40:03 actually here’s a question about newton-3. 00:40:06 Soft StringFreeze¶ 00:40:06 You are no longer allowed to accept proposed changes containing modifications in user-facing strings. Such changes should be rejected by the review team and postponed until the next series development opens (which should happen when RC1 is published). 00:40:28 does that mean no changes to docs or just no changes to program output? 00:40:49 Pretty sure that's program output 00:41:01 ok good =) 00:41:12 That's for translation, I'm pretty sure. It gives the translation team time to internationalize the messages. 00:41:13 “user-facing strings” is a bit ambiguous. 00:41:39 Docs always lag behind code, so freezing the docs before the code doesn't make sense 00:42:07 We haven't gotten around to internationalizing, so the string freeze doesn't apply 00:42:33 Here's my first question: are we stable without HAHT deployments? I.e. if we deploy in 1 process? 00:43:08 i’ve only done very light testing so far. 00:43:20 We have tempest tests running, so that gives us some confidence 00:43:40 I'd say that once we hit newton-3, we should do some of that testing immediately. 00:44:27 The goal then is to get the HAHT code in by newton-3 and use the remainder of the cycle to find/fix bugs. 00:44:50 It looks like all the code is underway. 00:45:00 Any worries about landing the ones in progress? 00:45:24 The synchronizer update for distributed deployment (ramineni_)? 00:45:37 datasource add/delete (ekcs)? 00:46:29 no notable concern on my end. 00:46:37 i have patch for that already 00:47:14 Awesome! 00:47:37 I hope we can get all tests in by newton-3 too, to expose bugs as early as possible. should be doable. 00:47:55 ekcs: +1 00:48:07 Maybe what I can do is save up some Congress time and once those two land start manual testing 00:48:40 new devstack will be interesting. 00:48:57 s/new/fresh 00:49:18 That's true. That'll take some time to get set up. 00:49:40 Is our devstack plugin stable at this point? 00:50:02 I could start getting that running now and then just upgrade the Congress code once it comes in 00:50:53 thinrichs: sorry, I have to leave now, will catchup later 00:51:05 masahito: are you expecting your patches to land in time? 00:51:14 ramineni_: sure. Thanks. See you later 00:51:26 thinrichs: yap. 00:52:10 Great! 00:52:18 That's all I have for today. 00:52:25 Anything else to discuss? 00:52:28 #topic open discussion 00:52:33 i think I realized that ramineni_ meant on ha tempest test 00:53:08 we’re trying to reuse existing tempest tests on new deployment configuration. so we’re not targeting specific nodes anyway. 00:53:34 ekcs: that makes sense. Reusing existing tests 00:54:03 So I guess we'll need new variants of tests for the HA stuff 00:54:28 I guess those should be focused on testing just the HAHT features. We can leave the more basic functionality to the existing tests 00:54:35 so the tests themselves won’t target specific nodes. but we could still write a new layer that sends requests to diff ports in a deterministic fashion. 00:55:20 right. and from our discussion last time, i think the conclusion is that most of the detailed HA feature testing should be done in test_congress style unit tests. 00:55:39 ekcs: makes sense 00:55:56 so it may or may not be worth doing extra work to adapt the existing tempest tests to distribute requests to diff ports. 00:56:34 Agreed 00:56:39 determinism still nice. but seems like we can go either way. 00:57:12 I'd say start with something that tests multiple PEs and if nondeterminism becomes a problem address it then 00:57:15 agreed. 00:57:21 agreed. 00:57:49 So it seems we're agreed about the API + PE + PE + DSD setup for running tempest tests 00:58:06 makes sense to me. 00:58:08 ekcs: how thorough would you say our test-congress tests are for HAHT? 00:58:33 2 min left 00:58:34 not thorough at all at the moment. 00:59:02 I was thinking specific failover tests would be in tempest. but now targeting unittests after discussion. 00:59:27 So one of those bugs is actually targeting test-congress tests, instead of tempest 00:59:29 not thorough in terms of simulating actual failvover scenario. 00:59:48 pretty good for testing specific behavior. 00:59:56 basically their more unit tests than integrationtests. 01:00:01 ekcs: got it. Good to know 01:00:15 yes. i’ll change the bug. 01:01:03 Out of time. Thanks all! 01:01:05 #endmeeting