00:01:31 #startmeeting CongressTeamMeeting 00:01:32 Meeting started Thu Jul 28 00:01:31 2016 UTC and is due to finish in 60 minutes. The chair is thinrichs. Information about MeetBot at http://wiki.debian.org/MeetBot. 00:01:33 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 00:01:35 The meeting name has been set to 'congressteammeeting' 00:01:41 hi 00:01:58 Agenda for today: (i) mascots, (ii) status 00:02:01 Anything else? 00:02:58 No ekcs yet. Maybe we should hold off on the mascot topic. 00:03:13 We need to decide for sure today, so let's not forget. 00:03:21 Let's do status updates. 00:03:23 #topic status 00:03:27 ramineni: want to start? 00:03:33 thinrichs: sure 00:03:48 sorry delayed by software weirdness. 00:04:24 thinrichs: completed synchronizer migration and horizon plugin model 00:04:39 thinrichs: bth looks ready to merge 00:04:54 thinrichs: thats it from my side 00:05:10 Synchronizer code is merged 00:05:39 oh great , havent checked it yet 00:05:53 Looks like 1 more typo in the horizon plugin. Then we should merge it. 00:06:13 ok, will have a look 00:06:13 very minor typo (sorry!) 00:06:25 ramineni: Nice work getting those two big changes in 00:07:17 aimeeu: want to go next? 00:07:22 sure 00:07:50 I haven't had much time this week to work on Congress. I started the HAHT overview guide and plan work on the python-congressclient support for identity v3 token unless ramineni wants to take that over. I'm having trouble getting Tempest to work on devstack. 00:08:48 and I thought a tempest test would be good to test identity v3 in the python-congressclient 00:09:01 that's all for me 00:09:04 We all have trouble getting tempest working. 00:09:29 ah, good to know - I've disabled the service for now 00:09:49 and plan to make code changes and test manually 00:10:02 A v3 identity test for the client would be good. Is there a way to do it with a unit test? It wouldn't be a live test, but might get us partway. 00:10:12 Tempest test would also be good 00:10:25 I'll look into that - thanks for the suggestion! 00:11:01 masahito: want to go next? 00:11:06 ok 00:11:40 Pacemaker RA and its deploy guide for HAHT were merged. 00:11:59 And I pushed first code for lazy datasource driver. 00:12:46 The patch got tempest failure, so I'm investigating it. 00:12:54 That's from my side. 00:13:33 Need feedback on the lazy driver? Or is it too early? 00:14:11 Need review. 00:14:32 Will do. 00:14:40 The tempest failure is in ha tests that we all hit. 00:16:14 Okay. Have those tempest gate tests been pretty reliable lately? Or are they still flakey sometimes? 00:17:41 if its timeout issue, Eric´s patch should solve it , else have to look at it 00:17:42 ekcs: want to go next? 00:18:02 ramineni: good to know. Thought it was all pretty solid at this point. 00:18:03 sure. 00:18:32 API routing patch done but stuck at a gate failure. I think the failure is unrelated to the patch and probably occassional (checks passed, but gate failed). But I'm not sure what the cause is. 00:18:33 thinrichs: for dse1 test it's reliable, for dse2 test the log don't show the details. 00:18:43 I did a lot of thinking and testing about the push data persistence design. Writing table as string to mysql or writing table as mysql tables. 00:19:31 masahito: got it. 00:19:39 ekcs: what did you conclude about the push data persistence? 00:20:29 Using 100kb json string (500 rows), writing a whole new json string is faster than adding 5 and deleting 5 rows. (20ms vs 100ms on very slow laptop HDD, 10 times faster on SSD) 00:21:27 Interesting stats 00:21:31 as we keep increasing table size, writing whole json string will eventually lose to adding/deleting rows. I think it happens around 1000kb. 00:22:36 So if people are pushing 1MB+ of data, it makes more sense to compute the diff and update individual rows in the DB. 00:23:05 Did you think through the issues around upgrade? 00:24:07 yes. But I think the best approach is to write string not rows, but allow writing string for incremental update (say 100 updates before writing a full snapshot and getting rid of those updates). 00:25:32 for upgrade, I think either approach could be made to work. writing string into single table is easier than writing each ds table into db table. 00:26:18 and the decision around when to write full snapshot and when to write diff can be made on a case by case basis if we write string rather than db rows 00:26:35 Not sure I follow this exactly. 00:26:43 So there's 1 DB table for all push datasources 00:27:10 That table has a column for datasource-name, datasource-state-as-a-string 00:27:30 yes. 00:27:31 Where do the deltas come in? 00:28:30 and for handling large tables (1m+), we can in addition store datasource-state-diff-as-string, indexed increasing number since the last snapshot. 00:29:24 in another table 00:29:55 the key is that this is usually a write only store, we very rarely expect to have to read. 00:30:02 Then when we write a new snapshot we delete all the rows in that table for that dsname 00:30:15 yes. 00:30:42 Another thought… we could add a col to the original table for 'diff-since-last-snapshot-string' 00:30:55 and then update that each time 00:31:13 yea that’s a good option to consider too. 00:31:30 Hmmm…that might be awkward though b/c the delta wouldn't be based on the in-memory snapshot, but rather a snapshot from the past. 00:32:16 the intuition around it is that mysql table is optimized for read that’s why it’s slow even to add&delete 10 rows. we really only care about write. 00:33:02 I'd imagine starting with just writing the snapshot as a string. Then optimizing as needed. 00:33:11 yup my thought too. 00:33:33 ekcs: good observation that we're not reading from this table very often. We've effectively got an in-memory cache. 00:34:12 ekcs: Thanks for digging into that! 00:34:16 one problem. to store long string in column, 00:34:34 we need to use special DB specific datatype (longtext, clob, etc.). 00:34:51 so we may lose the ability to arbitrarily swap mysql for postgres etc. 00:34:57 i’ll need to look at that deeper. 00:35:04 Is that supported by oslo-db? 00:35:32 I think so (supported by sqlalchemy). but maybe I need to make sure. 00:35:46 for sure supported by sqlalchemy. 00:35:57 just not sure if the oslo-db version restricts. 00:36:32 oslo-db is the OpenStack database layer, right? If that does it, I'd say it's fine. If not, I'm less certain. 00:36:42 ok. 00:36:51 i’ll make sure. that’s all from me. 00:37:40 Thanks! 00:37:44 Here's my status update. 00:38:33 I've tweaked my patch that disables DSE1 a bit. Had mistakenly removed some tests when moving tests2 to tests. 00:39:03 Now I'm working on rebasing the patch on top of the few that went in recently. 00:39:09 Enough changes that it's nontrivial. 00:39:44 This patch is large by its nature. 00:39:57 Future patches that finish off disabling dse1 will be easier to contain 00:40:20 So I'm looking forward to getting this merged 00:40:28 That's it from me. 00:41:06 2 more topics… 00:41:16 s/topics/agenda items/ 00:41:20 #topic mascots 00:41:28 We need to finalize our mascot decision today. 00:41:44 I gave our suggestions to get a little early feedback. here's what they said... 00:42:05 We're not allowed to choose a specific land item like Aeropagus 00:42:24 We could choose a large rock, but they thought it would just look like a blob and recommended against it. 00:43:22 There were a few lizard-like proposals from other teams, so they might have trouble differentiating a salamander from those. 00:43:30 The Raven seemed somewhat unique. 00:43:39 No one chose a monkey/baboon 00:44:19 They say all of those will work. 00:44:45 Personally I'm leaning toward raven and then salamander 00:44:49 Any other thoughts? 00:45:16 I think so too. 00:45:18 I agree with the raven 00:45:29 I agree the order. 00:45:50 but I really really don’t want a monkey or baboon. 00:45:57 ekcs: agreed 00:46:00 we have to submit three things right? 00:46:02 ekcs: agreed on the no baboon 00:46:20 I think just 2 is fine 00:46:26 They're going to pick 1, after all 00:46:42 And they've already told us both the raven and salamander are fine 00:47:21 got it. 00:47:35 +1 00:47:41 Okay, so I'll send them (1) raven and (2) salamander 00:48:08 final topic… 00:48:13 #topic Barcelona 00:48:21 Voting for talks is open. 00:48:29 * thinrichs looking for link 00:48:59 #link https://www.openstack.org/summit/barcelona-2016/vote-for-speakers/ 00:49:12 If you search for Congress on the left-hand side, you can see the talks people have submitted 00:49:40 #topic open discussion 00:49:49 Anything we should discuss? 00:50:19 aimeeu: one thing before I forget. ramineni and I ran across some low-hanging fruit you might be interested in. 00:50:33 excellent 00:51:03 we noticed that we're using 'e.message' in a bunch of places, where e is an exception. That works in Python2 but not Python3. We should be using 'str(e)' instead. 00:51:32 I could use a morale-boosting easy fix! (and one that lets me read lots of code) 00:51:41 aimeeu: :) 00:51:52 has a bug been created? 00:52:10 Not yet. 00:52:40 any thoughts on the gate failure? #link https://review.openstack.org/#/c/341904/In setting up test_datasource_db_sync_remove, datasource 'fake' seems to already exists. A wild guess is some kind of leftover state that carried forward from one test to the next, but the frequency of failure seems too low for that to be the case. 00:52:55 thinrichs: I can create the bug and assign it to myself. 00:52:59 ekcs: does that Py2 vs Py3 behavior sound right to you? I don't understand why we didn't hit the e.message problem before this 00:53:05 aimeeu: that would be great! 00:53:14 yes that’s a py2vs3 thing. 00:53:33 probbaly because we just haven’t hit those errors in testing. 00:53:51 That's a bit scary. We use that in like 50 places 00:54:44 yea. I think most people still do manual testing in py2 00:55:13 ekcs: that's the synchronizer test. All kinds of potential race conditions I'd imagine. 00:56:12 I dug through some code to understand the setup and teardown. and I can’t seem to understand exactly what may persist from test to test. it’s the same devstack instance right not a new one? 00:56:46 ramineni's the expert here 00:57:29 :) , not sure , ill have a look at it today , is it because of new synchronizer code mergd? 00:58:17 oh hmm that’s possible. it was the last patch to go in before the failures. 00:58:31 ekcs: yep, ill take a look today, and let you know if I find anything 00:58:39 great thanks! 00:59:15 Out of time for today. Thanks all! 00:59:24 thanks all 00:59:47 bye! 01:01:08 #endmeeting