00:01:31 <thinrichs> #startmeeting CongressTeamMeeting
00:01:32 <openstack> Meeting started Thu Jul 28 00:01:31 2016 UTC and is due to finish in 60 minutes.  The chair is thinrichs. Information about MeetBot at http://wiki.debian.org/MeetBot.
00:01:33 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
00:01:35 <openstack> The meeting name has been set to 'congressteammeeting'
00:01:41 <masahito> hi
00:01:58 <thinrichs> Agenda for today: (i) mascots, (ii) status
00:02:01 <thinrichs> Anything else?
00:02:58 <thinrichs> No ekcs yet.  Maybe we should hold off on the mascot topic.
00:03:13 <thinrichs> We need to decide for sure today, so let's not forget.
00:03:21 <thinrichs> Let's do status updates.
00:03:23 <thinrichs> #topic status
00:03:27 <thinrichs> ramineni: want to start?
00:03:33 <ramineni_> thinrichs: sure
00:03:48 <ekcs> sorry delayed by software weirdness.
00:04:24 <ramineni_> thinrichs: completed synchronizer migration and horizon plugin model
00:04:39 <ramineni_> thinrichs: bth looks ready to merge
00:04:54 <ramineni_> thinrichs: thats it from my side
00:05:10 <thinrichs> Synchronizer code is merged
00:05:39 <ramineni_> oh great , havent checked it yet
00:05:53 <thinrichs> Looks like 1 more typo in the horizon plugin.  Then we should merge it.
00:06:13 <ramineni_> ok, will have a look
00:06:13 <aimeeu> very minor typo (sorry!)
00:06:25 <thinrichs> ramineni: Nice work getting those two big changes in
00:07:17 <thinrichs> aimeeu: want to go next?
00:07:22 <aimeeu> sure
00:07:50 <aimeeu> I haven't had much time this week to work on Congress. I started the HAHT overview guide and plan work on the python-congressclient support for identity v3 token unless ramineni wants to take that over. I'm having trouble getting Tempest to work on devstack.
00:08:48 <aimeeu> and I thought a tempest test would be good to test identity v3 in the python-congressclient
00:09:01 <aimeeu> that's all for me
00:09:04 <thinrichs> We all have trouble getting tempest working.
00:09:29 <aimeeu> ah, good to know - I've disabled the service for now
00:09:49 <aimeeu> and plan to make code changes and test manually
00:10:02 <thinrichs> A v3 identity test for the client would be good.  Is there a way to do it with a unit test?  It wouldn't be a live test, but might get us partway.
00:10:12 <thinrichs> Tempest test would also be good
00:10:25 <aimeeu> I'll look into that - thanks for the suggestion!
00:11:01 <thinrichs> masahito: want to go next?
00:11:06 <masahito> ok
00:11:40 <masahito> Pacemaker RA and its deploy guide for HAHT were merged.
00:11:59 <masahito> And I pushed first code for lazy datasource driver.
00:12:46 <masahito> The patch got tempest failure, so I'm investigating it.
00:12:54 <masahito> That's from my side.
00:13:33 <thinrichs> Need feedback on the lazy driver?  Or is it too early?
00:14:11 <masahito> Need review.
00:14:32 <thinrichs> Will do.
00:14:40 <masahito> The tempest failure is in ha tests that we all hit.
00:16:14 <thinrichs> Okay.  Have those tempest gate tests been pretty reliable lately?  Or are they still flakey sometimes?
00:17:41 <ramineni_> if its timeout issue, Eric´s patch should solve it , else have to look at it
00:17:42 <thinrichs> ekcs: want to go next?
00:18:02 <thinrichs> ramineni: good to know.  Thought it was all pretty solid at this point.
00:18:03 <ekcs> sure.
00:18:32 <ekcs> API routing patch done but stuck at a gate failure. I think the failure is unrelated to the patch and probably occassional (checks passed, but gate failed). But I'm not sure what the cause is.
00:18:33 <masahito> thinrichs: for dse1 test it's reliable, for dse2 test the log don't show the details.
00:18:43 <ekcs> I did a lot of thinking and testing about the push data persistence design. Writing table as string to mysql or writing table as mysql tables.
00:19:31 <thinrichs> masahito: got it.
00:19:39 <thinrichs> ekcs: what did you conclude about the push data persistence?
00:20:29 <ekcs> Using 100kb json string (500 rows), writing a whole new json string is faster than adding 5 and deleting 5 rows. (20ms vs 100ms on very slow laptop HDD, 10 times faster on SSD)
00:21:27 <thinrichs> Interesting stats
00:21:31 <ekcs> as we keep increasing table size, writing whole json string will eventually lose to adding/deleting rows. I think it happens around 1000kb.
00:22:36 <thinrichs> So if people are pushing 1MB+ of data, it makes more sense to compute the diff and update individual rows in the DB.
00:23:05 <thinrichs> Did you think through the issues around upgrade?
00:24:07 <ekcs> yes. But I think the best approach is to write string not rows, but allow writing string for incremental update (say 100 updates before writing a full snapshot and getting rid of those updates).
00:25:32 <ekcs> for upgrade, I think either approach could be made to work. writing string into single table is easier than writing each ds table into db table.
00:26:18 <ekcs> and the decision around when to write full snapshot and when to write diff can be made on a case by case basis if we write string rather than db rows
00:26:35 <thinrichs> Not sure I follow this exactly.
00:26:43 <thinrichs> So there's 1 DB table for all push datasources
00:27:10 <thinrichs> That table has a column for datasource-name, datasource-state-as-a-string
00:27:30 <ekcs> yes.
00:27:31 <thinrichs> Where do the deltas come in?
00:28:30 <ekcs> and for handling large tables (1m+), we can in addition store datasource-state-diff-as-string, indexed increasing number since the last snapshot.
00:29:24 <ekcs> in another table <dsname, update #, ds-state-delta-as-string>
00:29:55 <ekcs> the key is that this is usually a write only store, we very rarely expect to have to read.
00:30:02 <thinrichs> Then when we write a new snapshot we delete all the rows in that table for that dsname
00:30:15 <ekcs> yes.
00:30:42 <thinrichs> Another thought… we could add a col to the original table for 'diff-since-last-snapshot-string'
00:30:55 <thinrichs> and then update that each time
00:31:13 <ekcs> yea that’s a good option to consider too.
00:31:30 <thinrichs> Hmmm…that might be awkward though b/c the delta wouldn't be based on the in-memory snapshot, but rather a snapshot from the past.
00:32:16 <ekcs> the intuition around it is that mysql table is optimized for read that’s why it’s slow even to add&delete 10 rows. we really only care about write.
00:33:02 <thinrichs> I'd imagine starting with just writing the snapshot as a string.  Then optimizing as needed.
00:33:11 <ekcs> yup my thought too.
00:33:33 <thinrichs> ekcs: good observation that we're not reading from this table very often.  We've effectively got an in-memory cache.
00:34:12 <thinrichs> ekcs: Thanks for digging into that!
00:34:16 <ekcs> one problem. to store long string in column,
00:34:34 <ekcs> we need to use special DB specific datatype (longtext, clob, etc.).
00:34:51 <ekcs> so we may lose the ability to arbitrarily swap mysql for postgres etc.
00:34:57 <ekcs> i’ll need to look at that deeper.
00:35:04 <thinrichs> Is that supported by oslo-db?
00:35:32 <ekcs> I think so (supported by sqlalchemy). but maybe I need to make sure.
00:35:46 <ekcs> for sure supported by sqlalchemy.
00:35:57 <ekcs> just not sure if the oslo-db version restricts.
00:36:32 <thinrichs> oslo-db is the OpenStack database layer, right?  If that does it, I'd say it's fine.  If not, I'm less certain.
00:36:42 <ekcs> ok.
00:36:51 <ekcs> i’ll make sure. that’s all from me.
00:37:40 <thinrichs> Thanks!
00:37:44 <thinrichs> Here's my status update.
00:38:33 <thinrichs> I've tweaked my patch that disables DSE1 a bit.  Had mistakenly removed some tests when moving tests2 to tests.
00:39:03 <thinrichs> Now I'm working on rebasing the patch on top of the few that went in recently.
00:39:09 <thinrichs> Enough changes that it's nontrivial.
00:39:44 <thinrichs> This patch is large by its nature.
00:39:57 <thinrichs> Future patches that finish off disabling dse1 will be easier to contain
00:40:20 <thinrichs> So I'm looking forward to getting this merged
00:40:28 <thinrichs> That's it from me.
00:41:06 <thinrichs> 2 more topics…
00:41:16 <thinrichs> s/topics/agenda items/
00:41:20 <thinrichs> #topic mascots
00:41:28 <thinrichs> We need to finalize our mascot decision today.
00:41:44 <thinrichs> I gave our suggestions to get a little early feedback.  here's what they said...
00:42:05 <thinrichs> We're not allowed to choose a specific land item like Aeropagus
00:42:24 <thinrichs> We could choose a large rock, but they thought it would just look like a blob and recommended against it.
00:43:22 <thinrichs> There were a few lizard-like proposals from other teams, so they might have trouble differentiating a salamander from those.
00:43:30 <thinrichs> The Raven seemed somewhat unique.
00:43:39 <thinrichs> No one chose a monkey/baboon
00:44:19 <thinrichs> They say all of those will work.
00:44:45 <thinrichs> Personally I'm leaning toward raven and then salamander
00:44:49 <thinrichs> Any other thoughts?
00:45:16 <ekcs> I think so too.
00:45:18 <aimeeu> I agree with the raven
00:45:29 <masahito> I agree the order.
00:45:50 <ekcs> but I really really don’t want a monkey or baboon.
00:45:57 <thinrichs> ekcs: agreed
00:46:00 <ekcs> we have to submit three things right?
00:46:02 <thinrichs> ekcs: agreed on the no baboon
00:46:20 <thinrichs> I think just 2 is fine
00:46:26 <thinrichs> They're going to pick 1, after all
00:46:42 <thinrichs> And they've already told us both the raven and salamander are fine
00:47:21 <ekcs> got it.
00:47:35 <masahito> +1
00:47:41 <thinrichs> Okay, so I'll send them (1) raven and (2) salamander
00:48:08 <thinrichs> final topic…
00:48:13 <thinrichs> #topic Barcelona
00:48:21 <thinrichs> Voting for talks is open.
00:48:29 * thinrichs looking for link
00:48:59 <thinrichs> #link https://www.openstack.org/summit/barcelona-2016/vote-for-speakers/
00:49:12 <thinrichs> If you search for Congress on the left-hand side, you can see the talks people have submitted
00:49:40 <thinrichs> #topic open discussion
00:49:49 <thinrichs> Anything we should discuss?
00:50:19 <thinrichs> aimeeu: one thing before I forget.  ramineni and I ran across some low-hanging fruit you might be interested in.
00:50:33 <aimeeu> excellent
00:51:03 <thinrichs> we noticed that we're using 'e.message' in a bunch of places, where e is an exception.  That works in Python2 but not Python3.  We should be using 'str(e)' instead.
00:51:32 <aimeeu> I could use a morale-boosting easy fix! (and one that lets me read lots of code)
00:51:41 <thinrichs> aimeeu: :)
00:51:52 <aimeeu> has a bug been created?
00:52:10 <thinrichs> Not yet.
00:52:40 <ekcs> any thoughts on the gate failure? #link https://review.openstack.org/#/c/341904/In setting up test_datasource_db_sync_remove, datasource 'fake' seems to already exists. A wild guess is some kind of leftover state that carried forward from one test to the next, but the frequency of failure seems too low for that to be the case.
00:52:55 <aimeeu> thinrichs: I can create the bug and assign it to myself.
00:52:59 <thinrichs> ekcs: does that Py2 vs Py3 behavior sound right to you?  I don't understand why we didn't hit the e.message problem before this
00:53:05 <thinrichs> aimeeu: that would be great!
00:53:14 <ekcs> yes that’s a py2vs3 thing.
00:53:33 <ekcs> probbaly because we just haven’t hit those errors in testing.
00:53:51 <thinrichs> That's a bit scary.  We use that in like 50 places
00:54:44 <ekcs> yea. I think most people still do manual testing in py2
00:55:13 <thinrichs> ekcs: that's the synchronizer test.  All kinds of potential race conditions I'd imagine.
00:56:12 <ekcs> I dug through some code to understand the setup and teardown. and I can’t seem to understand exactly what may persist from test to test. it’s the same devstack instance right not a new one?
00:56:46 <thinrichs> ramineni's the expert here
00:57:29 <ramineni_> :) , not sure , ill have a look at it today , is it because of new synchronizer code mergd?
00:58:17 <ekcs> oh hmm that’s possible. it was the last patch to go in before the failures.
00:58:31 <ramineni_> ekcs: yep, ill take a look today, and let you know if I find anything
00:58:39 <ekcs> great thanks!
00:59:15 <thinrichs> Out of time for today.  Thanks all!
00:59:24 <masahito> thanks all
00:59:47 <aimeeu> bye!
01:01:08 <thinrichs> #endmeeting