00:05:03 <thinrichs> #startmeeting CongressTeamMeeting
00:05:04 <openstack> Meeting started Thu May 26 00:05:03 2016 UTC and is due to finish in 60 minutes.  The chair is thinrichs. Information about MeetBot at http://wiki.debian.org/MeetBot.
00:05:05 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
00:05:07 <openstack> The meeting name has been set to 'congressteammeeting'
00:05:20 <thinrichs> Agenda for the week is to go over status updates.
00:05:30 <thinrichs> masahito: want to start?
00:05:47 <masahito> sure
00:06:28 <masahito> I pushed a new tempest test for push type driver this week.
00:07:54 <masahito> and for dse2, I'm working to remove self.running defined in deepsix
00:08:06 <masahito> that's from my side.
00:08:10 <thinrichs> Looks like the tempest test is failing devstack tests in an odd way.
00:08:19 <thinrichs> Any idea why?
00:08:42 <thinrichs> #link https://review.openstack.org/#/c/319173/
00:09:37 <thinrichs> Hmmm…I remember seeing the db_sync test failing in another review recently
00:09:41 <ekcs> I've been looking at it too, but I don't know.
00:09:59 <ramineni_> thinrichs: ya, its failing in my patch also
00:10:03 <ramineni_> same error
00:10:22 <thinrichs> Have we broken sync somehow?
00:10:59 <ekcs> Last time I saw the same failure, it passed on recheck.
00:11:23 <ekcs> Seems to be non-deterministic somehow. because it passes in jenkins check and then immediately after fails in jenkins gate.
00:11:48 <ramineni_> thinrichs: hmm , relica congress service fails to start here
00:12:08 <ramineni_> replica**
00:13:00 <thinrichs> ramineni_: did you find that in the logs/screen-congress.txt.gz file or somewhere else?
00:13:05 <thinrichs> http://logs.openstack.org/73/319173/6/gate/gate-congress-dsvm-api/63e2b50/logs/screen-congress.txt.gz
00:13:52 <thinrichs> BTW just noticed an error when polling keystone:
00:13:54 <thinrichs> #link http://logs.openstack.org/73/319173/6/gate/gate-congress-dsvm-api/63e2b50/logs/screen-congress.txt.gz#_2016-05-25_23_39_34_555
00:14:18 <ramineni_> thinrichs: i reproduced in my env and relica service itself fails to start saying DatasourceNameInUSe error ,
00:14:46 <ramineni_> thinrichs: i could investigate more yesterday, ill try to debug today
00:15:05 <ramineni_> culdn´t *
00:15:06 <thinrichs> ramineni_: that sounds like a race condition bug to me.  More investigation is definitely worthwhile.
00:15:34 <ramineni_> thinrichs: ya, will check today
00:15:46 <masahito> thinrichs: yes. I'll check it, too.
00:15:58 <thinrichs> ramineni_, masahito: great!
00:16:23 <thinrichs> masahito: is self.running just unnecessary and needs to be removed, or is it causing problems?
00:16:56 <masahito> I found one problem.
00:17:38 <masahito> datasource service starts before DseNode starts the service
00:18:29 <masahito> because old deepsix uses self.running but new dataservice uses self._running to manage its status.
00:19:04 <thinrichs> got it.  Thanks.
00:19:15 <thinrichs> ramineni_: want to give a status update?
00:19:32 <ramineni_> thinrichs: ok
00:20:08 <ramineni_> thinrichs: Migration of test-congress got merged this week. I tried to test congress with backend driver rabbit, it worked without any issue
00:20:27 <ramineni_> thinrichs: so raised a patch to make it cnfigurable
00:21:00 <ramineni_> thinrichs: but the HA tests are failing irrespective of the driver
00:21:06 <thinrichs> ramineni_: cool that we're working with rabbit!
00:21:34 <thinrichs> ramineni_: which HA tests are failing?  The ones that test the synchronizer?
00:21:48 <ramineni_> thinrichs: yes
00:22:02 <ramineni_> thinrichs: and also, we need to migrate synchronizer
00:22:07 <ramineni_> to dse2
00:22:22 <ramineni_> thinrichs: i think its not done
00:23:02 <thinrichs> I wouldn't be surprised if the synchronizer were broken
00:23:12 <thinrichs> I don't think anyone dug into making sure it was working.
00:23:23 <ramineni_> thinrichs: so raised a bug to track it
00:23:31 <thinrichs> In fact, we may need to rework it since there's no longer a datasource manager.
00:23:48 <ramineni_> thinrichs: yes
00:23:48 <thinrichs> ramineni_: makes sense to have a bug
00:24:16 <thinrichs> ramineni_: anything else?
00:24:48 <ramineni_> thinrichs: no, done from my side
00:25:07 <thinrichs> ekcs: you're up
00:27:20 <thinrichs> Maybe he stepped away for a moment.
00:27:45 <thinrichs> I've just pushed the Upgrade instructions to the readme.
00:28:01 <ekcs> Sorry something went wrong with my client =(
00:28:02 <thinrichs> We had someone ask for them, and I couldn't find the instructions elsewhere.
00:28:04 <ekcs> had to restarte
00:28:06 <thinrichs> Did I miss them?
00:28:23 <thinrichs> ekcs: no problem.  I'm just mentioning the Upgrade instruction patch.
00:29:07 <ekcs> I'd never seen them.
00:29:52 <thinrichs> I think the only thing we need to do really is run the DB schema migration scripts
00:31:25 <ekcs> anyway my status update.
00:31:28 <ekcs> I revamped the HA spec, incorporating ideas in comments and proposing a simpler way to do replicated deployment (symmetric nodes with driver deactivation). #link: https://review.openstack.org/#/c/318383/  Also continuing to read up on Pacemaker and HAProxy to understand the details. And still trying to get connected with someone at my organization who can lend me his Pacemaker experience and expertise.
00:32:07 <ekcs> It's a whole complicated mess when you get down to the nitty gritty =o
00:32:30 <thinrichs> ekcs: I failed to get you comments—my apologies.  I'll do it this week.
00:33:09 <ekcs> that's all from me.
00:34:24 <thinrichs> HA isn't easy.  That's part of why we wanted to make progress on the DSE2 so that we had a more concrete feel for how it would work.
00:35:18 <thinrichs> Another round or two of comments on the spec before next week then.
00:35:30 <thinrichs> That's it from me too.
00:35:34 <thinrichs> #topic open discussion
00:35:40 <thinrichs> Anything else to discuss this week?
00:37:24 <thinrichs> Let's finish up early then.
00:37:39 <thinrichs> Thanks all!
00:37:52 <thinrichs> #endmeeting