00:01:27 <thinrichs> #startmeeting CongressTeamMeeting
00:01:28 <bryan_att> hi thinrichs
00:01:28 <openstack> Meeting started Thu Apr 14 00:01:27 2016 UTC and is due to finish in 60 minutes.  The chair is thinrichs. Information about MeetBot at http://wiki.debian.org/MeetBot.
00:01:29 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
00:01:33 <openstack> The meeting name has been set to 'congressteammeeting'
00:01:39 <thinrichs> bryan_att, tsandall, masahito: hi
00:02:19 <ramineni_> hi
00:02:38 <thinrichs> Light agenda from me today..
00:02:46 <thinrichs> 1. Quick update on release
00:02:58 <thinrichs> 2. Gate issues
00:03:14 <thinrichs> 3. bryan_att emails
00:03:16 <thinrichs> Anything else?
00:04:07 <thinrichs> #topic releases
00:04:20 <thinrichs> Today I did a release of stable/liberty, as we discussed last time.
00:04:31 <thinrichs> Nothing unusual to report
00:04:55 <thinrichs> I also went through and did some cleanup of our bugs, marking some closed as part of the Mitaka release.
00:05:11 <thinrichs> Also cleaned up launchpad so that the Mitaka branch is closed, and the Newton branch is open.
00:05:22 <thinrichs> As we're targeting blueprints/bugs, we can now start using the newton series.
00:06:07 <thinrichs> Finally, as we discussed last week, kilo is end-of-lifed in May, so we're not backporting bug fixes anymore.
00:06:16 <thinrichs> Questions/comments?
00:06:26 <ekcs> Great!
00:06:45 <masahito> one thing about release
00:07:46 <masahito> I can't find a release note in official release site
00:07:48 <masahito> http://releases.openstack.org/mitaka/index.html
00:08:10 <thinrichs> I don't think we've set that up yet, or even know how.
00:08:15 <masahito> Have any one worked for it?
00:08:26 <thinrichs> There's a release-notes job, but I know nothing about how it works.
00:09:07 <thinrichs> Anyone know how to submit release notes?  And whether we can still do it for Mitaka?
00:09:18 <thinrichs> We should definitely do it for Newton.
00:09:37 <thinrichs> I think I'd added ReleaseNotes to our docs in the past.
00:10:07 <thinrichs> I must have missed that this time.
00:10:48 <thinrichs> Anyone want to volunteer to look into the release notes and how we're supposed to be writing/releasing them?
00:11:15 <ramineni_> i can check with infra
00:11:15 <masahito> ok, I'll do it.
00:11:24 <ramineni_> masahito: go ahead :)
00:11:46 <masahito> thnaks :-)
00:12:01 <masahito> I'll do
00:12:08 <thinrichs> Speaking of volunteers, I think we had some items from last meeting that people were looking into….
00:12:37 <thinrichs> ramineni_: remind me what you were looking into
00:13:15 <thinrichs> Was it the docs and getting Congress on the list of projects?
00:13:30 <ramineni_> thinrichs: ya, its done now
00:13:55 <ramineni_> thinrichs: we are listed in http://docs.openstack.org/developer/openstack-projects.html
00:14:29 <thinrichs> ramineni_: Great!
00:14:46 <thinrichs> masahito: did you get a chance to look into the API guide and how we contribute our APIs to it?
00:15:08 <masahito> yes.
00:15:55 <thinrichs> #action masahito will look into release notes
00:16:01 <thinrichs> (Forgot to do that earlier.)
00:16:11 <masahito> we can write API references by updating api-site project
00:16:13 <masahito> https://github.com/openstack/api-site
00:16:59 <masahito> but, now coding style change is under the discussion at Doc team.
00:17:53 <masahito> so I'll also hear doc team when we should start to update the the repository.
00:18:26 <thinrichs> Sounds good.
00:18:58 <thinrichs> masahito: thanks for following up with that
00:19:16 <thinrichs> #action masahito will follow the doc team and report about how/if the api docs are changing
00:19:29 <thinrichs> Anything else on the release front?
00:19:42 <ramineni_> thinrichs: one thing
00:20:05 <ramineni_> thinrichs: its about stable/liberty and mitaka
00:20:36 <ramineni_> thinrichs: tempest plans to remove all the ceilometer related tests in their repository , that might break our branches
00:20:50 <ramineni_> https://review.openstack.org/#/c/304992/
00:20:51 <patchbot> ramineni_: patch 304992 - tempest - Remove Ceilometer tempest tests
00:21:15 <gmann> thinrichs: yea, i talked to ramineni_ about those and need to find the better way for that in congress
00:21:20 <gmann> #link https://review.openstack.org/#/c/304992/1
00:21:20 <patchbot> gmann: patch 304992 - tempest - Remove Ceilometer tempest tests
00:21:46 <gmann> So ceilometer has plugin and tempest going to remove all ceilometer tests, config options, client  etc
00:22:30 <thinrichs> gmann: understood
00:22:46 <ramineni_> thinrichs: we use it here https://github.com/openstack/congress/blob/stable/liberty/contrib/tempest/tempest/scenario/congress_datasources/test_ceilometer.py#L41
00:22:47 <gmann> congress use those which can be moved to ceilometer plugin from Mitaka onwards but liberty/Kilo needs to fix in other was as ceilometer plugin in not present on those branches
00:23:36 <gmann> thinrichs: ramineni_ one was is to consider ceilometer as not-available if no config option (as done for murano option) and make those tests skip for liberty/kilo
00:23:50 <gmann> if it is ok to skip those for kilo/liberty
00:24:13 <thinrichs> gmann: that sounds reasonable.
00:24:30 <thinrichs> ramineni_: we're forcing ceilometer to be enabled in kilo/liberty, right?
00:24:37 <gmann> thinrichs: cool
00:24:44 <ramineni_> thinrichs: ya
00:24:49 <thinrichs> So it's just the test for whether they are enabled that we need to fix.
00:25:24 <ramineni_> thinrichs: there is only one test for ceilometer ,
00:25:33 <thinrichs> ramineni_: is this a quick fix we can add to both kilo/liberty?
00:25:42 <gmann> yup just 1 line one
00:25:55 <ramineni_> thinrichs: ya ,we can skip the test
00:26:08 <gmann> if not CONF.service_available.ceilometer:  -> if not getattr(CONF.service_available, 'ceilometer', False):
00:26:18 <gmann> ramineni_: ^^ that should work right
00:26:43 <ramineni_> gmann: yes
00:26:49 <thinrichs> ramineni_: want to put those fixes in?
00:27:20 <ramineni_> thinrichs: but for mitaka too, instead of relying on tempest , may be we should directly call ceilometer client for retrieving info
00:27:38 <gmann> ramineni_: thinrichs and m putting fix for master(which can directly be backport to mitaka) to switch to plugin
00:27:51 <gmann> to avoid any gate break before my patch gets merged
00:28:39 <thinrichs> gmann: are you saying you're preparing a patch for master, and the same patch will work in mitaka?
00:28:40 <gmann> ramineni_: you can take over that if needed
00:28:46 <ramineni_> gmann: great , ill backport to liberty and kilo
00:29:07 <gmann> thinrichs: yea, master fix will work for Mitaka just cherry pick
00:29:38 <thinrichs> All sounds good to me.
00:29:39 <gmann> ramineni_: no, for libertry/kilo you need to change differently as plugin is not available  there
00:30:13 <ramineni_> gmann: ok, got it .. but for mitaka we should change the client altogether
00:30:17 <gmann> ramineni_: if not CONF.service_available.ceilometer:  -> if not getattr(CONF.service_available, 'ceilometer', False):
00:30:28 <ramineni_> gmann: as cls.os.telemetry client is removed too
00:30:35 <ramineni_> in tempest
00:30:41 <gmann> ramineni_: this will skip tests on liberty/kilo as tempest would nto register the CONF.service_available.ceilometer
00:30:58 <gmann> ramineni_: yea thats fine as you are going to skip tests
00:31:51 <ramineni_> gmann: diff fixes for mater/mitaka and kilo/liberty, if you already have patch, pls post it, ill take it from there
00:31:56 <ramineni_> gmann: thanks
00:32:43 <gmann> ramineni_: yea, sure
00:32:52 <thinrichs> #action ramineni will apply getattr() patch to kilo/liberty and will cherry-pick gmann's change on master to mitaka (once that change arrives)
00:33:09 <gmann> thinrichs: +1 Thanks :) really help
00:33:22 <thinrichs> gmann: np
00:33:38 <thinrichs> Any other questions/comments about releases?
00:34:15 <ramineni_> no
00:34:23 <thinrichs> #topic Gate
00:34:38 <thinrichs> Seems we have a gate breakage.
00:34:49 <thinrichs> ekcs: looks like you've been working on it.
00:35:01 <thinrichs> Anyone tracked down the issue?
00:35:24 <ekcs> thinrichs: I was confirming it, but I didn’t figure out why the failure.
00:35:36 <ekcs> failure is in tempest TestHA
00:35:49 <thinrichs> Eric's debugging patch:
00:35:49 <thinrichs> https://review.openstack.org/#/c/254429/
00:35:50 <patchbot> thinrichs: patch 254429 - congress - DO NOT MERGE: Gating troubleshooting only
00:36:06 <thinrichs> Looks like a timeout
00:37:19 <ramineni_> thinrichs: looks like recheck not solving the problem
00:37:35 <ramineni_> same error on 2 patches
00:37:39 <thinrichs> right
00:38:03 <thinrichs> urllib3.exceptions.MaxRetryError
00:38:10 <thinrichs> Max retries exceeded with url: /v1/data-sources/365de9b4-183d-4c35-a85c-0a7d7d252465/status
00:39:12 <thinrichs> I remember seeing that one time, but I think it cleared up on its own.
00:39:21 <ramineni_> thinrichs: ya, recheck solved
00:39:30 <ramineni_> thinrichs: ill have look into this failure
00:39:50 <thinrichs> ramineni_: thanks!
00:39:55 <ekcs> thanks!
00:40:27 <thinrichs> Anything else on the Gate?
00:40:59 <thinrichs> #topic bryan_att emails
00:41:03 <thinrichs> bryan_att: you around?
00:41:06 <bryan_att> yes
00:41:30 <thinrichs> I saw masahito and ramineni were responding to your emails.
00:41:35 <thinrichs> Did everything get sorted out?
00:41:44 <bryan_att> yes, and I think I'm OK with the responses - appreciate the fast turnaround
00:42:00 <bryan_att> I still am trying to figure out the horizon integration though
00:42:17 <bryan_att> And have a pesky recurring issue where glance images don't show up in the glance table
00:42:25 <bryan_att> I don't know how to debug that one
00:42:33 <thinrichs> bryan_att: are you using liberty?
00:42:42 <bryan_att> yes, stable/liberty
00:42:56 <bryan_att> I will move to mitaka soon
00:42:58 <thinrichs> We found a subtle bug just before releasing mitaka with Glance
00:43:09 <thinrichs> That could be the issue
00:43:33 <thinrichs> I'm checking if we already backported it...
00:43:52 <ramineni_> thinrichs: only mitaka has the fix i suppose
00:44:08 <ramineni_> ekcs: correct?
00:44:12 <ekcs> yea I just checked we haven’t backported it.
00:44:24 <ekcs> guess we were rushing to get it in the mitaka and forgot.
00:44:52 <bryan_att> is there a way I could test to see if it resolves the issue?
00:45:21 <thinrichs> ekcs: could bryan_att cherry-pick the change (cleanly) onto stable/liberty?
00:46:02 <bryan_att> or if I know where it was (and it wasn't huge), could I just modify the source file in my clone?
00:46:16 <bryan_att> for a test
00:46:20 <ekcs> thinrichs: I think so.
00:46:40 <ekcs> bryan_att: before you try that though, I think we should first check if you have that error.
00:46:59 <ekcs> bryan_att: in the congress log and glance log you should see http authentication errors for glance.
00:47:07 <bryan_att> yes, I see them
00:47:10 <ekcs> bryan_att: let me get more exact details.
00:47:25 <ekcs> bryan_att: oh perfect. then that’s almost certainly the issue.
00:47:46 <bryan_att> there is no more detailed info I can see though, and I think I see the auth errors in both the case where the tables get populated and when they dont
00:48:08 <bryan_att> by checking the glance logs in both cases
00:49:13 <ekcs> here’s the patch for glance. https://review.openstack.org/#/c/299950/
00:49:13 <patchbot> ekcs: patch 299950 - congress (stable/mitaka) - Use keystone sessions to authenticate (MERGED)
00:49:37 <ramineni_> direct cherry-pick not working
00:50:01 <ramineni_> ekcs : we need to manually raise a patch for liberty i think
00:50:16 <thinrichs> It should be easy for us to prepare a patch for bryan_att and put it on Gerrit to try.
00:50:26 <thinrichs> bryan_att: 2 choices it seems…
00:50:33 <ramineni_> thinrichs: yes
00:50:42 <ekcs> thinrichs: yup.
00:51:05 <thinrichs> (i) check that we have properly identified the problem by applying a patch to your Congress install and looking to see if it solves the problem
00:51:17 <thinrichs> (ii) upgrade your Congress install to a version that already has the patch
00:51:41 <thinrichs> (i) would be a temporary solution, and if it succeeded you'd want to do (ii)
00:51:50 <thinrichs> If (i) doesn't work, we haven't identified the problem yet.
00:52:23 <bryan_att> OK, whichever I can figure out how to do, with a little help maybe
00:52:43 <thinrichs> bryan_att: you recently upgraded from kilo to liberty, right?
00:52:57 <bryan_att> I've been on liberty since Dec
00:53:15 <thinrichs> Are you planning to upgrade to mitaka anytime soon?
00:53:37 <bryan_att> Yes, very soon. We are closing Brahmaputra at the end of the month
00:53:48 <bryan_att> Colorado will be Mitaka-based
00:53:59 <thinrichs> Then I'd wait until you upgrade to Mitaka, since that has the patch anyway.
00:54:21 <bryan_att> btw, I just reverified that I see the auth errors in the glance log even when the congress glance table has the images listed
00:54:40 <ekcs> thinrichs: are we thinking we won’t backport to liberty then?
00:54:40 <thinrichs> Assuming we're right about the problem, this'll be one of those fix-via-upgrade problems.
00:54:43 <bryan_att> "WARNING keystonemiddleware.auth_token [-] Authorization failed for token"
00:54:52 <thinrichs> ekcs; we should also backport to liberty
00:55:16 <bryan_att> I didn't check to see if that error occurs *before* I install Congress though
00:55:48 <ekcs> bryan_att: right that’s the issue. authentication with glance times out at a certain point. if the image updates happened before then, they get in. after that, they don’t.
00:56:25 <bryan_att> OK, that's all for me - thanks for all the help this week. I'm making good progress on creating Congress tests for OPNFV
00:56:42 <ekcs> bryan_att: the auth error should show up at regular interval (every poll)
00:56:48 <thinrichs> bryan_att: Great!
00:56:56 <thinrichs> Anything else in the last few minutes?
00:57:00 <thinrichs> #topic open discussion
00:57:27 <ekcs> I have a patch for doing update ordering and differential update.  comments appreciated. https://review.openstack.org/#/c/304991/
00:57:27 <patchbot> ekcs: patch 304991 - congress - dse2 sequenced, differential pub-sub
00:58:00 <thinrichs> Great that you're picking back up on the distributed arch!
00:58:08 <thinrichs> I'll take a look tomorrow
00:59:02 <ekcs> bryan_att: question: which liberty version are you on?
00:59:13 <bryan_att> stable
00:59:26 <bryan_att> as of a couple of days ago
00:59:29 <thinrichs> ekcs: 2.0.0
00:59:46 <thinrichs> bryan_att: are you pulling the latest from stable/liberty via git?
01:00:05 <bryan_att> yes, cloning from github
01:00:18 <thinrichs> So my 2.0.0 answer was wrong.
01:00:42 <ekcs> bryan_att: ok great. i’ll prepare the backport patch so you can try it if you like.
01:00:53 <bryan_att> sure, thanks
01:01:04 <thinrichs> Out time.  Thanks all!
01:01:06 <thinrichs> #endmeeting