14:01:09 <dhellmann> #startmeeting oslo
14:01:09 <openstack> Meeting started Fri Feb 14 14:01:09 2014 UTC and is due to finish in 60 minutes.  The chair is dhellmann. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:01:10 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:01:12 <openstack> The meeting name has been set to 'oslo'
14:01:18 <dhellmann> #link https://wiki.openstack.org/wiki/Meetings/Oslo#Agenda_for_Next_Meeting
14:01:29 <dhellmann> lots to cover today
14:01:33 <dhellmann> #topic Review action items from previous meeting
14:01:41 <dhellmann> 1. adopting stackforge libs
14:01:47 <dhellmann> this is nearly complete, just a few requirements changes to iron out and then we need to set up gate test jobs for the libraries
14:01:57 <dhellmann> 2. log translation
14:02:04 <dhellmann> we need to merge https://review.openstack.org/#/c/70455/ and then we can update the tools that manage the catalog extraction
14:02:43 <dhellmann> I don't expect to make any build system changes around the i-3 deadline, so it would be really good to have that merge soon, hint hint :-)
14:02:57 <dhellmann> 3. qpid unit tests in oslo.messaging
14:03:07 <dhellmann> I'm not aware of any change on this one, did anyone pick that up?
14:03:14 <dhellmann> I may just have missed a changeset
14:03:24 <bnemec> Someone assigned the bug to themself last night.
14:03:30 <dhellmann> cool
14:03:46 <markmc> no unit tests yet
14:03:51 <dhellmann> #action dhellmann track down owner of qpid test bug
14:03:53 <markmc> and reviewing sileht's patches
14:03:59 <markmc> I found a qpid regression
14:04:02 <markmc> so, we need those tests
14:04:14 <dhellmann> is sileht the owner, then?
14:04:17 <bnemec> #link https://bugs.launchpad.net/oslo.messaging/+bug/1255239
14:04:24 <dhellmann> thanks, bnemec
14:04:59 <dhellmann> so that is marked as high priority, should I put it in the milestone?
14:05:15 <dhellmann> it sounds like more than one person is working on it, if sileht has changes for review too
14:05:52 <bnemec> I think markmc was saying he found a regression while reviewing sileht's other patches.
14:05:55 <markmc> right
14:06:01 <dhellmann> ah, ok
14:06:18 <dhellmann> I'll talk to Numan about whether to include it in the release plan, then
14:06:26 <dhellmann> 4. test suite parallelization (bnemec)
14:06:45 <dhellmann> the change to run most of the tests in parallel landed, so that's excellent work bnemec !
14:06:53 <bnemec> \o/
14:07:06 <dhellmann> and the rpc tests in oslo.messaging run in parallel, so I'm not worried at all about the ones in the incubator
14:07:07 <bnemec> Found a few legitimate bugs in the process too.
14:07:19 <dhellmann> indeed
14:07:34 <dhellmann> is there any work left to do? do you have any reviews still open?
14:07:43 <dhellmann> I think not but just to be sure...
14:08:03 <bnemec> I think all of my changes landed.
14:08:07 <dhellmann> good
14:08:09 <dhellmann> 5. oslo.config doc update for "type" arg (merged)
14:08:15 <dhellmann> that change is in, so that action is done
14:08:20 <dhellmann> on to new business...
14:08:25 <dhellmann> #topic icehouse-3 status review
14:08:30 <dhellmann> #link https://launchpad.net/oslo/+milestone/icehouse-3
14:08:36 <dhellmann> #link https://launchpad.net/oslo.messaging/+milestone/icehouse-3
14:08:44 <dhellmann> we have quite a few items open with pending reviews, please focus your review work on anything we have in the i-3 milestone and prioritize that over changes that don’t have blueprints or bugs
14:09:25 <dhellmann> I have this as a separate agenda item, but it makes sense to discuss it now: would it help to schedule a "review day" to push some of these changes through?
14:10:29 <dhellmann> I have a feeling if we don't close some soon, ttx is going to suggest dumping them from the milestone
14:10:56 <dhellmann> at least anything marked as high priority
14:11:13 <dhellmann> I'm going to spend time this morning reviewing, after the meeting
14:11:13 <bnemec> I have a bad feeling about the models and migrations one.
14:11:34 <rpodolyaka> I'm going to step up and take it
14:11:34 <bnemec> I think almost all of those patches got auto-abandoned months ago.
14:11:37 <rpodolyaka> yeah...
14:11:39 <dhellmann> yes, I need to talk to svetlana
14:11:42 <rpodolyaka> I have a simpler patch
14:11:46 <bnemec> rpodolyaka: Nice!
14:11:48 <rpodolyaka> (in Nova right now)
14:11:52 <dhellmann> good!
14:11:55 <rpodolyaka> https://review.openstack.org/#/c/69124/1
14:12:24 <rpodolyaka> I'll reuse opportunistic test cases implemented by ipekelny
14:12:36 <rpodolyaka> will be ready for review by monday
14:12:42 <dhellmann> excellent
14:12:52 <bnemec> Simpler is good.  Some of those patches were...large. ;-)
14:12:58 <rpodolyaka> yeah :(
14:13:21 <viktors> bnemec: yes, it was very hard to review them
14:13:52 <markmc> ok, I've dropped secure messaging and amqp-1.0 from oslo.messaging icehouse for now
14:13:52 <dhellmann> after talking with boris-42_ and rpodolyaka a day or two ago, we agreed the goal is to have oslo.db as a library by the end of the release cycle, and ready to start integrating with other projects early in juno
14:13:57 <dhellmann> markmc: ok, thanks
14:14:04 <boris-42_> dhellmann hi
14:14:08 <dhellmann> hi, boris-42_
14:14:20 <dhellmann> we'll talk more about oslo.db in a second
14:14:26 <boris-42_> dhellmann yeah absolutelly agree =)
14:14:29 <boris-42_> dhellmann with road map
14:14:39 <dhellmann> #topic oslo.sphinx rename
14:14:39 <boris-42_> dhellmann make lib, and Juno switch to it
14:14:42 <jd__> I'm blocked about posix-ipc because of infra
14:14:49 <dhellmann> #link https://bugs.launchpad.net/oslo/+bug/1277168
14:14:53 <dhellmann> oops
14:14:54 <dhellmann> #undo
14:14:55 <openstack> Removing item from minutes: <ircmeeting.items.Link object at 0x2fd1890>
14:14:58 <dhellmann> #undo
14:15:00 <openstack> Removing item from minutes: <ircmeeting.items.Topic object at 0x31d3a90>
14:15:02 <jd__> sorry my fault, I'm late
14:15:10 <dhellmann> jd__: what's blocking?
14:15:15 <jd__> just wanted to mention it though
14:15:26 <jd__> posix_ipc is not hosted on PyPI so it needs to be whitelisted
14:15:39 <jd__> I've asked the upstream author to move to PyPI but he's "thinking about it"
14:15:41 <dhellmann> have you spoken to the author about uploading a copy to pypi?
14:15:43 <dhellmann> ugh
14:15:44 <dhellmann> ok
14:15:53 <jd__> so I've sent patches to infra to whitelist it but got not review
14:15:57 <jd__> #link https://review.openstack.org/#/c/69743/
14:16:04 <jd__> #link https://review.openstack.org/#/c/68945/
14:16:18 <jd__> ends of the heads-up
14:16:37 <dhellmann> jd__: ok, I'll ask jeblair et al to take a look
14:16:57 <dhellmann> #action dhellmann talk to infra about whitelisting posix-ipc
14:17:15 <jd__> thanks :)
14:17:32 <dhellmann> ok, anything else about i-3?
14:17:54 <dhellmann> right
14:17:55 <dhellmann> #topic oslo.sphinx rename
14:18:02 <dhellmann> #link https://bugs.launchpad.net/oslo/+bug/1277168
14:18:08 <dhellmann> the old repo is renamed, and I’ve seen patches to start to update the consuming projects
14:18:52 <dhellmann> some of those changes aren't linked back to this original bug, I think -- I know there was one for ceilometer, for example
14:19:02 <markmc> fwiw, I put a bunch of details into the bug report
14:19:09 <markmc> took me an hour or more to dig them up
14:19:14 <dhellmann> markmc: I saw, thanks
14:19:28 * dhellmann needs to remember to write things down
14:19:29 <bnemec> Ah, sorry.  I guess I was relying on the ML discussion to document the problem.
14:19:54 <markmc> yeah, mailing list is fine
14:20:01 <markmc> and the bug actually linked to the thread
14:20:06 <markmc> but the thread crossed months so ...
14:20:19 <bnemec> yeah
14:20:23 <markmc> anyway, part stupidity on my fault
14:20:30 <markmc> oh, also the thread didn't use the [oslo] topic
14:21:06 <dhellmann> it looks like it started out as a general plea for help, so that's not a surprise
14:21:20 <dhellmann> we should probably have started a new thread when we started working out the solution
14:22:16 <dhellmann> some of the changes in the other projects are being submitted by dirk mueller, but I don't know his irc handle
14:22:25 <markmc> dmllr or something
14:22:34 <dhellmann> thanks
14:22:40 <markmc> or maybe that's his launched
14:22:43 <markmc> launchpad
14:23:09 <dhellmann> I'll email him
14:23:43 <dhellmann> I want to make sure I know when it's safe to remove the old oslo.sphinx from pypi, because I just saw a project try to start using it
14:23:43 <markmc> ah, it's just dirk ... but looks like he doesn't email much
14:23:58 <markmc> Jul 19 15:23:04 -->	dirk (dmueller@kde/mueller)
14:24:08 <dhellmann> thanks
14:24:18 <markmc> dhellmann, what's the rush in removing it?
14:24:25 <markmc> dhellmann, you're eager to break old tarballs and such?
14:24:27 <markmc> :)
14:24:34 <dhellmann> I want to prevent further instances of the original bug
14:24:43 <dhellmann> break?
14:24:56 <dhellmann> oh, I guess packaging stable branches with docs?
14:25:00 <markmc> right
14:25:06 <dhellmann> hmm
14:25:08 <markmc> any published tarballs that use it
14:25:25 <markmc> not saying we keep it around forever
14:25:27 <markmc> but ...
14:25:30 <bnemec> Does pypi have any way to mark a package deprecated?
14:25:32 <dhellmann> ok, I guess then just removing it from our global requirements list
14:25:40 <dhellmann> bnemec: I can look into that
14:26:14 <dhellmann> removing it from the global requirements would prevent integrated projects from using it, and we can fix any incubated projects as they graduate
14:26:34 <dhellmann> bnemec: I could also edit the README, I suppose
14:26:44 <dhellmann> #action dhellmann look into deprecating oslo.sphinx on pypi without deleting it
14:27:07 <bnemec> Well, removing it from global requirements should be fine.  Nobody outside openstack will be using it anyway.
14:27:19 * dhellmann hopes not
14:27:37 <dhellmann> #topic oslo.test graduation
14:27:43 <dhellmann> #link https://github.com/dhellmann/oslo.test
14:27:48 <dhellmann> #link https://blueprints.launchpad.net/oslo/+spec/graduate-oslo-test
14:28:01 <dhellmann> the oslo.test library will contain test base class(es) and some fixtures (not all)
14:28:26 <dhellmann> I have a repo set up and ready to import, but need to coordinate that with the infra team because the automated process broke so they have to do some of the steps by hand
14:28:38 <bnemec> What are we doing with the other fixtures?
14:28:55 <dhellmann> some of the fixtures have dependencies on other libraries, like messaging, so I was going to put them in those libraries
14:29:07 <bnemec> Ah, makes sense.
14:29:14 <dhellmann> if you want to write tests using a FancyFixture you probably already depend on FancyLib
14:29:27 <dhellmann> I guess that should be oslo.fancy
14:30:09 <dhellmann> I've tested the new library with oslo.config and oslo.messaging, so I have some changes ready for those repos when the new library is available
14:30:39 <dhellmann> any other questions about oslo.test?
14:31:03 <markmc> dhellmann, did you mean to put the mox/mock fixtures under oslo.test.fixture ?
14:31:21 * dhellmann looks
14:31:44 <markmc> other than that, looks good
14:32:04 <dhellmann> it looks like I forgot to delete the __init__ in fixtures
14:32:09 <markmc> oh, BaseTestCase.tempdirs looks unused ?
14:32:24 <dhellmann> I thought a flatter hierarchy made sense, since it is a separate library now
14:32:37 <markmc> yeah, sounds good
14:32:53 <dhellmann> #action dhellmann remove oslo/test/fixtures/__init__.py
14:33:01 <dhellmann> #action dhellmann remove tempdirs from BaseTestCase
14:33:12 <ekarlso> when any idea on availability of oslo.test?
14:33:13 * dhellmann was focused on file layout more than content
14:33:49 <dhellmann> ekarlso: I need to coordinate with the infra team, but I hope to have the new repo set up early next week so I can finish the process of publishing an initial version
14:34:49 <dhellmann> I'll have the new repo set up with review acls just like the incubator, fwiw
14:35:09 <dhellmann> ok, let's talk about oslo.db
14:35:10 <dhellmann> #topic oslo.db graduation (boris-42, rpodolyaka)
14:35:18 <viktors> #link https://github.com/malor/oslo.db
14:35:21 <viktors> #link https://blueprints.launchpad.net/oslo/+spec/oslo-db-lib
14:35:42 <viktors> at the moment there are two critical patches on review for oslo.db graduate
14:35:42 <dhellmann> as I mentioned, talking with boris-42_  and rpodolyaka earlier this week, we agreed to try to have an oslo.db ready by the end of the cycle
14:35:55 <rpodolyaka> btw, do we have freeze in incubator?
14:36:32 <dhellmann> IIRC, we don't worry as much about that, because the other projects can just not adopt changes
14:36:35 <dhellmann> markmc, is that right?
14:36:53 <rpodolyaka> my question is more like, should we rush graduation of oslo.db or we can push changes to incubator for now?
14:36:59 <boris-42_> dhellmann I think until we have libs yes
14:37:12 <dhellmann> boris-42_: yes, libs will definitely need to honor the feature freeze
14:37:16 <bnemec> I seem to recall that we did freeze incubator last release.
14:37:23 <markmc> dhellmann, freezing oslo-incubator, in general ?
14:37:26 <dhellmann> rpodolyaka: I don't want to rush graduation
14:37:31 <markmc> dhellmann, yeah, we have been doing that
14:37:33 <rpodolyaka> dhellmann: me too
14:37:56 <dhellmann> markmc: ah, ok, that may mean adjusting some plans
14:38:07 <markmc> dhellmann, so that e.g. if you have critical fixes, they go into incubator and projects sync them across
14:38:13 <dhellmann> markmc: right
14:38:25 <markmc> dhellmann, rather than post-freeze, push a bunch of features in, then a critical fix, then ... OH CRAP :)
14:38:34 <dhellmann> markmc: that makes sense
14:39:09 <dhellmann> so for oslo.db, we could retarget any work that doesn't land before the freeze to the library instead of the incubator
14:39:13 <markmc> dhellmann, https://github.com/dhellmann/oslo.test/pull/1
14:39:21 <dhellmann> but we would want to land anything that affects the API
14:39:47 <bnemec> Could we fork oslo.db but not release it right away?
14:40:23 <dhellmann> bnemec: yes, but we want to avoid the forked version having an API that diverges too much from the incubator version
14:40:23 <rpodolyaka> fork/graduate so we could actually improve and cleanup API, if needed
14:40:30 <dhellmann> at least at first
14:40:43 <dhellmann> I want to avoid the lengthy adoption cycle we had with oslo.messaging, if we can
14:40:50 <bnemec> +1
14:41:13 <rpodolyaka> sure, but without some changes to API oslo.db has no sense as a library
14:41:17 <rpodolyaka> e.g. global engines instances
14:41:20 <rpodolyaka> which we removed
14:41:27 <markmc> so why move out into oslo.db at all yet ?
14:41:32 <dhellmann> rpodolyaka: yes, with the facade to make adoption easier
14:41:34 <markmc> keep working in incubator until the new API is ready?
14:42:05 <dhellmann> markmc: because we also have an issue with other projects being reluctant to merge changes
14:42:20 <markmc> dhellmann, merge and adapt to API changes, or ?
14:42:23 <boris-42_> dhellmann ^ +100500
14:42:26 <boris-42_> markmc no
14:42:26 <viktors> at the moment, me with rpodolyaka trying to run some OS projects with oslo.db library to find some hidden issues
14:42:29 <boris-42_> markmc without any changes
14:42:35 <boris-42_> markmc just syncing code takes for months
14:42:44 <boris-42_> markmc and it is really hard to push them to upstream
14:42:47 <bnemec> Nobody likes reviewing oslo syncs
14:42:47 <dhellmann> viktors: excellent
14:42:50 <dhellmann> right
14:42:59 <markmc> dhellmann's point about oslo-incubator API not diverging from oslo.db ...
14:43:10 <boris-42_> markmc actually there is not too much difference
14:43:13 <markmc> is so that projects can adapt gradually to the API changes
14:43:17 <boris-42_> markmc it is not the same as with oslo.messaging
14:43:24 <boris-42_> markmc that changes everything=)
14:43:44 <dhellmann> markmc: I think we have the API we want with a compatibility layer to make adoption easier
14:43:50 <boris-42_> markmc here are some small nits around removing singleton of engine and moving it to the project code
14:43:52 <dhellmann> rpodolyaka, viktors : do you expect any other API changes in the incubator?
14:44:00 <jd__> sounds like always the same problem that people don't want to sync regularly :)
14:44:09 <boris-42_> jd__ yaa
14:44:10 * bnemec still wants the sql_mode change to land before graduation
14:44:11 <viktors> dhellmann: I hope, not
14:44:15 <viktors> let rpodolyaka fix me if I’m wrong
14:44:17 <rpodolyaka> dhellmann: I think, no. just fixes for lazy loading and dshulyak patches
14:44:26 <dhellmann> bnemec: is there a bug or blueprint for that?
14:44:27 <rpodolyaka> dhellmann: both are already on review
14:44:43 <dhellmann> viktors, rpodolyaka : ok,good
14:44:55 <viktors> dhellmann: FYI:  https://review.openstack.org/#/c/72984/ (Restore the ability to load the DB backend lazily) and https://review.openstack.org/#/c/71874/ (Refactor database migration manager to use single engine)
14:45:04 <bnemec> dhellmann: Just a bug report https://bugs.launchpad.net/oslo/+bug/1271706
14:45:06 <dhellmann> viktors: can you email the dev list with a list of the changes you think we need to land before we can split oslo.db out?
14:45:22 <dhellmann> bnemec: should I add that to i-3?
14:45:34 <bnemec> dhellmann: I think so.
14:45:34 <dhellmann> viktors: or are those the only 2?
14:45:47 <viktors> dhellmann: yes
14:45:48 <dhellmann> bnemec: added
14:45:49 <bnemec> I've already +2'd all of the patches that I can.
14:46:02 <bnemec> So they should be pretty close to ready.
14:46:12 <dhellmann> bnemec: ok, I'll try to look today or monday
14:46:17 <bnemec> dhellmann: Thanks
14:46:39 <rpodolyaka> yeah, and this one, completely forgot about it :(
14:46:48 <viktors> bnemec: I have a doubt about the first patch in chain... Will re review
14:47:09 <dhellmann> there are a lot of database code changes going on, so it would help to have a good priority list so we can focus review effort
14:47:45 <bnemec> viktors: Sure, thanks.
14:48:28 <dhellmann> when the oslo.db is set up, I'm going to ask infra to create oslo-db-core and oslo-db-milestone groups
14:48:45 <dhellmann> we'll put oslo-core in both, but we may also have some reviewers only interested in oslo-db
14:49:14 <dhellmann> we need a lead maintainer, and viktors has said he is willing and interested
14:49:50 <viktors> dhellmann: ok :)
14:49:51 <dhellmann> this is a new step for us, having a lead maintainer who is not oslo-core
14:50:09 <dhellmann> does anyone have an issue with that, regardless of who the nominee is?
14:50:22 <bnemec> dhellmann: Isn't Thierry the same for rootwrap?
14:50:33 <dhellmann> bnemec: ah, true, I forgot about that one
14:50:44 <dhellmann> ok, so not a new step :-)
14:50:53 <bnemec> :-)
14:51:55 <dhellmann> I think we're about ready to move on, unless there are other comments?
14:52:39 <dhellmann> ok
14:52:40 <dhellmann> #topic safe_unicode (bnemec)
14:52:48 <dhellmann> #link https://review.openstack.org/#/c/71362/
14:53:02 <dhellmann> bnemec, the floor is yours
14:53:13 <bnemec> Yeah, so there's some debate over what to do with that.
14:53:45 <bnemec> Part of the problem is that it's addressing a pretty obscure edge case - translated strings that don't encode to unicode properly.
14:54:13 <bnemec> It really shouldn't happen, but as I said on the review, our translation test coverage is basically nonexistent so it's probably good to be overly safe.
14:54:37 <jungleboyj_> I had thought that that was likely to happen in projects but that seems less likely.
14:55:07 <jungleboyj_> harlowja_away however is concerned it is needed for the libraries.
14:55:28 <bnemec> jungleboyj_: Why specific to libraries?
14:55:39 <dhellmann> is there an example of a case where this failure to encode could come up?
14:56:00 <jungleboyj_> less control over what is being sent in.  can't guarantee the source of a string.
14:56:38 <bnemec> jungleboyj_: Right, so that was my point about this being for user input.
14:56:43 <jungleboyj_> dhellmann there is a test case with the patch.
14:56:53 <bnemec> But harlowja_away also said this was needed for translations too.
14:57:02 <dhellmann> jungleboyj_: a test case demonstrates the fix, I meant a real life case where we've seen an issue
14:57:42 <jungleboyj_> dhellmann I haven't seen a case.  for translation we just need make sure the string is Unicode.
14:58:23 <jungleboyj_> bnemec working on getting all logs and exceptions to be Unicode instead of using str()
14:58:47 <dhellmann> jungleboyj_: would we eventually want to remove the str() calls from projects when they log objects?
14:59:13 <bnemec> We really should.
14:59:30 <bnemec> I seem to recall someone (I think Victor Stinner) saying that we shouldn't be using str() on exceptions ever.
14:59:35 <dhellmann> where is safe_unicode() meant to be used? in the projects instead of str() when the log call is made?
14:59:38 <jungleboyj_> dhellmann yes.  I am working on Cinder and jecarey on Nova.
15:00:02 <dhellmann> bnemec: yes, that's right, because that breaks translations and causes unicode issues, among other things
15:00:07 <jungleboyj_> dhellmann that was the original intent but that may be overkill.
15:00:43 <dhellmann> jungleboyj_: which was the original intent, changing str() calls to this new function?
15:00:52 <bnemec> Right.
15:01:03 <jungleboyj_> it seems it may be a good idea for Oslo to use it though.
15:01:23 <jungleboyj_> dhellmann yes.
15:01:24 <dhellmann> I don't understand why this is better than ensuring the objects return unicode properly and stop using byte strings
15:01:31 <dhellmann> ok, we're out of time, though
15:01:37 <dhellmann> please start a mailing list thread on this
15:01:51 <dhellmann> one other note before we wrap
15:01:51 <dhellmann> #topic oslo.vmware status (dims)
15:01:55 <jungleboyj_> dhellmann Ok.  thanks.
15:02:11 <dhellmann> dims is blocked on the same infra issue that I am with oslo.test, so those imports are likely to happen at the same time
15:02:32 <dhellmann> we'll have separate oslo-vmware acl groups like with oslo-db
15:02:41 <dhellmann> it's not clear yet whether dims will be the lead, or vipin
15:02:49 <dhellmann> dims is handling the import, though
15:03:09 <dhellmann> any questions on the vmware lib?
15:03:44 <DinaBelova> dhellmann, sorry to interrupt, but I'm just curious if you, folks, will finish your meeting soon :) I should have my one now :)
15:03:50 <dhellmann> oh, yeah, we're going to have the vmware folks commit directly to the new library and not worry about the incubator -- they say that code is fairly stable and they want to be able to use it in cinder as well as nova
15:03:56 <dhellmann> DinaBelova: we're seconds away
15:04:04 <DinaBelova> ok, thanks :)
15:04:12 <dhellmann> in fact...
15:04:14 <dhellmann> #endmeeting