14:01:09 #startmeeting oslo 14:01:09 Meeting started Fri Feb 14 14:01:09 2014 UTC and is due to finish in 60 minutes. The chair is dhellmann. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:01:10 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 14:01:12 The meeting name has been set to 'oslo' 14:01:18 #link https://wiki.openstack.org/wiki/Meetings/Oslo#Agenda_for_Next_Meeting 14:01:29 lots to cover today 14:01:33 #topic Review action items from previous meeting 14:01:41 1. adopting stackforge libs 14:01:47 this is nearly complete, just a few requirements changes to iron out and then we need to set up gate test jobs for the libraries 14:01:57 2. log translation 14:02:04 we need to merge https://review.openstack.org/#/c/70455/ and then we can update the tools that manage the catalog extraction 14:02:43 I don't expect to make any build system changes around the i-3 deadline, so it would be really good to have that merge soon, hint hint :-) 14:02:57 3. qpid unit tests in oslo.messaging 14:03:07 I'm not aware of any change on this one, did anyone pick that up? 14:03:14 I may just have missed a changeset 14:03:24 Someone assigned the bug to themself last night. 14:03:30 cool 14:03:46 no unit tests yet 14:03:51 #action dhellmann track down owner of qpid test bug 14:03:53 and reviewing sileht's patches 14:03:59 I found a qpid regression 14:04:02 so, we need those tests 14:04:14 is sileht the owner, then? 14:04:17 #link https://bugs.launchpad.net/oslo.messaging/+bug/1255239 14:04:24 thanks, bnemec 14:04:59 so that is marked as high priority, should I put it in the milestone? 14:05:15 it sounds like more than one person is working on it, if sileht has changes for review too 14:05:52 I think markmc was saying he found a regression while reviewing sileht's other patches. 14:05:55 right 14:06:01 ah, ok 14:06:18 I'll talk to Numan about whether to include it in the release plan, then 14:06:26 4. test suite parallelization (bnemec) 14:06:45 the change to run most of the tests in parallel landed, so that's excellent work bnemec ! 14:06:53 \o/ 14:07:06 and the rpc tests in oslo.messaging run in parallel, so I'm not worried at all about the ones in the incubator 14:07:07 Found a few legitimate bugs in the process too. 14:07:19 indeed 14:07:34 is there any work left to do? do you have any reviews still open? 14:07:43 I think not but just to be sure... 14:08:03 I think all of my changes landed. 14:08:07 good 14:08:09 5. oslo.config doc update for "type" arg (merged) 14:08:15 that change is in, so that action is done 14:08:20 on to new business... 14:08:25 #topic icehouse-3 status review 14:08:30 #link https://launchpad.net/oslo/+milestone/icehouse-3 14:08:36 #link https://launchpad.net/oslo.messaging/+milestone/icehouse-3 14:08:44 we have quite a few items open with pending reviews, please focus your review work on anything we have in the i-3 milestone and prioritize that over changes that don’t have blueprints or bugs 14:09:25 I have this as a separate agenda item, but it makes sense to discuss it now: would it help to schedule a "review day" to push some of these changes through? 14:10:29 I have a feeling if we don't close some soon, ttx is going to suggest dumping them from the milestone 14:10:56 at least anything marked as high priority 14:11:13 I'm going to spend time this morning reviewing, after the meeting 14:11:13 I have a bad feeling about the models and migrations one. 14:11:34 I'm going to step up and take it 14:11:34 I think almost all of those patches got auto-abandoned months ago. 14:11:37 yeah... 14:11:39 yes, I need to talk to svetlana 14:11:42 I have a simpler patch 14:11:46 rpodolyaka: Nice! 14:11:48 (in Nova right now) 14:11:52 good! 14:11:55 https://review.openstack.org/#/c/69124/1 14:12:24 I'll reuse opportunistic test cases implemented by ipekelny 14:12:36 will be ready for review by monday 14:12:42 excellent 14:12:52 Simpler is good. Some of those patches were...large. ;-) 14:12:58 yeah :( 14:13:21 bnemec: yes, it was very hard to review them 14:13:52 ok, I've dropped secure messaging and amqp-1.0 from oslo.messaging icehouse for now 14:13:52 after talking with boris-42_ and rpodolyaka a day or two ago, we agreed the goal is to have oslo.db as a library by the end of the release cycle, and ready to start integrating with other projects early in juno 14:13:57 markmc: ok, thanks 14:14:04 dhellmann hi 14:14:08 hi, boris-42_ 14:14:20 we'll talk more about oslo.db in a second 14:14:26 dhellmann yeah absolutelly agree =) 14:14:29 dhellmann with road map 14:14:39 #topic oslo.sphinx rename 14:14:39 dhellmann make lib, and Juno switch to it 14:14:42 I'm blocked about posix-ipc because of infra 14:14:49 #link https://bugs.launchpad.net/oslo/+bug/1277168 14:14:53 oops 14:14:54 #undo 14:14:55 Removing item from minutes: 14:14:58 #undo 14:15:00 Removing item from minutes: 14:15:02 sorry my fault, I'm late 14:15:10 jd__: what's blocking? 14:15:15 just wanted to mention it though 14:15:26 posix_ipc is not hosted on PyPI so it needs to be whitelisted 14:15:39 I've asked the upstream author to move to PyPI but he's "thinking about it" 14:15:41 have you spoken to the author about uploading a copy to pypi? 14:15:43 ugh 14:15:44 ok 14:15:53 so I've sent patches to infra to whitelist it but got not review 14:15:57 #link https://review.openstack.org/#/c/69743/ 14:16:04 #link https://review.openstack.org/#/c/68945/ 14:16:18 ends of the heads-up 14:16:37 jd__: ok, I'll ask jeblair et al to take a look 14:16:57 #action dhellmann talk to infra about whitelisting posix-ipc 14:17:15 thanks :) 14:17:32 ok, anything else about i-3? 14:17:54 right 14:17:55 #topic oslo.sphinx rename 14:18:02 #link https://bugs.launchpad.net/oslo/+bug/1277168 14:18:08 the old repo is renamed, and I’ve seen patches to start to update the consuming projects 14:18:52 some of those changes aren't linked back to this original bug, I think -- I know there was one for ceilometer, for example 14:19:02 fwiw, I put a bunch of details into the bug report 14:19:09 took me an hour or more to dig them up 14:19:14 markmc: I saw, thanks 14:19:28 * dhellmann needs to remember to write things down 14:19:29 Ah, sorry. I guess I was relying on the ML discussion to document the problem. 14:19:54 yeah, mailing list is fine 14:20:01 and the bug actually linked to the thread 14:20:06 but the thread crossed months so ... 14:20:19 yeah 14:20:23 anyway, part stupidity on my fault 14:20:30 oh, also the thread didn't use the [oslo] topic 14:21:06 it looks like it started out as a general plea for help, so that's not a surprise 14:21:20 we should probably have started a new thread when we started working out the solution 14:22:16 some of the changes in the other projects are being submitted by dirk mueller, but I don't know his irc handle 14:22:25 dmllr or something 14:22:34 thanks 14:22:40 or maybe that's his launched 14:22:43 launchpad 14:23:09 I'll email him 14:23:43 I want to make sure I know when it's safe to remove the old oslo.sphinx from pypi, because I just saw a project try to start using it 14:23:43 ah, it's just dirk ... but looks like he doesn't email much 14:23:58 Jul 19 15:23:04 --> dirk (dmueller@kde/mueller) 14:24:08 thanks 14:24:18 dhellmann, what's the rush in removing it? 14:24:25 dhellmann, you're eager to break old tarballs and such? 14:24:27 :) 14:24:34 I want to prevent further instances of the original bug 14:24:43 break? 14:24:56 oh, I guess packaging stable branches with docs? 14:25:00 right 14:25:06 hmm 14:25:08 any published tarballs that use it 14:25:25 not saying we keep it around forever 14:25:27 but ... 14:25:30 Does pypi have any way to mark a package deprecated? 14:25:32 ok, I guess then just removing it from our global requirements list 14:25:40 bnemec: I can look into that 14:26:14 removing it from the global requirements would prevent integrated projects from using it, and we can fix any incubated projects as they graduate 14:26:34 bnemec: I could also edit the README, I suppose 14:26:44 #action dhellmann look into deprecating oslo.sphinx on pypi without deleting it 14:27:07 Well, removing it from global requirements should be fine. Nobody outside openstack will be using it anyway. 14:27:19 * dhellmann hopes not 14:27:37 #topic oslo.test graduation 14:27:43 #link https://github.com/dhellmann/oslo.test 14:27:48 #link https://blueprints.launchpad.net/oslo/+spec/graduate-oslo-test 14:28:01 the oslo.test library will contain test base class(es) and some fixtures (not all) 14:28:26 I have a repo set up and ready to import, but need to coordinate that with the infra team because the automated process broke so they have to do some of the steps by hand 14:28:38 What are we doing with the other fixtures? 14:28:55 some of the fixtures have dependencies on other libraries, like messaging, so I was going to put them in those libraries 14:29:07 Ah, makes sense. 14:29:14 if you want to write tests using a FancyFixture you probably already depend on FancyLib 14:29:27 I guess that should be oslo.fancy 14:30:09 I've tested the new library with oslo.config and oslo.messaging, so I have some changes ready for those repos when the new library is available 14:30:39 any other questions about oslo.test? 14:31:03 dhellmann, did you mean to put the mox/mock fixtures under oslo.test.fixture ? 14:31:21 * dhellmann looks 14:31:44 other than that, looks good 14:32:04 it looks like I forgot to delete the __init__ in fixtures 14:32:09 oh, BaseTestCase.tempdirs looks unused ? 14:32:24 I thought a flatter hierarchy made sense, since it is a separate library now 14:32:37 yeah, sounds good 14:32:53 #action dhellmann remove oslo/test/fixtures/__init__.py 14:33:01 #action dhellmann remove tempdirs from BaseTestCase 14:33:12 when any idea on availability of oslo.test? 14:33:13 * dhellmann was focused on file layout more than content 14:33:49 ekarlso: I need to coordinate with the infra team, but I hope to have the new repo set up early next week so I can finish the process of publishing an initial version 14:34:49 I'll have the new repo set up with review acls just like the incubator, fwiw 14:35:09 ok, let's talk about oslo.db 14:35:10 #topic oslo.db graduation (boris-42, rpodolyaka) 14:35:18 #link https://github.com/malor/oslo.db 14:35:21 #link https://blueprints.launchpad.net/oslo/+spec/oslo-db-lib 14:35:42 at the moment there are two critical patches on review for oslo.db graduate 14:35:42 as I mentioned, talking with boris-42_ and rpodolyaka earlier this week, we agreed to try to have an oslo.db ready by the end of the cycle 14:35:55 btw, do we have freeze in incubator? 14:36:32 IIRC, we don't worry as much about that, because the other projects can just not adopt changes 14:36:35 markmc, is that right? 14:36:53 my question is more like, should we rush graduation of oslo.db or we can push changes to incubator for now? 14:36:59 dhellmann I think until we have libs yes 14:37:12 boris-42_: yes, libs will definitely need to honor the feature freeze 14:37:16 I seem to recall that we did freeze incubator last release. 14:37:23 dhellmann, freezing oslo-incubator, in general ? 14:37:26 rpodolyaka: I don't want to rush graduation 14:37:31 dhellmann, yeah, we have been doing that 14:37:33 dhellmann: me too 14:37:56 markmc: ah, ok, that may mean adjusting some plans 14:38:07 dhellmann, so that e.g. if you have critical fixes, they go into incubator and projects sync them across 14:38:13 markmc: right 14:38:25 dhellmann, rather than post-freeze, push a bunch of features in, then a critical fix, then ... OH CRAP :) 14:38:34 markmc: that makes sense 14:39:09 so for oslo.db, we could retarget any work that doesn't land before the freeze to the library instead of the incubator 14:39:13 dhellmann, https://github.com/dhellmann/oslo.test/pull/1 14:39:21 but we would want to land anything that affects the API 14:39:47 Could we fork oslo.db but not release it right away? 14:40:23 bnemec: yes, but we want to avoid the forked version having an API that diverges too much from the incubator version 14:40:23 fork/graduate so we could actually improve and cleanup API, if needed 14:40:30 at least at first 14:40:43 I want to avoid the lengthy adoption cycle we had with oslo.messaging, if we can 14:40:50 +1 14:41:13 sure, but without some changes to API oslo.db has no sense as a library 14:41:17 e.g. global engines instances 14:41:20 which we removed 14:41:27 so why move out into oslo.db at all yet ? 14:41:32 rpodolyaka: yes, with the facade to make adoption easier 14:41:34 keep working in incubator until the new API is ready? 14:42:05 markmc: because we also have an issue with other projects being reluctant to merge changes 14:42:20 dhellmann, merge and adapt to API changes, or ? 14:42:23 dhellmann ^ +100500 14:42:26 markmc no 14:42:26 at the moment, me with rpodolyaka trying to run some OS projects with oslo.db library to find some hidden issues 14:42:29 markmc without any changes 14:42:35 markmc just syncing code takes for months 14:42:44 markmc and it is really hard to push them to upstream 14:42:47 Nobody likes reviewing oslo syncs 14:42:47 viktors: excellent 14:42:50 right 14:42:59 dhellmann's point about oslo-incubator API not diverging from oslo.db ... 14:43:10 markmc actually there is not too much difference 14:43:13 is so that projects can adapt gradually to the API changes 14:43:17 markmc it is not the same as with oslo.messaging 14:43:24 markmc that changes everything=) 14:43:44 markmc: I think we have the API we want with a compatibility layer to make adoption easier 14:43:50 markmc here are some small nits around removing singleton of engine and moving it to the project code 14:43:52 rpodolyaka, viktors : do you expect any other API changes in the incubator? 14:44:00 sounds like always the same problem that people don't want to sync regularly :) 14:44:09 jd__ yaa 14:44:10 * bnemec still wants the sql_mode change to land before graduation 14:44:11 dhellmann: I hope, not 14:44:15 let rpodolyaka fix me if I’m wrong 14:44:17 dhellmann: I think, no. just fixes for lazy loading and dshulyak patches 14:44:26 bnemec: is there a bug or blueprint for that? 14:44:27 dhellmann: both are already on review 14:44:43 viktors, rpodolyaka : ok,good 14:44:55 dhellmann: FYI: https://review.openstack.org/#/c/72984/ (Restore the ability to load the DB backend lazily) and https://review.openstack.org/#/c/71874/ (Refactor database migration manager to use single engine) 14:45:04 dhellmann: Just a bug report https://bugs.launchpad.net/oslo/+bug/1271706 14:45:06 viktors: can you email the dev list with a list of the changes you think we need to land before we can split oslo.db out? 14:45:22 bnemec: should I add that to i-3? 14:45:34 dhellmann: I think so. 14:45:34 viktors: or are those the only 2? 14:45:47 dhellmann: yes 14:45:48 bnemec: added 14:45:49 I've already +2'd all of the patches that I can. 14:46:02 So they should be pretty close to ready. 14:46:12 bnemec: ok, I'll try to look today or monday 14:46:17 dhellmann: Thanks 14:46:39 yeah, and this one, completely forgot about it :( 14:46:48 bnemec: I have a doubt about the first patch in chain... Will re review 14:47:09 there are a lot of database code changes going on, so it would help to have a good priority list so we can focus review effort 14:47:45 viktors: Sure, thanks. 14:48:28 when the oslo.db is set up, I'm going to ask infra to create oslo-db-core and oslo-db-milestone groups 14:48:45 we'll put oslo-core in both, but we may also have some reviewers only interested in oslo-db 14:49:14 we need a lead maintainer, and viktors has said he is willing and interested 14:49:50 dhellmann: ok :) 14:49:51 this is a new step for us, having a lead maintainer who is not oslo-core 14:50:09 does anyone have an issue with that, regardless of who the nominee is? 14:50:22 dhellmann: Isn't Thierry the same for rootwrap? 14:50:33 bnemec: ah, true, I forgot about that one 14:50:44 ok, so not a new step :-) 14:50:53 :-) 14:51:55 I think we're about ready to move on, unless there are other comments? 14:52:39 ok 14:52:40 #topic safe_unicode (bnemec) 14:52:48 #link https://review.openstack.org/#/c/71362/ 14:53:02 bnemec, the floor is yours 14:53:13 Yeah, so there's some debate over what to do with that. 14:53:45 Part of the problem is that it's addressing a pretty obscure edge case - translated strings that don't encode to unicode properly. 14:54:13 It really shouldn't happen, but as I said on the review, our translation test coverage is basically nonexistent so it's probably good to be overly safe. 14:54:37 I had thought that that was likely to happen in projects but that seems less likely. 14:55:07 harlowja_away however is concerned it is needed for the libraries. 14:55:28 jungleboyj_: Why specific to libraries? 14:55:39 is there an example of a case where this failure to encode could come up? 14:56:00 less control over what is being sent in. can't guarantee the source of a string. 14:56:38 jungleboyj_: Right, so that was my point about this being for user input. 14:56:43 dhellmann there is a test case with the patch. 14:56:53 But harlowja_away also said this was needed for translations too. 14:57:02 jungleboyj_: a test case demonstrates the fix, I meant a real life case where we've seen an issue 14:57:42 dhellmann I haven't seen a case. for translation we just need make sure the string is Unicode. 14:58:23 bnemec working on getting all logs and exceptions to be Unicode instead of using str() 14:58:47 jungleboyj_: would we eventually want to remove the str() calls from projects when they log objects? 14:59:13 We really should. 14:59:30 I seem to recall someone (I think Victor Stinner) saying that we shouldn't be using str() on exceptions ever. 14:59:35 where is safe_unicode() meant to be used? in the projects instead of str() when the log call is made? 14:59:38 dhellmann yes. I am working on Cinder and jecarey on Nova. 15:00:02 bnemec: yes, that's right, because that breaks translations and causes unicode issues, among other things 15:00:07 dhellmann that was the original intent but that may be overkill. 15:00:43 jungleboyj_: which was the original intent, changing str() calls to this new function? 15:00:52 Right. 15:01:03 it seems it may be a good idea for Oslo to use it though. 15:01:23 dhellmann yes. 15:01:24 I don't understand why this is better than ensuring the objects return unicode properly and stop using byte strings 15:01:31 ok, we're out of time, though 15:01:37 please start a mailing list thread on this 15:01:51 one other note before we wrap 15:01:51 #topic oslo.vmware status (dims) 15:01:55 dhellmann Ok. thanks. 15:02:11 dims is blocked on the same infra issue that I am with oslo.test, so those imports are likely to happen at the same time 15:02:32 we'll have separate oslo-vmware acl groups like with oslo-db 15:02:41 it's not clear yet whether dims will be the lead, or vipin 15:02:49 dims is handling the import, though 15:03:09 any questions on the vmware lib? 15:03:44 dhellmann, sorry to interrupt, but I'm just curious if you, folks, will finish your meeting soon :) I should have my one now :) 15:03:50 oh, yeah, we're going to have the vmware folks commit directly to the new library and not worry about the incubator -- they say that code is fairly stable and they want to be able to use it in cinder as well as nova 15:03:56 DinaBelova: we're seconds away 15:04:04 ok, thanks :) 15:04:12 in fact... 15:04:14 #endmeeting