16:00:46 #startmeeting oslo 16:00:47 Meeting started Fri Jun 27 16:00:46 2014 UTC and is due to finish in 60 minutes. The chair is dhellmann. Information about MeetBot at http://wiki.debian.org/MeetBot. 16:00:49 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 16:00:51 The meeting name has been set to 'oslo' 16:00:57 hi 16:00:57 who's around for the oslo meeting today? 16:01:04 hi 16:01:05 Hi! 16:01:10 our agenda: 16:01:11 #link https://wiki.openstack.org/wiki/Meetings/Oslo 16:01:57 There 16:02:36 bnemec, markmc, flaper87, dims, jd__ : ping? 16:02:40 o/ 16:02:42 o/ 16:03:02 hi, all, let's get started 16:03:04 #topic Review action items from previous meeting 16:03:10 dhellmann clarify language about when to use _() vs. _LE() 16:03:31 aloha! 16:03:33 I think I need to carry this one over to next week. There are a couple of doc changes for oslo.i18n up, but not this 16:03:37 #action dhellmann clarify language about when to use _() vs. _LE() 16:03:51 Project liaisons review app-agnostic-logging-parameters: https://review.openstack.org/95281 16:04:02 * flaper87 clicks 16:04:03 that's merged now 16:04:12 Project liaisons review oslo-config-generator: https://review.openstack.org/100946 16:04:25 that has some good reviews, but no core reviewers :-) 16:04:37 #action review oslo-config-generator spec https://review.openstack.org/#/c/100946/ 16:04:43 the tests will amaze you 16:05:07 #undo 16:05:08 Removing item from minutes: 16:05:17 #action review oslo config generator patch https://review.openstack.org/#/c/100946/ 16:05:18 pong 16:05:22 that's the implementation, not the spec 16:05:43 I think that's it for old business; did I miss anything? 16:06:25 #topic Some Oslo hacking at Sprints/ParisJuno2014 16:06:33 jd__, do you want to say anything about the sprints? 16:06:45 #link https://wiki.openstack.org/wiki/Sprints/ParisJuno2014 16:06:51 * flaper87 confirms his presence 16:07:08 I'm not going to be able to make it, unfortunately, but I will be there in spirit and on irc 16:07:53 nothing special to say, I'll see with markmc what we'll do :) 16:08:15 it looks like the oslo.messaging crew, mostly, so maybe you guys can look at that amqp 1.0 spec and patchset? 16:08:31 sure 16:08:38 sileht won't be there unfortunately :( 16:08:58 ah 16:09:14 if anyone else is going, please add your name to the wiki page 16:09:19 #topic Red flags for/from liaisons 16:09:25 how are things going this week liaisons? 16:09:38 good 16:09:44 nothing to declare 16:09:55 oslo.messaging merged in heat finally 16:09:58 looks like there was a oslo.config released with the config fixture so I should be able to propose a patch to use it 16:10:02 \o/ 16:10:05 not sure if requirements was updated 16:10:07 some good patches for db still in progress, good progress 16:10:16 hello 16:10:26 bknudson: the requirements list should only have a lower bound for that, so you ought to be able to start using it and propose an update at the same time 16:10:51 therve: +1 w0000000000000t 16:11:00 glance reported some issues with unicode/byte string conflicts this week, did any other projects run into that problem? 16:11:32 I've seen some weird stuff, but couldn't reproduce reliably 16:12:06 #link http://lists.openstack.org/pipermail/openstack-dev/2014-June/038795.html 16:12:39 ok, keep an eye out therve and let us know 16:12:47 #topic Adoption status 16:12:48 so I can push forward the RPC removal in the incubator? 16:12:57 #undo 16:12:57 Removing item from minutes: 16:13:11 Is trove on oslo.messaging yet? 16:13:21 jd__: we had said we would wait to the end of the cycle 16:13:23 good question, bnemec 16:13:28 ok 16:13:53 bnemec, Apparently not 16:14:00 :-( 16:14:07 Trove is on its way 16:14:12 there's a patch proposed upstream already 16:14:17 * flaper87 gets the link 16:14:32 #link https://review.openstack.org/#/c/94484/ 16:14:40 thanks, flaper87 16:15:02 it'll likely be merged before j-2 is cut 16:15:12 ok 16:15:28 That's the last one using the old rpc module now, right? 16:15:43 we should probably wait until at least j3, in case projects end up with issues, but we will definitely remove that code from the incubator at the end of this cycle 16:15:57 it's the last integrated project, yes 16:16:21 Nice 16:16:46 Now I hear it's time for a from-scratch rewrite of oslo.messaging. ;-) 16:17:10 :-) 16:17:29 #topic Spec status 16:17:29 #info merged config filter - https://review.openstack.org/#/c/97228/ 16:17:29 #info merged oslo.serialization - https://review.openstack.org/#/c/97315/ 16:17:29 #info merged remove context adapter - https://review.openstack.org/#/c/95870/ 16:17:30 #info merged semver support in pbr - https://review.openstack.org/96608 16:17:31 #info merged ChainingRegExpFilter for rootwrap - https://review.openstack.org/98536 16:17:36 merged several more specs this week 16:17:54 bnemec, your concurrency spec is probably ready to go minus one comment and a rebase so it will merge cleanly 16:18:16 Yeah, been meaning to follow up on that. 16:18:35 #info merged app-agnostic-logging-parameters - https://review.openstack.org/95281 16:18:57 #info updated oslo.log spec to include context and local - https://review.openstack.org/102275 16:19:22 does anyone have any issues with specs they want to raise? 16:20:12 ok, moving on then 16:20:14 #topic oslo.i18n graduation status 16:20:32 I got distracted with some issues blocking pbr changes this week, so I didn't make the progress I was hoping to make. 16:21:05 I do have some minor API changes to make, so we can say "from oslo import i18n" instead of "from oslo.i18n import gettextutils", but after that I am going to try to tag a first release next week 16:21:28 w0000t 16:21:33 that sounds great to me! 16:21:35 the docs for the library are linked from the main project doc list now 16:21:39 #link http://docs.openstack.org/developer/openstack-projects.html 16:21:51 too bad I don't have i159 proposing a change to use oslo.i18n in keystone ... might have to do it myself 16:22:24 bknudson: I *think* the documentation explains it in a way that will make it fairly simple. Please let me know if you run into trouble. 16:23:01 #topic oslo.utils graduation status 16:23:10 dims has a repo up for review 16:23:10 #link https://github.com/dims/oslo.utils 16:23:10 maybe I can get started on it already. just have to figure out how to install it. 16:23:41 bknudson: hmm, propably I don't know what shoul I do. Should I? 16:23:47 bknudson: you should be able to add it to a virtualenv right from source, just to play with it 16:24:10 i159: don't worry about it if you don't have time. 16:24:13 cores, please look over that repository and provide feedback 16:24:19 although it's a lot easier if I can +2 it. 16:24:34 bknudson: ok 16:24:50 remember, if we can save changes for after the import that lets us review them and we can all contribute, so let's shoot for that in most cases 16:25:27 #topic oslo.concurrency graduation status 16:25:33 there's probably places that keystone could use oslo.utils 16:25:41 #undo 16:25:42 Removing item from minutes: 16:26:14 bknudson: we've added a few more modules to that one since the original plan was published, so there's a good chance that's true 16:26:37 we've got them in openstack.common 16:26:37 #topic oslo.concurrency graduation status 16:26:47 #undo 16:26:47 Removing item from minutes: 16:26:52 cool 16:26:54 that was it 16:27:01 :-) 16:27:03 #topic oslo.concurrency graduation status 16:27:27 I left comments on this one about a dependency issue, but it looks like the data I was reviewing was wrong or old, so it just needs a rebase I think 16:27:45 oh, hang on, no, that was the oslo.vmware thing 16:27:49 this is the one that is really a problem 16:28:00 the lockutils module relies on the fileutils module, which is meant to go into oslo.io which is not on our graduation list for this cycle 16:28:00 should we add it, use a copy of fileutils in oslo.concurrency, or solve it some other way? 16:28:18 bnemec: I'm inclined to just copy it from the incubator for now, what do you think? 16:28:40 dhellmann: Yeah, that sounds good to me. 16:28:52 It's going to be dependency nightmare to try to make sure all of the libs graduate along with every single dep. 16:28:58 ok, would you make a note of that in the spec? we'll want a bug to remember to fix it next cycle, too 16:29:25 yeah, it took me ages to build that graph last cycle and then I went and prioritized concurrency over io :-/ 16:29:33 dhellmann: Yep, will do. 16:29:36 thanks 16:29:44 #topic release review 16:30:00 markmc released oslo.messaging this week 16:30:09 I'm planning a pbr release monday 16:30:13 * dhellmann doesn't like to release code on fridays 16:30:25 is anything else ready? oslo.db? 16:30:45 i thought pbr was a beverage 16:30:46 I know we had that file encoding issue in the fake driver in oslo.messaging, so I'll talk to mark about another release 16:31:00 #action dhellmann talk to markmc about releasing file encoding fix in fake driver in oslo.messaging 16:31:02 I'm sorry, I don't know. But it seems to me not. 16:31:24 i159: ok, that's not a problem, we just want to give people notice when we know we are planning releases 16:31:37 bknudson: mordred came up with that name, iirc 16:32:01 * dhellmann crosses his fingers for an oslo.i18n release next week too 16:32:02 dhellmann: viktors and rpodolyaka know much more about releases 16:32:13 i159: ok, I'm sure they would have said something if they were ready :-) 16:32:36 ok, moving on 16:32:39 #topic review priorities for this week 16:32:51 db migration test issues (devananda) https://bugs.launchpad.net/ironic/+bug/1327397 fixed in nova and ironic, not oslo. 16:32:51 https://bugs.launchpad.net/nova/+bug/1328997 partial-fix in nova. incorrectly closed in oslo.db. 16:32:51 both should be fixed by the “opportunistic migration tests” fix: 16:32:52 #link https://review.openstack.org/#/c/93424/ 16:32:53 Launchpad bug 1327397 in oslo "No notice given when db migrations are not run due to missing engine" [High,In progress] 16:32:54 Launchpad bug 1328997 in nova "Unit test failure: openstack_citest" is being accessed by other users\nDETAIL: There are 1 other session(s) using the database." [Critical,In progress] 16:33:05 these are carried over from last week 16:33:37 #info oslo.utils repo mentioned earlier 16:33:45 what did I do? 16:33:52 you named pbr, right? 16:33:53 oh. 16:33:54 yes 16:34:07 hm i thought i reviewed this but apparently i am wrong 16:34:38 I know that db test issue bit the ironic folks, and we expect it might be hiding issues in other projects, too 16:34:39 dhellmann: #link https://review.openstack.org/#/c/93424/ still in progress, but pretty close to be merged 16:34:48 so, fixing these bugs in ironic/nova by implementing in oslo.db, are ironic/nova using this aspect of oslo.db yet? 16:35:15 zzzeek: I think they've worked around it in their own code base for now 16:35:33 we want to fix it in oslo.db so when they migrate it doesn't re-introduce the problem 16:35:34 dhellmann: OK, so here we’re looking at setting up a best practice pattern in oslo.db and that’s all ? 16:35:37 ok 16:35:51 i159: is that your understanding, too? 16:35:58 do you guys monkeypatch oslo.db into nova to integration test it? 16:36:08 zzzeek: no, we're not doing anything like that 16:36:30 dhellmann: OK so, does that mean it’s not clear if the fix as implemented in oslo.db actually works in nova? 16:36:44 or do we just test enough in oslo.db that we are confident 16:37:06 dhellmann: yes, sure. btw, 93424 had been tested in several projects by victors, it works good 16:37:12 we should be able to tell if it's working when we move nova over and run the tests 16:37:25 im asking somewaht selfishly as i am still mystified as to what code I should be actually hacking on 16:37:32 ok, so viktors has tested it with oslo.db in nova 16:37:53 zzzeek: oslo.db is the future, though adoption might not be finished until the end of K 16:37:53 dhellmann: yes, as I know 16:38:11 i159: thanks 16:38:11 dhellmann, Should we start looking at migrating to oslo.db? 16:38:30 therve: yes, there is a release out there and you should at least be experimenting with it 16:38:37 OK cool 16:38:54 i159: does the db team feel oslo.db is "ready" for real use? 16:39:08 therve: this work alredy in progress in several projects 16:39:33 I haven't seen anything yet in Heat, will ask around 16:39:44 dhellmann: yes, I thik issues is possible, but yes 16:39:57 i159: ok, good, I didn't want to say that too soon :-) 16:40:31 therve: I don't think that Heat on it 16:40:35 therve: we will have a 1.0 of oslo.db for juno, and at the end of k we will remove the db code from the incubator (following our established deprecation policy) 16:40:54 dhellmann, Sounds good 16:41:02 * zzzeek scratches head, “k” ? 16:41:13 the next release hasn't been named, but it will start with k 16:41:22 deep 16:41:29 KRACKEN 16:41:29 we're in juno now 16:41:35 as in “release the" 16:41:54 oh its the alphabett…….oooohhhhhh 16:41:55 I think Krull to keep with the bad movie theme 16:41:55 there's a list of names starting with k related to paris and france in general -- watch for the poll on the -dev list 16:42:13 heh 16:42:20 havana icehouse juno OHHHHHH 16:42:29 * zzzeek fails pattern recognition 16:42:40 enikanorov_: ping 16:42:55 we pick a name having to do with the geography of the area where we have the summit 16:43:08 armax: we're in a meeting here, can you go to #openstack-dev? 16:43:24 dhellmann: sorry my bad, wrong window :) 16:43:29 armax: no problem! :-) 16:43:41 #topic open discussion 16:43:54 that's all I had for the formal agenda, does anyone have anything else to bring up? 16:44:05 so I was talking to some nova folks yestederday, and this issue of “we want to remove the SQLAclehym ORM” came up again 16:44:19 so I really want to target the performance problems theyre having 16:44:30 that sounds like good work 16:44:42 some of it should probably happen right in nova, I would imagine 16:44:54 I think ORM use patterns can be mostly maintained if we just iron out the parts they dont like, so far it seems like the reliance on session.flush() when they just need to updaet a row is a big one 16:45:10 well ive noted from the start the object.save() method in oslo.db, and nova uses that 16:45:21 zzzeek: I’m actually surprised I have not yet seen a openstack ORM library project announcement... 16:45:27 heh 16:45:29 i can make that much much faster by bypassing the unit of worok 16:45:37 salv-orlando: oslo.db will *not* be an ORM 16:45:42 i dont think nova needs the UOW very much so we shoudl just bypass it 16:46:07 but i need to know, is that the main perf problem, or is object fetching hitting htem as well? they make heavy use of relationship() and replacing the ORM is not going to be very easy 16:46:10 dhellmann: I know that. I hope that we’ll never see an openstack ORM project. 16:46:14 zzzeek: are those changes in nova's use of sqla, or is there anything you want to put in oslo.db to make that easier across all of the projects 16:46:15 can we solve it just move Nova on Oslo.db? 16:46:17 salv-orlando: :-) 16:46:17 zzzeek, i think keystone can benefit form this as well. I'll keep an eye on changes you're making, what has to be updated in nova, and see if we can leverage it as well 16:46:30 dhellmann: I want oslo.db to have the patterns that nova needs for high performance, then everyone can use them 16:46:40 zzzeek: ok, that sounds like a good plan 16:46:44 but i need to learn what the bottlenecks are, so far im just guessing 16:47:01 and to that end….who / how should i ask, im told comstud is one such person 16:47:01 zzzeek: boris-42 has done some interesting profiling work with rally, you should talk to him about that 16:47:05 ok 16:47:22 I'm not sure boris-42 is a nova internals expert, but he has some useful tools and output to share 16:47:36 comstud from the Nova team was driving a lot of the db stuff in the past. 16:47:43 OK my other question, is status of Alembic vs. SQLalchemy migrate, is it the goal of oslo.db to standardize on alembic? 16:47:44 Don't know if that's still the case though. 16:47:50 zzzeek: can you tell us what are we need in oslo.db for perfomance increasing? 16:47:52 zzzeek: yes 16:48:06 dhellmann, zzzeek, https://review.openstack.org/#/c/102362/ adding the profiler (not just rally stuff) to global reqs 16:48:15 i159: I need to know when they say “the orm is too slow”, what are the operations theyre doing that need to be faster, so i can come upw ith alternate approaches 16:48:16 dhellmamm: that’s also a requirement for neutron’s full adoption of oslo.db 16:48:25 hard to believe that some python code would cause a performance problem for db ... isn't it the query that takes forever? 16:48:35 morganfainberg: +2 16:48:42 or are we not able to write a decent query using sqlalchemy orm? 16:48:44 dhellmann: Got some pushback on alembic from nova peeps as well yesteday, I’m going to write up what alembic definitely needs 16:48:54 bknudson, When you load thousand of rows, not necessarily 16:49:08 salv-orlando: we're supporting both now, I thought? 16:49:11 zzzeek: ping us (me, rpodolyaka, viktors) when you will know what are we need, please 16:49:16 therve: y, I can see that being a lot of work 16:49:18 bknudson, there is a lot of overhead when you do a lot of object conversions 16:49:33 bknudson: objecvt fetching is very slow beacuse of the bookkeeping that SQLAclhemy does, which is towards the goal of being able to manipulate and manage those objects. when those features arent needed, the overhead is wasteful 16:49:41 I spoke with jlibosva on monday - he said alembic support was still experiemntal in oslo.db. Or did I not get that right? 16:49:59 i159: can you clarify the status of alembic in oslo.db? 16:50:21 not suro if it was jlibosva or rpodolyaka actuall 16:50:25 dhellmann: I'm testing it on Ironic just now 16:50:27 * salv-orlando confused 16:50:33 i159: the alembic discussion is ongoing at https://review.openstack.org/#/c/102545/ 16:50:47 salv-orlando: Is Neutron not using the oslo-incubator db code for its migrations now? oslo.db shouldn't be any more experimental than that was. 16:51:19 at one point neutron was using alembic but skipping some migrations, right? do you still need that? 16:51:26 (the ability to skip) 16:51:33 bnemec: neutron has its own code for alembic migrations. We wrote that for the Grizzly release. I will see how much is that different from oslo-incubator 16:51:39 dhellmann zzzeek: I mean this patch https://review.openstack.org/#/c/99965 16:51:53 dhellmann: we’re working on removing the ability to skip 16:52:01 that’s a J-2 target 16:52:01 zzzeek, i want to bug you about the object overhead stuff outside this meeitng if you dont mind me picking your brain a little 16:52:05 salv-orlando: ok, good 16:52:09 morganfainberg: please do 16:52:27 i159: i think this patch looked fine i more had questions about it 16:52:53 zzzeek: it would be good to see some detailed examples of patterns we're using that may cause slowdowns, when you have that info will you post to the mailing list? 16:53:08 zzzeek: feel free to ask in comments or irc 16:53:16 dhellmann: I was thinking I can illustrate my proposal for a fast save() operation pretty easily 16:53:26 zzzeek: sounds good 16:53:46 dhellmann: as far as object fetching, that is more murky as it interacts with that whole “values dicitonary” thing I see in nova and also ive seen some datatype mismatches there 16:54:03 i159: was looking for you yesterday :) 16:54:47 zzzeek: I'm in +2:00 GMT, could be difficult to meet 16:54:51 dhellmann: when I’m approaching SQLAclhemy patterns here what SQLA version are we targeting, e.g. how long do the 0.8.x versions have to be supported? 16:55:02 i159, zzzeek : use the mailing list, if that works better than irc 16:55:15 * dhellmann looks at the global requirements list 16:55:23 dhellmann: ok 16:55:24 bcause i can add things to SQLAclhemy easily but it seems like i have to come up with 0.8 workarounds for them to be feasible 16:55:48 zzzeek: we have "SQLAlchemy>=0.7.8,!=0.9.5,<=0.9.99" right now 16:55:55 heh 16:56:01 I suspect that lower bound is due to a distro such as RHEL 16:56:02 the version-that-shall-not-be-named 16:56:24 so the thing to do is check with the distro folks about their plans for their next long-term-support versions 16:56:42 dhellmann: im pretty sure the 0.7 thing can be dropped from what ive heard 16:57:18 most of the distros are building separate repositories with versions for running openstack on their long-term releases, so we may be able to bump up but we should run it past them 16:57:29 dhellmann: so as I think we discussed earlier, my idea is to have new SQLA features and then workarounds for older versions that make use of semi-private APIs that I can guarantee are frozen 16:57:54 zzzeek: either ttx or the infra team should be able to help you find the reps you need to talk to for verification 16:58:26 zzzeek: yes, I remember that conversation, and that's still OK as long as we clearly document it in our code 16:59:04 we should put code like that in oslo.db, where it can be removed or simplified later on when we update sqla versions again 16:59:08 yes 16:59:09 ok, we're about out of time 16:59:17 thank you all, this was a good meeting! 16:59:52 #endmeeting