Wednesday, 2014-07-02

« Tuesday, 2014-07-01 Index Thursday, 2014-07-03 »
openstackgerritJoshua Harlow proposed a change to openstack/taskflow: Use a common message dispatcher  https://review.openstack.org/10109400:09
openstackgerritJoshua Harlow proposed a change to openstack/taskflow: Allow handlers to provide validation callables  https://review.openstack.org/10167900:09
openstackgerritJoshua Harlow proposed a change to openstack/taskflow: WBE notification message validation  https://review.openstack.org/10365700:09
openstackgerritJoshua Harlow proposed a change to openstack/taskflow: WBE response message validation  https://review.openstack.org/10369000:10
openstackgerritJoshua Harlow proposed a change to openstack/taskflow: WBE request message validation  https://review.openstack.org/10369500:10
openstackgerritJoshua Harlow proposed a change to openstack/taskflow: Add basic WBE validation sanity tests  https://review.openstack.org/10401600:10
openstackgerritZhongyue Luo proposed a change to openstack/oslo-incubator: Use timestamp in loopingcall  https://review.openstack.org/10370400:13
*** ajo|mac_ has quit IRC00:13
openstackgerritZhongyue Luo proposed a change to openstack/oslo-incubator: Use timestamp in loopingcall  https://review.openstack.org/10370400:16
openstackgerritZhongyue Luo proposed a change to openstack/oslo-incubator: Changes calcuation of variable delay  https://review.openstack.org/10370800:19
*** dims__ has quit IRC00:25
openstackgerritJoshua Harlow proposed a change to openstack/taskflow: Improve WBE testing coverage  https://review.openstack.org/10171700:32
openstackgerritSean Dague proposed a change to openstack/oslo-incubator: update new requests logger to default WARN  https://review.openstack.org/10396000:32
*** dims__ has joined #openstack-oslo00:43
openstackgerritJoshua Harlow proposed a change to openstack/oslo-incubator: Use the standard python logging in network_utils  https://review.openstack.org/10404100:47
*** dstanek is now known as dstanek_zzz00:50
openstackgerritJoshua Harlow proposed a change to openstack/oslo.messaging: Add a thread + futures executor based executor  https://review.openstack.org/7091400:51
openstackgerritJoshua Harlow proposed a change to openstack/oslo-incubator: Use the standard python logging in network_utils  https://review.openstack.org/10404101:16
*** dstanek_zzz is now known as dstanek01:17
*** zzzeek has joined #openstack-oslo01:36
*** mriedem has left #openstack-oslo01:53
*** mriedem has quit IRC01:53
*** praneshp has quit IRC01:54
*** zzzeek has quit IRC02:00
openstackgerritjichenjc proposed a change to openstack/oslo-incubator: Add lock_path as param to remove_external function  https://review.openstack.org/10373202:08
openstackgerritjichenjc proposed a change to openstack/oslo-incubator: Add lock_path as param to remove_external function  https://review.openstack.org/10373202:16
*** zhiyan_ is now known as zhiyan02:22
*** dstanek is now known as dstanek_zzz02:26
*** arnaud has quit IRC02:29
*** zhiyan is now known as zhiyan_02:30
*** zhiyan_ is now known as zhiyan02:42
*** dstanek_zzz is now known as dstanek02:48
*** harlowja is now known as harlowja_away02:50
*** ajc_ has joined #openstack-oslo02:50
*** praneshp has joined #openstack-oslo02:52
*** dims__ has quit IRC03:09
*** zhiyan is now known as zhiyan_03:22
*** zhiyan_ is now known as zhiyan03:24
*** zhiyan has quit IRC03:30
*** zhiyan has joined #openstack-oslo03:37
*** zhiyan is now known as zhiyan_03:39
*** zhiyan_ is now known as zhiyan03:40
*** SridharG has joined #openstack-oslo03:45
*** zhiyan is now known as zhiyan_03:46
*** zhiyan_ is now known as zhiyan03:50
*** arnaud has joined #openstack-oslo03:50
openstackgerritjichenjc proposed a change to openstack/oslo-incubator: Add lock_path as param to remove_external function  https://review.openstack.org/10373203:54
*** praneshp_ has joined #openstack-oslo04:01
*** tong_ has joined #openstack-oslo04:02
*** tongli has quit IRC04:03
*** praneshp has quit IRC04:04
*** praneshp_ is now known as praneshp04:04
*** tong_ has quit IRC04:33
*** dims__ has joined #openstack-oslo04:37
*** dims__ has quit IRC04:42
*** lbragstad is now known as lbragstad_04:45
*** arnaud has quit IRC04:47
*** arnaud has joined #openstack-oslo04:49
*** dims__ has joined #openstack-oslo05:02
*** dstanek is now known as dstanek_zzz05:04
*** dims__ has quit IRC05:07
*** dstanek_zzz is now known as dstanek05:51
*** dstanek is now known as dstanek_zzz06:01
*** dims__ has joined #openstack-oslo06:03
*** dstanek_zzz is now known as dstanek06:06
*** dims__ has quit IRC06:08
*** dstanek is now known as dstanek_zzz06:15
openstackgerritAbhijeet Malawade proposed a change to openstack/oslo.messaging: Cleanup listener after stopping rpc server  https://review.openstack.org/10318606:20
*** jaosorior has joined #openstack-oslo06:34
*** tkelsey has joined #openstack-oslo06:51
*** GheRivero_afk is now known as GheRivero06:56
*** mrda is now known as mrda_away07:11
*** flaper87|afk is now known as flaper8707:19
*** ajo|mac has joined #openstack-oslo07:24
*** arnaud has quit IRC07:31
*** flaper87 is now known as flaper87|afk07:35
*** pblaho has joined #openstack-oslo07:38
*** praneshp has quit IRC07:38
*** ildikov has joined #openstack-oslo07:51
*** markmc has joined #openstack-oslo08:00
*** nacim has joined #openstack-oslo08:02
*** dims__ has joined #openstack-oslo08:06
*** dims__ has quit IRC08:11
*** oomichi has quit IRC08:14
*** flaper87|afk is now known as flaper8708:17
openstackgerritRadoslav Gerganov proposed a change to openstack/oslo.vmware: Refactor the PBM support  https://review.openstack.org/10248008:17
*** ihrachyshka has joined #openstack-oslo08:19
openstackgerritAngus Lees proposed a change to openstack/oslo-incubator: Test that concurrent sqlalchemy transactions don't block  https://review.openstack.org/10411108:27
openstackgerritIlya Pekelny proposed a change to openstack/oslo.db: Add a base test case for DB schema comparison  https://review.openstack.org/9339808:38
openstackgerritGordon Sim proposed a change to openstack/oslo-specs: Added spec for proposed AMQP 1.0 driver  https://review.openstack.org/9672908:52
openstackgerritJuan Antonio Osorio Robles proposed a change to openstack/oslo.messaging: Raise if invalid conf is given to get_transport  https://review.openstack.org/10386609:06
*** dims__ has joined #openstack-oslo09:07
*** dims__ has quit IRC09:12
*** ajo has joined #openstack-oslo09:18
openstackgerritGordon Sim proposed a change to openstack/oslo-specs: Added spec for proposed AMQP 1.0 driver  https://review.openstack.org/9672909:29
boris-42markmc hi, did you have a chanche to take a look at my spec?09:30
boris-42markmc https://review.openstack.org/#/c/103825/09:30
*** ujjain has quit IRC09:42
*** nacim has quit IRC09:48
*** mkoderer has quit IRC09:52
*** nacim has joined #openstack-oslo09:58
openstackgerritGordon Sim proposed a change to openstack/oslo-specs: Added spec for proposed AMQP 1.0 driver  https://review.openstack.org/9672910:02
*** dims__ has joined #openstack-oslo10:08
*** dims__ has quit IRC10:13
*** ujjain has joined #openstack-oslo10:23
*** ujjain has quit IRC10:24
*** ujjain has joined #openstack-oslo10:24
*** pcm_ has joined #openstack-oslo10:25
*** pcm_ has quit IRC10:26
*** pcm_ has joined #openstack-oslo10:26
*** ajo has quit IRC10:31
*** dims__ has joined #openstack-oslo10:32
*** ajo has joined #openstack-oslo10:34
*** SridharG has quit IRC10:42
*** Alexei_9871 has joined #openstack-oslo10:43
*** ildikov_ has joined #openstack-oslo10:57
*** ildikov has quit IRC11:00
*** i159 has joined #openstack-oslo11:21
*** tkelsey has quit IRC11:32
openstackgerritAlex Xu proposed a change to openstack/oslo-specs: Support policy configuration directories  https://review.openstack.org/10415711:38
*** mkoderer has joined #openstack-oslo11:40
boris-42markmc please=) don't ignore me=)11:47
openstackgerritA change was merged to openstack/oslo-incubator: Updated from global requirements  https://review.openstack.org/10402211:50
openstackgerritA change was merged to openstack/oslo-incubator: Use the standard python logging in network_utils  https://review.openstack.org/10404111:50
*** SridharG has joined #openstack-oslo11:52
*** markmcclain has joined #openstack-oslo11:53
openstackgerritGraham Hayes proposed a change to openstack/oslosphinx: Added a incubating project config option  https://review.openstack.org/10393511:55
*** ajc_ has quit IRC11:58
*** markmcclain1 has joined #openstack-oslo11:59
*** markmcclain has quit IRC11:59
*** markmcclain1 has quit IRC12:02
*** alexpilotti has joined #openstack-oslo12:15
dims__python33 jobs should be unclogged now12:24
*** dstanek_zzz is now known as dstanek12:29
flaper87dhellmann: ping12:32
flaper87I'd like to make processutils use standard python logging but there's 1 function it is using from oslo's log module that we could probably move elsewhere. processutils uses `mask_password` to hide passwords in commands log messages12:34
flaper87I think that function could go into the strutils module, which means it could go in the str transformation module proposed for oslo-utils12:34
flaper87Does that make sense to you? (and other folks reading this)12:35
flaper87We'll have to do that or something similar when moving away from oslo-inc log anyway12:36
*** erecio has joined #openstack-oslo12:43
openstackgerritAlex Xu proposed a change to openstack/oslo-specs: Support policy configuration directories  https://review.openstack.org/10415712:47
ihrachyshkaflaper87: makes sense, it's not directly related to logging12:51
*** dims__ has quit IRC12:52
*** dims__ has joined #openstack-oslo12:53
dims__flaper87, sounds good to me12:57
*** pblaho has quit IRC12:59
*** tkelsey has joined #openstack-oslo13:05
openstackgerritA change was merged to openstack/oslo-incubator: Python 3: replace "im_self" by "__self__"  https://review.openstack.org/7672513:06
openstackgerritA change was merged to openstack/oslo-incubator: Make periodic tasks run on regular spacing interval  https://review.openstack.org/9969513:06
openstackgerritA change was merged to openstack/oslo.config: Add CLI option support to config fixture  https://review.openstack.org/10251513:15
*** i159 has quit IRC13:19
*** rdopieralski has joined #openstack-oslo13:19
rdopieralskispeaking of oslo.config, I wonder how useful it would be do add a helper method that does something like _all_sections here: https://review.openstack.org/#/c/100521/11/openstack_dashboard/utils/settings.py13:20
*** i159 has joined #openstack-oslo13:21
rdopieralski(I don't know the section names in there up front, so I have to peek, and then register the right groups)13:21
rdopieralskior is there a better way of doing it?13:21
markmcrdopieralski, heh13:21
markmcrdopieralski, quick question ...13:21
markmcin python, if an attribute is prefixed with underscore, what does it mean? :)13:22
*** mriedem has joined #openstack-oslo13:22
markmcrdopieralski, I'm talking about conf._namespace - you really, really, really shouldn't be looking at that13:23
rdopieralskimarkmc: it means that it's an internal implementation detail of whatever scope it is in, and that it shouldn't be accessed from outside of it, or at least, the programmer who accesses it takes the risk of it changing from under him13:23
markmcrdopieralski, right :)13:23
rdopieralskimarkmc: yeah, but there is no other way13:23
markmcrdopieralski, what exactly is the use case here?13:23
markmcok, it's groups that begin with 'dashboard:'13:24
markmcyou're trying to discover them13:24
rdopieralskimarkmc: https://review.openstack.org/#/c/100521/11/openstack_dashboard/settings.conf13:24
rdopieralskimarkmc: so, I was thinking about adding an apropriate method to oslo.config, so that I don't have to do this hack :)13:26
rdopieralskiobviously, it won't work with command line options13:26
markmcrdopieralski, what is the canonical list of dashboards ?13:28
markmcis it from import_submodules(module)13:28
rdopieralskimarkmc: there is none, you can create new dashboards and name them whatever you want them13:28
dhellmannflaper87: yeah, moving that function makes sense. Maybe import it into the logging code in the incubator for backwards-compat, with a comment to remove that when oslo.log graduates?13:29
rdopieralskimarkmc: that import_submodules is a leftover from how we did it previously, using python files for configuration13:29
flaper87dhellmann: +113:30
markmcrdopieralski, ok, so previously a dashboard was enabled simply by being on the filesystem ?13:30
markmcrdopieralski, now it is enabled by adding a [dashboard:foo] group ?13:30
flaper87dhellmann: I'll move it now since the oslo-utils spec hasn't been approved13:30
flaper87dhellmann: we should probably mention this function in the spec too13:30
flaper87thoughts?13:30
* flaper87 sad panda, the spec already has 2 +2s13:30
dhellmannflaper87: yes, definitely, can you do that?13:30
flaper87:P13:30
markmcrdopieralski, (ignoring 'disabled' for now)13:30
flaper87dhellmann: yup13:30
dhellmannflaper87: it's not approved until I merge it :-)13:30
rdopieralskimarkmc: no, by having a config file, like here: https://github.com/openstack/horizon/blob/master/openstack_dashboard/enabled/_20_admin.py13:30
dhellmannflaper87: thanks for spotting this and working out the solution :-)13:31
rdopieralskimarkmc: now it would also add its own config file, with that group13:31
flaper87dhellmann: my pleasure :)13:31
markmcrdopieralski, ok, the common pattern for this would be to do13:31
markmc[horizon]13:31
markmc(one sec)13:31
markmcsorry13:32
markmc[horizon]13:32
markmcdashboards = foo,bar13:32
markmc[dashboard:foo]13:32
markmc...13:32
markmc[dashboard:bar]13:32
markmc...13:32
markmcat least nova, neutron, cinder do something very similar13:33
rdopieralskithat won't work for us, because then installing a plugin would require you to modify a single central file13:33
markmcok, then you could use MultiStrOpt13:33
markmc[horizon]13:33
markmcdashboard = foo13:33
markmcdashboard = bar13:33
markmc[dashboard:foo]13:33
markmc...13:33
markmc[dashboard:bar]13:34
markmc...13:34
rdopieralskihow do you disable a dashboard then?13:34
markmc[dashboard:bar]13:34
markmcdisabled = True13:34
openstackgerritFlavio Percoco proposed a change to openstack/oslo-specs: graduate-oslo-utils  https://review.openstack.org/9843113:34
rdopieralskibut it still won't work with command line options13:34
flaper87dhellmann: ^ added mask_password assuming a not-yet-propossed patch that moves it into strutils13:34
dhellmannrdopieralski, markmc : can't the dashboard plugins register their own options?13:34
flaper87I'll work on that patch right away13:34
rdopieralskiso the only "advantage" is that you have to type the names of the dashboards twice13:35
*** jecarey has quit IRC13:35
rdopieralskidhellmann: they would need to execute some code to do that13:35
markmcrdopieralski, this doesn't seem like the kind of thing you want to do on the command line13:35
rdopieralskimarkmc: yeah, sure13:35
rdopieralskimarkmc: that's not really an issue13:35
openstackgerritGordon Sim proposed a change to openstack/oslo.messaging: Possible performance optimisation for publish  https://review.openstack.org/10419413:36
markmcrdopieralski, if you really want to do it your way, use ConfigParser (or whatever it's called) to parse the files listed in CONF.config_files and list their sections13:36
rdopieralskiso every dahboard config file would have this boilerplate:  [horizon] dashoard = foo [dashboad:foo] ...13:36
dhellmannrdopieralski: they're being imported13:36
rdopieralskimarkmc: I would do that, but oslo.config had the bright idea of writing their own ConfigParser, and the syntax differs in obscure corner cases13:37
markmcrdopieralski, that ConfigParser is part of the public API13:37
rdopieralskidhellmann: they are being imported after being listed in the settings13:37
rdopieralskimarkmc: then I introduce a race condition13:39
markmcrdopieralski, the files list passed to load_config() - what does that look like?13:39
dhellmannrdopieralski: on closer reading, I see you're having them provide values for options for values that something else is using13:40
rdopieralskimarkmc: _files = [os.path.join(ROOT_PATH, 'settings.conf'), os.path.join(LOCAL_PATH, 'local_settings.conf')]13:41
rdopieralski_files.extend(sorted(glob.glob(os.path.join(ENABLED_PATH, '*.conf'))))13:41
markmcENABLED_PATH is where the per-dashboard config files are?13:41
rdopieralskiyes13:41
rdopieralskibut ideally I don't want to have two different file formats13:42
rdopieralskiand I want to be able to add several dashboards and panels in one file13:42
dhellmannwow this dashboard discovery code is hairy13:43
rdopieralskiit is13:43
rdopieralskiit was a stopgap solution13:43
markmcrdopieralski, ok, if it was dashboard-per-file I'd suggest using the basename of each file to look for [dashboard:] sections13:43
dhellmannyou could probably replace a lot of this with stevedore, which would give you an API entry point to invoke to register options, which would avoid the need to explicitly scan the config settings for the dashboards13:43
rdopieralskimarkmc: why do I need to have the name twice if it's there already?13:44
dhellmannthere's even a manager for plugins that know how to figure out if they are enabled or not, which would automatically ignore the ones with "disabled = True" in their config section13:44
rdopieralskidhellmann: that sounds interesting13:44
markmcrdopieralski, the notion of listing unregistered groups doesn't exist in the API13:44
dhellmannhttp://stevedore.readthedocs.org/en/latest/managers.html#enabledextensionmanager13:44
markmcrdopieralski, so, an explicit list of dashboards or parse the files to list their sections13:45
rdopieralskidhellmann: thank you, I will see what I can do with that13:45
markmcyeah, stevedore is a good idea too13:45
rdopieralskiwell, I've heard the name a couple of times, but it's hard to guess what it does13:45
dhellmannupdate_dashboards() I think could be replaced by a loop over the manager calling methods on the plugins like "get_installed_apps" and "get_angular_modules" to retrieve those settings13:45
dhellmannhttp://en.wikipedia.org/wiki/Stevedore13:46
rdopieralskiif it was named oslo.plugins ;)13:46
dhellmannrdopieralski: it's not limited to use in openstack, and started life outside of oslo :-)13:47
rdopieralskiI will definitely try to use that, it looks like it will solve a lot of our problems13:47
rdopieralskithank you13:47
rdopieralskiI should come here more often ;)13:47
dhellmannrdopieralski: let me know if I can help you find the right use pattern13:47
dims__dhellmann, about the i18n review. did you get a chance look at the instances of gettextutils.Message in nova?13:49
dhellmanndims__: They're instantiating Message directly?13:49
dims__y13:50
dhellmannwhy?13:50
dims__or at least referring to them13:50
dims__    elif isinstance(value, gettextutils.Message):13:50
dims__        return unicode(value).encode('utf-8')13:50
dhellmannthat's what translate() is for13:50
dims__        elif isinstance(value, gettextutils.Message):13:50
dims__            return value.data13:50
dhellmannok, let me look...13:50
dims__thanks13:50
dims__biab13:52
dhellmanndims__: ok, the case in the API code can be replaced with a call to translate(), since it does the same checks13:56
dhellmanndims__: the case in the test may be more trouble, but I think we can work around that, too13:57
dhellmannmarkmc: do you have any opinion on whether Message should be part of the public API of oslo.i18n? I'm trying to treat it as an implementation detail.13:59
*** zzzeek has joined #openstack-oslo14:01
markmcdhellmann, no opinion I'm afraid14:06
markmcdhellmann, keep it private until there's an obvious use case for it, I guess14:06
boris-42markmc  dhellmann btw guys can we discuss about stevedore14:06
boris-42markmc dhellmann  do we actually need it?14:06
boris-42markmc dhellmann  I mean there is a simple way to make autodiscovring that doesn't require specifying everything14:07
boris-42markmc dhellmann  in conf file14:07
boris-42markmc dhellmann  e.g. you need to create 1 class as a base14:07
boris-42and iterate through all his childs until you find the class with proper name14:08
boris-42why this doesn't work for example for oslo.messaging ?14:08
boris-42maybe I am missing something14:08
boris-42(it's not hollywar, just a question)14:09
*** erecio has quit IRC14:10
zzzeekwhat do we need to bump SQLA requirements to >= 0.8 ?14:18
boris-42zzzeek hm add patch to global requreiments14:21
boris-42zzzeek https://github.com/openstack/requirements14:21
zzzeekand see what everyone says14:21
zzzeekwell yes i knew it was there :)14:21
boris-42zzzeek so everybody will get updates14:21
boris-42zzzeek to update their version to it14:21
boris-42zzzeek and we will see what will say gates of these projects=)14:22
zzzeekboris-42: OK but in reality this is more of a vendor/customer thing b.c. some distros might still be putting out 0.7.814:22
boris-42zzzeek btw14:25
boris-42zzzeek could I ask you for some help14:25
zzzeeksure14:25
ihrachyshkazzzeek: which of them, specifically? for RHOSP5 for RHEL 6.5 it's already 0.8.4-something, and it's quite an old platform14:26
boris-42zzzeek probably we can get some more interesting data14:26
boris-42zzzeek https://github.com/stackforge/osprofiler/blob/master/osprofiler/sqlalchemy.py14:26
boris-42zzzeek during profiling14:26
zzzeekihrachyshka: as new as I can get is the best, if we made it 0.8.4, terrific14:26
zzzeekboris-42: i might profile SQL at the cursor.execute level14:28
rpodolyakabtw, there already is a patch on review bumping alembic and sa versions in global requirements - https://review.openstack.org/#/c/101918/14:28
boris-42zzzeek what is the difference ?14:28
zzzeekboris-42: well you get the raw thing that is being sent to the cursor, at the very least it means your str() call there won’t potentially cause a SQL Core element to recompile itself14:28
zzzeekboris-42: which takes up a significant portion of time14:29
boris-42zzzeek hm interesting14:29
boris-42zzzeek yep seems reasonable14:29
zzzeekboris-42: DB level tracking, transactions are interesting also, rollbacks which occur when connections are returned to the conn pool are often a performance issue14:30
zzzeekboris-42: OS’s patterns of many small transactions adds up a lot14:30
zzzeekboris-42: transactions are not cheap on MySQL14:30
boris-42zzzeek btw if you'll have some time14:33
boris-42zzzeek It will be nice if you improve this part a bit=)14:33
zzzeekthis file here looks pretty straightforward14:34
boris-42zzzeek lol=)14:36
boris-42zzzeek I like when things are simple=)14:36
boris-42zzzeek btw I have issue with this when I am running14:37
boris-42zzzeek these data is creating a lot of load on collector14:37
boris-42zzzeek cause sql queires are quite loooong14:37
openstackgerritlawrancejing proposed a change to openstack/oslo-incubator: Use immutable arg rather mutable arg  https://review.openstack.org/10369814:38
boris-42zzzeek e.g something like that http://paste.openstack.org/show/85328/14:38
zzzeekwell that is a pretty terrible query, left outer joining 5 times is pretty unnecessary14:40
zzzeeksomeone is conflating joinedload() with “magic pixie performance dust"14:40
zzzeekpro tip:  put it in an editor with word wrap, search for the first “FROM”, add a line break right there14:41
zzzeekfrom FROM on down, that’s really the query14:41
openstackgerritYuriy Taraday proposed a change to openstack/oslo.config: Allow to reference values outside DEFAULT in substitution  https://review.openstack.org/10388414:42
openstackgerritFlavio Percoco proposed a change to openstack/oslo-incubator: Move `mask_password` to strutils  https://review.openstack.org/10420914:43
boris-42zzzeek yep14:43
boris-42zzzeek that is why we need this stuff=)14:43
zzzeekjoinedload() supports inner=True so those are at least inner joins14:43
boris-42zzzeek so I am not speaking about that query =)14:44
boris-42zzzeek I am speaking about osprofiler lib =)14:44
boris-42zzzeek and what we can collect more related to sqlalchemy14:44
boris-42zzzeek and how to make it better=)14:44
boris-42zzzeek I don't want to fix anything in openstack, unit I finish work on osproifler14:45
zzzeeki know that, but thats the query they’re writing, that’s it.  not sure what improvement there could be14:45
boris-42zzzeek cause it's waste of time=)14:45
boris-42zzzeek in osproifler?14:45
boris-42zzzeek or in nova?)14:45
zzzeekwhen you say, “the query is looming"14:45
boris-42zzzeek oh sorry14:45
boris-42zzzeek forgot abou tthat=)14:46
boris-42zzzeek is there anything interesting data then we can fetch?14:46
boris-42zzzeek e.g. amount of updated/changed rows and so on?14:46
zzzeeknot without querying the database14:46
zzzeeki guess you could log session size14:46
zzzeekyou want to catch flush() calls and count session.new / dirty / deleted / identity_map ?14:47
boris-42zzzeek yep getting info about session14:48
boris-42zzzeek can help sometimes14:48
openstackgerritA change was merged to openstack/oslo.config: Add oslo-config-generator  https://review.openstack.org/10094614:51
*** tkelsey has quit IRC14:54
*** dims__ has quit IRC14:55
openstackgerritFlavio Percoco proposed a change to openstack/oslo-incubator: Remove `processutils` dependency on `log`  https://review.openstack.org/10421314:55
dhellmannmarkmc: ok, that's basically what I was trying to do14:56
dhellmannboris-42: scanning subclasses only works if the code containing those subclasses has already been imported. stevedore finds plugins defined in other packages14:56
boris-42dhellmann so we resolve this issue by forcing importing these modules14:57
dhellmannboris-42: how can I import your code from my application if I don't know your code exists?14:57
openstackgerritFlavio Percoco proposed a change to openstack/oslo-incubator: Remove `processutils` dependency on `log`  https://review.openstack.org/10421314:58
openstackgerritFlavio Percoco proposed a change to openstack/oslo-incubator: Move `mask_password` to strutils  https://review.openstack.org/10420914:58
boris-42dhellmann not sure that I get=)14:58
dhellmannboris-42: stevedore will load plugins from third-party packages14:58
boris-42dhellmann we had the same issue with rally plugins14:58
boris-42dhellmann we are just importing .py files14:58
flaper87dhellmann: no need to review now, just a heads up. those 2 last patches are for the mask_password thing and making processutils use stdlib logging14:59
boris-42dhellmann that are specified if they are specified14:59
boris-42dhellmann e.g. rally imports all .py files from .rally/plugins directory14:59
flaper87(I'll comment on the spec-utils mentioning the review)14:59
dhellmannboris-42: yeah, you're going backwards in usability, potentially introducing import issues, and requiring explicit configuration -- all of those are things setuptools entry points were meant to fix14:59
boris-42dhellmann backward?15:00
dhellmannboris-42: what you're suggesting does not actually solve all of the use cases that entry points solve15:00
boris-42dhellmann imho configuration options are backward15:00
boris-42dhellmann for example in rally we have more then 50 classes15:00
dhellmannboris-42: you need to look at the history before we continue this conversation15:00
boris-42dhellmann probably15:00
*** jecarey has joined #openstack-oslo15:00
boris-42dhellmann any interesting post about that?15:00
boris-42dhellmann as I said probably I don't know something15:01
dhellmannboris-42: most of what I've written is in the stevedore docs15:01
boris-42dhellmann but currently it doesn't looks nice15:01
boris-42dhellmann e.g. if you have 200 classes15:01
boris-42dhellmann you'll need 200 options15:01
boris-42dhellmann if you have 2000 you'll need 200015:01
boris-42dhellmann to write any plugin you need to change configuration15:02
boris-42dhellmann but okay I'll try to find historical reasons of stevedore15:02
dhellmannhttp://stevedore.readthedocs.org/en/latest/essays/pycon2013.html15:02
*** tsekiyama has joined #openstack-oslo15:03
boris-42dhellmann thanks15:03
boris-42dhellmann btw I decided to change one name in public API of osprofiler15:04
boris-42dhellmann that is used by oslo.messaging15:04
boris-42dhellmann as I need in any case to push 0.2.0 version15:04
boris-42dhellmann but I don't think that it should be blocker for that spec ..15:04
dhellmannboris-42: i get nervous about making oslo.messaging rely on something that doesn't have a stable api15:05
boris-42dhellmann it's just changing of name..15:05
boris-42dhellmann get_profiler -> profiler()15:05
dhellmannboris-42: if the old name no longer works, that's an API change15:05
boris-42dhellmann I will make all changes15:06
boris-42dhellmann before any of patches in OpenSack15:06
dhellmannboris-42: also, functions and methods should have verbs in their name15:06
boris-42dhellmann sorry15:06
boris-42dhellmann get_profiler() -> get()15:06
dhellmannreally?15:06
boris-42dhellmann it's done to make it more pretty15:06
boris-42from osproifler import profiler15:07
boris-42profiler.get()15:07
*** tkelsey_ has joined #openstack-oslo15:07
boris-42instead of profiler.get_proifler()15:07
boris-42so there are methods init(), get(), start() and stop()15:07
dhellmannflaper87: what effect will the deprecation warning in https://review.openstack.org/#/c/104209/2 have on log output for tests -- we get complaints about oslo changing logging behavior15:08
flaper87dhellmann: tests were moved to strutils15:10
flaper87oh wait, you mean nova and other apps tests15:10
* flaper87 slaps himself15:10
dhellmannflaper87: yeah15:11
flaper87mmh, that'sa good point. I've no idea but I'll try to find out15:12
flaper87I actually was wondering whether keeping this method there makes sense at all15:12
flaper87we're moving it and someone syncing from oslo should notice that, I think15:13
flaper87(last famous words)15:13
*** i159 has quit IRC15:13
flaper87Based on other services tests, it'll cause quite some noise15:13
dhellmannflaper87: yeah, I would like to put off forcing the import change until they migrate to oslo.log, to avoid complaints about making syncs harder15:14
flaper87dhellmann: makes sense15:14
dhellmannflaper87: if it makes the logs noisy, we should take out the warning and leave a comment15:14
flaper87dhellmann: what about just getting rid of the warning?15:14
flaper87LOL15:14
flaper87yeah, we'll force the change when switching over to oslo.log15:15
dhellmannI had really meant for you to do something like "from openstack.common.strutils import mask_password # noqa"15:15
dhellmannthe warning would be good, except for the complaining :-/15:15
flaper87dhellmann: I did that but I then wrote that wrapper+warning to give people a heads up about the change15:15
flaper87ok, I'll change that with the import from strutils15:16
*** nacim has quit IRC15:16
openstackgerritFlavio Percoco proposed a change to openstack/oslo-incubator: Remove `processutils` dependency on `log`  https://review.openstack.org/10421315:17
openstackgerritFlavio Percoco proposed a change to openstack/oslo-incubator: Move `mask_password` to strutils  https://review.openstack.org/10420915:17
dhellmannflaper87: yeah, if you add "if you don't like this warning message complain to flaper87 instead of dhellmann" then I can +2 it :-)15:18
openstackgerritMark McLoughlin proposed a change to openstack/oslo.config: generator: refactor MultiStrOpt handling  https://review.openstack.org/10422515:18
openstackgerritMark McLoughlin proposed a change to openstack/oslo.config: cfg,generator: add Opt.sample_default attribute  https://review.openstack.org/10422615:18
openstackgerritMark McLoughlin proposed a change to openstack/oslo.config: generator: remove the default sanitizer feature  https://review.openstack.org/10422715:18
openstackgerritYuriy Taraday proposed a change to openstack/oslo-specs: Add rootwrap-daemon-mode blueprint  https://review.openstack.org/9461315:18
flaper87dhellmann: LOOOOOOL, I prefer the silent, nasty, noqa import15:18
flaper87:P15:18
dhellmannflaper87: I thought you might see it my way :-)15:19
openstackgerritFlavio Percoco proposed a change to openstack/oslo-incubator: Remove `processutils` dependency on `log`  https://review.openstack.org/10421315:20
openstackgerritFlavio Percoco proposed a change to openstack/oslo-incubator: Move `mask_password` to strutils  https://review.openstack.org/10420915:20
flaper87ok, the last one fixes pep8 and adds a comment to the import regarding oslo.log graduation15:20
openstackgerritMark McLoughlin proposed a change to openstack/oslo-specs: Add oslo-config-generator  https://review.openstack.org/10254815:20
dhellmannflaper87: should we link those commits to both graduation blueprints?15:21
flaper87I didn't do it because I considered this change a tiny step forward so, nothing really really related to the oslo.log blueprint. I can do it if you think otherwise15:22
flaper87dhellmann: ^15:22
*** dims__ has joined #openstack-oslo15:23
dhellmannflaper87: I'm probably just being pedantic about tracking; we can leave them as they are15:23
dhellmannflaper87: while you wait for those tests to run, do you have a few minutes to look over https://review.openstack.org/#/c/103224/ for me? That's the last patch to review before releasing oslo.i18n15:24
* flaper87 clicks and reviews dhellmann's patch15:25
* flaper87 needs to improve his oslo's queries on gerrit15:25
dhellmannflaper87: https://wiki.openstack.org/wiki/Oslo#Review_Links15:26
dims__dhellmann, sounds good15:27
*** dims__ has quit IRC15:27
*** dims__ has joined #openstack-oslo15:27
dhellmanndims__: thanks for bringing that up, though, I should add it to the migration notes15:28
dims__dhellmann, +115:29
flaper87dhellmann: oooohhhhhh, that link is useful.15:30
openstackgerritDoug Hellmann proposed a change to openstack/oslo-specs: Update adoption notes about Message objects  https://review.openstack.org/10423415:32
dhellmanndims__: ^^15:32
*** james_li has joined #openstack-oslo15:36
*** SridharG has quit IRC15:43
*** markmcclain has joined #openstack-oslo15:45
openstackgerritFlavio Percoco proposed a change to openstack/oslo-incubator: Remove `processutils` dependency on `log`  https://review.openstack.org/10421315:46
openstackgerritFlavio Percoco proposed a change to openstack/oslo-incubator: Move `mask_password` to strutils  https://review.openstack.org/10420915:46
* flaper87 swears he tested pep8 this time15:46
*** ihrachyshka has quit IRC15:51
boris-42dhellmann btw can I make one another spec15:58
boris-42dhellmann called "Integration of OSprofiler in projects"15:58
boris-42dhellmann cause the process is the same for all projects, and it's hard for me to get the similar spec in all projects specs stuff15:58
boris-42dhellmann and some of projects are asking spec15:59
dhellmannboris-42: do they want an oslo spec, or do they want a spec in their own repo where they can vote?16:05
boris-42dhellmann I would like to have one spec16:06
boris-42dhellmann to call all PTL and cores that are interested16:06
boris-42dhellmann and make one spec that works for all16:06
dhellmannboris-42: I understand that, but what are you being *asked* to do?16:06
boris-42dhellmann so to have some spec that will be discussed16:06
boris-42dhellmann and approved16:06
boris-42dhellmann not all projects are asking but for some of these projects is intereting to see that this is accepted by community16:07
dhellmannboris-42: is that not https://review.openstack.org/#/c/103825/ ?16:07
boris-42dhellmann nope16:07
boris-42dhellmann it's related to oslo.messaging16:07
boris-42dhellmann it's too specific16:07
boris-42dhellmann integrating in oslo.messaging and in projects like cinder is quite different16:08
dhellmannboris-42: why not add details to ^^?16:08
boris-42dhellmann but there is no difference between integrating in cider and nova16:08
boris-42dhellmann so then I should chose another name for BP16:08
boris-42dhellmann e.g. Integration of OSprofiler in OpenStack16:08
*** arnaud has joined #openstack-oslo16:08
*** ildikov has joined #openstack-oslo16:09
boris-42dhellmann I will do what you'll prefer16:09
boris-42dhellmann to put it in one or in 2 specs16:09
dhellmannisn't the oslo.messaging work an implementation aspect of enabling the cross-service profiling?16:09
boris-42dhellmann oslo.messaging is just used to sync profiler instances in different services (of one project)16:09
boris-42dhellmann but enabling osproifler in project means to add one small patch in python client16:10
boris-42dhellmann and one middleware in code base16:10
dhellmannboris-42: ok, try to write one spec describing the changes needed, and if you have to rename the spec that's ok16:10
dhellmannboris-42: include *all* of the changes (client, API, messaging, whatever)16:10
*** ildikov|2 has joined #openstack-oslo16:11
boris-42dhellmann okay there are just those 316:11
*** ildikov has quit IRC16:11
boris-42dhellmann actually 4 (oslo.db can be as well changed) but it's not essential16:11
boris-42dhellmann okay I will do16:11
dims__flaper87, don't you have to fix processutils.py too? (in https://review.openstack.org/#/c/104209)16:11
dhellmannboris-42: *all* of the changes :-)16:12
*** ildikov_ has quit IRC16:12
boris-42dhellmann okay, I'll ping you when I finish16:12
boris-42dhellmann thanks16:12
flaper87dims__: there's a follow-up patch for that16:12
flaper87dims__: this patch imports mask_password from strutils so processutils is fine16:13
dims__flaper87, ah ok.16:13
flaper87dims__: since you're on it. This is the follow-up patch: https://review.openstack.org/#/c/104213/16:14
dims__y, found it16:14
dims__just as a note, when we sync to nova, we'll need to fix 3 references to logging.mask_password16:15
dhellmanndims__: we left an import in log.py so we didn't need to worry about that on sync, but we will need to do it when moving to oslo.log16:16
dims__sounds good. thx16:16
flaper87dims__: thanks for the reviews16:18
*** mriedem has quit IRC16:19
dims__flaper87, my pleasure16:20
*** jaosorior has quit IRC16:22
openstackgerritA change was merged to openstack/oslo-incubator: Move `mask_password` to strutils  https://review.openstack.org/10420916:23
openstackgerritA change was merged to openstack/oslo-incubator: Remove `processutils` dependency on `log`  https://review.openstack.org/10421316:23
*** SridharG has joined #openstack-oslo16:25
*** markmc has quit IRC16:31
*** ildikov|2 has quit IRC16:33
*** flaper87 is now known as flaper87|afk16:34
zhiyanhi folks, does oslo.config has a routine release schedule? how can i get a notice for that? check it by maillist?16:38
dhellmannzhiyan: we plan to tag alphas as needed throughout the cycle, and release the next non-alpha version at the end of juno16:39
dhellmannzhiyan: I try to mention planned releases in the project meetings on tuesday and we announce new releases on the dev mailing list, but don't necessarily plan out dates for alphas very far in advance16:39
openstackgerritYuriy Taraday proposed a change to openstack/oslo.config: Allow to reference values outside DEFAULT in substitution  https://review.openstack.org/10388416:46
openstackgerritA change was merged to openstack/oslo-incubator: Use lockutils wrapper to set OSLO_LOCK_PATH for run_tests.sh  https://review.openstack.org/8563716:47
*** markmcclain has quit IRC16:48
*** zhiyan is now known as zhiyan_16:50
*** zhiyan_ is now known as zhiyan16:51
*** harlowja_away is now known as harlowja16:52
*** tkelsey_ has quit IRC16:52
*** markmcclain has joined #openstack-oslo16:58
*** markmcclain has quit IRC16:58
*** AAzza has joined #openstack-oslo16:59
*** markmcclain has joined #openstack-oslo16:59
AAzzahi, all. I want to help with Python 3 support. Currenty I'm searching for problems for myself, but maybe you can suggest some modules it is worth to pay attention?17:05
*** zhiyan is now known as zhiyan_17:08
*** Alexei_9871 has quit IRC17:09
*** praneshp has joined #openstack-oslo17:14
*** tmcpeak has joined #openstack-oslo17:15
*** tmcpeak has left #openstack-oslo17:16
*** AAzza has left #openstack-oslo17:18
*** AAzza has joined #openstack-oslo17:18
*** mriedem has joined #openstack-oslo17:25
*** AAzza is now known as AAzza_afk17:29
dims__AAzza_afk, if you check the tox.ini you will see that only a subset of test cases are enabled for py3317:32
*** dstanek is now known as dstanek_lunch17:41
*** openstackgerrit has quit IRC17:49
*** openstackgerrit has joined #openstack-oslo17:49
*** arnaud has quit IRC17:59
*** i159 has joined #openstack-oslo18:03
openstackgerritA change was merged to openstack/oslo.i18n: Shift public API to top level package  https://review.openstack.org/10322418:14
*** AAzza_afk is now known as AAzza18:25
*** rdopieralski has quit IRC18:28
openstackgerritAnne Gentle proposed a change to openstack/oslosphinx: Added a incubating project config option  https://review.openstack.org/10393518:28
*** zzzeek has quit IRC18:30
*** zzzeek has joined #openstack-oslo18:34
*** dims__ has quit IRC18:37
*** i159 has quit IRC18:39
*** SridharG has quit IRC18:41
*** tongli has joined #openstack-oslo18:43
*** dims__ has joined #openstack-oslo18:47
*** james_li has quit IRC18:49
openstackgerritDoug Hellmann proposed a change to openstack/oslo.i18n: Document how to add import exceptions  https://review.openstack.org/10321318:49
openstackgerritDoug Hellmann proposed a change to openstack/oslo.i18n: Remove mention of Message objects from public docs  https://review.openstack.org/10281918:49
*** AAzza- has joined #openstack-oslo18:54
*** dstanek_lunch is now known as dstanek_lunch_zz18:55
*** james_li has joined #openstack-oslo18:55
*** AAzza has quit IRC18:57
*** AAzza- is now known as AAzza18:57
*** harlowja is now known as harlowja_away18:59
*** dstanek_lunch_zz is now known as dstanek18:59
*** AAzza has quit IRC19:03
*** harlowja_away is now known as harlowja19:03
*** AAzza has joined #openstack-oslo19:08
dhellmannoslo.i18n 0.1.0 is released!19:12
dims__w00t!19:15
dims__dhellmann, have a nova review in progress? should i try?19:16
dhellmanndims__: we need to get it into the global requirements list, first, but if you want to start working on a port locally that would be great19:16
bknudsonnow I have to get working on keystone using it!19:17
dims__dhellmann, will do19:17
dhellmannbknudson: +119:18
dhellmanndims__: thanks19:18
openstackgerritDoug Hellmann proposed a change to openstack/oslo-incubator: Mark gettextutils obsolete  https://review.openstack.org/10430719:20
*** AAzza is now known as AAzza_afk19:22
*** dims__ has quit IRC19:33
*** zzzeek has quit IRC19:41
*** dims has joined #openstack-oslo19:44
*** dims_ has joined #openstack-oslo19:48
*** praneshp has quit IRC19:49
*** dims has quit IRC19:49
*** praneshp has joined #openstack-oslo19:54
*** dims_ has quit IRC19:54
*** dims has joined #openstack-oslo19:54
*** dims_ has joined #openstack-oslo19:58
*** dims has quit IRC20:00
*** dims_ has quit IRC20:05
*** dims has joined #openstack-oslo20:06
*** harlowja is now known as harlowja_away20:09
*** ihrachyshka has joined #openstack-oslo20:21
*** james_li has quit IRC20:22
*** james_li has joined #openstack-oslo20:22
*** ihrachyshka has quit IRC20:26
*** alexpilotti has quit IRC20:31
*** harlowja_away is now known as harlowja20:39
*** dims has quit IRC20:55
openstackgerritKen Giusti proposed a change to openstack/oslo.messaging: An initial implementation of a 1.0 based messaging driver  https://review.openstack.org/7581520:58
*** dims has joined #openstack-oslo21:00
*** mrda_away is now known as mrda21:02
*** jecarey has quit IRC21:06
*** dims_ has joined #openstack-oslo21:13
*** dims has quit IRC21:15
*** lbragstad_ is now known as lbragstad21:17
openstackgerritJoshua Harlow proposed a change to openstack/taskflow: Update oslo incubator code to commit 0b02fc0f36814968  https://review.openstack.org/10434521:17
openstackgerritA change was merged to openstack/oslosphinx: Added a incubating project config option  https://review.openstack.org/10393521:18
*** dims_ has quit IRC21:18
*** james_li has quit IRC21:20
*** jecarey has joined #openstack-oslo21:20
openstackgerritA change was merged to openstack/oslo-specs: graduate-oslo-utils  https://review.openstack.org/9843121:21
*** ajo|mac has quit IRC21:21
*** markmcclain has quit IRC21:25
*** dims has joined #openstack-oslo21:31
bknudsondhellmann: question about switch to oslo.i18n --21:37
bknudsonthere's still "from keystone.openstack.common.gettextutils import _LI" in keystone.openstack.common21:38
dhellmannbknudson: yeah, I have a note to work on updating the incubator, we might need to do that before you can adopt it21:39
bknudsondhellmann: ok, sounds complicated21:39
dhellmannbknudson: I was going to try to do that in the morning when I'm fresh, because I need to figure out how to get the new imports right21:39
bknudsonif incubator uses it then keystone will have to import even if we don't switch to oslo.i18n21:40
dhellmannbknudson: I was thinking about adding an integration module to the incubator, and letting projects sync that instead of gettextutils. I haven't tried it to see if it works, though.21:40
bknudsondhellmann: that's what I was thinking you might be able to do21:41
dhellmannI think we'd eventually be able to get rid of it, after we remove the incubated copy of gettextutils21:42
dhellmannalthough maybe not, since other projects are likely to bring code with translations back into the incubator21:43
bknudsonall the oslo-incubator stuff should need is _, _LI, etc.21:44
bknudsonif the application provided those then it could set them to either oslo-incubator gettextutils or its own from oslo.i18n21:45
dhellmannbknudson: the problem is the imports in the incubated modules21:45
bknudsonmaybe oslo-incubator changes to "from oslo.i18n import _"21:46
bknudsonthen when I import that into keystone it becomes "from keystone.i18n import _"21:46
dhellmannthere's no _ in oslo.i18n, though, that's the thing -- it will only exist in the integration module within the project21:46
bknudsonwhich is the name of my integration module anyways21:46
bknudsonsince update.py changes oslo to keystone21:47
bknudsonthis would require everyone to name their integration module <base>.i18n21:47
dhellmannthat works except for 2 things: there is no instance of the translator factory in oslo.i18n, because that can't be a global the library creates, and oslo.i18n is a special case that isn't replaced21:48
dhellmannhowever, you're on the right track21:48
dhellmannan openstack/common/i18n.py could have all of those things21:49
dhellmannand "from openstack.common import i18n" would be changed to "from keystone.openstack.common import i18n", and the contents of that file would also have oslo replaced with keystone21:49
dhellmannthat's what I am going to experiment with tomorrow morning21:49
openstackgerritAbhishek Chanda proposed a change to openstack/stevedore: Migrate tox to use testr  https://review.openstack.org/10167321:50
bknudsondhellmann: ok, thanks21:51
dhellmannbknudson: I'll add you to the review so you can see what I mean21:52
openstackgerritJoshua Harlow proposed a change to openstack/taskflow: Remove need to do special exception catching in parse_uri  https://review.openstack.org/10435421:53
*** james_li has joined #openstack-oslo21:55
*** tongli has quit IRC22:01
*** pcm_ has quit IRC22:02
*** zzzeek has joined #openstack-oslo22:22
openstackgerritDoug Hellmann proposed a change to openstack/oslo-specs: Update adoption notes about Message objects  https://review.openstack.org/10423422:33
*** tsekiyam_ has joined #openstack-oslo22:42
*** tsekiyama has quit IRC22:45
openstackgerritJoshua Harlow proposed a change to openstack/taskflow: Update oslo incubator code to commit 0b02fc0f36814968  https://review.openstack.org/10434522:47
openstackgerritJoshua Harlow proposed a change to openstack/taskflow: Remove need to do special exception catching in parse_uri  https://review.openstack.org/10435422:47
*** tsekiyam_ has quit IRC22:47
openstackgerritJoshua Harlow proposed a change to openstack/taskflow: Update oslo incubator code to commit 0b02fc0f36814968  https://review.openstack.org/10434522:55
openstackgerritJoshua Harlow proposed a change to openstack/taskflow: Remove need to do special exception catching in parse_uri  https://review.openstack.org/10435422:55
*** jecarey has quit IRC22:57
*** jecarey has joined #openstack-oslo22:57
openstackgerritA change was merged to openstack/oslo-incubator: Mark gettextutils obsolete  https://review.openstack.org/10430723:09
openstackgerritA change was merged to openstack/oslo-incubator: Use timestamp in loopingcall  https://review.openstack.org/10370423:13
openstackgerritA change was merged to openstack/oslo-incubator: update new requests logger to default WARN  https://review.openstack.org/10396023:17
*** james_li has quit IRC23:27
openstackgerritA change was merged to openstack/oslo-incubator: Changes calcuation of variable delay  https://review.openstack.org/10370823:44
*** oomichi has joined #openstack-oslo23:45
« Tuesday, 2014-07-01 Index Thursday, 2014-07-03 »

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!