| *** tsekiyam_ has joined #openstack-oslo | 00:02 | |
| *** tsekiyama has quit IRC | 00:06 | |
| *** tsekiyam_ has quit IRC | 00:07 | |
| *** pblaho has quit IRC | 00:12 | |
| openstackgerrit | A change was merged to openstack/pycadf: Debug env for tox https://review.openstack.org/109903 | 00:37 |
|---|---|---|
| *** zzzeek has quit IRC | 00:45 | |
| *** celttechie has joined #openstack-oslo | 00:53 | |
| *** Ish__ has joined #openstack-oslo | 01:01 | |
| *** mriedem has joined #openstack-oslo | 01:11 | |
| *** Ish__ has quit IRC | 01:34 | |
| *** mriedem has quit IRC | 01:44 | |
| *** jgrimm has joined #openstack-oslo | 01:45 | |
| *** mriedem has joined #openstack-oslo | 01:51 | |
| *** mriedem has left #openstack-oslo | 01:55 | |
| *** mriedem has quit IRC | 01:55 | |
| *** SridharG has joined #openstack-oslo | 01:56 | |
| *** SridharG has quit IRC | 01:58 | |
| openstackgerrit | A change was merged to openstack/oslo-incubator: Fix E126 pep8 errors https://review.openstack.org/110892 | 02:46 |
| *** arnaud has quit IRC | 03:01 | |
| *** jecarey has joined #openstack-oslo | 03:34 | |
| *** oomichi has quit IRC | 03:48 | |
| *** arnaud has joined #openstack-oslo | 04:16 | |
| *** Ish__ has joined #openstack-oslo | 04:19 | |
| *** Ish__ has left #openstack-oslo | 04:20 | |
| openstackgerrit | A change was merged to openstack/oslo.messaging: Enable check for E226 https://review.openstack.org/108278 | 04:29 |
| *** jaosorior has joined #openstack-oslo | 04:48 | |
| *** bknudson has quit IRC | 05:35 | |
| *** mrda-away is now known as mrda | 06:00 | |
| *** k4n0 has joined #openstack-oslo | 06:02 | |
| *** celttechie has quit IRC | 06:05 | |
| openstackgerrit | Christian Berendt proposed a change to openstack/oslo.messaging: Enable PEP8 check E241 https://review.openstack.org/109077 | 06:06 |
| openstackgerrit | Christian Berendt proposed a change to openstack/oslo.messaging: Enable PEP8 check E265 https://review.openstack.org/109079 | 06:07 |
| openstackgerrit | Christian Berendt proposed a change to openstack/oslo.messaging: Enable PEP8 check E714 https://review.openstack.org/109080 | 06:07 |
| openstackgerrit | OpenStack Proposal Bot proposed a change to openstack/oslo.utils: Imported Translations from Transifex https://review.openstack.org/112766 | 06:10 |
| *** morrocan has joined #openstack-oslo | 06:33 | |
| *** morrocan has quit IRC | 06:45 | |
| *** i159 has joined #openstack-oslo | 06:50 | |
| *** ildikov has joined #openstack-oslo | 06:51 | |
| *** ihrachyshka has joined #openstack-oslo | 07:05 | |
| *** ihrachyshka has quit IRC | 07:09 | |
| *** morganfainberg is now known as morganfainberg_Z | 07:17 | |
| *** AAzza_afk is now known as AAzza | 07:27 | |
| *** flaper87|afk is now known as flaper87 | 07:29 | |
| *** morganfainberg_Z is now known as morganfainberg | 07:34 | |
| *** ihrachyshka has joined #openstack-oslo | 07:38 | |
| *** arnaud has quit IRC | 07:42 | |
| *** stevemar has quit IRC | 08:03 | |
| *** Alexei_987 has joined #openstack-oslo | 08:29 | |
| *** flaper87 is now known as flaper87|afk | 08:47 | |
| *** pblaho has joined #openstack-oslo | 08:51 | |
| *** ihrachyshka has quit IRC | 08:53 | |
| *** ihrachyshka has joined #openstack-oslo | 09:13 | |
| *** AAzza_ has joined #openstack-oslo | 10:02 | |
| *** AAzza_afk has joined #openstack-oslo | 10:04 | |
| *** ihrachyshka has quit IRC | 10:05 | |
| *** AAzza has quit IRC | 10:05 | |
| *** AAzza_afk is now known as AAzza | 10:05 | |
| *** AAzza_ has quit IRC | 10:08 | |
| *** alexpilotti has joined #openstack-oslo | 10:22 | |
| *** tsufiev has quit IRC | 10:23 | |
| *** ujjain has quit IRC | 10:23 | |
| *** tsufiev has joined #openstack-oslo | 10:25 | |
| *** ujjain has joined #openstack-oslo | 10:26 | |
| *** pcm_ has joined #openstack-oslo | 10:33 | |
| *** pcm__ has joined #openstack-oslo | 10:34 | |
| *** pcm_ has quit IRC | 10:38 | |
| *** abhishekk has joined #openstack-oslo | 10:47 | |
| *** AAzza is now known as AAzza_afk | 11:08 | |
| *** AAzza_afk is now known as AAzza | 11:22 | |
| *** pcm__ has quit IRC | 11:31 | |
| *** pcm_ has joined #openstack-oslo | 11:56 | |
| *** pcm_ has quit IRC | 11:56 | |
| *** pcm_ has joined #openstack-oslo | 11:56 | |
| *** ihrachyshka has joined #openstack-oslo | 12:05 | |
| *** ihrachyshka has quit IRC | 12:10 | |
| *** pcm_ has quit IRC | 12:12 | |
| openstackgerrit | Ilya Pekelny proposed a change to openstack/oslo-specs: Alembic environment runner specification https://review.openstack.org/112842 | 12:22 |
| *** bknudson has joined #openstack-oslo | 12:32 | |
| *** rpodolyaka has quit IRC | 12:34 | |
| *** bogdando has quit IRC | 12:34 | |
| *** bogdando has joined #openstack-oslo | 12:35 | |
| *** rpodolyaka has joined #openstack-oslo | 12:36 | |
| *** bknudson has quit IRC | 12:37 | |
| *** AAzza is now known as AAzza_afk | 12:38 | |
| openstackgerrit | Roman Vasilets proposed a change to openstack/oslo.db: Extract logging setup into a separate function https://review.openstack.org/106992 | 12:45 |
| *** pcm_ has joined #openstack-oslo | 12:46 | |
| *** dhellmann_ is now known as dhellmann | 12:47 | |
| *** bknudson has joined #openstack-oslo | 12:54 | |
| *** AAzza_afk is now known as AAzza | 12:54 | |
| *** gordc has joined #openstack-oslo | 12:56 | |
| *** linkid has joined #openstack-oslo | 13:00 | |
| linkid | hi | 13:00 |
| linkid | I have a question about oslo incubator | 13:01 |
| linkid | about apiclient/exceptions.py | 13:02 |
| dhellmann | hi, linkid, ask away | 13:04 |
| linkid | is it this channel or is there another channel for oslo-incubator ? | 13:04 |
| linkid | ok | 13:04 |
| linkid | the from_response function is weird to me | 13:04 |
| linkid | especially that : https://github.com/openstack/oslo-incubator/blob/master/openstack/common/apiclient/exceptions.py#L451 | 13:05 |
| linkid | because there is a get applied to a list | 13:06 |
| linkid | and when you look at wsme, it is not the same body type as expected here | 13:08 |
| linkid | so I look at ceilometer, to know how they do, but the from_response function were overrided, I think | 13:09 |
| linkid | https://github.com/openstack/python-ceilometerclient/blob/master/ceilometerclient/exc.py#L120 | 13:10 |
| *** zzzeek has joined #openstack-oslo | 13:10 | |
| dhellmann | the line in apiclient/exceptions.py is turning body.values() into a list (because in python 3 it is an iterable view) and then the [0] at the end pulls out the first element | 13:11 |
| dhellmann | that value is saved as error, and the get() calls are against that, so I guess the list is assumed to have a list of dictionaries | 13:11 |
| dhellmann | it's possible the ceilometer client code is based on an older (or newer) version of the apiclient stuff -- there have been a few different people working on bringing the most current code into the incubator to be shared, but I don't know how much progress they've made | 13:12 |
| linkid | yes, but when wsme raise an error, there is no list of dictionnaries in the first element of body.values(), for example | 13:12 |
| dhellmann | what's there instead? | 13:13 |
| linkid | it is something like that : {"debuginfo": <string>, "faultcode": <string>, "faultstring": <string>} | 13:14 |
| dhellmann | so a single dictionary, rather than a list? | 13:14 |
| linkid | so, the line 451 will return a string | 13:15 |
| linkid | yes | 13:15 |
| dhellmann | maybe that's why ceilometer's from_response() is different, then | 13:15 |
| linkid | yes, I think so too | 13:15 |
| linkid | so my question is : is it a bug in oslo-incubator or do I have to override the from_response function like in ceilometer ? | 13:16 |
| linkid | because maybe, it is used by another lib… | 13:17 |
| dhellmann | linkid: well, the different services apparently return errors in different ways, so if we want to use the incubator code for all of them it will have to know how to figure out what the error response looks like | 13:18 |
| linkid | ok, I see | 13:19 |
| linkid | but maybe I could add a test in the from_response function, non ? | 13:19 |
| dhellmann | linkid: yes, that might do it. Are you trying to make the ceilometer client code use the incubated version of apiclient? | 13:20 |
| linkid | dhellmann: nope, I'm currently working on another client | 13:21 |
| dhellmann | linkid: for a service that also uses wseme? (I'm trying to understand the source of your questions. :-) | 13:21 |
| linkid | but I will submit a ticket for this test | 13:21 |
| linkid | dhellmann: yes | 13:21 |
| dhellmann | ok | 13:22 |
| zzzeek | rpodolyaka: transparent reader/writer solving problems that dont exist, eh ? :) | 13:22 |
| rpodolyaka | zzzeek: heh :) | 13:23 |
| linkid | dhellmann: it is for the cloudkitty client which I'm currently working on | 13:23 |
| *** mriedem has joined #openstack-oslo | 13:24 | |
| linkid | dhellmann: for oslo-incubator, it is the same launchpad for oslo ? | 13:24 |
| dhellmann | linkid: yes, please use the oslo launchpad project for bugs and blueprints for the incubator | 13:26 |
| dhellmann | linkid: I think cloudkitty is the 3rd billing-related project I've learned about this week. I sense a meme. :-) | 13:27 |
| openstackgerrit | Ilya Pekelny proposed a change to openstack/oslo-specs: Alembic environment runner specification https://review.openstack.org/112842 | 13:31 |
| openstackgerrit | Ilya Pekelny proposed a change to openstack/oslo.db: Implementation of Alembic as migration engine https://review.openstack.org/99965 | 13:31 |
| openstackgerrit | Ilya Pekelny proposed a change to openstack/oslo.db: Common alembic environment runner. https://review.openstack.org/112859 | 13:31 |
| zzzeek | “H703 Multiple positional placeholders" wtf is that ? | 13:35 |
| rpodolyaka | zzzeek: "%s something %s" % (a, b) | 13:36 |
| zzzeek | whats wrong with that ?!?!!? | 13:36 |
| rpodolyaka | "%(a)s something %(b)s" % {'a': 1, 'b': 2} is preferred :) | 13:36 |
| zzzeek | how are we making flake8 do all these extra things and where is that coming from ? | 13:37 |
| zzzeek | is this “hacking”? how does that work? | 13:37 |
| rpodolyaka | yep, it's hacking, afaik | 13:37 |
| zzzeek | this is new right? H703 ? | 13:37 |
| rpodolyaka | it's something like an additional set of rules we check | 13:37 |
| rpodolyaka | yep, I think we've merged a new hacking version recently | 13:38 |
| rpodolyaka | and removed a few warning from the list of warning being ignored | 13:38 |
| *** abhishekk has quit IRC | 13:47 | |
| *** AAzza is now known as AAzza_afk | 13:49 | |
| *** jecarey has quit IRC | 13:49 | |
| *** russellb is now known as rustlebee | 13:50 | |
| zzzeek | this completely blows because we can no longer defer argument stringification for python logging | 13:51 |
| rpodolyaka | hmm, can't you pass a mapping as an argument? | 13:52 |
| zzzeek | ah, single dictioanry, OK | 13:52 |
| zzzeek | thx | 13:52 |
| rpodolyaka | np | 13:53 |
| openstackgerrit | Michael Bayer proposed a change to openstack/oslo.db: Reorganize DbTestCase to use provisioning completely https://review.openstack.org/110170 | 13:55 |
| openstackgerrit | Michael Bayer proposed a change to openstack/oslo.db: Use testr instance provisioning to lazily create databases https://review.openstack.org/110486 | 13:55 |
| *** AAzza_afk is now known as AAzza | 14:00 | |
| zzzeek | rpodolyaka: i have a completely different organization in provision.py now, shoudl be easier to follow | 14:01 |
| rpodolyaka | zzzeek: ok, cool! | 14:04 |
| zzzeek | huh wonder how this: logging_name="%s@%s" % (self.drivername, ident) is making it through. bet it fails | 14:08 |
| *** tsekiyama has joined #openstack-oslo | 14:10 | |
| openstackgerrit | Vlad Okhrimenko proposed a change to openstack/oslo.db: Create class TestInnoDB https://review.openstack.org/108994 | 14:18 |
| linkid | dhellmann: do I have to use a tag to specify that my bug concerns oslo-incubator ? | 14:18 |
| dhellmann | linkid: no, we assume the incubator by default | 14:18 |
| linkid | ok, thanks | 14:19 |
| *** mriedem is now known as SteelyDan | 14:24 | |
| *** SteelyDan is now known as mriedem | 14:25 | |
| bnemec | zzzeek: The logging_name line is fine because it isn't translated. | 14:25 |
| zzzeek | oh hacking is only looking inside of _LI() ? OK | 14:26 |
| bnemec | The multiple positional arguments is disallowed because it can break translations if the arguments are swapped during translation. | 14:26 |
| *** celttechie has joined #openstack-oslo | 14:26 | |
| bnemec | zzzeek: Yep | 14:26 |
| *** stevemar has joined #openstack-oslo | 14:28 | |
| *** k4n0 has quit IRC | 14:39 | |
| *** i159 has quit IRC | 14:41 | |
| zzzeek | bnemec: that’s a great reason | 14:41 |
| *** celttechie has quit IRC | 14:51 | |
| *** ildikov has quit IRC | 14:56 | |
| zzzeek | woop big green checkmark! | 14:57 |
| *** jecarey has joined #openstack-oslo | 14:57 | |
| *** markmcclain has joined #openstack-oslo | 14:58 | |
| dhellmann | team meeting starting in a couple of minutes in #openstack-meeting-alt | 14:59 |
| rpodolyaka | isn't it starting in one hour? or I've missed something? :) | 15:00 |
| dhellmann | is my clock messed up? | 15:01 |
| dhellmann | thanks, rpodolyaka, you're right -- back in ~1 hr | 15:03 |
| rpodolyaka | heh :) | 15:03 |
| dhellmann | apparently I need more human interaction :-) | 15:03 |
| rpodolyaka | :) | 15:03 |
| openstackgerrit | Vlad Okhrimenko proposed a change to openstack/oslo.db: Create get_non_innodb_tables in utils. https://review.openstack.org/108994 | 15:04 |
| *** Alexei_987 has quit IRC | 15:05 | |
| openstackgerrit | Doug Hellmann proposed a change to openstack/oslo-specs: clean up toctree now that we are publishing https://review.openstack.org/112317 | 15:06 |
| *** celttechie has joined #openstack-oslo | 15:11 | |
| openstackgerrit | Roman Podoliaka proposed a change to openstack/oslo.db: Add get_non_innodb_tables() to utils https://review.openstack.org/108994 | 15:13 |
| rpodolyaka | oh, gerrit doesn't automatically wrap lines if you edit the commit message via the web interface... | 15:14 |
| openstackgerrit | Roman Podoliaka proposed a change to openstack/oslo.db: Add get_non_innodb_tables() to utils https://review.openstack.org/108994 | 15:17 |
| openstackgerrit | Michael Bayer proposed a change to openstack/oslo.db: Use testr instance provisioning to lazily create databases https://review.openstack.org/110486 | 15:22 |
| rpodolyaka | can I get another core review for two very simple patches - https://review.openstack.org/#/c/112730/1 and https://review.openstack.org/#/c/106992/ ? :) | 15:22 |
| *** linkid has left #openstack-oslo | 15:23 | |
| morganfainberg | dhellmann, does this sound to you like something wonky going on in oslo.config: https://bugs.launchpad.net/keystonemiddleware/+bug/1354269 ? it seems like they are getting a non-int back for an IntType option (I haven't confirmed this is really the case yet) | 15:25 |
| morganfainberg | dhellmann, but it *looks* like it. | 15:26 |
| openstackgerrit | Michael Bayer proposed a change to openstack/oslo.db: Use testr instance provisioning to lazily create databases https://review.openstack.org/110486 | 15:26 |
| boris-42 | dhellmann hi | 15:28 |
| boris-42 | dhellmann As far we merged in glacne osprofiler seems like bumping version of osprofiler is essential https://review.openstack.org/#/c/104691/ | 15:28 |
| boris-42 | dhellmann somebody already faced issue https://review.openstack.org/#/c/112925/ | 15:28 |
| bknudson | morganfainberg: that may not be going through oslo.config | 15:30 |
| bknudson | if it's reading from the api-paste.ini | 15:30 |
| morganfainberg | bknudson, the options should be loaded from the conf object | 15:31 |
| dhellmann | the middleware options don't come from the paste config object? | 15:31 |
| morganfainberg | dhellmann, but the *options* are defined in oslo manner? | 15:31 |
| morganfainberg | is that just... for show? | 15:32 |
| dhellmann | morganfainberg: I don't know how paste and oslo.config interact here | 15:32 |
| dhellmann | maybe that was just for generating docs? | 15:32 |
| bknudson | here's _conf_get: http://git.openstack.org/cgit/openstack/keystonemiddleware/tree/keystonemiddleware/auth_token.py#n540 | 15:32 |
| morganfainberg | maybe | 15:32 |
| bknudson | self._conf is from the conf that's passed in | 15:32 |
| bknudson | which I assume is a dict of strings | 15:33 |
| morganfainberg | so.. the options defined at the top are *only* for doc generation really (and/or defaults)? | 15:33 |
| dhellmann | yeah, and that would be the dict that paste sends | 15:33 |
| bknudson | it's not something I've looked into | 15:33 |
| bknudson | I've always wondered if there was something fishy there. | 15:33 |
| morganfainberg | that sounds like something we should push through the config options we define | 15:33 |
| dhellmann | morganfainberg: based on that source ^^ if the options are set in the keystone.conf file they will be converted properly | 15:33 |
| morganfainberg | dhellmann, right. | 15:34 |
| morganfainberg | dhellmann, well nova.conf in this case | 15:34 |
| dhellmann | ok, yeah | 15:34 |
| morganfainberg | dhellmann, we don't use auth_token middleware in keystone | 15:34 |
| dhellmann | right | 15:34 |
| morganfainberg | bknudson, i think this is all around fishy and needs to be solved (if possible) by getting the config info into the real config object not just whatever paste sends in | 15:35 |
| bknudson | morganfainberg: it would be cool if we could do that. | 15:35 |
| morganfainberg | bknudson, dhellmann, thanks for the eyes, i'll update the bug to reflect this information | 15:35 |
| dhellmann | I wonder if you could use setdefaults with the values that come from paste to inject them into the config, or if they need to have the right type | 15:35 |
| bknudson | I think there's already a fix proposed... not sure what approach was taken. | 15:36 |
| morganfainberg | bknudson, calling int() around conf.geT() | 15:36 |
| bknudson | classy. | 15:36 |
| morganfainberg | bknudson, it was also proposed to keystoneclient (so it got a -2 until it lands in keystonemiddleware, we may need to evalutate if we want that one in keystoneclient's version) | 15:37 |
| morganfainberg | bknudson, or whatever the fix ends up being | 15:37 |
| bknudson | it's not a security fix | 15:37 |
| morganfainberg | bknudson, ++ thats my view. | 15:37 |
| bknudson | and there's a simple workaround | 15:37 |
| morganfainberg | bknudson, but i am keeping an open mind :P someone may *REALY* need it. | 15:37 |
| morganfainberg | ok updated the bug with the info. thanks for the eyes on it :) | 15:40 |
| openstackgerrit | Michael Bayer proposed a change to openstack/oslo.db: Use testr instance provisioning to lazily create databases https://review.openstack.org/110486 | 15:40 |
| *** bknudson has left #openstack-oslo | 15:44 | |
| zzzeek | /j #openstack_meeting_alt | 15:45 |
| zzzeek | whups | 15:45 |
| *** Ish__ has joined #openstack-oslo | 15:45 | |
| * dhellmann has the sense that the meeting today is going to be a bit fraught, between TZ and IRC user errors :-) | 15:50 | |
| *** AAzza is now known as AAzza_afk | 15:52 | |
| *** bknudson has joined #openstack-oslo | 15:54 | |
| YorikSar | zzzeek: s/_/-/ in channel name ;) | 15:57 |
| *** i159 has joined #openstack-oslo | 15:57 | |
| *** bnemec is now known as beekneemech | 16:02 | |
| zzzeek | arg | 16:05 |
| *** jogo is now known as flashgordon | 16:26 | |
| *** SridharG has joined #openstack-oslo | 16:29 | |
| openstackgerrit | A change was merged to openstack/oslo.db: Restore correct source file encodings https://review.openstack.org/112730 | 16:33 |
| *** pcm_ has left #openstack-oslo | 16:56 | |
| *** pcm_ has joined #openstack-oslo | 16:56 | |
| pcm_ | Can anyone tell me if it is possible to override cfg.CONF.config_file in UTs, and if so how to do that? | 16:58 |
| *** i159 has quit IRC | 17:00 | |
| YorikSar | dhellmann: https://review.openstack.org/81798 - CR to oslo.rootwrap that adds daemon mode and drags in some eventlet handling. | 17:02 |
| *** pblaho has quit IRC | 17:02 | |
| YorikSar | dhellmann: Although I've added eventlet import to too common code that ends up imported from rootwrap daemon itself (and runs as root) and that should be fixed. | 17:03 |
| *** bknudson has quit IRC | 17:03 | |
| dhellmann | pcm_: in UTs? | 17:04 |
| pcm_ | dhellmann: yes | 17:04 |
| dhellmann | pcm_: what is "UTs"? | 17:04 |
| pcm_ | dhellmann: Unit Tests | 17:04 |
| dhellmann | pcm_: ah, yes, you can definitely do that: http://docs.openstack.org/developer/oslo.config/fixture.html | 17:05 |
| pcm_ | dhellmann: I have a vendor config file that is read at agent init time (Neutron) using cfg.CONF.config_file | 17:05 |
| pcm_ | dhellmann: I'd like to set a temp config_file and then use that in the UTs. | 17:07 |
| dhellmann | why is the test reading a file to get the configuration? | 17:07 |
| *** zzzeek has quit IRC | 17:09 | |
| *** zzzeek has joined #openstack-oslo | 17:09 | |
| pcm_ | The production code has the config in a vendor config file (this is an interim solution). File is read at agent startup to provide info for 1+ routers (username, password, port, IP, ...) | 17:10 |
| pcm_ | For UT, I want to create a dummy config file and then test the code that loads the configuration file. | 17:10 |
| openstackgerrit | A change was merged to openstack/oslo-specs: clean up toctree now that we are publishing https://review.openstack.org/112317 | 17:12 |
| pcm_ | For existing tests, I create temp file, and pass the path to the function that reads the config file. Production code passes in cfg.CONF.config_file | 17:13 |
| pcm_ | However, I'm creating a new method that will do several things and as part of that, call the function to read the config files for these settings. | 17:13 |
| YorikSar | beekneemech: It looks like Nova relies on file locking for image cache only | 17:14 |
| beekneemech | YorikSar: Right | 17:14 |
| YorikSar | beekneemech: And it happens just to allow image cache to be mounted from remote host | 17:14 |
| pcm_ | I could pass the filename to this new method, but I was wondering if I could override the cfg.CONF.config_file | 17:14 |
| pcm_ | dhellmann: Did I make sense? | 17:14 |
| YorikSar | beekneemech: So in the long run that can be replaced with tooz I suppose. | 17:14 |
| beekneemech | YorikSar: Yes, it would be good to get them away from that. I don't know that file locks are even guaranteed to work on all distributed filesystems. | 17:15 |
| YorikSar | beekneemech: They should work over very good old fiel sharing protocols. | 17:16 |
| dhellmann | pcm_: I'm not quite sure. It seems like you should have the config file passed to the application and let it read the files one time, and not have the vendor code reading new files at all. | 17:16 |
| YorikSar | beekneemech: But with morden new ones - not so much. | 17:16 |
| pcm_ | dhellmann: From the link... would I use the Config class's config() method to set config_file then? | 17:16 |
| dhellmann | pcm_: well, the point is I don't think you want to be changing that at all | 17:17 |
| dhellmann | pcm_: is the vendor code creating a new ConfigOpts instance? | 17:17 |
| pcm_ | dhellmann: It is passed, the app (the VPN agent), will load (dynamically) a device driver. The device driver uses the config_file config to find the config file and load the configuraiton. | 17:18 |
| YorikSar | dhellmann, beekneemech: Which brings me to another question. Can we change lockutils API during graduation? I'd like to make a very clear separation of file locks and semaphores. | 17:18 |
| pcm_ | dhellmann: no | 17:18 |
| pcm_ | dhellmann: https://github.com/openstack/neutron/blob/master/neutron/services/vpn/device_drivers/cisco_ipsec.py#L209 | 17:18 |
| YorikSar | E.g. add a flag "file_lock=True" that will force usage of file locks, and stuff like lock_path will be ignored otherwise. | 17:19 |
| pcm_ | dhellmann: This is existing code for the device driver. It was a proof of concept implementation to do VPN. | 17:19 |
| dhellmann | YorikSar: changing the API is going to need a spec so we can hash out the details. | 17:19 |
| dhellmann | pcm_: reading... | 17:19 |
| YorikSar | dhellmann: And we just agreed to no more specs for now, right. Ok, I'll shelve that thought. | 17:20 |
| dhellmann | YorikSar: well, "for now", so you could still write it up while the ideas are fresh in your mind | 17:20 |
| beekneemech | YorikSar: I really, really, really don't want to make changes to lockutils if we can possibly help it. | 17:21 |
| YorikSar | dhellmann: Yes, I will. Not today though... | 17:21 |
| beekneemech | Every time we do it causes havoc. | 17:21 |
| dhellmann | pcm_: cfg.CONF.config_file is going to be the main configuration file for the application, right? why not just use cfg.CONF directly? | 17:21 |
| pcm_ | dhellmann: Here are the UTs: https://github.com/openstack/neutron/blob/master/neutron/tests/unit/services/vpn/device_drivers/test_cisco_ipsec.py#L1539 | 17:21 |
| dhellmann | pcm_: oh, it *does* create a new config parser instance | 17:22 |
| *** jaosorior has quit IRC | 17:22 | |
| YorikSar | beekneemech: I don't want to change any implementation, just the API. And since it's going to graduate, we can easily switch to this modified API during adoption. | 17:23 |
| dhellmann | pcm_: https://github.com/openstack/neutron/blob/master/neutron/services/vpn/device_drivers/cisco_ipsec.py#L82 | 17:23 |
| beekneemech | YorikSar: I really, really, really don't want to change _anything_ :-) | 17:23 |
| dhellmann | YorikSar: that library is particularly brittle :-/ | 17:23 |
| pcm_ | dhellmann: Ah. Yes. | 17:23 |
| beekneemech | This is a deeply embedded piece of OpenStack and even little changes cause major issues. | 17:23 |
| pcm_ | dhellmann: So this is upstream. End goal is to instead of doing this. Read the router information dynamically via Neutron calls from the service driver (and pass down to device driver). | 17:24 |
| beekneemech | The problem is lockutils has had issues forever. Every attempt to fix them breaks something else. | 17:24 |
| dhellmann | pcm_: I don't understand this code at all. It takes the list of parsed config files out of one config parser, re-parses them, then scans them to find the settings it wants? | 17:25 |
| YorikSar | While fighting with tests I just stumbled upon the fact that even for POSIX locks we add /tmp/somerandomstring/anotherrandomstring/ prefix although it's not required. | 17:25 |
| dhellmann | pcm_: what I can't figure out is why it's not just looking at cfg.CONF directly | 17:25 |
| beekneemech | If we can fix this posix locks bug we will be as bug-free as lockutils has ever been. | 17:25 |
| dhellmann | pcm_: maybe it has something to do with the MultiConfigParser acting differently | 17:25 |
| pcm_ | dhellmann: Partly, if we wanted to update that config file and then get the new info (by restarting the agent). | 17:26 |
| beekneemech | YorikSar: Also, just a note in general, we don't want to alter API's during graduation. | 17:26 |
| beekneemech | YorikSar: API changes should happen in incubator. That's what it's there for. | 17:26 |
| dhellmann | pcm_: but the exact same config file would have been parsed to give you the list of config files in cfg.CONF.config_file | 17:27 |
| YorikSar | beekneemech: Well... Maybe I'll be able to add some functional tests there that would ensure its stability :) | 17:27 |
| beekneemech | YorikSar: Only for the cases you know to test for. There will be others, trust me. :-) | 17:27 |
| pcm_ | dhellmann: right. I mean change the config file contents. | 17:27 |
| dhellmann | YorikSar: if the only issue is that we're making ugly names for locks, then let's just leave it alone | 17:27 |
| beekneemech | dhellmann: Posix locks may be going away anyway. | 17:28 |
| dhellmann | pcm_: when is find_available_csrs_from_config() called? | 17:28 |
| beekneemech | dhellmann: They're completely broken right now if a service gets killed in a way that doesn't allow cleanup. | 17:28 |
| dhellmann | pcm_: it looks like it is only called when the driver is initialized | 17:28 |
| pcm_ | in the existing code, when the device driver is loaded at startup. | 17:28 |
| dhellmann | beekneemech: ok, all the more reason, then | 17:28 |
| YorikSar | beekneemech: Yeah, that's fair. I guess I can suggest to not see adding lock_path to POSIX/SysV locks' names as a part of API :) (j/k) | 17:29 |
| dhellmann | pcm_: right, so at startup, after parsing the configuration files, the driver is loaded and then it asks the *object that has the parsed configuration data* to reparse the configuration files | 17:29 |
| YorikSar | beekneemech: The same stands for SysV locks - name generation part is the same. | 17:29 |
| pcm_ | dhellmann: So what you're saying is that when we restart the agent, it would reread and we could use the cfg.CONF | 17:29 |
| beekneemech | YorikSar: That's not part of the lock api, it's part of the lockutils API. | 17:29 |
| dhellmann | pcm_: I think so, yes. Unless there is something special about the fact that the MultiConfigParser is being used -- that class doesn't have a docstring, and I don't know what it does that is special | 17:30 |
| beekneemech | YorikSar: (or it shouldn't be anyway) | 17:30 |
| *** markmcclain has quit IRC | 17:30 | |
| pcm_ | dhellmann: So...what we're working towards is no config file at all, and obtain the info from Neutron calls, most importantly, in a dynamic basis (the existing is static) | 17:31 |
| dhellmann | pcm_: ok, that's fine | 17:31 |
| YorikSar | beekneemech: Hm... Not sure I understand last 2 messages. The fact that CONF.lock_path is added to POSIX lock names _is_ part of the lockutils API, right? | 17:31 |
| pcm_ | dhellmann: However, that has a dependency on another BP that is currently struggling. So, I'm thinking how to mitigate the risk and what work-arounds I could do. | 17:32 |
| beekneemech | YorikSar: Right, lockutils. Not the lock. consumers use lockutils, lockutils uses locks. | 17:32 |
| dhellmann | pcm_: for your tests, you should be able to replace the value of config_file with a new list using the fixture's config() method -- let me know if that doesn't work | 17:33 |
| pcm_ | dhellmann: One thought (kick me if it doesn't make sense), would be to try to read the config file dynamically, when a new device is needed. | 17:33 |
| pcm_ | dhellmann: To do that, I need to be able to re-read the config file. Which comes around to the question on how to do that in the UTs. | 17:34 |
| dhellmann | pcm_: I don't think you want to treat the configuration file like a database. | 17:34 |
| YorikSar | beekneemech: Yeah. I wonder if anyone actually relies on having that prefix... | 17:34 |
| pcm_ | dhellmann: Yeah I don't, really. Just looking for ways to remove the dependency. | 17:34 |
| dhellmann | pcm_: well, if you design an API that takes a ConfigParser as an argument, then your tests can pass in a new one with the values you need pre-configured | 17:35 |
| pcm_ | dhellmann: Thought was could read the file, then, when the other blueprint is upstreamed, pull that code and use the Neutron calls (idea is to use the same call) | 17:35 |
| dhellmann | pcm_: that's better than looking at the global one or reading files | 17:35 |
| dhellmann | pcm_: yeah, using the neutron calls when they are available makes sense, too | 17:36 |
| beekneemech | YorikSar: Oh, that's just because we're using the file lock path as the name for posix locks. | 17:36 |
| pcm_ | dhellmann: So I'm thinking make a method with same API as the neutron call, but have it read a config file instead (until the real method is avail) | 17:37 |
| pcm_ | s/API/signature/ | 17:37 |
| dhellmann | pcm_: sure, that makes sense | 17:37 |
| beekneemech | YorikSar: Basically that's required because InterProcessLock on Windows is still a file lock: https://github.com/openstack/oslo-incubator/blob/master/openstack/common/lockutils.py#L191 | 17:37 |
| pcm_ | dhellmann: Interested in your comment on the ConfigParser argument. Any examples I could look at to better understand (if you think that could be used here)? | 17:38 |
| YorikSar | beekneemech: Yes, but... Unless you need to lock on the specific file (like with imagecache), you don't need that prefix, and so that config variable looses its value... | 17:38 |
| YorikSar | beekneemech: Oh, right, Windows... I wonder if there's a lib that wraps Windows semaphores... | 17:39 |
| alexpilotti | YorikSar: posix_ipc to some extent does | 17:39 |
| YorikSar | alexpilotti: POSIX ones aren't good. | 17:41 |
| YorikSar | alexpilotti: I'd expect their behavior on Windows to be the same as on Linux. | 17:41 |
| alexpilotti | YorikSar: posix_ipc wrapos on teh WIn32 ones | 17:41 |
| alexpilotti | YorikSar: for the rest we usually use ctypes on WIn32 | 17:42 |
| dhellmann | pcm_: oslo.messaging has the caller pass the config in, instead of looking at the global: http://git.openstack.org/cgit/openstack/oslo.messaging/tree/oslo/messaging/transport.py#n137 | 17:43 |
| dhellmann | YorikSar: I would rather have the lock stuff be a little ugly than broken | 17:44 |
| YorikSar | alexpilotti: I grep for CreateSemaphore and find nothing. | 17:44 |
| alexpilotti | YorikSar: I have unfortunately to run now, but if you could send me an email with your requirements I’d happily work on a patch | 17:44 |
| pcm_ | dhellmann: thanks. | 17:44 |
| alexpilotti | YorikSar: a pointer to teh corresponding Linux code would be enough | 17:45 |
| YorikSar | dhellmann: Yeah, sure. I'm not proposing to change API anymore :) | 17:45 |
| alexpilotti | YorikSar: we had quite some portability issues in the past due to this in the Hyper-V CI, so if can help in fixing it it’d be great :-) | 17:45 |
| YorikSar | alexpilotti: The main issue with POSIX semaphores that has been uncovered recently - they're not being released if the process is brutally killed (say kill -9). | 17:46 |
| alexpilotti | YorikSar: you mean on WIndows or in general? | 17:46 |
| dhellmann | YorikSar: ok :-) | 17:46 |
| YorikSar | alexpilotti: In general. So if Windows implements POSIX API, it should be the same on Windows. | 17:47 |
| YorikSar | alexpilotti: I don't know if that's the case for CreateSemaphore API. If it's not, we might be able to replace (most of) file locks on Windows with semaphores as well. | 17:48 |
| pcm_ | dhellmann: Thanks for the pointers! | 17:49 |
| dhellmann | pcm_: sure thing! | 17:50 |
| alexpilotti | YorikSar: no, WIndows dows not implement POSIX API (athough there an old layer, but that’s a different story) | 17:52 |
| alexpilotti | YorikSar: posix_ipc is wrapping WIn32 APIs in Posix ones, so that they have the same signature and (similar) semantyc | 17:53 |
| YorikSar | alexpilotti: Somehow I can't find references to Win32 API then... | 17:53 |
| alexpilotti | YorikSar: if you could just point me to the Linux equivalent I could probably help on this | 17:54 |
| alexpilotti | YorikSar: I have to run now, sorry! | 17:55 |
| YorikSar | alexpilotti: See you around | 17:55 |
| YorikSar | alexpilotti: It's sem_get and others. | 17:55 |
| *** alexpilotti has quit IRC | 17:55 | |
| amrith | YorikSar, at ur convenience, would you please re-review https://review.openstack.org/#/c/110933/ | 17:57 |
| amrith | also YorikSar, I got to the bottom of the trove issue with communicate | 17:57 |
| amrith | between this change, and another that I proposed for trove, I think we have 2/3 of the problem worked out. | 17:57 |
| amrith | YorikSar ^^ | 17:57 |
| YorikSar | amrith: Great! I'll take another look at the change today, but later. | 17:58 |
| amrith | thanks! have a good weekend YorikSar | 17:59 |
| YorikSar | amrith: You too :) | 17:59 |
| openstackgerrit | Ben Nemec proposed a change to openstack/oslo.serialization: Migrate to oslo.i18n https://review.openstack.org/112990 | 18:15 |
| openstackgerrit | Ben Nemec proposed a change to openstack/oslo.serialization: Migrate to oslo.utils https://review.openstack.org/112991 | 18:15 |
| *** jaosorior has joined #openstack-oslo | 18:26 | |
| openstackgerrit | Michael Bayer proposed a change to openstack/oslo.db: Use dialect dispatch for engine initiailization. https://review.openstack.org/110446 | 18:29 |
| *** SridharG has quit IRC | 18:33 | |
| openstackgerrit | Michael Bayer proposed a change to openstack/oslo.db: Use dialect dispatch for engine initiailization. https://review.openstack.org/110446 | 18:33 |
| *** stevemar has quit IRC | 18:43 | |
| *** stevemar has joined #openstack-oslo | 18:44 | |
| openstackgerrit | Michael Bayer proposed a change to openstack/oslo.db: Use testr instance provisioning to lazily create databases https://review.openstack.org/110486 | 18:51 |
| *** ildikov has joined #openstack-oslo | 19:28 | |
| *** Ish__ has quit IRC | 19:33 | |
| openstackgerrit | A change was merged to openstack/oslo-incubator: Use oslosphinx to generate documentation https://review.openstack.org/111910 | 19:38 |
| *** ildikov has quit IRC | 19:41 | |
| *** morganfainberg has quit IRC | 19:43 | |
| *** morganfainberg has joined #openstack-oslo | 19:43 | |
| *** Ish__ has joined #openstack-oslo | 19:43 | |
| *** openstackgerrit has quit IRC | 20:20 | |
| *** markmcclain has joined #openstack-oslo | 20:25 | |
| *** stevemar has quit IRC | 20:55 | |
| *** tsekiyam_ has joined #openstack-oslo | 21:03 | |
| *** gordc has quit IRC | 21:07 | |
| *** tsekiyama has quit IRC | 21:07 | |
| *** tsekiyam_ has quit IRC | 21:08 | |
| YorikSar | zzzeek: Ping | 21:16 |
| zzzeek | hi | 21:16 |
| YorikSar | zzzeek: Just watched your talk at PyCon Canada. I finally understand how sessions work and why they issue statements (mostly) only at commit()/flush() time! | 21:17 |
| zzzeek | great | 21:17 |
| YorikSar | zzzeek: You're awesome presenter :) | 21:17 |
| zzzeek | thanks! | 21:17 |
| YorikSar | zzzeek: Oh, crap... I wanted to ask smth, but forgot what exactly... Let me scroll through the video... | 21:18 |
| YorikSar | zzzeek: Here it is. Does SQLAlchemy use tricky constructs like UPDATE IF EXISTS or INSERT ON DUPLICATE UPDATE? | 21:21 |
| zzzeek | no | 21:21 |
| YorikSar | I wonder if you can lower amount of locking from the client side using those... | 21:21 |
| YorikSar | (locking on the server side) | 21:22 |
| zzzeek | those constructs are not available on most databases and have very complex behavioral patterns | 21:23 |
| YorikSar | zzzeek: Right. So for Session to use them, it needs to know too much about backend... | 21:23 |
| zzzeek | it would also generate tons of surprises for users as they would behave differently | 21:24 |
| YorikSar | zzzeek: Yeah... Ok, my dream wizard-like Session vanishes :) | 21:25 |
| zzzeek | YorikSar: if every DB had a somewhat similar MERGE type of command then maybe it woudl be easier to look into but for now it woudl be all MySQL only | 21:25 |
| zzzeek | YorikSar: and very complicated | 21:25 |
| zzzeek | YorikSar: there’s features coming down which will allow more granular INSERT patterns and maybe that would eventually be how these constructs might work at the ORM level | 21:26 |
| YorikSar | zzzeek: Don't Posgres have smth like that? | 21:26 |
| zzzeek | it does not | 21:26 |
| zzzeek | it doesn’t have MERGE or any equivalent | 21:27 |
| zzzeek | here’s the latest: https://wiki.postgresql.org/wiki/SQL_MERGE | 21:27 |
| zzzeek | http://www.depesz.com/2012/06/10/why-is-upsert-so-complicated/ | 21:28 |
| YorikSar | zzzeek: Oh, I see another talk I should take a look at - "Why upsert is weird" | 21:28 |
| YorikSar | zzzeek: And another question while I didn't forget. Does Session provide consistency across different DB transactions in one Session transaction? | 21:29 |
| zzzeek | it can work with multiple DB transactions simultaneously, but “consistency” is ambiguous in that context | 21:30 |
| zzzeek | it can provide atomicity with them by using two phase commit | 21:30 |
| YorikSar | zzzeek: Yeah, that what I meant. | 21:30 |
| zzzeek | that is built in, to the degree that drivers support 2pc | 21:30 |
| zzzeek | which, is very spotty :) | 21:30 |
| YorikSar | zzzeek: Consistency can be provided by client only in that case. | 21:30 |
| zzzeek | posgresql and mysql only, really | 21:31 |
| zzzeek | very limited support on oracle, sql server we haven’t been able to get it to work with most drivers | 21:31 |
| zzzeek | I’d not really use it. 2pc is scary | 21:31 |
| YorikSar | And it will work even for 2 different drivers at once? | 21:31 |
| zzzeek | not with 2pc, no | 21:31 |
| zzzeek | well | 21:31 |
| zzzeek | ok yes it will, sorry | 21:31 |
| zzzeek | these are not patterns anyone really does in real life though | 21:32 |
| YorikSar | Why is it scary? Isn't it just... like double commit? | 21:32 |
| zzzeek | the transaction and its locks live beyond the scope of the connection | 21:33 |
| zzzeek | your app can die totally and the transaction stays, the locks, your whole DB is deadlocked | 21:33 |
| YorikSar | Oh... Yeah... | 21:33 |
| zzzeek | until someone goes in, selects all the open 2pc’s from the correct system view, and rolls them back | 21:33 |
| zzzeek | so very bad deadlocks become a possibility when 2pc is used | 21:33 |
| zzzeek | if the application fails badly, that is | 21:34 |
| YorikSar | Ok, I guess as any distributed lock (which it basically is) it's really hard to get guarantees... | 21:35 |
| YorikSar | *as _with_ any | 21:35 |
| YorikSar | Well, thanks a lot for enlightenment! I'll go read some more on MERGE and upserts. | 21:37 |
| *** jecarey has quit IRC | 21:38 | |
| zzzeek | good luck | 21:38 |
| *** celttechie has quit IRC | 22:15 | |
| *** stevemar has joined #openstack-oslo | 22:25 | |
| *** openstackgerrit has joined #openstack-oslo | 22:38 | |
| openstackgerrit | A change was merged to openstack/oslo-specs: pylockfile adoption https://review.openstack.org/102202 | 23:02 |
| openstackgerrit | Doug Hellmann proposed a change to openstack/oslo-specs: fix the name of the pylockfile spec https://review.openstack.org/113043 | 23:03 |
| openstackgerrit | A change was merged to openstack/oslo-specs: fix the name of the pylockfile spec https://review.openstack.org/113043 | 23:05 |
| *** f13o_ has joined #openstack-oslo | 23:08 | |
| *** jokke_ has quit IRC | 23:13 | |
| *** stevemar has quit IRC | 23:40 | |
| *** jaosorior has quit IRC | 23:42 | |
| *** f13o_ has quit IRC | 23:44 | |
| *** markmcclain has quit IRC | 23:46 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!