| *** tosky has quit IRC | 00:12 | |
| *** threestrands has joined #openstack-oslo | 00:40 | |
| *** zhiguo has joined #openstack-oslo | 00:41 | |
| zhiguo | Hello, dose someone face this case: https://bugs.launchpad.net/oslo.messaging/+bug/1781876 | 00:43 |
|---|---|---|
| openstack | Launchpad bug 1781876 in oslo.messaging "An strange error in oslo.messaging" [Undecided,New] | 00:43 |
| *** dave-mccowan has joined #openstack-oslo | 00:55 | |
| *** dave-mccowan has quit IRC | 01:09 | |
| *** harlowja has quit IRC | 01:13 | |
| *** dave-mccowan has joined #openstack-oslo | 01:17 | |
| *** dave-mcc_ has joined #openstack-oslo | 01:21 | |
| *** dave-mccowan has quit IRC | 01:24 | |
| *** mriedem has quit IRC | 01:24 | |
| *** larainema has quit IRC | 01:46 | |
| *** larainema has joined #openstack-oslo | 02:22 | |
| *** gcb has joined #openstack-oslo | 02:22 | |
| openstackgerrit | Merged openstack/mox3 master: Switch to stestr https://review.openstack.org/581302 | 02:32 |
| openstackgerrit | Merged openstack/oslo.utils master: Handle non-string keys appropriately https://review.openstack.org/583304 | 03:43 |
| openstackgerrit | Merged openstack/taskflow master: Update various links in docs https://review.openstack.org/567181 | 03:48 |
| *** lpetrut has joined #openstack-oslo | 03:51 | |
| openstackgerrit | Merged openstack/debtcollector master: Switch to stestr https://review.openstack.org/580915 | 03:55 |
| openstackgerrit | Ha Manh Dong proposed openstack/oslo.config master: Switch to use stestr for unit test https://review.openstack.org/583453 | 04:22 |
| *** lpetrut has quit IRC | 04:39 | |
| *** lpetrut has joined #openstack-oslo | 04:42 | |
| *** lpetrut has quit IRC | 04:54 | |
| *** chhagarw has joined #openstack-oslo | 04:59 | |
| *** openstackgerrit has quit IRC | 05:10 | |
| *** lucasagomes_ has joined #openstack-oslo | 05:12 | |
| *** links has joined #openstack-oslo | 05:15 | |
| *** efried has quit IRC | 05:16 | |
| *** lxkong has quit IRC | 05:16 | |
| *** aspiers has quit IRC | 05:16 | |
| *** mwhahaha has quit IRC | 05:16 | |
| *** johnsom has quit IRC | 05:16 | |
| *** NobodyCam has quit IRC | 05:16 | |
| *** DuncanT has quit IRC | 05:16 | |
| *** lucasagomes has quit IRC | 05:16 | |
| *** sambetts_ has quit IRC | 05:18 | |
| *** toabctl has quit IRC | 05:18 | |
| *** toabctl has joined #openstack-oslo | 05:21 | |
| *** sambetts_ has joined #openstack-oslo | 05:23 | |
| *** zhiguo has quit IRC | 05:27 | |
| *** zhiguo has joined #openstack-oslo | 05:28 | |
| *** efried has joined #openstack-oslo | 05:35 | |
| *** lxkong has joined #openstack-oslo | 05:35 | |
| *** aspiers has joined #openstack-oslo | 05:35 | |
| *** mwhahaha has joined #openstack-oslo | 05:35 | |
| *** johnsom has joined #openstack-oslo | 05:35 | |
| *** NobodyCam has joined #openstack-oslo | 05:35 | |
| *** DuncanT has joined #openstack-oslo | 05:35 | |
| *** njohnston_ has joined #openstack-oslo | 05:37 | |
| *** njohnston has quit IRC | 05:38 | |
| *** ianychoi has joined #openstack-oslo | 05:43 | |
| *** njohnston has joined #openstack-oslo | 05:57 | |
| *** njohnston_ has quit IRC | 05:58 | |
| *** lpetrut has joined #openstack-oslo | 06:22 | |
| *** pcaruana has joined #openstack-oslo | 06:35 | |
| *** peereb has joined #openstack-oslo | 07:09 | |
| *** shardy has joined #openstack-oslo | 07:12 | |
| *** tesseract has joined #openstack-oslo | 07:18 | |
| *** rcernin has quit IRC | 07:28 | |
| *** AlexeyAbashkin has joined #openstack-oslo | 07:45 | |
| *** tobasco_away is now known as tobasco | 07:54 | |
| *** mattgo has joined #openstack-oslo | 08:05 | |
| *** tosky has joined #openstack-oslo | 08:06 | |
| *** e0ne has joined #openstack-oslo | 08:08 | |
| *** lucasagomes_ is now known as lucasagomes | 08:15 | |
| *** threestrands has quit IRC | 08:51 | |
| *** pbourke has quit IRC | 08:52 | |
| *** pbourke has joined #openstack-oslo | 08:52 | |
| *** openstackgerrit has joined #openstack-oslo | 09:05 | |
| openstackgerrit | Stephen Finucane proposed openstack/oslo.config master: sphinxext: Stop emitting lines multiple times https://review.openstack.org/583523 | 09:05 |
| openstackgerrit | Stephen Finucane proposed openstack/oslo.config master: sphinxext: Print output to logs, not to file https://review.openstack.org/583524 | 09:05 |
| openstackgerrit | Stephen Finucane proposed openstack/oslo.config master: sphinxext: Resolve FIXME https://review.openstack.org/583525 | 09:05 |
| stephenfin | dhellmann, mriedem: That was my bad. I've cleaned up the whole broken feature starting here https://review.openstack.org/583523 | 09:06 |
| stephenfin | dhellmann: I really need to sit down and write _proper_ Sphinx tests for that. Alas, I think doing so would involve bringing in pytest which I imagine mtreinish, mordred et al are loathe to do | 09:07 |
| openstackgerrit | Stephen Finucane proposed openstack-dev/pbr master: trivial: Increase integration pip timeout https://review.openstack.org/583178 | 09:10 |
| openstackgerrit | Stephen Finucane proposed openstack-dev/pbr master: trivial: Remove 'tools/releasenotes_tox.sh' https://review.openstack.org/580414 | 09:10 |
| openstackgerrit | Stephen Finucane proposed openstack-dev/pbr master: trivial: Fix file permissions https://review.openstack.org/580415 | 09:10 |
| openstackgerrit | Stephen Finucane proposed openstack-dev/pbr master: Switch to stestr https://review.openstack.org/580413 | 09:10 |
| openstackgerrit | Stephen Finucane proposed openstack-dev/pbr master: builddoc: Remove support for Sphinx < 1.6 https://review.openstack.org/580416 | 09:10 |
| openstackgerrit | Stephen Finucane proposed openstack-dev/pbr master: packaging: Remove support for pyN requirement files https://review.openstack.org/580417 | 09:10 |
| openstackgerrit | Stephen Finucane proposed openstack-dev/pbr master: doc: Fix formatting of packagers guide https://review.openstack.org/580418 | 09:10 |
| openstackgerrit | Stephen Finucane proposed openstack-dev/pbr master: doc: Add documentation for missing '[pbr]' options https://review.openstack.org/580419 | 09:10 |
| openstackgerrit | Stephen Finucane proposed openstack-dev/pbr master: Add Sphinx extension https://review.openstack.org/580420 | 09:10 |
| openstackgerrit | Stephen Finucane proposed openstack-dev/pbr master: Deprecate 'build_sphinx' integration https://review.openstack.org/580421 | 09:10 |
| openstackgerrit | Stephen Finucane proposed openstack-dev/pbr master: Deprecate 'test' integration https://review.openstack.org/580422 | 09:10 |
| openstackgerrit | Stephen Finucane proposed openstack-dev/pbr master: trivial: Remove 'tools/releasenotes_tox.sh' https://review.openstack.org/580414 | 09:15 |
| openstackgerrit | Stephen Finucane proposed openstack-dev/pbr master: trivial: Fix file permissions https://review.openstack.org/580415 | 09:15 |
| openstackgerrit | Stephen Finucane proposed openstack-dev/pbr master: Switch to stestr https://review.openstack.org/580413 | 09:15 |
| openstackgerrit | Stephen Finucane proposed openstack-dev/pbr master: builddoc: Remove support for Sphinx < 1.6 https://review.openstack.org/580416 | 09:15 |
| openstackgerrit | Stephen Finucane proposed openstack-dev/pbr master: packaging: Remove support for pyN requirement files https://review.openstack.org/580417 | 09:15 |
| openstackgerrit | Stephen Finucane proposed openstack-dev/pbr master: doc: Fix formatting of packagers guide https://review.openstack.org/580418 | 09:15 |
| openstackgerrit | Stephen Finucane proposed openstack-dev/pbr master: doc: Add documentation for missing '[pbr]' options https://review.openstack.org/580419 | 09:15 |
| openstackgerrit | Stephen Finucane proposed openstack-dev/pbr master: Add Sphinx extension https://review.openstack.org/580420 | 09:15 |
| openstackgerrit | Stephen Finucane proposed openstack-dev/pbr master: Deprecate 'build_sphinx' integration https://review.openstack.org/580421 | 09:15 |
| openstackgerrit | Stephen Finucane proposed openstack-dev/pbr master: Deprecate 'test' integration https://review.openstack.org/580422 | 09:15 |
| *** moguimar has joined #openstack-oslo | 09:21 | |
| *** zhiguo has quit IRC | 09:25 | |
| *** sambetts_ is now known as sambetts | 09:30 | |
| *** hoonetorg has quit IRC | 09:34 | |
| *** hoonetorg has joined #openstack-oslo | 09:48 | |
| *** dmellado has quit IRC | 10:36 | |
| *** dmellado has joined #openstack-oslo | 10:42 | |
| *** chhagarw has quit IRC | 11:10 | |
| *** raildo has joined #openstack-oslo | 11:53 | |
| openstackgerrit | Vipin Balachandran proposed openstack/oslo.vmware master: Utility method to check SDRS enabled status https://review.openstack.org/583556 | 12:01 |
| *** mattgo has quit IRC | 12:02 | |
| *** mattgo has joined #openstack-oslo | 12:04 | |
| *** edmondsw has joined #openstack-oslo | 12:11 | |
| mordred | stephenfin: I don't know why writing proper sphinx tests would necessitate pytest | 12:23 |
| mordred | stephenfin: unless upstream sphinx has some test fixtures written that only work with pytest, in which case I'll just be sad and judgemental | 12:24 |
| mordred | stephenfin: but as long as our tests are still _run_ by stestr, I don't think bringing in pytest via a transitive dependency on sphinx's test suite (if that's the issue) would be an issue | 12:26 |
| *** lbragstad has quit IRC | 13:19 | |
| dhellmann | stephenfin : why would adding tests require pytest? | 13:25 |
| stephenfin | dhellmann, mordred: Yeah, they've written their test framework with pytest | 13:26 |
| stephenfin | So we can reinvent the wheel or use their fixtures and drag in pytest | 13:26 |
| stephenfin | mordred: You know full well that as soon as pytest is let in the door, people will use it. Just be prepared :) | 13:27 |
| dhellmann | I wish we had just switched to pytest from testr to begin with, but whatever | 13:27 |
| stephenfin | dhellmann: yeah, so do I :( | 13:28 |
| dhellmann | I wonder if stestr can be made to load pytest test suites | 13:28 |
| dhellmann | I guess alternatively we could segregate the 2 types of tests into their own directories so we can use different tools to run each | 13:29 |
| mordred | if pytest wasn't broken, we would have | 13:29 |
| mordred | but pytest is broken in the same way nose is broken | 13:29 |
| dhellmann | which is? | 13:29 |
| stephenfin | nose is dead, testr is dead, pytest is supported by PyPA, PyQA, PyCA etc. | 13:29 |
| stephenfin | mordred: Oh? | 13:29 |
| mordred | it completely violates the python testing protocol, and it conflates test runner with test support library | 13:29 |
| mordred | also, it's impossible to folow what's going on - but that's just a personal preference | 13:30 |
| mordred | the python unittest protocol allows people to use whatever test runner they like - and our tests current allow that, because we ahere to it | 13:30 |
| stephenfin | mtreinish: Yup. I was trying to recall what you'd brought up in the stestr session and that was it | 13:30 |
| dhellmann | ah | 13:31 |
| stephenfin | Whoops, *mordred: | 13:31 |
| *** gcb has quit IRC | 13:31 | |
| mordred | we use stestr/testr because they have the ability to support streaming test results in a multi-threaded/multi-process manner | 13:31 |
| mordred | NOW - at some point someone did some initial work on adding subunit output support to pytest, but I think nobody ever really cared | 13:31 |
| stephenfin | The dependency injection (?) pattern they use is odd too, I'll admit, but the stack inspection they do to allow you to use plain old asserts is awesome | 13:31 |
| mordred | you can use plain asserts with our current code | 13:32 |
| stephenfin | mordred: This is something more than pytest-xdist would do, I assume? | 13:32 |
| mordred | stephenfin: yes. subunit is a streaming test result format | 13:32 |
| * stephenfin has little to no understanding of what subunit does for us | 13:32 | |
| mordred | pytest-xdist is a pytest plugin that does what testr does - namely spawn subprocesses | 13:32 |
| stephenfin | and there's my answer, heh | 13:33 |
| mordred | subunit allows for streaming interleaved output from multiple concurrent tests | 13:33 |
| stephenfin | mordred: You can use plain asserts but the assert exceptions have no context, right? | 13:33 |
| stephenfin | hence the hacking checks to ensure assertIsNone, assertTrue etc. are used | 13:33 |
| mordred | dunno. honestly I have never desired to use them myself | 13:33 |
| mordred | yeah - assertIsNone are convenience methods provided by testtools | 13:34 |
| mordred | they are not related to testr/stestr | 13:34 |
| stephenfin | Hmm | 13:34 |
| mordred | stestr is a test runner. testtools is a testing support library | 13:34 |
| mordred | pytest is both combined | 13:34 |
| stephenfin | I need to investigate that so. See what things look like | 13:34 |
| mordred | which is how nose was also designed | 13:34 |
| mordred | and one of the reasons we moved off of nose | 13:34 |
| mordred | nose's support library did "interesting" things to help, which wound up modifying the code that was under test | 13:35 |
| *** bobh has joined #openstack-oslo | 13:35 | |
| mordred | all that said- I do not think re-using sphinx's test fixtures would be a violation of anything - I do think that if we are required to use pytest as atest runner in pbr in order to do so that it would be a problem | 13:36 |
| stephenfin | Then that's something I need to investigate. I wrote some tests for one of my extensions and wound up using pytest because it was available anyway | 13:36 |
| stephenfin | But hopefully it's not necessary here | 13:37 |
| mordred | of course, I *think* pytest test fixtures have an unnatural reliance on pytest as a test runner because they python testing protocol (which controls how discovery is run and tests are loaded) wasn't good enough | 13:37 |
| stephenfin | Things like needing to put everything inside a class, for one? | 13:38 |
| stephenfin | Or other stuff | 13:38 |
| mordred | also - (sorry, you tickled my rage button this morning) I **LOVE** that py.test completely ignores the PEP-defined and standard library supported python testing protocol but it's popular and everyone loves it, but pbr is different from setuptools having attempted to follow where the peps are going and people point their fingers at us and yell because we're "not doing things right? | 13:38 |
| mordred | stephenfin: things do not need to be in a class for stestr or the python testing protocol to operate | 13:38 |
| mordred | things being in a class is an organizational choice testtools provides and that we have adopted as a strategy | 13:39 |
| mordred | however, things being in a class allows for defining things like shared setUp/tearDown methods | 13:39 |
| *** bobh has quit IRC | 13:40 | |
| stephenfin | mordred: But who listens to the haters, eh? | 13:40 |
| mordred | exactly :) | 13:40 |
| mordred | stephenfin: ranting aside - let's see if we can figure out how to reuse appropriate sphinx testing fixtures without having to follow the fashion trends of popular test runners | 13:41 |
| mordred | I think the idea of using them is certainly the right choice | 13:41 |
| mordred | and maybe if we figure it out we can show people a pattern for doing so if it comes up again | 13:42 |
| stephenfin | indeed | 13:42 |
| stephenfin | Though I honestly think I was the first person outside Sphinx to use these tests, so adopting them elsewhere is definitely going to present bug/documentation issues which I'll have to fix. I always did like a challenge... | 13:43 |
| *** lbragstad has joined #openstack-oslo | 13:46 | |
| mordred | stephenfin: I concur- you and challenges seem to go hand in hand | 13:46 |
| mordred | stephenfin: re-using test suite fixtures in dependent projects seems to be a concept people haven't fully wrapped their head around | 13:47 |
| stephenfin | bnemec: I assume it's too late to get https://review.openstack.org/#/q/topic:the-great-cleanup+status:open in? | 13:53 |
| *** bobh has joined #openstack-oslo | 13:57 | |
| *** bobh has quit IRC | 14:01 | |
| bnemec | stephenfin: Yeah, technically we're in feature freeze for Oslo already, although I haven't done the final releases yet because of the partially merged oslo.config series. | 14:03 |
| bnemec | It would be nice to get those deprecations in this cycle though. :-/ | 14:03 |
| *** spilla has joined #openstack-oslo | 14:03 | |
| bnemec | I suppose there's an argument that pbr as a more standalone tool could observe the non-client libraries freeze instead. | 14:04 |
| stephenfin | bnemec: I'd be eager to deprecate it now and...encourage people to move off the tools. You're the boss though so your call :) | 14:05 |
| bnemec | stephenfin: Is there any way we could move those earlier in the series so we don't have to land the whole thing? The removals in particular make me a little nervous. | 14:10 |
| stephenfin | Was thinking the same thing myself. Sure, lemme do that | 14:10 |
| bnemec | Most of the rest of it seems like minor fixes and cleanups which don't bother me. | 14:11 |
| stephenfin | Means this probably doesn't need a major version bump then either | 14:11 |
| stephenfin | Not yet, anyway | 14:11 |
| bnemec | The sphinx extension is pretty big. Is that neede for the build_sphinx deprecation? | 14:11 |
| bnemec | *needed | 14:11 |
| stephenfin | Ideally, yes, but not by anyone in OpenStack | 14:12 |
| stephenfin | It's a duplicate of functionality provided by openstackdocstheme, which all official OpenStack projects should be using | 14:12 |
| bnemec | Okay | 14:14 |
| openstackgerrit | Stephen Finucane proposed openstack-dev/pbr master: doc: Fix formatting of packagers guide https://review.openstack.org/580418 | 14:16 |
| openstackgerrit | Stephen Finucane proposed openstack-dev/pbr master: doc: Add documentation for missing '[pbr]' options https://review.openstack.org/580419 | 14:16 |
| openstackgerrit | Stephen Finucane proposed openstack-dev/pbr master: Add Sphinx extension https://review.openstack.org/580420 | 14:16 |
| openstackgerrit | Stephen Finucane proposed openstack-dev/pbr master: Deprecate 'build_sphinx' integration https://review.openstack.org/580421 | 14:16 |
| openstackgerrit | Stephen Finucane proposed openstack-dev/pbr master: Deprecate 'test' integration https://review.openstack.org/580422 | 14:16 |
| openstackgerrit | Stephen Finucane proposed openstack-dev/pbr master: builddoc: Remove support for Sphinx < 1.6 https://review.openstack.org/580416 | 14:16 |
| openstackgerrit | Stephen Finucane proposed openstack-dev/pbr master: packaging: Remove support for pyN requirement files https://review.openstack.org/580417 | 14:16 |
| bnemec | It would be good to have mordred sign off on this too, as the resident pbr expert. | 14:17 |
| stephenfin | bnemec: Yup, he's gone though it once before. Would be good to go again | 14:19 |
| stephenfin | bnemec: What's the next version going to be though. I've referenced 5.0 since I was assuming we'd be removing stuff and would need a major bump. Not so now, I guess? https://review.openstack.org/#/c/580420/9/doc/source/user/features.rst | 14:20 |
| bnemec | stephenfin: Yeah, looks like it would be 4.2.0. | 14:21 |
| openstackgerrit | Stephen Finucane proposed openstack-dev/pbr master: Add Sphinx extension https://review.openstack.org/580420 | 14:29 |
| openstackgerrit | Stephen Finucane proposed openstack-dev/pbr master: Deprecate 'build_sphinx' integration https://review.openstack.org/580421 | 14:29 |
| openstackgerrit | Stephen Finucane proposed openstack-dev/pbr master: Deprecate 'test' integration https://review.openstack.org/580422 | 14:29 |
| openstackgerrit | Stephen Finucane proposed openstack-dev/pbr master: packaging: Remove support for pyN requirement files https://review.openstack.org/580417 | 14:29 |
| openstackgerrit | Stephen Finucane proposed openstack-dev/pbr master: builddoc: Remove support for Sphinx < 1.6 https://review.openstack.org/580416 | 14:29 |
| stephenfin | bnemec: Done and done ^ | 14:29 |
| bnemec | stephenfin: Okay, I'm working my way through the series. | 14:32 |
| *** namnh has joined #openstack-oslo | 14:45 | |
| *** lpetrut has quit IRC | 14:47 | |
| openstackgerrit | Stephen Finucane proposed openstack/oslo.config master: sphinxext: Handle multi-line group descriptions https://review.openstack.org/583525 | 14:49 |
| *** peereb has quit IRC | 14:51 | |
| *** pcaruana has quit IRC | 14:56 | |
| *** moguimar has quit IRC | 15:00 | |
| raildo | dhellmann, bnemec hey, can I get an workflow on that again? https://review.openstack.org/#/c/578144/ I rebased this patch yesterday, since we fixed that issue with that test | 15:06 |
| bnemec | raildo: That's the last patch from the series, right? | 15:10 |
| raildo | yep | 15:10 |
| bnemec | Cool. Fingers crossed that ci cooperates. :-) | 15:11 |
| raildo | bnemec, ++ | 15:11 |
| *** namnh has quit IRC | 15:13 | |
| *** lpetrut has joined #openstack-oslo | 15:16 | |
| *** rpioso|afk is now known as rpioso | 15:20 | |
| *** lpetrut has quit IRC | 15:37 | |
| *** lpetrut has joined #openstack-oslo | 15:55 | |
| *** links has quit IRC | 15:55 | |
| *** pcaruana has joined #openstack-oslo | 16:00 | |
| *** lbragstad has quit IRC | 16:02 | |
| *** efried is now known as efried_rollin | 16:13 | |
| *** lpetrut has quit IRC | 16:17 | |
| *** AlexeyAbashkin has quit IRC | 16:36 | |
| *** AlexeyAbashkin has joined #openstack-oslo | 16:39 | |
| *** njohnston has quit IRC | 16:39 | |
| *** AlexeyAbashkin has quit IRC | 16:43 | |
| *** mattgo has quit IRC | 16:44 | |
| *** spilla_ has joined #openstack-oslo | 16:46 | |
| *** spilla has quit IRC | 16:46 | |
| *** lbragstad has joined #openstack-oslo | 16:50 | |
| *** e0ne has quit IRC | 16:51 | |
| *** njohnston has joined #openstack-oslo | 16:57 | |
| *** shardy has quit IRC | 17:15 | |
| *** mattgo has joined #openstack-oslo | 17:20 | |
| *** tesseract has quit IRC | 17:35 | |
| *** e0ne has joined #openstack-oslo | 17:39 | |
| *** spilla has joined #openstack-oslo | 17:42 | |
| *** spilla_ has quit IRC | 17:42 | |
| *** e0ne has quit IRC | 17:44 | |
| *** e0ne has joined #openstack-oslo | 17:48 | |
| *** e0ne has quit IRC | 17:51 | |
| *** pcaruana has quit IRC | 18:08 | |
| *** e0ne has joined #openstack-oslo | 18:10 | |
| *** mattgo has quit IRC | 18:15 | |
| openstackgerrit | Merged openstack/oslo.config master: New cache layer for external sources https://review.openstack.org/578144 | 18:15 |
| *** e0ne has quit IRC | 18:17 | |
| *** sambetts is now known as sambetts|afk | 18:21 | |
| *** lpetrut has joined #openstack-oslo | 18:28 | |
| *** vivsoni__ has joined #openstack-oslo | 18:33 | |
| vivsoni__ | hi team | 18:34 |
| vivsoni__ | can you please help me to get rid off this error http://paste.openstack.org/show/726221/ | 18:35 |
| vivsoni__ | UnicodeEncodeError: 'utf-8' codec can't encode character '\\udc82' in position 1365: surrogates not allowed\x1b[00m" | 18:35 |
| *** harlowja has joined #openstack-oslo | 18:54 | |
| dhellmann | vivsoni__ : hmm, that's a new one to me. You might want to post the details to the openstack-dev mailing list | 18:58 |
| vivsoni__ | dhellmann: ok | 18:59 |
| dhellmann | vivsoni__ : it would be interesting to see if you could produce the same error using a small python program that doesn't use privsep or msgpack or anything -- just try to encode the string you're working with | 19:03 |
| vivsoni__ | dhellmann: sure i will dig into it a bit more to reproduce and will share the details | 19:04 |
| bnemec | FWIW, stackoverflow came up with this: https://stackoverflow.com/questions/27366479/python-3-os-walk-file-paths-unicodeencodeerror-utf-8-codec-cant-encode-s#27367173 | 19:04 |
| dhellmann | vivsoni__ : sounds good | 19:04 |
| bnemec | But I don't understand why it would trigger inside of msgpack since it shouldn't be printing anything. | 19:04 |
| dhellmann | ah, interesting | 19:05 |
| dhellmann | no, but I wonder if it is doing something similar where it encodes without the surrogateescape setting | 19:06 |
| dhellmann | I wouldn't expect stdout to use that setting, for example | 19:07 |
| *** mattgo has joined #openstack-oslo | 19:13 | |
| rook | bnemec: hey - just saw your reply to the oslo policy... is there a way to mnkey patch your change in to see if it helps? | 19:21 |
| * rook hasn't seen the patch | 19:21 | |
| *** vivsoni__ has quit IRC | 19:21 | |
| bnemec | rook: The functional part of the patch is like 4 lines, so it shouldn't be too hard. | 19:22 |
| bnemec | It's just the changes in this file: https://review.openstack.org/#/c/511426/4/oslo_policy/policy.py | 19:22 |
| rook | hm | 19:23 |
| rook | that doens't seem to be the hot function we are seeing | 19:23 |
| rook | maybe i am misunderstanding bnemec | 19:23 |
| rook | Enforcer._undefined_check and _cycle_check seem to be the big hitters | 19:24 |
| *** njohnston has quit IRC | 19:24 | |
| rook | maybe they get called so often due to reloadin the rules | 19:24 |
| rook | maybe that is the gain here | 19:24 |
| openstackgerrit | Ben Nemec proposed openstack/oslo.policy master: Avoid redundant policy syntax checks https://review.openstack.org/511426 | 19:25 |
| bnemec | rook: Right, those are called in check_rules, which gets called every single time enforce() is called. | 19:26 |
| bnemec | See line 566 and 569 of the change in policy.py. | 19:26 |
| bnemec | By only calling them when necessary it should massively decrease the amount of time spent in those functions. | 19:27 |
| bnemec | It might be more accurate to say that enforce() always calls load_rules(), which always calls check_rules(), which leads to a lot of calls to those functions. | 19:30 |
| bnemec | I think the assumption is that load_rules is basically a noop if the policy hasn't changed, but that wasn't entirely true. | 19:31 |
| dhellmann | it probably was a noop before the validation was added | 19:34 |
| dhellmann | it would also be interesting to see how big the policy file in question is. some of this stuff was designed before we had such large APIs | 19:34 |
| *** e0ne has joined #openstack-oslo | 19:39 | |
| openstackgerrit | Ben Nemec proposed openstack/oslo.policy master: Avoid redundant policy syntax checks https://review.openstack.org/511426 | 19:40 |
| bnemec | It also seems to have more impact when there are a lot of instances or ports or whatever. | 19:43 |
| bnemec | I couldn't reproduce it in a fresh devstack install, but then that had very little to actually check. | 19:44 |
| bnemec | And I will say that I find horizon to be painfully slow when you have a lot of instances. | 19:44 |
| *** efried_rollin is now known as efried | 19:49 | |
| rook | bnemec: ack | 19:51 |
| rook | bnemec: I will see if Anil can try your patch | 19:51 |
| bnemec | Sounds good | 19:51 |
| *** e0ne has quit IRC | 19:52 | |
| openstackgerrit | Merged openstack-dev/pbr master: trivial: Remove 'tools/releasenotes_tox.sh' https://review.openstack.org/580414 | 20:34 |
| openstackgerrit | Merged openstack-dev/pbr master: trivial: Fix file permissions https://review.openstack.org/580415 | 20:48 |
| *** lpetrut has quit IRC | 20:48 | |
| *** raildo has quit IRC | 21:06 | |
| *** njohnston_ has joined #openstack-oslo | 21:38 | |
| *** edmondsw has quit IRC | 21:54 | |
| *** mattgo has quit IRC | 21:54 | |
| *** njohnston_ has quit IRC | 21:54 | |
| *** njohnston_ has joined #openstack-oslo | 22:17 | |
| *** lbragstad has quit IRC | 22:29 | |
| *** rcernin has joined #openstack-oslo | 22:32 | |
| *** njohnston_ has quit IRC | 22:33 | |
| mtreinish | mordred, stephenfin: I wrote a blog post about the testing stack we use few months ago: https://blog.kortar.org/?p=370 (it's probably a bit too high level though) | 22:58 |
| mordred | mtreinish: it's a good post and useful for people who don't have all the background I thnk | 23:02 |
| clarkb | heh just catching up on that discussion, I'm too impatient for py.test. Working on pip is soo slooooowwwww | 23:02 |
| clarkb | but the good news is you can totally run openstack tests under py.test because we use standard compliant libs | 23:02 |
| clarkb | unlike if we went in reverse | 23:02 |
| mtreinish | clarkb: heh, I actually found one case where pytest didn't work with our tests. In tempest we forgot to delete 1 file when we wiped out a dir (which deleted the __init__.py) | 23:03 |
| mtreinish | unittest discovery just skipped the leftover file | 23:04 |
| mtreinish | but because pytest does a recursive directory scan instead of importing the code it found the file and failed when running those dead tests | 23:04 |
| clarkb | huh however thats a case of them not following the standard right? | 23:04 |
| mtreinish | yeah | 23:04 |
| mtreinish | it was just a weird edge case because pytest discovery is just basically "ls -R | grep test_*py" | 23:05 |
| clarkb | wow | 23:05 |
| clarkb | I would be curious for someone to compare nova or neutron's test suite runtime between py.test and testr | 23:06 |
| clarkb | and wonder if people would want to switch after :) | 23:06 |
| clarkb | in any case we don't stop you from using py.test locally, but we use test runner that allows for throughput in the gate because it helps get work done quicker | 23:08 |
| clarkb | mordred: not to induce further rage but they added setup.cfg support to setuptools which is yay until you realize its sort of intentionally not compatible with our setup.cfg | 23:09 |
| clarkb | which was based on pep whatever it was in the first place | 23:10 |
| *** tosky has quit IRC | 23:20 | |
| mordred | clarkb: yeah. don't even get me started on that | 23:24 |
| *** rpioso is now known as rpioso|afk | 23:48 | |
| *** threestrands has joined #openstack-oslo | 23:50 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!