openstackgerrit | Joshua Harlow proposed a change to openstack/taskflow: WBE response message validation https://review.openstack.org/103690 | 00:01 |
---|---|---|
*** czzl has quit IRC | 00:22 | |
*** markmcclain has quit IRC | 00:24 | |
*** markmcclain has joined #openstack-oslo | 00:26 | |
*** markmcclain has quit IRC | 00:27 | |
*** markmcclain has joined #openstack-oslo | 00:27 | |
*** praneshp has quit IRC | 00:27 | |
*** erecio has joined #openstack-oslo | 00:28 | |
zzzeek | heya dhellmann : if we move forward with deeper SQLA extensions in oslo.db, what do you think of this change to tox.ini, is this feasible and how would downstream test runners need to change: http://paste.openstack.org/show/85190/ | 00:30 |
zzzeek | i tried other ways of making that work but right now you can’t have the conflicting requiremnents all as “deps” in tox | 00:31 |
openstackgerrit | Boris Pavlovic proposed a change to openstack/oslo.messaging: Enable cross service profiling https://review.openstack.org/103271 | 00:40 |
*** GheRivero is now known as GheRivero_afk | 00:40 | |
openstackgerrit | Boris Pavlovic proposed a change to openstack/oslo.messaging: Enable cross service profiling https://review.openstack.org/103271 | 00:45 |
openstackgerrit | Boris Pavlovic proposed a change to openstack/oslo.messaging: Enable cross service profiling https://review.openstack.org/103271 | 00:49 |
*** markmcclain has quit IRC | 00:56 | |
zzzeek | well im going to stick it in a gerrit tomorrow :) | 00:56 |
*** HenryG has quit IRC | 00:57 | |
*** HenryG has joined #openstack-oslo | 00:59 | |
openstackgerrit | Joshua Harlow proposed a change to openstack/taskflow: WBE response message validation https://review.openstack.org/103690 | 01:04 |
openstackgerrit | Joshua Harlow proposed a change to openstack/taskflow: WBE request message validation https://review.openstack.org/103695 | 01:04 |
*** dstanek is now known as dstanek_zzz | 01:06 | |
*** jecarey has joined #openstack-oslo | 01:11 | |
*** yamahata has joined #openstack-oslo | 01:18 | |
*** erecio has quit IRC | 01:23 | |
*** HenryG has quit IRC | 01:30 | |
*** HenryG has joined #openstack-oslo | 01:30 | |
openstackgerrit | xu-haiwei proposed a change to openstack/oslo-incubator: Fix .conf.sample file generation bug https://review.openstack.org/103473 | 01:32 |
*** dstanek_zzz is now known as dstanek | 01:36 | |
openstackgerrit | lawrancejing proposed a change to openstack/oslo-incubator: Use immutable arg rather mutable arg https://review.openstack.org/103698 | 01:43 |
openstackgerrit | Xavier Barbosa proposed a change to openstack/cliff: Inject default values to subcommands. https://review.openstack.org/103700 | 01:52 |
*** gcb has joined #openstack-oslo | 01:59 | |
*** tongli has quit IRC | 02:24 | |
openstackgerrit | A change was merged to openstack/oslo-incubator: Fix broken formatting of processutils.execute log statement https://review.openstack.org/103569 | 02:25 |
openstackgerrit | Zhongyue Luo proposed a change to openstack/oslo-incubator: Use timestamp in loopingcall https://review.openstack.org/103704 | 02:26 |
*** zzzeek has quit IRC | 02:29 | |
*** zhiyan_ is now known as zhiyan | 02:38 | |
openstackgerrit | Zhongyue Luo proposed a change to openstack/oslo-incubator: Use timestamp in loopingcall https://review.openstack.org/103704 | 02:42 |
*** dims_ has quit IRC | 02:46 | |
*** zhiyan is now known as zhiyan_ | 02:50 | |
*** zhiyan_ is now known as zhiyan | 02:51 | |
*** mriedem has left #openstack-oslo | 03:00 | |
*** harlowja is now known as harlowja_away | 03:02 | |
*** arnaud has quit IRC | 03:05 | |
*** arnaud__ has quit IRC | 03:05 | |
openstackgerrit | Zhongyue Luo proposed a change to openstack/oslo-incubator: Use timestamp in loopingcall https://review.openstack.org/103704 | 03:07 |
*** harlowja_away is now known as harlowja | 03:09 | |
*** praneshp has joined #openstack-oslo | 03:10 | |
*** harlowja is now known as harlowja_away | 03:10 | |
*** harlowja_away is now known as harlowja | 03:11 | |
*** praneshp_ has joined #openstack-oslo | 03:13 | |
*** praneshp has quit IRC | 03:16 | |
*** praneshp_ is now known as praneshp | 03:16 | |
*** HenryG has quit IRC | 03:17 | |
*** HenryG has joined #openstack-oslo | 03:20 | |
openstackgerrit | Zhongyue Luo proposed a change to openstack/oslo-incubator: Changes calcuation of variable delay https://review.openstack.org/103708 | 03:27 |
openstackgerrit | Joshua Harlow proposed a change to openstack/taskflow: Have the dispatch_job function return a future https://review.openstack.org/103712 | 03:34 |
openstackgerrit | Joshua Harlow proposed a change to openstack/taskflow: WBE request message validation https://review.openstack.org/103695 | 03:45 |
*** gcb has quit IRC | 03:59 | |
openstackgerrit | Joshua Harlow proposed a change to openstack/taskflow: Add a conductor considerations section https://review.openstack.org/103267 | 04:05 |
openstackgerrit | Joshua Harlow proposed a change to openstack/taskflow: Add a conductor considerations section https://review.openstack.org/103267 | 04:05 |
*** sileht has quit IRC | 04:06 | |
*** rpodolyaka has joined #openstack-oslo | 04:06 | |
*** sileht has joined #openstack-oslo | 04:09 | |
*** jogo_awa1 has quit IRC | 04:11 | |
*** ildikov has quit IRC | 04:11 | |
*** pcm__ has quit IRC | 04:11 | |
*** ekarlso has quit IRC | 04:11 | |
*** SlickNik has quit IRC | 04:11 | |
*** jd__ has quit IRC | 04:11 | |
*** mrda has quit IRC | 04:12 | |
*** arnaud has joined #openstack-oslo | 04:13 | |
*** arnaud__ has joined #openstack-oslo | 04:14 | |
*** erecio has joined #openstack-oslo | 04:21 | |
*** mrda has joined #openstack-oslo | 04:22 | |
*** jogo_awa1 has joined #openstack-oslo | 04:22 | |
*** ildikov has joined #openstack-oslo | 04:22 | |
*** pcm__ has joined #openstack-oslo | 04:22 | |
*** ekarlso has joined #openstack-oslo | 04:22 | |
*** SlickNik has joined #openstack-oslo | 04:22 | |
*** jd__ has joined #openstack-oslo | 04:22 | |
*** ildikov has quit IRC | 04:31 | |
*** ildikov has joined #openstack-oslo | 04:32 | |
*** jogo_awa1 has quit IRC | 04:32 | |
*** jogo_away has joined #openstack-oslo | 04:32 | |
*** ildikov has quit IRC | 04:49 | |
*** erecio has quit IRC | 04:53 | |
*** ajc_ has joined #openstack-oslo | 04:59 | |
*** dstanek is now known as dstanek_zzz | 05:05 | |
*** harlowja is now known as harlowja_away | 05:06 | |
openstackgerrit | jichenjc proposed a change to openstack/oslo-incubator: Add lock_path as param to remove_external function https://review.openstack.org/103732 | 05:06 |
*** zzzeek has joined #openstack-oslo | 05:11 | |
*** dstanek_zzz is now known as dstanek | 05:13 | |
*** ildikov has joined #openstack-oslo | 05:23 | |
*** dstanek is now known as dstanek_zzz | 05:28 | |
*** tkelsey has joined #openstack-oslo | 05:32 | |
openstackgerrit | Pranesh Pandurangan proposed a change to openstack/taskflow: Convert usage of some state strings to objects https://review.openstack.org/103382 | 05:39 |
*** markmc has joined #openstack-oslo | 05:52 | |
openstackgerrit | Joshua Harlow proposed a change to openstack/taskflow: Add a conductor considerations section https://review.openstack.org/103267 | 06:06 |
*** mkoderer has joined #openstack-oslo | 06:24 | |
*** dstanek_zzz is now known as dstanek | 06:29 | |
*** SridharG has joined #openstack-oslo | 06:30 | |
*** rpodolyaka has quit IRC | 06:43 | |
*** zzzeek has quit IRC | 06:43 | |
*** zzzeek has joined #openstack-oslo | 06:47 | |
*** pblaho has joined #openstack-oslo | 06:52 | |
*** zzzeek has quit IRC | 06:53 | |
*** i159 has joined #openstack-oslo | 06:56 | |
*** i159_ has joined #openstack-oslo | 07:02 | |
*** i159 has quit IRC | 07:02 | |
*** arnaud__ has quit IRC | 07:06 | |
*** arnaud has quit IRC | 07:06 | |
*** morganfainberg is now known as morganfainberg_Z | 07:12 | |
*** arnaud has joined #openstack-oslo | 07:14 | |
*** arnaud__ has joined #openstack-oslo | 07:15 | |
*** arnaud__ has quit IRC | 07:20 | |
*** arnaud has quit IRC | 07:20 | |
*** arnaud has joined #openstack-oslo | 07:20 | |
*** arnaud__ has joined #openstack-oslo | 07:20 | |
*** dstanek is now known as dstanek_zzz | 07:24 | |
*** praneshp has quit IRC | 07:27 | |
*** rdopieralski has joined #openstack-oslo | 07:37 | |
*** nacim has joined #openstack-oslo | 07:38 | |
*** rdopieralski has quit IRC | 07:40 | |
*** mrda is now known as mrda-away | 07:49 | |
*** erecio has joined #openstack-oslo | 07:51 | |
openstackgerrit | lawrancejing proposed a change to openstack/taskflow: Fix the section name in CONTRIBUTING.rst https://review.openstack.org/103780 | 07:51 |
*** flaper87|afk is now known as flaper87 | 07:55 | |
*** morganfainberg_Z is now known as morganfainberg | 07:55 | |
*** ihrachyshka has joined #openstack-oslo | 08:08 | |
boris-42 | markmc hi | 08:11 |
boris-42 | markmc I should write spec to https://github.com/openstack/oslo-specs ?? | 08:11 |
boris-42 | markmc or oslo.messaging has separated repo? | 08:11 |
*** dstanek_zzz is now known as dstanek | 08:15 | |
*** ihrachyshka has quit IRC | 08:15 | |
*** ihrachyshka has joined #openstack-oslo | 08:16 | |
*** nacim has quit IRC | 08:20 | |
rpodolyaka1 | boris-42: oslo-specs repo is used to track specs to all projects within oslo program | 08:22 |
boris-42 | rpodolyaka1 ah great thanks | 08:23 |
rpodolyaka1 | boris-42: np | 08:23 |
*** erecio has quit IRC | 08:24 | |
*** dstanek is now known as dstanek_zzz | 08:25 | |
*** arnaud__ has quit IRC | 08:29 | |
*** arnaud has quit IRC | 08:29 | |
*** ujjain has quit IRC | 08:39 | |
*** ujjain has joined #openstack-oslo | 08:39 | |
*** ildikov has quit IRC | 08:42 | |
openstackgerrit | garyk proposed a change to openstack/oslo.vmware: Translations: make use of _LE, _LI and _LW https://review.openstack.org/101821 | 08:42 |
*** viktors has joined #openstack-oslo | 08:49 | |
*** yamahata has quit IRC | 09:12 | |
*** flaper87 is now known as flaper87|afk | 09:13 | |
*** ihrachyshka has quit IRC | 09:13 | |
openstackgerrit | A change was merged to openstack/taskflow: Clarify locked decorator is for instance methods https://review.openstack.org/98691 | 09:16 |
*** dstanek_zzz is now known as dstanek | 09:16 | |
openstackgerrit | A change was merged to openstack/taskflow: Fix traces left in zookeeper https://review.openstack.org/100415 | 09:19 |
*** lbragstad_ has quit IRC | 09:22 | |
*** lbragstad_ has joined #openstack-oslo | 09:24 | |
*** dstanek is now known as dstanek_zzz | 09:26 | |
*** oomichi has quit IRC | 09:30 | |
*** dstanek_zzz is now known as dstanek | 09:48 | |
openstackgerrit | Boris Pavlovic proposed a change to openstack/oslo-specs: Enable cross service profiling https://review.openstack.org/103825 | 09:51 |
boris-42 | markmc ^ I added spec | 09:52 |
openstackgerrit | A change was merged to openstack/oslo.messaging: Remove duplicate docs for MessageHandlingServer https://review.openstack.org/102731 | 09:57 |
*** dstanek is now known as dstanek_zzz | 09:58 | |
*** ajo|mac has joined #openstack-oslo | 10:08 | |
*** ihrachyshka has joined #openstack-oslo | 10:16 | |
openstackgerrit | Ilya Pekelny proposed a change to openstack/oslo.db: Opportunistic migration tests https://review.openstack.org/93424 | 10:18 |
openstackgerrit | Ilya Pekelny proposed a change to openstack/oslo.db: Implementation Alembic as migration engine https://review.openstack.org/99965 | 10:18 |
*** dstanek_zzz is now known as dstanek | 10:49 | |
*** openstack has joined #openstack-oslo | 10:51 | |
openstackgerrit | Boris Pavlovic proposed a change to openstack/oslo.messaging: Enable cross service profiling https://review.openstack.org/103271 | 10:56 |
openstackgerrit | Boris Pavlovic proposed a change to openstack/oslo.messaging: Enable cross service profiling https://review.openstack.org/103271 | 10:57 |
*** dstanek is now known as dstanek_zzz | 10:59 | |
*** jaosorior has joined #openstack-oslo | 11:06 | |
jaosorior | Anybody here working with oslo.messaging? | 11:06 |
*** dims_ has joined #openstack-oslo | 11:07 | |
ihrachyshka | jaosorior: the question is ambiguous, but yes | 11:15 |
*** tkelsey has quit IRC | 11:17 | |
*** erecio has joined #openstack-oslo | 11:22 | |
*** nacim has joined #openstack-oslo | 11:33 | |
dhellmann | viktors: it looks like https://blueprints.launchpad.net/oslo/+spec/oslo-db-lib is pretty close to done (some of the TODO items like ACLs are done). Can you review it and update its status, please? | 11:41 |
viktors | dhellmann: hi! I think, that the single TODO is left - Update consumers | 11:45 |
viktors | not all projects use oslo.db - only keystone and neutron at the moment | 11:45 |
viktors | can I close this BP without the lats item& | 11:46 |
viktors | ? | 11:46 |
dhellmann | viktors: yes, you've released the library so let's close that bp and use something else to track adoption (bps in each project, perhaps) | 11:49 |
dhellmann | viktors: I'm going to remove that todo item from the template list on the wiki page | 11:50 |
*** dstanek_zzz is now known as dstanek | 11:50 | |
viktors | dhellmann: ok, I'll close this BP | 11:50 |
dhellmann | viktors: if you keep https://etherpad.openstack.org/p/juno-oslo-adoption-status up to date, I'll be happy with the tracking and the PTLs of the other projects can decide if they want blueprints or not | 11:51 |
viktors | dhellmann: oh, I missed this etherpad. Will do | 11:52 |
dhellmann | viktors: when we move to storyboard, it will be a little easier to track all of this in one place, but I'm not sure about our ability to have cross-project bp dependencies in lp | 11:52 |
viktors | dhellmann: also we are suppose to release oslo.db 0.3.0 after we will get merged i159_'s patches with opportunistic migration tests for sqla-migrate and alembic | 11:56 |
dhellmann | viktors: ok, thanks, I'll mention that today in the project meeting to give the other teams a little notice | 11:56 |
dhellmann | viktors: do you have an estimate for when that will be? this week? next week? | 11:57 |
boris-42 | dhellmann Hi | 11:59 |
*** dstanek is now known as dstanek_zzz | 12:00 | |
viktors | dhellmann: as soon as we'll merge some patches. I really hope, this week, but I'm not sure :) | 12:00 |
viktors | Also we with rpodolyaka1 also want to ask some question to Mike about migration tests for alembic implementation. | 12:00 |
dhellmann | viktors: ok, I'll say "soon" :-) | 12:00 |
viktors | dhellmann: :) | 12:00 |
*** rpodolyaka1 has left #openstack-oslo | 12:01 | |
*** rpodolyaka1 has joined #openstack-oslo | 12:01 | |
*** rpodolyaka1 is now known as rpodolyaka | 12:01 | |
boris-42 | dhellmann hi=) | 12:01 |
*** rpodolyaka1 has joined #openstack-oslo | 12:03 | |
*** rpodolyaka has quit IRC | 12:05 | |
*** ajo|mac has joined #openstack-oslo | 12:06 | |
*** rpodolyaka1 is now known as rpodolyaka | 12:06 | |
*** viktors is now known as viktors|afk | 12:08 | |
boris-42 | dhellmann could you pls take a look at my spec for osprofiler | 12:12 |
boris-42 | dhellmann https://review.openstack.org/#/c/103825/ | 12:12 |
openstackgerrit | A change was merged to openstack/oslo-specs: Update the sphinx version of the module https://review.openstack.org/103387 | 12:13 |
*** ihrachyshka has quit IRC | 12:16 | |
dhellmann | boris-42: I'm about to head in to the office for the day, but I will look at it. | 12:16 |
boris-42 | dhellmann thanks | 12:16 |
jaosorior | yeah, so is the coverage in oslo.messaging working for anyone? haven't gotten it to give results :/ | 12:16 |
*** ajc_ has quit IRC | 12:17 | |
boris-42 | markmc pls could you take a look at my spec as well.. | 12:17 |
*** dhellmann is now known as dhellmann_ | 12:19 | |
*** dims_ has quit IRC | 12:32 | |
*** dims_ has joined #openstack-oslo | 12:32 | |
*** ihrachyshka has joined #openstack-oslo | 12:36 | |
openstackgerrit | Juan Antonio Osorio Robles proposed a change to openstack/oslo.messaging: Raise if invalid conf is given to get_transport https://review.openstack.org/103866 | 12:41 |
*** tkelsey has joined #openstack-oslo | 12:43 | |
openstackgerrit | Christian Berendt proposed a change to openstack-dev/cookiecutter: Use spaces instead of tabs for indentation https://review.openstack.org/103869 | 12:46 |
*** alexpilotti has joined #openstack-oslo | 12:47 | |
*** dstanek_zzz is now known as dstanek | 12:50 | |
*** SridharG has quit IRC | 12:53 | |
openstackgerrit | Christian Berendt proposed a change to openstack-dev/oslo-cookiecutter: Use spaces instead of tabs for indentation https://review.openstack.org/103872 | 12:56 |
*** dstanek is now known as dstanek_zzz | 13:00 | |
openstackgerrit | Gordon Sim proposed a change to openstack/oslo-specs: Added spec for proposed AMQP 1.0 driver https://review.openstack.org/96729 | 13:02 |
*** pblaho_ has joined #openstack-oslo | 13:09 | |
*** pblaho has quit IRC | 13:09 | |
openstackgerrit | OpenStack Proposal Bot proposed a change to openstack/oslo.db: Updated from global requirements https://review.openstack.org/102132 | 13:11 |
*** markmcclain has joined #openstack-oslo | 13:15 | |
*** mriedem has joined #openstack-oslo | 13:22 | |
*** mriedem has left #openstack-oslo | 13:22 | |
*** mriedem has quit IRC | 13:22 | |
*** mriedem has joined #openstack-oslo | 13:22 | |
openstackgerrit | Yuriy Taraday proposed a change to openstack/oslo.config: Allow to reference values outside DEFAULT in substitution https://review.openstack.org/103884 | 13:23 |
*** jecarey has quit IRC | 13:31 | |
*** pblaho_ is now known as pblaho | 13:33 | |
*** pblaho_ has joined #openstack-oslo | 13:33 | |
*** pblaho has quit IRC | 13:37 | |
*** dstanek_zzz is now known as dstanek | 13:51 | |
*** SridharG has joined #openstack-oslo | 13:58 | |
*** pblaho_ is now known as pblaho | 14:02 | |
*** zzzeek has joined #openstack-oslo | 14:04 | |
*** flaper87|afk is now known as flaper87 | 14:06 | |
*** erecio has quit IRC | 14:19 | |
*** nacim has quit IRC | 14:24 | |
*** dims_ has quit IRC | 14:28 | |
*** nacim has joined #openstack-oslo | 14:35 | |
openstackgerrit | Ilya Pekelny proposed a change to openstack/oslo.db: Add a base test case for DB schema comparison https://review.openstack.org/93398 | 14:39 |
*** dims_ has joined #openstack-oslo | 14:41 | |
*** jecarey has joined #openstack-oslo | 14:52 | |
*** james_li has joined #openstack-oslo | 14:55 | |
*** flaper87 is now known as flaper87|afk | 14:57 | |
*** Alexei_987 has joined #openstack-oslo | 14:58 | |
rpodolyaka | zzzeek: hey! are you around? | 14:58 |
zzzeek | heya | 14:59 |
rpodolyaka | a quick question on alembic | 14:59 |
rpodolyaka | to make sure I understand things correctly | 14:59 |
rpodolyaka | some time ago we had a patch to oslo.db to add an API layer to unify alembic/migrate migrations | 14:59 |
rpodolyaka | so that I could pass a migrations repo dir and a Connection/Engine instance and run migrations scripts for this particular connection | 15:00 |
rpodolyaka | for migrate this is quite straightforward | 15:00 |
rpodolyaka | but for alembic we had to use some private API functions | 15:00 |
zzzeek | the connecvtion/engine setup is done by env.py | 15:01 |
rpodolyaka | to run migration scripts programatically | 15:01 |
zzzeek | you have control over that | 15:01 |
rpodolyaka | yep | 15:01 |
rpodolyaka | https://review.openstack.org/#/c/71874 | 15:01 |
rpodolyaka | but env.py is bound to a particular migration repo, right? | 15:01 |
rpodolyaka | so the question is more like, I can't provide one within oslo.db | 15:01 |
*** dstanek is now known as dstanek_zzz | 15:02 | |
rpodolyaka | as I can't know in advance how it will be used in a consuming project like e.g. nova | 15:02 |
zzzeek | the front end env.py is in a migration repo but you can certainly control what it does externally | 15:02 |
zzzeek | i dont see why you had to take apart command.upgrade here | 15:03 |
*** flaper87|afk is now known as flaper87 | 15:03 | |
rpodolyaka | afair, we coundn't do something like: with environment.EnvironmentContext(...): upgrade() | 15:04 |
rpodolyaka | passing an existing Connection instance | 15:04 |
zzzeek | you can put a Connection into the config and have your env.py read it | 15:05 |
zzzeek | its so ironic that you say migrate is easier. the whole point of env.py was because of hte massive monkeypatching i had to do to get migrate to do the simplest things | 15:05 |
zzzeek | when i used migrate, it called create_engine() in no less than three places | 15:05 |
rpodolyaka | heh | 15:05 |
zzzeek | it was utterly impossible to control where it did anything with connections and transactions were a total non starter | 15:06 |
rpodolyaka | yep | 15:06 |
zzzeek | with env.py i said, never again. i will give a script to the user, that they can do anything with | 15:06 |
zzzeek | so you can pass anything at all to env.py, you are the one calling it! | 15:06 |
zzzeek | and writing it! | 15:06 |
*** viktors|afk is now known as viktors | 15:06 | |
rpodolyaka | ok, so if I understand you correctly, you say that a consuming project must be somehow 'ready' for passing a connection to env.py? | 15:07 |
zzzeek | it woudl be the most straightforward way, i could look some more and see other ways also | 15:07 |
zzzeek | the env.py is responsible for figuring out where the connection comes from | 15:08 |
zzzeek | by deafult its looking in alembic.ini | 15:08 |
zzzeek | im sure openstack projects dont use that | 15:08 |
*** dstanek_zzz is now known as dstanek | 15:09 | |
rpodolyaka | yeah, I'm looking at neutron code right now and they use their own configs, parse them with oslo.config and create an Engine instance based on that url | 15:09 |
zzzeek | im not sure how this patch woudl even work, woudlnt a non-cooperating env.py just override the connection you’re putting in there? | 15:09 |
zzzeek | projects using oslo.db should absolutely be using an oslo.db env.py, their own env.py should be: “from oslo.db import env; env.run()” | 15:10 |
zzzeek | just like you see in everyone’s setup.py using “pbr” | 15:10 |
rpodolyaka | cool, we should add one :) | 15:10 |
zzzeek | that’s waht i usually do, the env.py in the migrate dir just becomes a stub | 15:10 |
zzzeek | pulls in a more authoriative env.py from elsewhere | 15:10 |
rpodolyaka | seems that we'll need this for https://review.openstack.org/#/c/99965/13/oslo/db/sqlalchemy/test_migrations.py | 15:14 |
rpodolyaka | and until consuming projects don't cooperate with us (until we don't provide common env.py), it seems that we'll need to monkey-patch projects env.py in tests | 15:14 |
rpodolyaka | in order to pass the Engine instance we want | 15:15 |
zzzeek | OK well env.py is totally responsible for setting up environment. here we’re getting into, you’re trying to skip that. and there’s no API for skipping env.py | 15:15 |
zzzeek | b.c. the idea of env.py is that there could be totally crazy things going on like, it connects to three different DBs and runs them all. but. what if a program wants to override env.py, lets look | 15:15 |
zzzeek | all you’d really want is to be able to inject ScriptDirectory.run_env() | 15:17 |
zzzeek | not a big deal | 15:17 |
*** praneshp has joined #openstack-oslo | 15:20 | |
*** i159_ has quit IRC | 15:21 | |
*** ihrachyshka has quit IRC | 15:24 | |
rpodolyaka | zzzeek: ok, will take a look alembic API once again, thanks a lot! | 15:31 |
zzzeek | rpodolyaka: if we need enhancement to change run_env(), let me know | 15:32 |
rpodolyaka | zzzeek: ok | 15:32 |
zzzeek | rpodolyaka: i think having everyone use oslo.db.env.py is best but we can do either | 15:32 |
*** erecio has joined #openstack-oslo | 15:33 | |
*** eezhova has joined #openstack-oslo | 15:36 | |
*** flaper87 is now known as flaper87|afk | 15:40 | |
*** flaper87|afk is now known as flaper87 | 15:42 | |
*** dstanek is now known as dstanek_zzz | 15:47 | |
*** nacim has quit IRC | 15:49 | |
*** markmcclain has quit IRC | 15:52 | |
*** markmc has quit IRC | 15:55 | |
*** praneshp has quit IRC | 15:57 | |
*** praneshp has joined #openstack-oslo | 15:59 | |
*** stannie1 has joined #openstack-oslo | 16:01 | |
*** stannie1 is now known as stannie42 | 16:01 | |
*** viktors is now known as viktors|afk | 16:06 | |
openstackgerrit | Graham Hayes proposed a change to openstack/oslosphinx: Added a incubating project config option https://review.openstack.org/103935 | 16:10 |
*** zhiyan is now known as zhiyan_ | 16:19 | |
*** arnaud has joined #openstack-oslo | 16:20 | |
*** erecio has quit IRC | 16:21 | |
*** dstanek_zzz is now known as dstanek | 16:28 | |
*** arnaud has quit IRC | 16:30 | |
*** arnaud has joined #openstack-oslo | 16:33 | |
*** Alexei_987 has quit IRC | 16:37 | |
*** tkelsey has quit IRC | 16:40 | |
*** dims_ has quit IRC | 16:41 | |
*** dims_ has joined #openstack-oslo | 16:42 | |
*** erecio has joined #openstack-oslo | 16:48 | |
*** zzzeek has quit IRC | 16:50 | |
*** jaosorior has quit IRC | 16:52 | |
*** pblaho has quit IRC | 16:56 | |
*** ihrachyshka has joined #openstack-oslo | 16:56 | |
*** arnaud has quit IRC | 16:57 | |
openstackgerrit | Pranesh Pandurangan proposed a change to openstack/taskflow: Convert usage of some state strings to objects https://review.openstack.org/103382 | 16:59 |
*** markmcclain has joined #openstack-oslo | 17:01 | |
*** markmcclain1 has joined #openstack-oslo | 17:03 | |
*** markmcclain has quit IRC | 17:05 | |
*** harlowja_away is now known as harlowja | 17:06 | |
*** flaper87 is now known as flaper87|afk | 17:06 | |
openstackgerrit | Pranesh Pandurangan proposed a change to openstack/taskflow: Convert usage of some state strings to objects https://review.openstack.org/103382 | 17:07 |
*** rpodolyaka1 has joined #openstack-oslo | 17:18 | |
*** james_li has quit IRC | 17:21 | |
openstackgerrit | Pranesh Pandurangan proposed a change to openstack/taskflow: Convert usage of some state strings to objects https://review.openstack.org/103382 | 17:22 |
*** dstanek is now known as dstanek_zzz | 17:22 | |
boris-42 | dhellmann_ hi | 17:23 |
*** rpodolyaka1 has quit IRC | 17:23 | |
*** dstanek_zzz is now known as dstanek | 17:25 | |
*** lbragstad_ is now known as lbragstad | 17:31 | |
*** tmcpeak has joined #openstack-oslo | 17:32 | |
tmcpeak | I know jogo is the main guy to talk to for hacking stuff, but I've thrown together a PoC for a security check, was wondering if anybody can tell me if I'm headed in the right direction and help me sort out some next steps | 17:35 |
*** erecio has quit IRC | 17:35 | |
*** pblaho has joined #openstack-oslo | 17:39 | |
*** arnaud has joined #openstack-oslo | 17:48 | |
*** arnaud__ has joined #openstack-oslo | 17:48 | |
*** rpodolyaka1 has joined #openstack-oslo | 17:49 | |
*** erecio has joined #openstack-oslo | 17:50 | |
*** dhellmann_ is now known as dhellmann | 17:53 | |
*** markmcclain1 has quit IRC | 17:53 | |
*** markmcclain has joined #openstack-oslo | 17:53 | |
*** tongli has joined #openstack-oslo | 17:56 | |
tongli | trying to use oslo.messaging , producer post a message, wanting to have 2 or more consumers to consume same messages. | 17:57 |
openstackgerrit | Davanum Srinivas (dims) proposed a change to openstack/oslo-incubator: Ability to customize default_log_levels for each project https://review.openstack.org/94001 | 17:57 |
tongli | can anyone tell me how this is done using oslo.messaging? | 17:58 |
*** zzzeek has joined #openstack-oslo | 18:00 | |
*** rpodolyaka1 has quit IRC | 18:00 | |
*** harlowja is now known as harlowja_away | 18:10 | |
tmcpeak | jogo_away: you really away? :) | 18:12 |
*** arnaud__ has quit IRC | 18:13 | |
*** arnaud has quit IRC | 18:13 | |
*** tmcpeak has left #openstack-oslo | 18:21 | |
openstackgerrit | Sean Dague proposed a change to openstack/oslo-incubator: update new requests logger to default WARN https://review.openstack.org/103960 | 18:26 |
*** erecio has quit IRC | 18:28 | |
*** james_li has joined #openstack-oslo | 18:29 | |
dhellmann | dims, flaper87|afk, markmc, jd__, or haypo : do any of you have time for a spot check of this spec? https://review.openstack.org/#/c/97296/4 | 18:30 |
boris-42 | dhellmann I know you are busy a lot | 18:31 |
openstackgerrit | A change was merged to openstack/oslo-specs: Add API details section to graduation spec https://review.openstack.org/102269 | 18:31 |
boris-42 | dhellmann but it will be soon feature freeze and all that crap | 18:31 |
boris-42 | dhellmann and this small patch in oslo.messaging is essential -> to be able to show others to osprofiler works | 18:32 |
dhellmann | boris-42: sorry, I was out yesterday so I'm still catching up with everything I missed | 18:32 |
boris-42 | dhellmann yep sure =) | 18:32 |
*** harlowja_away is now known as harlowja | 18:33 | |
*** erecio has joined #openstack-oslo | 18:37 | |
*** ihrachyshka has quit IRC | 18:37 | |
*** ajo|mac has joined #openstack-oslo | 18:41 | |
boris-42 | dhellmann I really don't know why people is afraid so | 18:42 |
boris-42 | dhellmann about profiler | 18:42 |
boris-42 | dhellmann it doesn't add any overhead if you don't use it... | 18:43 |
boris-42 | dhellmann even if it is turned on.. | 18:43 |
dhellmann | boris-42: because not everyone has time to read the code to verify that claim | 18:43 |
*** arnaud has joined #openstack-oslo | 18:44 | |
boris-42 | dhellmann I replied on your comments | 18:49 |
*** ajo|mac has quit IRC | 18:50 | |
boris-42 | dhellmann should I fix somehow description? | 18:51 |
*** jecarey has quit IRC | 18:54 | |
*** alexpilotti has quit IRC | 18:57 | |
boris-42 | dhellmann so actually I should change API | 18:59 |
boris-42 | dhellmann in osprofiler | 18:59 |
boris-42 | dhellmann to move one patch from oslo.messaging | 18:59 |
dhellmann | boris-42: please put this info in the spec or the review comments | 19:00 |
boris-42 | dhellmann but it's just 0.1.3 not 0.2.0 version | 19:00 |
boris-42 | dhellmann I already I am going to put in spec | 19:00 |
dhellmann | boris-42: ok | 19:01 |
boris-42 | dhellmann is it point one clear now? | 19:01 |
boris-42 | dhellmann about message? | 19:01 |
dhellmann | boris-42: I'll have to get back around to look at the spec again in a bit | 19:01 |
harlowja | zzzeek where u located (out of curosity) | 19:06 |
zzzeek | brooklyn ny | 19:06 |
harlowja | nice, good ole brooklyn | 19:06 |
zzzeek | come on over there’s great bars here | 19:08 |
openstackgerrit | Boris Pavlovic proposed a change to openstack/oslo-specs: Enable cross service profiling https://review.openstack.org/103825 | 19:08 |
morganfainberg | zzzeek, wanted to ask you, do you prefer when people use sqlalchemy.and_ / .or_ + session.query().filter or using the session.query().filter_by mechanisms? | 19:12 |
*** ildikov has joined #openstack-oslo | 19:12 | |
zzzeek | do you mean multple filter calls vs. using and_() ? | 19:12 |
*** jecarey has joined #openstack-oslo | 19:12 | |
morganfainberg | zzzeek, .filter_by(thing=<value>) or .filter(and_(MODEL.id == <val>, etc)) | 19:13 |
zzzeek | they are all fine, filter_by() is a little less reliable, fixed some bugs in it recently | 19:13 |
morganfainberg | right | 19:14 |
morganfainberg | ok | 19:14 |
morganfainberg | i prefer filter, but flake8 balks at 'Model.Column == None' | 19:14 |
morganfainberg | says you need to use 'is None' | 19:14 |
zzzeek | we should remove that flake8 rule | 19:14 |
zzzeek | ive seen it removed in some suites already | 19:14 |
morganfainberg | zzzeek, i'll propse that on the ML | 19:15 |
morganfainberg | propose* | 19:15 |
zzzeek | if you really want to avoid the == you can say Column.is_(None) | 19:15 |
zzzeek | or Column == sql.null() | 19:15 |
zzzeek | or Column.is_(sql.null()) | 19:15 |
morganfainberg | ah | 19:15 |
zzzeek | :) | 19:15 |
zzzeek | all of which are more of a PITA | 19:15 |
morganfainberg | that might be a good alternative | 19:15 |
zzzeek | im not 100% sure those are all more backend-agnostic. i think they are. true()/ false() are more flaky | 19:15 |
zzzeek | like if you say column.is_(true()) | 19:16 |
zzzeek | that might fail on MySQL | 19:16 |
morganfainberg | zzzeek, have to say it's kinda cool being our own downstream :P | 19:16 |
zzzeek | yeah | 19:16 |
morganfainberg | get to ask all these questions :) | 19:16 |
*** ihrachyshka has joined #openstack-oslo | 19:16 | |
morganfainberg | i think the .and_() and .or_() + .filter() is way easier to read in code, which is why i was asking | 19:16 |
openstackgerrit | Michael Bayer proposed a change to openstack/oslo.db: Test for distinct SQLAlchemy major releases https://review.openstack.org/103975 | 19:16 |
zzzeek | OK who wants to be a reviewer on that | 19:17 |
morganfainberg | zzzeek, anyway thanks! | 19:17 |
morganfainberg | zzzeek, i need lunch :P catch ya a bit later on | 19:17 |
zzzeek | later | 19:17 |
*** arnaud has quit IRC | 19:20 | |
*** ildikov has quit IRC | 19:20 | |
openstackgerrit | Michael Bayer proposed a change to openstack/oslo.db: Test for distinct SQLAlchemy major releases https://review.openstack.org/103975 | 19:21 |
ihrachyshka | zzzeek, what,s your timezone? I have stuff to discuss with you. | 19:21 |
zzzeek | EDT | 19:21 |
ihrachyshka | ok, meaning you'll be avail the next hour :) correct? | 19:23 |
zzzeek | If im still sitting here…. :) | 19:23 |
zzzeek | what are we talking about | 19:23 |
zzzeek | and/or do you want to send me a review or somethign up front | 19:23 |
ihrachyshka | more of discussion regarding db lock timeouts we experience in neutron | 19:24 |
zzzeek | ok | 19:24 |
zzzeek | my short answer to that is, “dont use locks” :) | 19:25 |
ihrachyshka | you can check https://review.openstack.org/103540 to get the history of our failure :) | 19:28 |
harlowja | zzzeek when i get back in NY might take u up on that (most of my family is in NYC or upstate, aka the rest of NY, ha) | 19:29 |
dhellmann | flaper87|afk: the issue with duplicate option errors is likely caused by those options not having exactly the same settings (check the help text, for example) | 19:30 |
dhellmann | flaper87|afk: let me know if syncing the incubator into glance doesn't fix it | 19:30 |
*** markmcclain has quit IRC | 19:34 | |
zzzeek | ihrachyshka: my immediate thought there is that you are seeing traditional deadlocks. it’s not specific to using eventlet, though eventlet’s timing may increase the chance of such a thing happening. | 19:34 |
zzzeek | concurrent transactions happen regardless of eventlet or even threads being in use | 19:34 |
zzzeek | ihrachyshka: although, I guess if the other side of the deadlock is that eventlet won’t context switch back until the second actor is done, then yes eventlet is a player here, OK | 19:35 |
ihrachyshka | zzzeek, how so? if mysqldb blocks thread switch, where do concurrent transactions come from? | 19:35 |
zzzeek | a deadlock can occur just with two mutually dependent locks, shared between two processes. | 19:36 |
zzzeek | but youre rgiht, here, its just one lock and the other “lock” is eventlet not context switching back | 19:36 |
zzzeek | but, hmm | 19:37 |
zzzeek | wouldnt eventlet go to thread #2, do the thing, hit the lock, then that’s blocking IO, so it switches back again? | 19:37 |
ihrachyshka | it locks deep in c code in mysqldb, so no event | 19:38 |
zzzeek | OK | 19:38 |
ihrachyshka | et monkey patching | 19:38 |
zzzeek | well then this goes back to waht I’ve talked about a few weeks ago somehwere, one sec | 19:38 |
zzzeek | look into alternative drivers: http://www.diamondtin.com/2014/sqlalchemy-gevent-mysql-python-drivers-comparison/ | 19:38 |
ihrachyshka | aha, that's what I actually searched for | 19:39 |
zzzeek | or try to work aroudn the lock. are you using SELECT FOR UPDATE ? | 19:39 |
ihrachyshka | yes, we do use it | 19:39 |
zzzeek | is that the source of the row lock ? | 19:39 |
ihrachyshka | let me show one of the patches to workaround a particular deadlock scenario | 19:40 |
ihrachyshka | it's https://review.openstack.org/100934 | 19:41 |
ihrachyshka | yes, this is probably the source, we have multiple locks like that in neutron (don't ask me why) :) | 19:41 |
harlowja | zzzeek if u interested, https://bitbucket.org/eventlet/eventlet/pull-request/29/fix-use-of-semaphore-with-tpool-issue-137/diff#comment-126671 is the known issue with eventlet (i believe this is the right link) | 19:42 |
zzzeek | is the idea, send notifcations outside of transaction blocks ? | 19:42 |
harlowja | chris (comstud) can be found in the #openstack-nova channel afaik | 19:42 |
ihrachyshka | yes, it's the idea | 19:42 |
ihrachyshka | but the patch is ugly, so I don't want to extend the approach | 19:42 |
zzzeek | ihrachyshka: if you send a notifaction outside of the transaction block, then that whole thread is done, isnt the issue that the sending of that notifcation will interrupt some other thread? which then could be contending with a third? | 19:43 |
zzzeek | ihrachyshka: im not deeply experienced with async so i could be misunderstanding. but this code pattern, you could clean it up so that it isn’t “ugly” | 19:45 |
zzzeek | e.g. build it into “context” and the context manager | 19:45 |
zzzeek | just like a web framework, add “post-transaction” callables | 19:45 |
ihrachyshka | I guess thread1 opens transaction, locks on a row, then triggers rpc, it yields to thread2 that also locks on the row, the latter is blocked, but mysqdb does not yield since it's not under eventlet control | 19:46 |
zzzeek | so inside of self.disassociate_floatingips() -> context.post_transaction(self.notify_routers_updated, context, router_ids) | 19:46 |
ihrachyshka | zzzeek, that's exactly what my spec I've sent before is about :) | 19:47 |
zzzeek | ihrachyshka: OK….. hmm so you know for sure that context does not siwtch between thread1-> transaction completed and “triggers rpc” because there are no blocking operations | 19:47 |
ihrachyshka | hm. I didn't get the last one | 19:48 |
boris-42 | hey somebody | 19:48 |
boris-42 | stupid question | 19:48 |
boris-42 | when we are testing for example cinder | 19:49 |
boris-42 | we are using oslo.messaging from master in cinder gates? | 19:49 |
ihrachyshka | I know that in neutron, if you call anything that can yield under transaction, it may result in db lock timeout | 19:49 |
zzzeek | ihrachyshka: im trying to confirm in my head why we know that, thread1 completes transaction, thread2 starts new transaction, locks row, thread1 sends RPC, thread2 is bumped by the RPC call, thread3 then starts up, locks row, deadlock | 19:49 |
boris-42 | or not | 19:49 |
zzzeek | and i think the reason is, thread2 would never start before thread1 is finished because thread1 does not block on IO at that point | 19:49 |
zzzeek | ihrachyshka: or, RPC only interrupts the thread that it is actually called from | 19:50 |
ihrachyshka | thread1 hasn't completed transaction, it yielded to thread2 by calling to rpc | 19:50 |
zzzeek | ihrachyshka: yes, but im saying, with the fix in place | 19:50 |
zzzeek | ihrachyshka: where RPC call is outside of transaction | 19:50 |
ihrachyshka | if thread2 opens transaction, there is no way for it to yield to another thread, unless we call to smth that can yield. and we try to avoid that. | 19:52 |
zzzeek | ihrachyshka: but beyond that, if you already know that’s the fix, I think the pattern should be: “with context.transaction() as session: …. do stuff … context.post_exec(fn, *args)” | 19:52 |
zzzeek | ihrachyshka: yes that was what i was asking | 19:52 |
zzzeek | ihrachyshka: e.g. that when we deal with eventlet, it isn’t like traditional threads that are totlaly non-deterministic | 19:52 |
zzzeek | ihrachyshka: we know when it might be context switching | 19:52 |
zzzeek | ihrachyshka: which is odd! even though this is “implicit” async, it’s still more explicit than regular threads | 19:53 |
ihrachyshka | the fix is proved to be correct in local testing. and adding that post_exec thing is my goal with the spec | 19:53 |
ihrachyshka | yes, eventlet is explicit | 19:53 |
zzzeek | ihrachyshka: yes. i think code like this should have zero SQLAlcehmy-specific transaction code and context managers should handle all that. | 19:53 |
dhellmann | boris-42: the integration tests use master of oslo.messaging, but the unit tests do not | 19:53 |
zzzeek | ihrachyshka: “implicit” vs. “explicit” async is, gevent/eventlet vs. twisted / tornado / node.js | 19:53 |
ihrachyshka | it's either you call eventlet.sleep() or you call a monkey patched functiom, no randomness | 19:54 |
zzzeek | ihrachyshka: gottit | 19:54 |
zzzeek | ihrachyshka: there’s also an after_commit() event on the Session | 19:54 |
zzzeek | ihrachyshka: if you wanted to leverage that | 19:54 |
zzzeek | ihrachyshka: but id make that happen behind the scenes | 19:54 |
zzzeek | ihrachyshka: also, this all should be an oslo.db object | 19:55 |
ihrachyshka | + for after_commit hook, sounds like what I may need | 19:55 |
zzzeek | ihrachyshka: OK but dont litter all the code with @event.listen() :) do that in just one place | 19:56 |
ihrachyshka | what should be oslo.db object? | 19:56 |
zzzeek | ihrachyshka: this may be longer term, but ideally id like projects to say: from oslo.db import transaction; “with tranasction() as session:” | 19:56 |
ihrachyshka | you mean, implement it in oslo.db and then use in neutron? ok | 19:57 |
zzzeek | any kind of database pattern that is invented by a project should be maintained in oslo.db and called from there | 19:57 |
zzzeek | ihrachyshka: yes | 19:57 |
zzzeek | we need to take as much SQLAlchemy “creational” code as possible out of projects | 19:58 |
zzzeek | ihrachyshka: projects will of coruse be using query(), and flush() if they have to, even execute(), but the scoping and framework of things should follow shared patterns | 19:58 |
*** arnaud has joined #openstack-oslo | 19:58 | |
ihrachyshka | ok, so good pointers and discussion so far :) I think I can continue from here. thanks a lot and welcome on openstack board :) we need your expertise like hell. | 19:59 |
boris-42 | dhellmann ok, for me its important that dsvm are using master | 20:00 |
zzzeek | glad i could help | 20:00 |
boris-42 | dhellmann so we will be able to use osprofiler in gates even if there will be no new release of oslo.messaging for a while | 20:00 |
*** tcammann has joined #openstack-oslo | 20:00 | |
dhellmann | boris-42: now that we can make alpha releases, we can do that every week or so | 20:00 |
boris-42 | dhellmann ah ok | 20:01 |
boris-42 | dhellmann so I added extra details in spec that you asked* | 20:01 |
*** dims__ has joined #openstack-oslo | 20:15 | |
tcammann | Hello looking for a core reviewer to +2/+A a change thats been sitting around a while with few +1s https://review.openstack.org/#/c/99695/ | 20:16 |
*** markmcclain has joined #openstack-oslo | 20:16 | |
*** dims_ has quit IRC | 20:18 | |
*** dims__ has quit IRC | 20:20 | |
*** erecio has quit IRC | 20:21 | |
*** ajo|mac has joined #openstack-oslo | 20:29 | |
*** dims__ has joined #openstack-oslo | 20:33 | |
*** ihrachyshka has quit IRC | 20:40 | |
*** dims__ has quit IRC | 20:41 | |
*** ajo|mac has quit IRC | 20:45 | |
*** ajo|mac has joined #openstack-oslo | 20:47 | |
*** pblaho has quit IRC | 20:53 | |
openstackgerrit | A change was merged to openstack/taskflow: Make the expiring cache a top level cache type https://review.openstack.org/98565 | 21:13 |
*** ajo|mac_ has joined #openstack-oslo | 21:13 | |
*** ajo|mac has quit IRC | 21:15 | |
*** SridharG has quit IRC | 21:33 | |
*** dims__ has joined #openstack-oslo | 21:36 | |
*** james_li has quit IRC | 21:37 | |
*** markmcclain has quit IRC | 21:42 | |
*** stannie42 has left #openstack-oslo | 21:49 | |
*** mrda-away is now known as mrda | 22:02 | |
*** zzzeek has quit IRC | 22:14 | |
*** james_li has joined #openstack-oslo | 22:15 | |
*** pcm__ has quit IRC | 22:15 | |
*** dstanek is now known as dstanek_zzz | 22:28 | |
*** james_li has quit IRC | 22:43 | |
*** dims__ has quit IRC | 22:44 | |
*** dstanek_zzz is now known as dstanek | 22:49 | |
*** tcammann has quit IRC | 22:54 | |
*** dims__ has joined #openstack-oslo | 22:58 | |
*** oomichi has joined #openstack-oslo | 23:07 | |
openstackgerrit | Joshua Harlow proposed a change to openstack/taskflow: Add basic WBE validation sanity tests https://review.openstack.org/104016 | 23:19 |
openstackgerrit | OpenStack Proposal Bot proposed a change to openstack/oslo-incubator: Updated from global requirements https://review.openstack.org/104022 | 23:36 |
openstackgerrit | Joshua Harlow proposed a change to openstack/oslo.messaging: Add a thread + futures executor based executor https://review.openstack.org/70914 | 23:42 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!