Monday, 2016-03-14

*** Eva-i has left #openstack-oslo00:16
*** zz_dimtruck is now known as dimtruck00:23
*** dims_ has joined #openstack-oslo00:27
*** dims has quit IRC00:28
*** vilobhmm11 has joined #openstack-oslo00:28
*** dims has joined #openstack-oslo00:33
*** harlowja_at_home has joined #openstack-oslo00:33
*** dims_ has quit IRC00:33
*** dimtruck is now known as zz_dimtruck00:39
*** EinstCrazy has quit IRC00:40
openstackgerritDavanum Srinivas (dims) proposed openstack/oslo.messaging: Bump rabbit_transient_queues_ttl to 30 mins  https://review.openstack.org/29218100:44
*** david-lyle has quit IRC00:54
*** david-lyle has joined #openstack-oslo00:54
*** _amrith_ is now known as amrith01:35
*** suro-patz has quit IRC01:46
openstackgerritDavanum Srinivas (dims) proposed openstack/oslo.messaging: [WIP] Bump rabbit_transient_queues_ttl to 30 mins  https://review.openstack.org/29218101:49
*** zz_dimtruck is now known as dimtruck02:05
*** dims has quit IRC02:07
openstackgerritDavanum Srinivas (dims) proposed openstack/oslo.messaging: [WIP] Bump rabbit_transient_queues_ttl to 30 mins  https://review.openstack.org/29218102:11
*** dims has joined #openstack-oslo02:21
*** amrith is now known as _amrith_02:23
*** vilobhmm11 has quit IRC02:27
*** dims has quit IRC02:41
*** gcb has quit IRC02:43
*** gcb has joined #openstack-oslo02:46
*** yamamoto has quit IRC02:55
*** dimtruck is now known as zz_dimtruck03:02
openstackgerritChangBo Guo(gcb) proposed openstack/osprofiler: doc: Log warning when can't get informaiton from git  https://review.openstack.org/29219103:08
*** zz_dimtruck is now known as dimtruck03:12
openstackgerritChangBo Guo(gcb) proposed openstack/osprofiler: Dont claim copyright for future years  https://review.openstack.org/29219203:17
*** gcb has quit IRC03:18
*** gcb has joined #openstack-oslo03:32
*** gcb has quit IRC03:36
*** dkehn has quit IRC03:45
openstackgerritMerged openstack/oslo.config: Fix typo error in docstrings of oslo_config.cfg.py  https://review.openstack.org/29172203:48
*** links has joined #openstack-oslo03:50
*** dkehn has joined #openstack-oslo03:51
*** dimtruck is now known as zz_dimtruck03:52
openstackgerritMerged openstack/oslo.config: Explicitly exclude tests from bandit scan  https://review.openstack.org/29205303:56
*** yamamoto has joined #openstack-oslo03:56
openstackgerritMerged openstack/oslo.utils: Explicitly exclude tests from bandit scan  https://review.openstack.org/29205503:57
*** ekarlso- has quit IRC04:00
*** mtanino has quit IRC04:02
*** harlowja_at_home has quit IRC04:04
*** andymaier has joined #openstack-oslo04:07
*** ekarlso- has joined #openstack-oslo04:14
*** gcb has joined #openstack-oslo04:21
*** andymaier has quit IRC04:23
*** zz_dimtruck is now known as dimtruck04:27
openstackgerritChangBo Guo(gcb) proposed openstack/osprofiler: Remove outdated version  https://review.openstack.org/29220304:44
*** otherwiseguy has quit IRC05:02
*** boris-42 has joined #openstack-oslo05:06
*** vilobhmm11 has joined #openstack-oslo05:08
*** otherwiseguy has joined #openstack-oslo05:12
*** _amrith_ is now known as amrith05:29
*** EinstCrazy has joined #openstack-oslo05:31
*** EinstCrazy has quit IRC05:38
*** amrith is now known as _amrith_05:39
*** ihrachys has joined #openstack-oslo06:02
openstackgerritFeng Xi Yan proposed openstack/oslo.vmware: Add a locale param to allow vSphere SDK locale specification.  https://review.openstack.org/21875506:06
*** dimtruck is now known as zz_dimtruck06:19
*** ihrachys_ has joined #openstack-oslo06:19
*** ihrachys has quit IRC06:21
*** zz_dimtruck is now known as dimtruck06:23
*** dimtruck is now known as zz_dimtruck06:36
*** pcaruana has quit IRC06:39
*** vilobhmm111 has joined #openstack-oslo06:42
*** vilobhmm11 has quit IRC06:44
*** ihrachys has joined #openstack-oslo06:46
*** ihrachys_ has quit IRC06:47
*** ihrachys_ has joined #openstack-oslo06:53
*** ihrachys has quit IRC06:55
*** nkrinner has joined #openstack-oslo06:58
*** ihrachys_ has quit IRC07:04
*** patchbot has quit IRC07:04
*** patchbot` has joined #openstack-oslo07:04
*** patchbot` is now known as patchbot07:05
*** dmitryme2 has joined #openstack-oslo07:14
*** rpodolyaka_ has joined #openstack-oslo07:14
*** isq_ has quit IRC07:15
*** pkholkin__ has quit IRC07:15
*** eezhova has quit IRC07:15
*** sileht has quit IRC07:15
*** szaher has quit IRC07:15
*** rpodolyaka has quit IRC07:15
*** dmitryme has quit IRC07:15
*** mkoderer has quit IRC07:15
*** dstanek has quit IRC07:15
*** dmitryme2 is now known as dmitryme07:15
*** rpodolyaka_ is now known as rpodolyaka07:15
*** isq_ has joined #openstack-oslo07:16
*** pkholkin__ has joined #openstack-oslo07:16
*** eezhova has joined #openstack-oslo07:16
*** szaher has joined #openstack-oslo07:16
*** mkoderer has joined #openstack-oslo07:16
*** dstanek has joined #openstack-oslo07:16
*** sileht has joined #openstack-oslo07:16
*** ihrachys has joined #openstack-oslo07:30
*** tesseract has joined #openstack-oslo07:52
*** tesseract is now known as Guest5918707:52
*** vilobhmm111 has quit IRC08:03
*** vilobhmm11 has joined #openstack-oslo08:04
*** rcernin has joined #openstack-oslo08:25
*** dguitarbite has joined #openstack-oslo08:27
*** dtardivel has joined #openstack-oslo08:27
*** pcaruana has joined #openstack-oslo08:43
haypomorning08:46
*** permalac has joined #openstack-oslo08:50
*** vilobhmm11 has quit IRC09:01
*** gcb has quit IRC09:05
*** mhickey has joined #openstack-oslo09:06
*** ndipanov has joined #openstack-oslo09:14
*** gcb has joined #openstack-oslo09:18
*** lucas-afk is now known as lucasagomes09:19
*** yassine__ has joined #openstack-oslo09:27
*** pblaho has joined #openstack-oslo09:33
*** e0ne has joined #openstack-oslo09:52
*** gcb has quit IRC09:53
*** dims has joined #openstack-oslo10:09
*** sdague has joined #openstack-oslo10:13
*** dims has quit IRC10:21
*** cdent has joined #openstack-oslo10:28
*** dims has joined #openstack-oslo10:30
*** dims has quit IRC10:39
*** dims has joined #openstack-oslo10:40
*** e0ne has quit IRC10:45
*** yassine__ has quit IRC10:55
*** e0ne has joined #openstack-oslo10:56
openstackgerritMerged openstack/osprofiler: Remove outdated version  https://review.openstack.org/29220311:01
*** takedakn has joined #openstack-oslo11:02
*** _amrith_ is now known as amrith11:15
*** cdent has quit IRC11:17
*** ihrachys has quit IRC11:18
*** e0ne has quit IRC11:21
openstackgerritDavanum Srinivas (dims) proposed openstack/oslo.messaging: Bump rabbit_transient_queues_ttl to 30 mins  https://review.openstack.org/29218111:24
dimssileht : i was tracking this down all weekend. ^^11:25
dimsdmitryme : please take a look too11:25
silehtdims, well done11:26
*** e0ne has joined #openstack-oslo11:26
dimssileht : hopefully it will help11:27
*** cdent has joined #openstack-oslo11:27
dimsthanks sileht11:27
*** gordc has joined #openstack-oslo11:35
*** cdent_ has joined #openstack-oslo11:37
dimsgordc : dukhlov : does this look good to you? https://review.openstack.org/#/c/292181/11:37
patchbotdims: patch 292181 - oslo.messaging - Bump rabbit_transient_queues_ttl to 30 mins11:37
*** cdent has quit IRC11:37
*** cdent_ is now known as cdent11:37
*** ujjain- is now known as ujjain11:39
dimsdukhlov : ozamiatin_ : sileht : gordc : dmitryme : Now that we have stable/mitaka, i'd like to see what's the minimum reviews we absolutely need that was not in the last release of oslo.messaging and came up with these two - https://review.openstack.org/#/q/is:open+project:openstack/oslo.messaging+branch:stable/mitaka11:40
dimsdukhlov : ozamiatin_ : sileht : gordc : dmitryme : Are there others? (Objective is to make sure rabbitmq+kombu is production ready), the zmq and pika driver we can wait a bit for a release11:41
*** cdent has quit IRC11:41
silehtI have nothing in mind11:41
*** cdent has joined #openstack-oslo11:42
*** binarin has joined #openstack-oslo11:43
gordcdims: those two seem fine to me. not tracking anything else.11:44
*** cdent has quit IRC11:45
*** dims_ has joined #openstack-oslo11:45
*** dims has quit IRC11:47
dims_binarin : long story, but here's the short version, in https://review.openstack.org/#/c/243845/ we 'x-expires' in a queue and specified a ttl11:47
patchbotdims_: patch 243845 - oslo.messaging - Kombu: make reply and fanout queues expire instead... (MERGED)11:47
dims_binarin : so when exactly does the the queue expire? is the time counted from the last time something was sent using that queue?11:49
dims_binarin : what i see is that the ttl clock seems to start when the queue is created, not when it's last used11:50
dims_binarin : can you please check when you get a chance?11:50
binarindims_: Yes, I'm already looking into the rabbit source11:50
dims_thanks binarin : documentation says "Unused means the queue has no consumers, the queue has not been redeclared, and basic.get has not been invoked for a duration of at least the expiration period." https://www.rabbitmq.com/ttl.html11:51
*** cdent has joined #openstack-oslo11:51
dims_binarin : temporarily, i've bumped ttl in https://review.openstack.org/#/c/292181/ hopefully when it gets into a release it will help11:55
patchbotdims_: patch 292181 - oslo.messaging - Bump rabbit_transient_queues_ttl to 30 mins11:55
*** yassine__ has joined #openstack-oslo11:57
*** yamamoto has quit IRC12:00
*** ihrachys has joined #openstack-oslo12:01
*** ihrachys has quit IRC12:03
dmitrymedims: +1’ed https://review.openstack.org/#/c/292181/ . But frankly I hardly understand, how it happens that a queue lives 10 minutes without consumer.12:03
patchbotdmitryme: patch 292181 - oslo.messaging - Bump rabbit_transient_queues_ttl to 30 mins12:03
dmitrymeand nothing to add to stable/liberty from my side12:03
dims_dmitryme : right12:03
*** ihrachys has joined #openstack-oslo12:03
*** kgiusti has joined #openstack-oslo12:07
openstackgerritjaveme proposed openstack/oslo.messaging: Missing version parameter in can_send_version()  https://review.openstack.org/29038512:13
*** amrith is now known as _amrith_12:19
*** claudiub has joined #openstack-oslo12:27
*** edmondsw has joined #openstack-oslo12:31
*** boris-42 has quit IRC12:34
*** MVenesio has joined #openstack-oslo12:40
openstackgerritMerged openstack/oslo.messaging: Pika: fix sending fanout messages  https://review.openstack.org/28943012:40
*** links has quit IRC12:46
*** links has joined #openstack-oslo12:46
*** yamamoto has joined #openstack-oslo12:48
*** mtanino has joined #openstack-oslo12:52
*** lucasagomes is now known as lucas-hungry12:54
*** cdent has quit IRC13:01
*** cdent has joined #openstack-oslo13:02
*** EinstCrazy has joined #openstack-oslo13:04
*** rlrossit has joined #openstack-oslo13:05
*** cdent has quit IRC13:06
*** _amrith_ is now known as amrith13:16
*** takedakn has quit IRC13:21
dims_folks, please check your local times :) we are meeting at 1600 UTC13:29
*** krotscheck has joined #openstack-oslo13:32
*** gcb has joined #openstack-oslo13:33
*** cdent has joined #openstack-oslo13:35
*** jpena is now known as jpena|lunch13:48
krotscheckdhellmann: Just went through and rechecked the various patches that failed in the cors bug. Other than that, it looks like most of them are landing.13:52
*** sigmavirus24_awa is now known as sigmavirus2414:00
*** ig0r_ has joined #openstack-oslo14:02
*** lucas-hungry is now known as lucasagomes14:03
*** mriedem has joined #openstack-oslo14:04
*** boris-42 has joined #openstack-oslo14:05
*** amrith is now known as _amrith_14:06
*** _amrith_ is now known as amrith14:06
*** salv-orlando has joined #openstack-oslo14:08
*** zz_dimtruck is now known as dimtruck14:12
*** david-lyle has quit IRC14:16
*** david-lyle has joined #openstack-oslo14:19
*** yamamoto has quit IRC14:25
*** pcaruana has quit IRC14:28
*** ansiwen has quit IRC14:29
*** mriedem is now known as mriedem_meeting14:30
openstackgerritChangBo Guo(gcb) proposed openstack/oslo.db: Remove  sqlalchemy < 1.0.0 compatible layer  https://review.openstack.org/29241314:31
*** yamahata has quit IRC14:31
*** ansiwen has joined #openstack-oslo14:33
dhellmannkrotscheck : thanks for staying on top of those14:33
*** mtanino has quit IRC14:34
*** yamamoto has joined #openstack-oslo14:35
*** yamamoto has quit IRC14:36
krotscheckdhellmann: Of course.14:36
*** binarin has quit IRC14:37
openstackgerritOleksii Zamiatin proposed openstack/oslo.messaging: [zmq] Refactoring consumer side  https://review.openstack.org/28304914:41
*** gcb has quit IRC14:41
openstackgerritOleksii Zamiatin proposed openstack/oslo.messaging: [zmq] Refactoring consumer side  https://review.openstack.org/28304914:41
*** gcb has joined #openstack-oslo14:43
*** mriedem_meeting is now known as mriedem14:49
dims_oslo-cores, please help me fill in https://etherpad.openstack.org/p/mitaka-oslo-achievements14:49
haypodims_: is the death of openstack/common/ in oslo-incubator an archivement? :)14:51
openstackgerritJulien Danjou proposed openstack/oslo.middleware: cors: prevent WebOb setting a default Content-Type  https://review.openstack.org/29048214:51
dims_haypo : goes into the last bucket :)14:52
haypo(in fact, i'm not sure that it's 100% dead)14:52
haypodims_: is it dead or is it still moving?14:52
*** yamamoto has joined #openstack-oslo14:52
dims_haypo : we can say needs cleanup from projects14:53
dims_haypo : we have released all the code as libs. what's left is cleanup14:53
krotscheckdhellmann: Just to confirm - that default config hook name has to be unique, but the actual string doesn't matter, yes?14:54
*** jpena|lunch is now known as jpena14:56
dhellmannkrotscheck : the string does matter. it must match the name of one of the plugins registered for the application to return a list of options14:58
krotscheckdhellmann: ....RIGHT14:58
* krotscheck goes back to confirm his patches.14:58
dhellmannkrotscheck : if a project has 2 apps, and they want different defaults, it can register 2 functions and have 2 input files to the config generator using the different names14:58
dhellmanns/using/containing14:58
rbradforhaypo, has all code become libraries? from looking at what's still in use some does not seem to have an obvious migration path15:06
hayporbradfor: no. cliutils.py simply died :)15:06
hayporbradfor: it was decided to keep unsynchronized copies in each projects, and these projects must be updated to cliff15:07
rbradforhaypo, do you see all the client libs moving to cliff.  there is probably 15+ using cliutils15:10
dims_krotscheck : wanna add CORS stuff to https://etherpad.openstack.org/p/mitaka-oslo-achievements ?15:11
krotscheckdims_: Sure!15:11
dims_haypo : something about python3 readiness?15:11
hayporbradfor: sorry, is it a question?15:11
krotscheckdims_: New features?15:12
haypodims_: oslo was already ready in the previous cycle. but i added some helpers to make porting easier15:12
dims_krotscheck : sure15:12
dims_haypo : ok, counts as new features? :)15:12
dims_rbradfor : what else can we add?  https://etherpad.openstack.org/p/mitaka-oslo-achievements ?15:12
*** amotoki has quit IRC15:14
*** ozamiatin has joined #openstack-oslo15:16
rbradfordims_, cannot say a lot of my small contributions getting started this cycle are noteworthy15:23
*** links has quit IRC15:23
*** ig0r__ has joined #openstack-oslo15:23
*** regXboi has joined #openstack-oslo15:25
*** ig0r_ has quit IRC15:26
rbradforlittle consolation, however it does help me with what's important for a full newton cycle.15:29
*** mriedem is now known as mriedem_meeting15:30
*** yamamoto has quit IRC15:30
*** salv-orlando has quit IRC15:31
*** salv-orlando has joined #openstack-oslo15:33
*** harlowja_at_home has joined #openstack-oslo15:38
*** AJaeger has joined #openstack-oslo15:53
*** gcb has quit IRC15:54
dims_4 mins to oslo meeting15:56
openstackgerritMehdi Abaakouk (sileht) proposed openstack/oslo.serialization: msgpack: fix datetime serialization  https://review.openstack.org/29171515:56
harlowja_at_homemeeting!15:57
*** jecarey has joined #openstack-oslo15:57
harlowja_at_homesileht, thanks for fixing those, guess bugs i never saw :-P16:00
harlowja_at_home(when making said code)16:00
*** pcaruana has joined #openstack-oslo16:09
*** jaypipes has joined #openstack-oslo16:10
*** cdent has quit IRC16:12
*** nkrinner has quit IRC16:13
*** EinstCrazy has quit IRC16:14
*** mriedem_meeting is now known as mriedem16:21
*** yamamoto has joined #openstack-oslo16:31
*** Guest59187 has quit IRC16:31
hayposileht: i'm not sure  that https://review.openstack.org/#/c/291715/ is 100% safe, but i'm ok to come back to the code later if someone complains ;)16:34
patchbothaypo: patch 291715 - oslo.serialization - msgpack: fix datetime serialization16:34
hayposileht: (i just approved your change)16:34
silehthaypo: thx16:35
*** cdent has joined #openstack-oslo16:39
*** yamamoto has quit IRC16:41
*** sigmavirus24 is now known as sigmavirus24_awa16:46
hayposileht: it looks like timezone (used in OpenStack) are always encoded to ASCII16:48
hayposileht: hopefully, the issue goes away with python 3 :-D16:49
*** rcernin has quit IRC16:56
*** sputnik13 has joined #openstack-oslo17:04
*** vilobhmm11 has joined #openstack-oslo17:07
*** suro-patz has joined #openstack-oslo17:07
*** yamahata has joined #openstack-oslo17:07
*** dims has joined #openstack-oslo17:08
*** vilobhmm111 has joined #openstack-oslo17:09
*** dims_ has quit IRC17:09
*** vilobhmm11 has quit IRC17:11
*** yassine__ has quit IRC17:15
*** e0ne has quit IRC17:17
openstackgerritMerged openstack/oslo.messaging: Bump rabbit_transient_queues_ttl to 30 mins  https://review.openstack.org/29218117:40
*** jecarey_ has joined #openstack-oslo17:50
*** alejandrito has joined #openstack-oslo17:51
harlowja_at_homedims, http://lists.openstack.org/pipermail/openstack-dev/2016-March/089276.html (and others)17:51
harlowja_at_homeincluded a few newton references, lol17:52
dimslol17:52
*** jecarey has quit IRC17:53
*** salv-orl_ has joined #openstack-oslo17:54
*** salv-orlando has quit IRC17:57
harlowja_at_homeya, whoever names these summits needs to know all the secret meanings of these words, ha17:57
openstackgerritMerged openstack/oslo.serialization: msgpack: fix datetime serialization  https://review.openstack.org/29171517:57
harlowja_at_homefig newton, iaasac newton, apple newton17:58
harlowja_at_homeprobably other newtons, lol17:58
harlowja_at_homealthough 'apple newton' shouldn't be used, unless u want to talk about a failure17:58
* harlowja_at_home had one of those once17:58
*** browne has joined #openstack-oslo17:58
harlowja_at_homeapples first attempt at a pda that failed (iphone finally worked out)17:58
harlowja_at_homehttps://en.wikipedia.org/wiki/Apple_Newton17:58
*** salv-orl_ has quit IRC17:59
harlowja_at_homewayne newton18:00
harlowja_at_homelol18:00
dimshaha18:02
*** jpena is now known as jpena|off18:05
AJaegerfungi, rpodolyaka, zzzeek, dims: We (=OpenStack Infra) currently always set up for python27, python34 etc jobs MySQL and PostgreSQL adatabases. We want to only setup databases when needed and I need some help from you.18:05
AJaegerIs a project using oslo.db always needing a setup database? Or how to determine if one needs it?18:06
AJaegerI mean: During the test runs...18:06
*** vilobhmm111 has quit IRC18:07
fungiideal would be some pattern we can look for to tell whether projects are likely relying on opportunistic database testing with fixtures/framework support provided by oslo.db18:07
*** ihrachys has quit IRC18:10
*** e0ne has joined #openstack-oslo18:11
*** ozamiatin has quit IRC18:12
*** harlowja_at_home has quit IRC18:14
*** mhickey has quit IRC18:15
kevinbentonzzzeek: what do you suggest I do for https://review.openstack.org/#/c/291795/ ? The session has already been rolled back by the transaction context manager18:16
patchbotkevinbenton: patch 291795 - neutron - Call expunge_all on session before retrying requests18:16
*** permalac has quit IRC18:19
kevinbentonzzzeek: for context, this is the leftover object in the session that i was chatting with you about the other day that would go away when using pdb18:20
*** lucasagomes is now known as lucas-dinner18:25
*** cdent has quit IRC18:37
*** yamahata has quit IRC18:40
*** yamahata has joined #openstack-oslo18:40
*** ozamiatin has joined #openstack-oslo18:44
zzzeekkevinbenton: added a comment to the launchpad issue illustrating weakrefs should not matter18:47
*** vilobhmm11 has joined #openstack-oslo18:51
*** cdent has joined #openstack-oslo18:55
kevinbentonzzzeek: managed to repro with your example and a relationship. left code on the bug report19:18
zzzeekok19:18
zzzeekkevinbenton: passive_deletes = 'all' == probably wrong, why is that there?19:19
kevinbentonzzzeek: is that the cause. it's something that has been in the neutron model for quite some time. i will need to git blame to see why it's there19:20
kevinbentonzzzeek: but if that's it, that's great news because we can probably just dump it19:20
zzzeekkevinbenton: yes, because SQLA has no idea you're deleting those19:20
zzzeekkevinbenton: it should be:  cascade='all, delete-orphan' and passive_deletes=True19:20
kevinbentonzzzeek: ok. cool. let me figure out why that was done19:21
kevinbentonzzzeek: thx19:21
zzzeekkevinbenton: whoever put passive_deletes='all' there was basically doing the "try everything until it works" solution which is what I'm trying to interrupt here :)19:22
*** claudiub|2 has joined #openstack-oslo19:22
*** ozamiatin has quit IRC19:23
*** claudiub has quit IRC19:25
kevinbentonzzzeek: it looks like it was done as some of the quota work https://review.openstack.org/#/c/188883/19:25
patchbotkevinbenton: patch 188883 - neutron - Introduce usage data tracking for Neutron (MERGED)19:25
kevinbentonzzzeek: unfortunately nobody asked about it in the code review so i will need to get in touch with Salvatore to figure out why he added it19:26
kevinbentonzzzeek: (fixed_ips is the relationship that is causing the issue)19:27
*** suro-patz1 has joined #openstack-oslo19:28
*** suro-patz has quit IRC19:29
kevinbentonzzzeek: this still fails. http://paste.openstack.org/show/490430/19:36
*** fhermeni has joined #openstack-oslo19:37
kevinbentonzzzeek: it even fails with no passive_deletes option19:37
kevinbentonzzzeek: and also with no cascade option19:37
*** yamahata has quit IRC19:38
fhermenihi19:38
zzzeekkevinbenton: looking19:44
zzzeekkevinbenton: ok that is a much more subtle variant on the same thing19:46
zzzeekand ive actually never seen that one before19:46
kevinbentonzzzeek: the wonky thing is the lookup using the relationship19:47
zzzeekkevinbenton: well sort of19:47
zzzeekkevinbenton: i know what this is, and it is very easy to resolve, but im not sure what i can tell you about this never happening b.c. i think openstack's patterns lend themselves to this kind of thing19:48
zzzeekkevinbenton: if you do:19:48
zzzeek        s.flush()19:48
zzzeek        b1 = B(a_id=1, address='1234')19:48
zzzeek        s.add(b1)19:48
zzzeekthen its fine19:48
zzzeekbecause19:48
zzzeekit knows that b1 is part of that transaction.19:48
zzzeekif you do:  s.add(B(...))19:48
zzzeekthen it gets gc'ed19:48
zzzeekthen you load it again19:48
zzzeekthe Session has no idea it was the one you just added in that transaction19:48
zzzeekkevinbenton: there's something to be improved here in SQLA possibly, not usre19:50
kevinbentonzzzeek: ack. unfortunately holding that reference will have to carry through lots of code because the record creation for an allocation is isolated to a function https://github.com/openstack/neutron/blob/master/neutron/db/db_base_plugin_common.py#L94-L10919:52
zzzeekkevinbenton: yes i figured that's whats happening19:52
zzzeekkevinbenton: for your decorator I'd just put session.close() at the end19:53
zzzeekas long as its run at the toplevel transaction19:53
zzzeekif its doing savepoints then you'd not want that19:53
kevinbentonzzzeek: ack. it's not doing savepoints. we have been good about decorating at the very top-level19:53
zzzeekyeah id just close() it19:54
zzzeekor even just make a new Session19:54
kevinbentonzzzeek: yeah, at one point i was just discarding the session on the context so a new one would get created19:54
kevinbentonzzzeek: but it felt a little heavy-handed19:54
kevinbentonzzzeek: at least now i understand why it's happening19:55
kevinbentonzzzeek: and have a better justification for it19:55
zzzeekkevinbenton: on the SQLA side we don't hold onto thigns that arent referenced brcause if someone is inserting 100K recs it would be foolish for us to make strong refs19:55
zzzeekkevinbenton: you *can* just throw it in session.info also19:56
*** mtanino has joined #openstack-oslo19:56
kevinbentonzzzeek: ah, that might be less invasive19:56
zzzeekkevinbenton: e.g. this could be even on SQLA side "make sure this rolls back" type of thing19:56
zzzeekkevinbenton: but it will look strange to people19:56
kevinbentonzzzeek: well it will be isolated to one particular part of the code with a nice TODO to clean this up later19:56
zzzeekkevinbenton: OK, im jsut not sure what the long term solution to this one wuld be19:57
kevinbentonzzzeek: the decorator modification is messier because the decorator doesn't know which argument is the context19:57
kevinbentonzzzeek: but i think it makes sense to have a clean session in the long run just as a way to completely ensure everything is started from scratch19:57
*** dtardivel has quit IRC19:58
*** claudiub|2 has quit IRC20:01
zzzeekkevinbenton: so this works20:03
zzzeek        s.flush()20:03
zzzeek        b1 = B(a_id=1, address='1234')20:03
zzzeek        s.add(b1)20:03
zzzeek        s.info['my_b1'] = b120:03
zzzeek        del b120:03
*** ig0r_ has joined #openstack-oslo20:03
kevinbentonzzzeek: yep, just modified the logic in ipam to add it and it works as well20:05
kevinbentonzzzeek: adding a note now20:05
zzzeekkevinbenton: OK.  i just find it to bea shortcoming in SQLA that one even has to worry about a ref here20:05
*** ig0r__ has quit IRC20:07
kevinbentonzzzeek: can you leave a comment on the bug explaining that the root cause is because this is GC'ed?20:07
*** salv-orlando has joined #openstack-oslo20:08
*** dimtruck is now known as zz_dimtruck20:08
AJaegerzzzeek: did you see my question in the backscroll and can help infra, please?20:09
zzzeekkevinbenton: i cna but im doing 8 things at onec here, if you want to give it a shot20:10
kevinbentonzzzeek: no prob20:10
zzzeekAJaeger: my backscroll is unreliable let me check20:10
*** amrith is now known as _amrith_20:10
zzzeekAJaeger: negatory, if you want to repaste for me otherwise i have to go look ona  server at the logs20:11
zzzeekAJaeger: oh here, i was looking on openstack-infra20:11
zzzeekAJaeger: yes20:11
*** zz_dimtruck is now known as dimtruck20:12
zzzeekAJaeger: oslo.db has a front-to-back version of this but you have to buy into it a little bit20:12
*** MVenesio has quit IRC20:12
zzzeekAJaeger: well i guess you could use provisioning externally too... would need to know what you're trying to do20:13
*** kgiusti has quit IRC20:13
AJaegerzzzeek: here ;) Shall I repaste?20:17
*** AJaeger has quit IRC20:17
*** dguitarbite has quit IRC20:17
*** AJaeger has joined #openstack-oslo20:17
AJaegerzzzeek: sorry, lost connectivty but read your comment on eavesdrop20:17
AJaegerzzzeek, the problem that fungi and myself try to solve is: Currently python27 jobs setup databases for each repository.20:18
AJaegerBut most repos don't need a database for their python27 jobs.20:18
zzzeekAJaeger: yes are these tempest jobs20:18
AJaegernot tempest, we talk about unittests20:18
zzzeekAJaeger: OK20:18
AJaegerSo, gate-nova-python2720:18
AJaegerhow can we figure out which repos need database access - so that only these get database setup?20:19
zzzeekAJaeger: so how do we define what "setup" means?20:19
zzzeekAJaeger: just "CREATE DATABASE" "DROP DATABASE", or actualyl running a DB in some kind of container or something?20:20
AJaegersetup means: We create a gate-nova-python27-db job that sets up the database access as it's done right now and remove it from gate-nova-python2720:20
zzzeekAJaeger: OK is there a DB server already running, and you create a username, or theyre using openstack-citest ?20:21
AJaegersetup is http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/macros.yaml#n64320:21
fungiyeah, we install mysql and postgresql servers, precreate openstack_citest accounts with basically full database control, and also precreate openstack_citest databases which some jobs use20:21
AJaegerand http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/macros.yaml#n68120:21
fungiwondering whether there are some projects relying on testing features of oslo.db which we need to spot separately from just grepping for "openstack_citest" in their repos directly20:22
*** thingee has joined #openstack-oslo20:22
zzzeekAJaeger / fungi OK if the DB is already running, and I assume the test suite gets exclusive control of that DB, what is the downside to just having a persistent openstack_citest user?  concern that the DB messes up the user or some kind of security concern ?20:23
thingeefhermeni: hey are you interested in talking about this with a wider audience tomorrow at the cross-project meeting? https://wiki.openstack.org/wiki/Meetings/CrossProjectMeeting20:23
AJaegerzzzeek: we want to allow projects to install their own package lists. Right now since we always setup the database, they would need to install database packages even if they don't need them.20:24
fungizzzeek: we're whittling down what we preconfigure on our base images. we don't install any database services on them at all now, but we've temporarily worked around not knowing which jobs need database support to run their tests by having all unit test and coverage jobs install mysql and portgres and set them up similarly to how our old-style images did20:24
*** dguitarbite has joined #openstack-oslo20:25
AJaegerSo, we like to make it  possible that a project that doesn't need a database, has the option to install a package set iwthout database20:25
fungier, s/portgres/postgres/20:25
kevinbentonzzzeek: https://review.openstack.org/291795 thanks for all of your help as always!20:26
fungizzzeek: so the reason we're asking is that we're trying to figure out which jobs really do make use of these preconfigured databases. one way is to look for "openstack_citest" in all their repos and try to investigate from that angle, but we're worried there may be indirect use through oslo.db too20:27
fungiso we want to make sure we find those as well if so20:27
fhermenithingee: Hi thingee. I think I can, but Joshua is a bit more used to such procedures first20:27
zzzeekfungi: OK well if you have scripts that run outside of a test, and you want to be handed a repo and generically determine what kinds of databases that suite will want to see, that repo would have to publish that metadata somehow20:28
zzzeekfungi: i wouldn't want to do it by guessing20:28
zzzeekfungi: id say we'd be looking in either pbr / setup.cfg / tox.ini something like that for a known indicator20:28
thingeefhermeni: ok, either of you can lead it. Just need someone that understands the subject to brief introduce it and answer questions to the group.20:28
thingeefhermeni: it would also be good to identify the immediate projects this involves.20:29
zzzeekfungi: otherwise if you're looking for heuristics that would tell you from looking at the code itself if it needs a DB then I can figure that out for you but that seems a little more brittle20:29
fhermenithingee: It will be 9:00 UTC, so 10 am in France. My kid will be asleep, so I will be able to attend anyway20:29
fungizzzeek: i'll try to rephrase again20:29
*** jecarey_ has quit IRC20:29
fungizzzeek: does oslo.db provide opportunistic database testing fixtures/framework that may assume the presence of a database?20:30
zzzeekfungi: there is probably a function in there called "has_database" or somethign like that, i can check20:30
fhermenithingee: ok. I can discuss about the objective and what we had in mind. Some questions might however be too technical for me wrt. my knowlegde about OS architecture and principles. Joshua matters then for that second point20:30
zzzeekfungi: there's also the whole test harness that will run tests only if such a database is present20:31
thingeefhermeni: actually 21:00 utc so it'll be late in france20:31
zzzeekfungi: but that sounds like the opposite of what you're asking20:31
dimsfungi : keys off this environment variable http://codesearch.openstack.org/?q=OS_TEST_DBAPI_ADMIN_CONNECTION&i=nope&files=&repos=20:31
thingee22:00 I believe20:31
zzzeekfungi: like, if you run oslo.db tests on a machien that doesnt have mysql or pg installed, it detects that and skips all those tests20:31
fungizzzeek: thanks. and yeah that's what i was mostly asking about. because if we try to run these jobs without a database to discover whether or not they use one, then opportunistic testing will cause us to silently strip database tests upstream20:31
zzzeekfungi: all of the openstack projects that test against mysql/pg to my knowledge have such a system20:32
*** vilobhmm111 has joined #openstack-oslo20:32
fhermenithingee: indeed but sooner would then have been difficult as it is still partly a “family time”. So 22:00 is ok to me20:32
zzzeektehy all opportunistically check for the DB being present and should skip tests if not20:32
fungizzzeek: oslo.db tests but also tests of other projects using oslo.db which may use test features from oslo.db to the same end20:32
thingeefhermeni: excellent, I'll add it to the agenda. I will ping both you and josh20:32
zzzeekfungi: other openstack projets which use MySQLOpportunisticTestBase and PosgresqlOpportunisticTestBAse, yes20:32
fungizzzeek: eventually, i want to start a broader discussion across the project about figuroug out how to disable opportunism in tests, since it's terribly fragile20:33
zzzeekfungi: most of them use that, though not consistently20:33
AJaegerso, should we then say: If oslo.db is in requirements, let's assume it uses databases?20:33
dimsfungi : zzzeek : we look for urls in OS_TEST_DBAPI_ADMIN_CONNECTION env variable and check if any one of them works and then use that20:33
zzzeekAJaeger: sure20:33
dimsto run opportunistic tests20:33
fungiAJaeger: that seems like a safe addition to our initial list20:33
fungithanks zzzeek and dims!20:34
*** vilobhmm11 has quit IRC20:34
zzzeekAJaeger: there's the tiniest chance they only use SQLite but that is not currently the case for any projects20:34
AJaegerdo we need to check whether a project is pulling in oslo.db indirectly?20:34
fhermenithingee: about the people involved by the spec, it matters to exchange will people that are providing schedulers to ensure the language allows to express their feature and people talking to the scheduler to check for the language practical aspects20:35
*** edmondsw has quit IRC20:35
fungii'd love to come up with a way to make it possible for devs to skip database tests easily but still force them to run in our ci so that absence of database setup doesn't just cause them to be silently skipped20:35
fungiAJaeger: probably any project which declares a requirement on oslo.db, and any project qhich declares a requirement on any of those requirements, recursively until we find teh closed set20:36
*** dims_ has joined #openstack-oslo20:36
* AJaeger hates these closure calculations ;(20:36
lifelessfungi: env variables ftw20:37
thingeefhermeni: in the cross-project specs you want to get consensus on a concept. This can include technical details, but some implementation details can be handled within each project on their own. That doesn't fall on you to do each spec, but someone on the team that understands the project well to draft. You just want agreement at a higher level from the OpenStack projects.20:37
thingeeso each project this would matter to on agree. Just the immediate ones that come to mind. We can figure out the full list in meeting since each project should have a rep available.20:37
*** dims has quit IRC20:38
fungilifeless: right, we likely need a cross-project spec and/or pti update to provide an appropriate model for how to default database tests (and any other sort of currently opportunistic testing) to on and still be able to disable it with an envvar or whatever20:38
thingeefhermeni: in addition, the hope with cross-project specs is to surface existing solutions to solve to implement the concept. To make sure everyone knows and doesn't go off duplicating effort in their own project.20:39
AJaegerfungi, only 52 repos in openstack namespace import oslo.db directly20:39
dims_haha "only" :)20:39
fungithat's still quite a few20:39
fungidims_: well, that's out of 1010 repos we have in gerrit now20:39
fhermenithingee: looks ok to me20:39
fungiso ~0.5%20:40
dims_fungi : wow20:40
fungisorry, 5%20:40
AJaegerfungi: 5 percent!20:40
fungiindeed. i math terribly20:40
AJaegerbut you noticed quickly ;)20:40
fungii tend to recheck my calculations, i just often hit enter before i do so :/20:41
lifelessfungi: I was more thinking20:42
lifelessfungi: we define a variable meaning 'skipping db tests are errors'20:43
AJaegerfungi, dims_ , zzzeek http://paste.openstack.org/show/490434/ is result of grep20:43
AJaegerlifeless: yeah!20:43
lifelessfungi: and then look in the subunit output for matching skips20:43
lifelessfungi: so there's no per-project changes to do20:43
lifelessfungi: (potentially; we can of course do it locally to - agree that a spec probably needed)20:44
lifelessthough, I'd like to see someone JFDI it somewhere and use a spec to drive wider adoption20:44
AJaegerfungi, all these repos are "server" typo, so shouldn't show up in requirements...20:47
fungilifeless: right, we can implement on a limited scale first as a trial before trying to convince everyone else it's beneficial20:49
fungiAJaeger: i hope that's true, though it didn't used to be (for example we had things declaring glance as a requirement once upon a time)20:50
fungibut that was in the days before middlewares and such20:50
lifelessstadium...20:50
AJaegerfungi, yeah, we need to double check - I checked a few but should automate...20:51
fungioh, right, tons of neutron-stadium repos depend on neutron20:52
fungior did20:52
fungino idea whether they still do20:52
AJaegerthey use neutron-lib now -and that's in the list ;(20:54
*** bapalm has quit IRC20:55
*** bapalm has joined #openstack-oslo20:58
AJaegerfungi, so how to continue? I can change these repos to use the db jobs - and then remove the db from the default. Do you want to write an email or a spec?20:59
*** e0ne has quit IRC21:00
*** haypo has left #openstack-oslo21:04
zzzeekkevinbenton: I likely have a fix for this but it will be in 1.121:19
*** sdague has quit IRC21:19
*** ryansb has left #openstack-oslo21:22
kevinbentonzzzeek: cool. What's the fix? Keep a ref if a transaction is active?21:25
*** jdandrea_ is now known as jdandrea21:25
zzzeekkevinbenton: no, when i raise that error, check the row21:25
zzzeekkevinbenton: im working up a fix w/ an issue we can reference in the gerrit21:25
kevinbentonzzzeek: sgtm21:25
*** dimtruck is now known as zz_dimtruck21:27
*** zz_dimtruck is now known as dimtruck21:29
*** yamahata has joined #openstack-oslo21:36
*** vilobhmm111 has quit IRC21:37
*** vilobhmm11 has joined #openstack-oslo21:37
*** regXboi has quit IRC21:40
zzzeekkevinbenton: this is a huge fix so, this is why i always want to understand :)21:45
kevinbentonzzzeek: well now you have a test case at least :)21:46
kevinbentonzzzeek: I actually dislike our allocation pattern anyway because now it emits a select statement for something we just created :(21:51
kevinbentonzzzeek: regardless of this bug21:52
*** dims_ has quit IRC21:53
fungiAJaeger: with the number being that high, i'm unconvinced all 50+ repos are actually testing database interactions in their unit tests21:55
fungiAJaeger: so we probably should look for the other more direct markers mentioned (i.e., OS_TEST_DBAPI_ADMIN_CONNECTION and OpportunisticTestBase substrings as well as openstack_citest)21:57
*** jdandrea_ has joined #openstack-oslo21:57
*** jdandrea has quit IRC21:57
*** mriedem is now known as mriedem_away21:57
*** rlrossit has quit IRC22:03
kevinbentonzzzeek: the only thing I can't quite reconcile is how messing around in pdb was fixing it22:13
*** MVenesio has joined #openstack-oslo22:13
zzzeekkevinbenton: pdb changes gc stuff highly22:13
zzzeekobject was not getting gc'ed in that case22:14
kevinbentonzzzeek: I see. So by having the set_trace in that allocation code it was preventing gc from collecting when that function exited or something?22:15
zzzeekkevinbenton: likely22:15
*** MVenesio has quit IRC22:18
*** yamahata has quit IRC22:26
*** dims has joined #openstack-oslo22:26
*** dimtruck is now known as zz_dimtruck22:30
*** e0ne has joined #openstack-oslo22:34
*** cdent has quit IRC22:36
*** alejandrito has quit IRC22:42
*** e0ne has quit IRC22:49
*** dims has quit IRC22:50
*** yamahata has joined #openstack-oslo22:56
*** yamamoto_ has joined #openstack-oslo22:56
*** yamamoto_ has quit IRC23:00
*** pradk has quit IRC23:02
*** ozamiatin has joined #openstack-oslo23:06
*** harlowja_at_home has joined #openstack-oslo23:10
*** gordc has quit IRC23:11
fhermeniharlowja_at_home: hi !23:15
harlowja_at_homefhermeni, hey hey23:15
harlowja_at_homeisn't it late for u ;)23:15
fhermeniharlowja_at_home: indeed23:15
harlowja_at_homesoooooo scheduler stuff23:16
harlowja_at_homeill comment on that spec soon23:16
harlowja_at_homefhermeni, though if u want to attend a TC meeting23:18
harlowja_at_homeu can bug the TC about it, may or may not help ,lol23:18
fhermeniharlowja_at_home: that is my point. Your opinion ?23:19
fhermeniharlowja_at_home: as I still have a limited perspective on OS principles, it might not be easy to me23:19
harlowja_at_homethat's ok imho, someone who rocks the boat might not be bad23:20
harlowja_at_homesylvian is coming at the view point of nova23:20
harlowja_at_homefhermeni, maybe i'll showup, this boat imho needs rocking23:20
harlowja_at_homethere isn't time enough to wait for nova to do something23:20
harlowja_at_homeand wait for others to do something23:21
harlowja_at_homeand wait and wait23:21
harlowja_at_homeopenstack imho be dead by then23:21
harlowja_at_homelol23:21
fhermeniharlowja_at_home: :D23:21
harlowja_at_homebasically it will probably end up in the area of 'what is nova'23:21
harlowja_at_homebecause if nova wants to do cross-project scheduling, fine23:21
harlowja_at_homebut it doesn't23:21
*** dims has joined #openstack-oslo23:21
harlowja_at_homedims, howdy23:22
harlowja_at_homedims, jaypipes might have thoughts if they are around23:22
harlowja_at_homefhermeni,  jaypipes do u know when the 'nova scheduler' weekly meetings are?23:22
fhermeniharlowja_at_home: nova is also looking for moving our the driver parts as I remember ?23:23
harlowja_at_homefhermeni,  i'd like to see the documentation on that split-out also, idk what the up to date docs even are, lol23:23
harlowja_at_homefhermeni, there are many ambitious plans :-P23:23
harlowja_at_homethat ones been an ambition for years23:23
harlowja_at_homereality == (not so much afaik)23:23
harlowja_at_homejaypipes, dims btw, this is in regard to commentary at https://review.openstack.org/#/c/210549/ (if u get time to read it)23:24
patchbotharlowja_at_home: patch 210549 - openstack-specs - Super scheduling (WIP)23:24
harlowja_at_homesuper-duper-scheduling, lol23:24
fhermenidamn spam bot, there are everywhere :D23:25
dimsharlowja_at_home : i believe it's https://wiki.openstack.org/wiki/Meetings/Scheduler23:26
dimsfhermeni : i requested it for our channel :)23:26
*** dims has quit IRC23:27
harlowja_at_homeMondays at 14:00 UTC (#openstack-meeting-alt)23:28
harlowja_at_homeoops guess i missed it23:28
harlowja_at_homelol23:28
harlowja_at_homefhermeni,  that might be better in your tz23:29
harlowja_at_homeits 6am mine, lol23:29
harlowja_at_homei haven't seen 6am in a while, lol23:29
fhermeniharlowja_at_home: Very bad timing for you indeed. How is the time choosed ? Depending on the majority TZ ?23:30
harlowja_at_homefhermeni, usually ya23:30
fhermeniso bed time now. If you have some idea/points/.. that I should mention. Mail them to me23:33
*** fhermeni has quit IRC23:34
*** zz_dimtruck is now known as dimtruck23:41
*** sputnik13 has quit IRC23:46
kevinbentonzzzeek: ouch, now holding that reference is causing a test to fail that creates ports, deletes them, and re-creates them all using the same session23:47
openstackgerritJamie Lennox proposed openstack/oslo.config: Handle some native python types in config generation  https://review.openstack.org/29264723:48
*** sputnik13 has joined #openstack-oslo23:51
*** yamahata has quit IRC23:53
*** sputnik13 has quit IRC23:53
*** salv-orl_ has joined #openstack-oslo23:54
*** _amrith_ is now known as amrith23:55
*** salv-orlando has quit IRC23:57
*** yamamoto_ has joined #openstack-oslo23:57

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!