Tuesday, 2016-11-15

« Monday, 2016-11-14 Index Wednesday, 2016-11-16 »
*** jdandrea has joined #openstack-oslo00:02
*** cdent has quit IRC00:40
*** ansmith has quit IRC00:47
*** ansmith has joined #openstack-oslo01:03
*** tovin07_ has joined #openstack-oslo01:14
*** kfox1111 is now known as kfox1111_away01:51
*** gcb has joined #openstack-oslo02:00
*** mnaser has quit IRC02:24
*** mnaser has joined #openstack-oslo02:31
*** yamamoto_ has joined #openstack-oslo02:39
*** links has joined #openstack-oslo02:49
*** jamielennox is now known as jamielennox|away03:07
*** jamielennox|away is now known as jamielennox03:21
openstackgerritJamie Lennox proposed openstack/oslo.context: Move existing attributes to _id suffixed attributes  https://review.openstack.org/31532103:25
*** dimtruck is now known as zz_dimtruck03:27
*** Rockyg has quit IRC03:31
*** yamahata has quit IRC03:37
*** yamahata has joined #openstack-oslo03:53
*** jamielennox is now known as jamielennox|away04:15
*** chandankumar has joined #openstack-oslo04:15
*** zz_dimtruck is now known as dimtruck04:16
*** jamielennox|away is now known as jamielennox04:17
*** jamielennox is now known as jamielennox|away04:40
*** dimtruck is now known as zz_dimtruck04:57
*** jamielennox|away is now known as jamielennox05:03
*** khamtamtun has joined #openstack-oslo05:04
*** khamtamtun has quit IRC05:21
openstackgerritJamie Lennox proposed openstack/oslo.context: Add the service token details to context  https://review.openstack.org/39104505:41
openstackgerritJamie Lennox proposed openstack/oslo.context: Reformat the Context.__init__ arguments  https://review.openstack.org/39104405:41
*** khamtamtun has joined #openstack-oslo06:17
*** yamamoto_ has quit IRC06:27
*** khamtamtun has quit IRC06:35
*** khamtamtun has joined #openstack-oslo06:48
*** lpetrut has joined #openstack-oslo06:50
*** Shengping has joined #openstack-oslo06:53
*** Shengping has left #openstack-oslo06:54
*** hello has joined #openstack-oslo06:57
hellohello guys, I am using oslo_messaging. but I got some question about it, therefore I am coming here look for some helps.06:59
helloI am using oslo_messaging to test the batch_notification_listener, I write the simple endpoint.06:59
helloclass Test(object):06:59
hello   def info(self, messages):06:59
hello        for i in messages:06:59
hello            print i06:59
helloand simple get_server  with eventlet executor07:00
hellowhen I send the message to the topic. the server is blocked.07:01
hellobut when I using ‘threading’ or ‘blocking’ executor, everything is fine.07:02
helloI dont know why it is happend. is it a bug ?07:03
*** amotoki has joined #openstack-oslo07:09
*** eezhova has joined #openstack-oslo07:20
*** hello has left #openstack-oslo07:23
*** rcernin has quit IRC07:23
silehthecliunyx, did you eventlet monkeypatch first ?07:39
*** yamamot__ has joined #openstack-oslo07:46
*** lpetrut has quit IRC08:06
*** eezhova has quit IRC08:15
*** dtardivel has joined #openstack-oslo08:15
*** pcaruana has joined #openstack-oslo08:22
*** e0ne has joined #openstack-oslo08:24
*** tovin07_ has quit IRC08:24
*** e0ne has quit IRC08:25
*** ihrachys has joined #openstack-oslo08:32
*** shardy has joined #openstack-oslo08:35
*** jpena|off is now known as jpena08:37
*** eezhova has joined #openstack-oslo08:47
*** ihrachys has quit IRC08:56
*** ihrachys has joined #openstack-oslo08:56
*** zzzeek has quit IRC09:00
*** zzzeek has joined #openstack-oslo09:00
*** szaher has joined #openstack-oslo09:18
*** openstackgerrit has quit IRC09:18
*** openstackgerrit has joined #openstack-oslo09:18
*** lucas-afk is now known as lucasagomes09:28
*** yassine has joined #openstack-oslo09:29
*** yassine is now known as Guest1079209:29
*** mhickey has joined #openstack-oslo09:43
openstackgerritCao Xuan Hoang proposed openstack/tooz: Using assertIsNone() instead of assertEqual(None, ...)  https://review.openstack.org/39762509:50
duleksileht: Hi, do you remember bug 1630975?09:59
openstackbug 1630975 in Cinder "Inconsistent server name used for RPC-client and RPC-server" [Medium,In progress] https://launchpad.net/bugs/1630975 - Assigned to Michal Dulko (michal-dulko-f)09:59
silehtdulek, hi yes09:59
duleksileht: I'm trying to model Cinder's multibackend using Target.namespace instead of @backend suffix on Target.server.10:00
duleksileht: I've noticed that in RPCServer level namespaces are ignored. Are namespaces respected only on Endpoint level?10:00
silehtdulek, on endpoint you need to set target = oslomsg.Target(accepted_namespaces=["foo", "bar"])10:02
silehtdulek, if you have two rpc server listening on a topic they will be steal messages each other10:04
duleksileht: Actually looking at Target.__init__ I need to set namespace and eventual legacy_namespaces.10:04
duleksileht: Yeah, anyway we have two RPC servers, so namespaces are ignored on that level.10:05
silehtdulek, use namespace or legacy_namespaces10:05
silehtdulek, accepted_namespaces is internal concatenation of both arguments10:06
duleksileht: Yes, I've noticed. :)10:06
*** ihrachys has quit IRC10:07
duleksileht: I think we have some assumptions in Cinder that different backend managers are in separated processes. So using namespaces on endpoint (manager) level would be problematic.10:07
duleksileht: I wonder if we shouldn't just route messages to correct backend's manager using topics instead.10:07
silehtdulek, this is perhaps a better idea10:09
silehtdulek, namespace is more to spread all rpc methods is many classes10:09
sileht(and allow to have multiple class that define the same method)10:10
silehtI think nobody use namespace...10:10
duleksileht: Yup, makes sense. Okay, I'll see what I can do with topics.10:10
duleksileht: Hey, I've really tried using them! :D10:10
dulekKind of, in a wrong way…10:10
silehtdulek, thx for trying, I was not aware of the per process backend manager10:11
*** e0ne has joined #openstack-oslo10:19
*** ihrachys has joined #openstack-oslo10:20
*** ihrachys has quit IRC10:25
openstackgerritJulien Danjou proposed openstack/tooz: Add a hashring implementation  https://review.openstack.org/39726410:37
openstackgerritJulien Danjou proposed openstack/tooz: Move watchers implementation into watcher mixin  https://review.openstack.org/39726310:37
*** pblaho has quit IRC10:39
*** khamtamtun has quit IRC10:39
*** pblaho has joined #openstack-oslo10:40
*** lpetrut has joined #openstack-oslo10:59
*** ajo has quit IRC11:00
*** khamtamtun has joined #openstack-oslo11:01
*** ajo has joined #openstack-oslo11:03
*** ihrachys has joined #openstack-oslo11:04
duleksileht: One more question I've stumbled upon when trying to use topic here. We have one fanout call, that wants to address all the cinder-volumes, regardless of it's backend. Now if I'll append something to the topic, I would either need to iterate over these values on client side or spawn multiple servers listening on 'cinder-vol' and 'cinder-vol@backend' topics, or somehow make single server able to accept multiple topics.11:06
duleksileht: Or maybe use some kind of wildcard like 'cinder-vol*' on client side.11:07
duleksileht: Is any of two last options possible?11:07
*** yamahata has quit IRC11:21
openstackgerritRoman Podoliaka proposed openstack/oslo.db: Restore provisioning of DBs in legacy test base  https://review.openstack.org/39714411:22
*** ansmith has quit IRC11:32
*** boden has joined #openstack-oslo11:36
*** mhickey has quit IRC11:38
*** ansmith has joined #openstack-oslo11:43
*** links has quit IRC11:43
*** ansmith has quit IRC12:02
*** gcb has quit IRC12:19
*** yamamot__ has quit IRC12:25
*** rtheis has joined #openstack-oslo12:28
*** lucasagomes is now known as lucas-hungry12:54
*** e0ne has quit IRC12:56
*** e0ne has joined #openstack-oslo12:56
*** kgiusti has joined #openstack-oslo13:01
*** cdent has joined #openstack-oslo13:14
*** khamtamtun has quit IRC13:16
*** edmondsw has joined #openstack-oslo13:18
*** jpena is now known as jpena|lunch13:36
*** pradk has joined #openstack-oslo13:42
*** jeckersb is now known as jeckersb_gone13:48
*** khamtamtun has joined #openstack-oslo13:58
*** lucas-hungry is now known as lucasagomes14:02
*** pcaruana has quit IRC14:13
*** khamtamtun has quit IRC14:16
*** mriedem has joined #openstack-oslo14:16
*** pradk has quit IRC14:17
openstackgerritJulien Danjou proposed openstack/tooz: Add a hashring implementation  https://review.openstack.org/39726414:18
openstackgerritJulien Danjou proposed openstack/tooz: Move watchers implementation into watcher classer  https://review.openstack.org/39726314:18
*** mhickey has joined #openstack-oslo14:23
*** zz_dimtruck is now known as dimtruck14:26
*** jeckersb_gone is now known as jeckersb14:27
*** yamamoto has joined #openstack-oslo14:36
*** yamamoto_ has joined #openstack-oslo14:37
*** yamamoto has quit IRC14:41
*** jpena|lunch is now known as jpena14:42
*** dimtruck is now known as zz_dimtruck14:42
openstackgerritJulien Danjou proposed openstack/tooz: Add a hashring implementation  https://review.openstack.org/39726414:45
openstackgerritJulien Danjou proposed openstack/tooz: Factorize member_id in the base coordinator class  https://review.openstack.org/39777214:45
*** khamtamtun has joined #openstack-oslo14:45
*** khamtamtun has quit IRC14:50
*** yamamoto_ has quit IRC14:58
*** yamamoto has joined #openstack-oslo15:00
*** zz_dimtruck is now known as dimtruck15:01
*** yamamoto has quit IRC15:07
*** pradk has joined #openstack-oslo15:11
*** yamamoto has joined #openstack-oslo15:11
*** eezhova has quit IRC15:11
*** eezhova has joined #openstack-oslo15:12
*** yamamoto has quit IRC15:12
*** yamamoto has joined #openstack-oslo15:14
*** eezhova has quit IRC15:15
*** yamamoto has quit IRC15:15
*** yamamoto has joined #openstack-oslo15:23
*** yamamoto has quit IRC15:23
*** nkrinner_afk has quit IRC15:24
*** nkrinner_afk has joined #openstack-oslo15:30
*** yamamoto has joined #openstack-oslo15:31
*** yamamoto has quit IRC15:32
openstackgerritJulien Danjou proposed openstack/tooz: Add a hashring implementation  https://review.openstack.org/39726415:32
openstackgerritJulien Danjou proposed openstack/tooz: Factorize member_id in the base coordinator class  https://review.openstack.org/39777215:32
*** yamahata has joined #openstack-oslo15:32
openstackgerritJulien Danjou proposed openstack/tooz: Move watchers implementation into watcher classer  https://review.openstack.org/39726315:32
openstackgerritGevorg Davoian proposed openstack/oslo.messaging: [zmq] Implement MatchmakerRing  https://review.openstack.org/39779915:38
*** dimtruck is now known as zz_dimtruck15:45
*** openstackgerrit has quit IRC15:48
*** openstackgerrit has joined #openstack-oslo15:48
*** yamamoto has joined #openstack-oslo15:57
*** rpodolyaka_afk is now known as rpodolyaka15:58
*** yamamoto has quit IRC16:01
*** yamamoto has joined #openstack-oslo16:01
*** pcaruana has joined #openstack-oslo16:02
*** zz_dimtruck is now known as dimtruck16:02
*** dimtruck is now known as zz_dimtruck16:05
*** yamamoto has quit IRC16:06
*** gordc has joined #openstack-oslo16:14
*** chandankumar has quit IRC16:17
*** purplerbot has quit IRC16:21
*** purplerbot has joined #openstack-oslo16:22
*** ozamiatin has joined #openstack-oslo16:22
*** yamamoto has joined #openstack-oslo16:27
*** mhickey has quit IRC16:28
*** kfox1111_away is now known as kfox111116:28
*** e0ne has quit IRC16:33
*** e0ne has joined #openstack-oslo16:34
*** yamamoto has quit IRC16:35
*** rtheis_ has joined #openstack-oslo16:36
*** GMAzrael has joined #openstack-oslo16:38
*** GMAzrael has left #openstack-oslo16:38
*** rtheis has quit IRC16:39
*** zz_dimtruck is now known as dimtruck16:40
*** spzala has joined #openstack-oslo16:59
*** ozamiatin has quit IRC17:01
*** browne has joined #openstack-oslo17:13
*** yamamoto has joined #openstack-oslo17:21
*** e0ne has quit IRC17:21
*** yamamoto has quit IRC17:22
*** yamamoto has joined #openstack-oslo17:22
*** yamamoto has quit IRC17:34
*** amotoki has quit IRC17:44
*** yamahata has quit IRC17:48
*** lucasagomes is now known as lucas-afk17:48
*** jlvillal is now known as jlvill-travel17:50
*** chandankumar has joined #openstack-oslo17:55
harlowjakgiusti hey wasn't feeling good yesterday, sorry about that18:06
* harlowja took a day off18:06
harlowjahope u guys managed withot me :-P18:06
harlowjaamrith thx for taking that one over18:07
* harlowja had a stomach issue or something (not sure what)18:07
kgiustihope u r feeling better - those things can be nasty.18:08
harlowjaya, just rested, wasn't quite sure what it was18:08
amrithaha ... what's this day off thing? new oslo library?18:09
harlowjaoslo.take_a_day18:09
harlowjaha18:09
amrithsorry to hear about stomach issue. i have one too ... actually some say three or four.18:10
amrithworking on it :)18:10
harlowjahow many stomachs u got?18:10
harlowjau sure u are human?18:10
amrithone, large economy size.18:11
amrithno, i chew the cud18:11
harlowjacool18:11
*** e0ne has joined #openstack-oslo18:11
*** lpetrut has quit IRC18:11
harlowjalots more people will be doing things with grass where i live soon anyway18:11
amrithhope u feel better soon18:11
harlowjaso i guess i should get used to it18:11
harlowjathx18:11
amrithyes. same here18:12
*** pradk- has joined #openstack-oslo18:19
*** pradk has quit IRC18:22
*** e0ne has quit IRC18:23
*** chandankumar has quit IRC18:27
*** yamahata has joined #openstack-oslo18:27
*** ihrachys has quit IRC18:32
*** ihrachys has joined #openstack-oslo18:33
*** pradk- has quit IRC18:37
*** jpena is now known as jpena|off18:37
*** e0ne has joined #openstack-oslo18:37
*** harlowja has quit IRC18:38
*** shardy has quit IRC18:39
*** pradk has joined #openstack-oslo18:41
*** ihrachys has quit IRC18:44
*** chandankumar has joined #openstack-oslo18:55
*** chandankumar has quit IRC18:56
*** spzala has quit IRC19:00
*** rcernin has joined #openstack-oslo19:02
openstackgerritJohn Perkins proposed openstack/oslo.config: [WIP] oslo validator  https://review.openstack.org/38455919:06
*** harlowja has joined #openstack-oslo19:06
*** ajay_ has joined #openstack-oslo19:08
ajay_we are seeing an issue where "Reply timeout happens on nova compute" when we boot 10 instances at a time....19:08
ajay_increasing rpc_response_timeout=180 seems to help....19:08
ajay_what generally happens and why would just 10 VMs cause so much time to be spent in response code path19:09
ajay_note we are booting 10 instances on one compute node19:09
*** spzala has joined #openstack-oslo19:22
*** spzala has quit IRC19:23
*** spzala has joined #openstack-oslo19:23
*** cdent has quit IRC19:30
*** e0ne has quit IRC19:31
*** andymaier has joined #openstack-oslo19:36
*** amotoki has joined #openstack-oslo19:44
amrithajay_, this isn't an oslo issue, the issue is that something in nova is making a call with a timeout that is being exceeded becuase of the load you are placing on the system19:45
amrithfigure out why your nova call is taking so long to complete.19:46
amrithoslo is merely the messenger.19:46
amrithajay_ ^^19:46
ajay_hello amrith19:47
ajay_msg amrith19:48
amrithyes ajay_ we can chat here19:48
amrithno need to PM :)19:48
*** lpetrut has joined #openstack-oslo19:48
ajay_so basically it looks like after nova boots an instance it tells conductor that instance is active19:48
ajay_it does this through an RPC call19:49
amrithspecifically, an RPC call(), not to be confused with an RPC cast()19:49
ajay_its specifically an RPC call yes19:49
*** amotoki has quit IRC19:50
ajay_i sent a Traceback on openstack-dev but its pretty much a simple RPC call19:50
amrithok, I'm not sure that is true; just wanted to make the distinction between cast() and call().19:50
ajay_after that conductor replies instantly since its just  a simple db write but it looks like compute never gets to process it19:51
ajay_for a long time19:51
ajay_i had to literally increase rpc response timeout to 180 seconds to get the compute to process the reply19:52
amrithok, what was the timeout before19:52
amrithhow busy was the machine when you lauched 10 vm's on it?19:53
amrithI assume it got a bit warm19:53
ajay_60 first problem always happens 120 problem less likely 180 problem seems ok19:53
ajay_nope CPU is pretty idle not much happening19:53
ajay_question i had is in this same scenario sometimes heartbeat also times out19:53
ajay_is there a lock contention between heartbeat and data send threads19:54
amrithok, something has to be causing it to take that long for a reply to appear; I suggest you ask folks in #openstack-nova ...19:54
ajay_which forces them sometimes to wait long time for lock19:54
amrithif there is some thread starvation issue, that's not in o.m land, that's up above in the client19:54
amrithaka nova19:54
ajay_ok19:54
amrithyou sent your email to the ML less than 30m ago, give it a little time, you will get a reply there.19:55
kgiustiajay_: one other point (thanks amrith) - are the rabbitmq queues backing up?19:55
*** cdent has joined #openstack-oslo19:55
ajay_not much i see like 6 unacked messages max19:55
ajay_on any reply queue19:55
ajay_but they stick there for like 60+ seconds19:55
ajay_sometimes19:55
amrithare these the messages to the conductor that are sitting for 60m?19:56
amrithsorry 60s?19:56
ajay_i think its reverse19:57
amrithah19:57
amrithok, that makes more sense.19:57
ajay_its conductor sending back response to compute19:57
ajay_and response is sitting a while19:57
ajay_and on same scenario i see heartbeat timeouts at times19:58
ajay_possibly bcos of same reason19:58
kgiustiajay_: hrm - what's the CPU load on the client (reply  receiver)?19:58
ajay_not much at all19:58
ajay_also would be good to know how long heartbeat and driver thread wait to get the lock19:59
ajay_maybe they end up waiting for a long time19:59
ajay_bcos if lock is obtained than i suppose not much can go wrong to complete transaction?19:59
kgiustiajay_: I'm not really familiar with the locking behavior of the rabbit driver but that sounds probable20:01
jeckersbsometimes blocking i/o gets into the eventloop and plugs things up20:01
jeckersbthat will cause the heartbeat thread to miss20:01
kgiustiajay_: btw what release of oslo.messaging are you hitting this on?20:01
jeckersbi've seen it happen interacting with libvirt, gluster, ceph, probably others20:01
ajay_just libvirt20:01
ajay_for VM bring up20:01
ajay_i have the red hat release version of OSP8 python-oslo-messaging-2.5.0-7.el7ost.noarch20:03
ajay_for heartbeat timeout to happen if heartbeat threshold=60 and interval=2 i need to miss getting a single heartbeat in 120 seconds correct?20:03
ajay_am i reading that right20:03
ajay_amrith  how can nova cause issue in this case because using TCPdump i see that the response from nova conductor shows up on compute node20:06
ajay_so the reply is sent in less than 1 second20:06
ajay_response to the RPC call i mean20:06
kgiustiajay_: From the option text I think the threshold is the full timeout (60s).20:06
kgiustiajay_: btw I don't think heartbeating was fully baked back in 2.5.0 - jeckersb do you recall?20:07
amrithajay_, the fact that a packet (in tcpdump) appears to be sent means little; the messaging layer for a response is very light and from my experience with o.m with rabbit, the culprit is usually in the client application. Trove has a similar set of problems and we've never found the issue to be in oslo.messaging.20:08
jeckersbuh i think it was mostly fine by then20:08
ajay_ok20:09
kgiustiajay_: yeah the code looks like threshold is the actual deadline (60 secs) - the heart_rate determines how _often_ o.m. checks for the heartbeat.20:11
ajay_kguisti: thanks20:11
*** e0ne has joined #openstack-oslo20:13
jeckersbif threshold=60 and rate=2, it will heartbeat every 30s20:16
jeckersbif no response by 60s, it will disconnect20:16
ajay_jeckersb: thx20:19
*** Guest10792 has quit IRC20:27
*** browne has quit IRC20:30
*** lpetrut has quit IRC20:31
*** ihrachys has joined #openstack-oslo20:33
*** lpetrut has joined #openstack-oslo20:35
*** amotoki has joined #openstack-oslo20:50
*** dtardivel has quit IRC20:54
*** amotoki has quit IRC20:55
*** takedakn has joined #openstack-oslo20:59
*** takedakn has quit IRC20:59
*** e0ne has quit IRC21:04
*** cdent has quit IRC21:06
*** browne has joined #openstack-oslo21:10
openstackgerritKen Giusti proposed openstack/oslo.messaging: Document the transport backend driver interface  https://review.openstack.org/39673821:10
*** jeckersb is now known as jeckersb_gone21:20
openstackgerritBrian Stajkowski proposed openstack/oslo.config: [WIP] oslo validator  https://review.openstack.org/38455921:30
*** spzala has quit IRC21:32
*** pcaruana has quit IRC21:32
*** spzala has joined #openstack-oslo21:32
*** lpetrut has quit IRC21:34
*** lpetrut has joined #openstack-oslo21:34
*** kgiusti has left #openstack-oslo21:35
*** spzala has quit IRC21:37
*** rtheis_ has quit IRC21:41
*** johndperkins has joined #openstack-oslo21:50
*** dkehn__ has joined #openstack-oslo21:53
*** dkehn has quit IRC21:54
*** harlowja has quit IRC21:55
*** dkehn_ has quit IRC21:55
*** dkehn has joined #openstack-oslo21:56
*** ansmith has joined #openstack-oslo21:56
*** harlowja has joined #openstack-oslo22:13
*** lpetrut has quit IRC22:15
*** khamtamtun has joined #openstack-oslo22:29
harlowjazzzeek have u seen http://logs.openstack.org/periodic/periodic-nova-py27-with-oslo-master/e6bf84a/console.html#_2016-11-15_06_14_41_90022522:31
harlowjapossibly sqlalchemy issue?22:31
zzzeekharlowja: hmmm22:31
harlowjadamn None and nulls and crap22:32
harlowjaalways screwing up languages, lol22:32
zzzeekyep22:32
harlowjaseems like thats killing keystone to22:33
harlowjahttp://logs.openstack.org/periodic/periodic-keystone-py27-with-oslo-master/b4230a4/console.html#_2016-11-15_06_05_07_01448922:33
zzzeekharlowja: umm.  "SELECT DATABASE()" would have to return NULL I think for what we see her22:33
zzzeekhere22:33
zzzeekthere's no "current database".   MySQL URL perhaps has no dbname22:34
harlowjahmmmm, any recent change that altered that?22:34
zzzeekharlowja: not on my end22:34
zzzeekharlowja: this would be a URL like mysql://scott:tiger@localhost/22:35
harlowjacould https://github.com/openstack/oslo.db/commit/2ad571c8d7a7e1 have busted that?22:36
harlowjaseems like that (finally) just merged22:36
*** edmondsw has quit IRC22:36
zzzeekharlowja: it's not supposed to but seems likely22:37
harlowjaya seems like one of the few changes in that lib thats not released22:37
harlowjaand it looks like it might be in the fixture code22:37
zzzeekharlowja: so basiclly, the URL is always supposed to have a DB name token on it22:38
zzzeekthis system should not be generating a db-less URL22:38
harlowjaright22:38
harlowjai mean the python logging shows ' 2016-11-15 06:14:42,164 INFO [oslo_db.sqlalchemy.provision] DROP ALL OBJECTS, BACKEND mysql+pymysql://openstack_citest:openstack_citest@localhost/'22:38
harlowjawhich seems ok22:38
zzzeekharlowja: actually no that's exactly what would be wrong :)22:39
harlowjaoh22:39
harlowjaya no DB name, lol22:39
zzzeekit should be  mysql+pymysql://openstack_citest:openstack_citest@localhost/openstack_citest or some random generated name22:39
openstackgerritMichael Bayer proposed openstack/oslo.db: Adjust SAVEPOINT cause test for SQLA 1.1  https://review.openstack.org/39798122:40
zzzeekharlowja: my day is about over here can you ping me with a launchpad or something tomrorrow, or someone try to poke at it22:40
harlowjakk22:40
harlowjawill do22:41
zzzeekharlowja: but yeah probably someting in that patch, my fault22:41
*** mriedem has quit IRC22:41
zzzeeksave it for me if you can't immediately see anyhting22:41
harlowjaya, checking out that test22:41
harlowjajust incase its messed up22:41
*** rcernin has quit IRC22:42
harlowjanothing immediatly obvious zzzeek22:44
zzzeekbleh22:44
harlowjawill poke u tommorow22:44
harlowjahttps://github.com/openstack/nova/blob/master/nova/tests/unit/db/test_migration_utils.py#L6022:44
*** dimtruck is now known as zz_dimtruck22:44
harlowjathough that thing seems to be testing sqalchemy ^22:44
harlowjawhich is weird22:44
harlowjaseems to have verified that sqlememcy did an insert then a delete and a query22:45
harlowja*sqlalchemy22:45
ajay_hey folks regarding that earlier rpc timeout issue with nova we instrumented the impl_rabbit.py and we see that the lock is being held for really long time22:48
ajay_like 223 seconds in one case, 77 seconds in another22:48
ajay_and hence causes heartbeat and rpc timeout22:49
*** ihrachys has quit IRC22:52
zzzeekharlowja: oh22:54
zzzeekharlowja: wait22:54
harlowjai opened https://bugs.launchpad.net/oslo.db/+bug/1642070 zzzeek if u want it22:54
openstackLaunchpad bug 1642070 in oslo.db "Fixture replace none issue?" [Undecided,New]22:54
zzzeekharlowja: it's this https://review.openstack.org/#/c/397144/22:54
harlowjaah22:55
zzzeekharlowja: very likely it's that22:55
harlowjakk22:55
harlowjaapproving22:55
harlowjawill let the periodics run tonight and see22:55
harlowjathx :)22:56
zzzeekharlowja: I just added all the "probably"  / "possibly" :)22:57
harlowjalol22:57
zzzeekharlowja: could be somethign totally else who knos22:57
harlowjamostly22:57
harlowja:-P22:57
harlowjamostly probably right22:57
harlowjaha22:57
*** boden has quit IRC22:58
*** khamtamtun has quit IRC22:59
*** pradk has quit IRC23:35
*** pradk has joined #openstack-oslo23:38
*** zz_dimtruck is now known as dimtruck23:39
openstackgerritMerged openstack/oslo.db: Restore provisioning of DBs in legacy test base  https://review.openstack.org/39714423:39
*** pradk has quit IRC23:43
*** andymaier has quit IRC23:48
*** gordc has quit IRC23:52
« Monday, 2016-11-14 Index Wednesday, 2016-11-16 »

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!