Tuesday, 2017-08-29

« Monday, 2017-08-28 Index Wednesday, 2017-08-30 »
openstackgerritMerged openstack/neutron-lbaas master: Updated from global requirements  https://review.openstack.org/49705600:01
*** leitan has joined #openstack-lbaas00:01
*** yamamoto_ has joined #openstack-lbaas00:13
*** yamamoto_ has quit IRC00:13
openstackgerritJude Cross proposed openstack/octavia-tempest-plugin master: Create scenario tests for health monitors  https://review.openstack.org/49764300:33
*** leitan has quit IRC00:52
*** yamamoto_ has joined #openstack-lbaas01:14
*** yamamoto_ has quit IRC01:20
*** yamamoto_ has joined #openstack-lbaas02:16
*** yamamoto_ has quit IRC02:22
*** bzhao has joined #openstack-lbaas02:43
*** yamamoto has joined #openstack-lbaas02:49
*** sanfern has quit IRC02:51
*** bbzhao has joined #openstack-lbaas02:53
*** bzhao has quit IRC02:53
*** yamamoto has quit IRC03:12
*** links has joined #openstack-lbaas04:03
*** sanfern has joined #openstack-lbaas04:12
*** yamamoto has joined #openstack-lbaas04:13
*** yamamoto has quit IRC04:19
*** bcafarel has quit IRC04:57
*** bcafarel has joined #openstack-lbaas04:59
*** yamamoto has joined #openstack-lbaas05:15
*** yamamoto has quit IRC05:21
*** yamamoto has joined #openstack-lbaas06:13
*** aojea has joined #openstack-lbaas06:32
*** aojea has quit IRC06:32
*** aojea has joined #openstack-lbaas06:32
*** yamamoto has quit IRC06:38
*** yamamoto has joined #openstack-lbaas06:39
*** yamamoto has quit IRC06:39
*** pcaruana has joined #openstack-lbaas06:39
*** eezhova has joined #openstack-lbaas06:44
*** gcheresh has joined #openstack-lbaas06:51
*** rcernin has joined #openstack-lbaas07:10
*** aojea has quit IRC07:17
*** tesseract has joined #openstack-lbaas07:25
*** aojea has joined #openstack-lbaas07:29
*** yamamoto has joined #openstack-lbaas07:40
*** eezhova has quit IRC07:41
*** aojea has quit IRC07:41
*** yamamoto has quit IRC07:45
*** eezhova has joined #openstack-lbaas08:05
*** yamamoto has joined #openstack-lbaas08:42
*** aojea has joined #openstack-lbaas08:47
*** yamamoto has quit IRC08:47
*** reedip has quit IRC09:12
*** reedip has joined #openstack-lbaas09:13
openstackgerritsumitjami proposed openstack/neutron-lbaas master: fixed statuses, as it was using an undefined variable  https://review.openstack.org/49242909:18
openstackgerritsumitjami proposed openstack/neutron-lbaas master: tempest: healthmonitor is deleted via resource_cleanup  https://review.openstack.org/49212809:19
*** yamamoto has joined #openstack-lbaas09:43
*** eezhova is now known as eezhova_afk09:45
*** yamamoto has quit IRC09:49
*** reedip has quit IRC09:52
*** reedip has joined #openstack-lbaas09:53
*** sapd has quit IRC09:56
*** sapd has joined #openstack-lbaas09:57
*** sapd has quit IRC10:17
*** eezhova_afk is now known as eezhova10:21
*** sapd has joined #openstack-lbaas10:33
*** yamamoto has joined #openstack-lbaas10:45
*** yamamoto has quit IRC10:51
*** sanfern has quit IRC11:18
*** yamamoto has joined #openstack-lbaas11:47
*** yamamoto has quit IRC11:52
*** Dave has quit IRC12:02
*** atoth has joined #openstack-lbaas12:13
*** yamamoto has joined #openstack-lbaas12:49
*** sanfern has joined #openstack-lbaas12:52
*** yamamoto has quit IRC12:54
*** sanfern has quit IRC12:58
*** sanfern has joined #openstack-lbaas12:58
*** slaweq has quit IRC13:02
*** slaweq has joined #openstack-lbaas13:03
*** leitan has joined #openstack-lbaas13:12
*** links has quit IRC13:18
*** catintheroof has joined #openstack-lbaas13:19
*** slaweq has quit IRC13:29
*** slaweq has joined #openstack-lbaas13:30
*** slaweq has quit IRC13:30
*** armax has quit IRC13:31
*** slaweq has joined #openstack-lbaas13:32
*** sanfern has quit IRC13:33
*** sanfern has joined #openstack-lbaas13:34
*** fnaval has joined #openstack-lbaas13:44
*** yamamoto has joined #openstack-lbaas13:50
*** yamamoto has quit IRC13:56
*** armax has joined #openstack-lbaas14:08
*** aojea has quit IRC14:30
*** gcheresh has quit IRC14:40
*** chlong has joined #openstack-lbaas14:41
*** yamamoto has joined #openstack-lbaas14:52
xgerman_https://www.irccloud.com/pastebin/7LKfc6ix/14:54
xgerman_this is with rm_work’s script — so not sure if we have a problem with DIB or if it’s just me and my bad luck14:55
nmagnezixgerman_, which script?14:56
nmagnezixgerman_, hi btw :D14:56
xgerman_wget bit.do/devstack && chmod +x devstack && vim devstack && ./devstack14:56
xgerman_this is the way he installs devstack14:56
xgerman_it’s pretty plain… just wanted to see if somebody else has seen it…14:57
*** yamamoto has quit IRC14:58
nmagnezii simply 'git clone' .. ¯\_(ツ)_/¯15:01
johnsomxgerman_ Your error is above that output, but no, I haven't had any trouble with DIB recently15:04
xgerman_some curl timeout…15:09
*** dayou has joined #openstack-lbaas15:13
*** eezhova has quit IRC15:18
*** aojea has joined #openstack-lbaas15:37
*** slaweq has quit IRC15:38
*** aojea has quit IRC15:41
*** yamamoto has joined #openstack-lbaas15:54
*** eezhova has joined #openstack-lbaas15:57
*** eezhova has quit IRC15:57
*** yamamoto has quit IRC16:00
*** eezhova has joined #openstack-lbaas16:00
*** ssmith has joined #openstack-lbaas16:03
*** tesseract has quit IRC16:15
*** rcernin has quit IRC16:46
*** pcaruana has quit IRC16:48
*** links has joined #openstack-lbaas16:55
openstackgerritMichael Johnson proposed openstack/octavia master: Adds the ability to failover a load balancer  https://review.openstack.org/44434716:55
*** yamamoto has joined #openstack-lbaas16:56
*** yamamoto has quit IRC17:02
*** dayou has quit IRC17:08
*** eezhova has quit IRC17:13
*** links has quit IRC17:34
*** sanfern has quit IRC17:35
*** Dave has joined #openstack-lbaas17:42
*** catintheroof has quit IRC17:49
*** tongl has joined #openstack-lbaas17:50
*** eezhova has joined #openstack-lbaas17:56
*** aojea has joined #openstack-lbaas17:57
*** yamamoto has joined #openstack-lbaas17:58
*** eezhova has quit IRC17:58
*** sshank has joined #openstack-lbaas18:01
*** yamamoto has quit IRC18:04
*** dayou has joined #openstack-lbaas18:07
*** tomtomtom has quit IRC18:11
*** tomtomtom has joined #openstack-lbaas18:12
*** catintheroof has joined #openstack-lbaas18:23
*** sshank has quit IRC18:23
openstackgerritSwaminathan Vasudevan proposed openstack/octavia master: Add ability to build amp image on SUSE  https://review.openstack.org/48888518:44
openstackgerritSwaminathan Vasudevan proposed openstack/octavia master: (WIP):Support opensuse based Amphora Image for Octavia  https://review.openstack.org/49890918:44
*** eezhova has joined #openstack-lbaas18:53
*** aojea has quit IRC18:55
*** yamamoto has joined #openstack-lbaas18:59
*** cpusmith has joined #openstack-lbaas19:01
*** ssmith has quit IRC19:04
*** cpusmith_ has joined #openstack-lbaas19:04
*** yamamoto has quit IRC19:05
*** cpusmith has quit IRC19:08
*** rstarmer has joined #openstack-lbaas19:08
*** armax has quit IRC19:20
openstackgerritMichael Johnson proposed openstack/octavia master: Adds the ability to failover a load balancer  https://review.openstack.org/44434719:22
*** yamamoto has joined #openstack-lbaas19:27
*** catintheroof has quit IRC19:30
rm_workxgerman_: yeah what is above that?19:31
xgerman_curl timed out19:32
xgerman_might be my ISP lying…19:32
*** gcheresh has joined #openstack-lbaas19:34
rm_workare you on VPN?19:35
*** catintheroof has joined #openstack-lbaas19:36
nmagnezio/19:36
johnsomI just had a gate timeout getting the ubuntu image, so maybe a canonical issue.19:39
rm_workjohnsom: so why is it again that when you admin-disable a member, we remove it from the LB config entirely instead of having it in MAINT?19:40
johnsomWell, I think it was two fold.  It was easier and the health reporting wasn't setup for MAINT19:40
johnsomIDK19:41
xgerman_rm_work no, TWC/Spectrum19:41
*** rstarmer has quit IRC19:42
*** sshank has joined #openstack-lbaas19:45
*** fnaval has quit IRC19:51
*** rstarmer has joined #openstack-lbaas19:53
rm_worki might do a devstack build in a sec, i'll see if i see any weirdness20:14
xgerman_rm_work how do I pull the octavia-dashboard into your devstack script?20:19
rm_workhmmmm20:21
rm_workI can look at adding that20:22
xgerman_that would be grand… thinking about testing Michael’s latest & greatest20:22
johnsompip install octavia-dashboard20:23
johnsomcp octavia_dashboard/enabled/_1481_project_ng_loadbalancersv2_panel.py /opt/stack/horizon/openstack_dashboard/local/enabled20:23
johnsomcd /opt/stack/horizon20:23
johnsom./manage.py collectstatic20:23
johnsom./manage.py compress20:23
johnsomsudo service apache2 restart20:23
johnsomI don't know if the the devstack plugin works or not, haven't tried it yet20:23
rm_workjohnsom: hmmm k i figured i'd add horizon via devstack plugin20:23
xgerman_yeah, we should aim to get that working20:23
rm_workand then do the copywork for the dashboard file20:24
johnsomYeah, horizon works, just not sure about our dashboard20:24
rm_workbut if the devstack plugin works that'd be good20:24
rm_worki can try it20:24
xgerman_okey20:24
rm_workjohnsom: do you know how to enable horizon via devstack plugin correctly?20:25
rm_workjust enable_plugin?20:25
rm_workor does it also need ENABLED_SERVICES20:25
rm_workxgerman_: give me a sec and i'll push up a branch for it20:26
*** fnaval has joined #openstack-lbaas20:26
*** fnaval has quit IRC20:26
xgerman_nice!20:26
*** fnaval has joined #openstack-lbaas20:26
johnsomAt the moment I do this:20:26
johnsomENABLED_SERVICES+=,horizon20:26
johnsomHORIZON_REPO=https://github.com/openstack/horizon20:26
johnsomHORIZON_BRANCH=master20:26
johnsomBut I'm sure their plugin work now20:27
rm_workhmm i'll try it20:27
rm_worki'm a little confused though because20:27
rm_workhttps://docs.openstack.org/devstack/latest/20:27
rm_workseems to indicate horizon is default?20:27
johnsomI have a habit of just using things that keep working...  grin20:27
johnsomNo, I know it's not enabled by default20:28
johnsomI just stacked without those lines and didn't get it20:28
rm_workerr20:28
rm_workwell your stuff resets the ENABLED_SERVICES right?20:28
rm_worksame as mine20:28
johnsomNo20:28
rm_workno?20:28
johnsomOh, Yes, I guess it does20:28
johnsomHa, forgot, it's been that way so long20:29
rm_workyep looks like default has it20:29
rm_workhttps://github.com/openstack-dev/devstack/blob/master/stackrc#L6620:29
johnsomI wonder how much stuff you end up with if you run the default....20:29
johnsomUgh20:30
xgerman_I hope you get Octavia with the default - if not we shoud propose a patch ;-)20:31
rm_workok trying with the devstack plugin for our dashboard20:31
rm_workxgerman_: try with this20:32
rm_workhttps://raw.githubusercontent.com/rm-you/devstack_deploy/oct_dash/stackme.sh20:32
rm_workI will also20:32
xgerman_will do, wonder if I can get an env variable to specify the pacth under test for the UI ;-)20:36
rm_workwas there a specific patch you wanted to test?20:36
rm_workyes I set it20:37
rm_workor rather20:37
rm_workhmmm20:37
rm_workwell, I *set* it, so you have to set it in the script file20:37
rm_workbut i could change it to "defaulting"20:37
*** armax has joined #openstack-lbaas20:42
*** sshank has quit IRC20:43
*** cpusmith_ has quit IRC20:43
*** leitan has quit IRC20:46
*** leitan has joined #openstack-lbaas20:47
xgerman_I am tryin the script20:47
*** leitan has quit IRC20:47
*** leitan has joined #openstack-lbaas20:47
*** leitan has quit IRC20:47
*** leitan has joined #openstack-lbaas20:48
rm_workme too20:49
rm_workcrossing fingers20:49
*** gcheresh has quit IRC20:50
*** leitan has quit IRC20:52
*** sshank has joined #openstack-lbaas20:52
*** eezhova has quit IRC20:54
xgerman_rm-work bad news:20:54
xgerman_https://www.irccloud.com/pastebin/YYLaffOR/20:54
rm_workuhhh20:58
rm_workhmm20:58
rm_work]augh20:58
rm_workff yeah i messed up20:58
rm_workoh20:58
rm_workor no20:58
rm_workit's the plugin20:59
rm_workhttps://github.com/openstack/octavia-dashboard/blob/master/devstack/settings#L120:59
rm_workthis needs to be $DEST/octavia_dashboard20:59
rm_workT_T20:59
rm_workmaybe I caught it in time for my stack and changed it on the FS21:00
rm_worknope :(21:00
rm_workoh21:02
rm_workmaybe it's because of how i pulled it in21:02
rm_workok21:03
rm_worktrying this21:03
rm_workyep21:03
rm_worki messed up, one sec21:03
rm_workxgerman_: pushed an updated file21:03
xgerman_no worries just riding your coattails on the script — I could ave forked it, too21:04
rm_workthe issue was I used "octavia_dashboard" instead of "octavia-dashboard"21:04
rm_workin the enable_plugin line21:04
rm_worki fixed it21:04
*** yamamoto has quit IRC21:04
rm_worknext time you run it'll pull the correct local.conf21:04
*** yamamoto has joined #openstack-lbaas21:08
xgerman_nice21:08
*** aojea has joined #openstack-lbaas21:12
*** rstarmer has quit IRC21:20
*** rstarmer has joined #openstack-lbaas21:21
*** rstarmer has quit IRC21:21
*** rstarmer has joined #openstack-lbaas21:21
*** rstarmer has quit IRC21:22
*** rstarmer has joined #openstack-lbaas21:22
*** rstarmer has quit IRC21:23
*** rstarmer has joined #openstack-lbaas21:24
*** rstarmer has quit IRC21:24
*** rstarmer has joined #openstack-lbaas21:25
*** rstarmer has quit IRC21:25
*** yamamoto has quit IRC21:30
*** yamamoto has joined #openstack-lbaas21:32
*** yamamoto has quit IRC21:32
*** catintheroof has quit IRC21:34
*** fnaval has quit IRC21:43
rm_workthis is weird21:48
rm_workon a member create in the API, we're getting rare DB deadlocks on this query:21:48
johnsom"Keep OpenStack weird"21:48
rm_workhttp://paste.openstack.org/raw/619838/21:49
rm_workthat's on the amphora table?!?21:49
rm_workon the API side21:49
rm_workduring member create...21:49
rm_workbut why would that ever lock21:49
xgerman_so the script doesn’t bring up the LB screen21:49
rm_workxgerman_: ?21:49
xgerman_ran your script. wwnt to Horizon, no LBaaS panel21:50
rm_workhmm21:50
xgerman_guess I need to do Michael’s steps21:50
rm_workcan you post your devstack log21:50
johnsomYeah, that is a select with no for update, so, a bit odd21:50
rm_worki want to see what the devstack plugin for octavia did21:50
rm_workit may have done something wrong21:50
rm_workand we just need to fix it21:50
*** aojea has quit IRC21:51
rm_workxgerman_: I see the file in the /local/enabled dir21:52
rm_workso ... it should be there21:52
johnsomDid it just not restart apache?21:53
rm_workmaybe21:53
rm_worklet me see21:53
rm_workwhat is the port21:53
johnsom8021:53
rm_workah yeah lol21:53
rm_workdefault logins?21:53
johnsomWhatever is in your localrc21:54
rm_workah21:54
johnsomotherwise yes, a default21:54
xgerman_admin/secretadmin21:54
xgerman_so sdid Michael’s steps - still no O ctavia dashboard21:55
xgerman_I assume we show under networking?21:55
rm_workhmm yeah i am not seeing it21:55
rm_workbut the file is there21:55
rm_workand restarted apache21:55
xgerman_same21:55
johnsomOk, so the manage steps were skipped21:56
xgerman_I did the manage steps by hand21:56
johnsomThen restarted apache?21:56
rm_workjohnsom: BTW it's `attempting to add a new member to 2 different pools in the LB at the "same time"`21:57
johnsomIs octavia-dashboard listed in pip?  I.e. did it get installed?21:57
rm_workmaybe my DB has bad row-locking settings21:57
rm_workfor reads21:57
rm_workyes it is in pip21:57
xgerman_+121:58
johnsomrm_work how do you add a new member to two pools at the same time???21:58
rm_workthat is a good question21:58
rm_workthe DB should lock21:58
rm_workon LB21:58
*** KeithMnemonic2 has joined #openstack-lbaas21:58
rm_workmaybe the deadlock is happening before it can DO the lock?21:58
rm_workO_o21:58
rm_workgoing to look into it in a moment21:58
rm_workafter i figure out this dashboard thing21:58
johnsomSo, if it pip installed, the file is in enabled (1482 right?), both manage scripts were ran (answered yes), and apache was restarted, it should be under admin project, network load balancers21:59
xgerman_maybe systemd throws a wrench into apache restat22:00
rm_worklol22:00
rm_work_1481_project_ng_loadbalancersv2_panel.py22:00
rm_workthat's the one it installs22:00
rm_workT_T22:00
xgerman_yep22:00
rm_workfail22:00
johnsomYou grabbed master?22:00
rm_worki grabbed your patch22:01
rm_workoh22:01
rm_workor it didn't work :/22:01
rm_workso yes, master >_>22:01
*** KeithMnemonic1 has quit IRC22:01
xgerman_I grabbes whatever rm_work’s script does22:01
rm_workit's supposed to have your patch22:01
johnsomhttps://review.openstack.org/#/c/497642/1/devstack/settings22:01
rm_workyeah22:02
rm_workit was supposed to be on that patch22:02
rm_workbut it didn't load it22:02
rm_workwtf22:02
rm_workit looks like it didn't load my .profile22:03
rm_workwonderful22:03
rm_workwell i'm restacking22:03
xgerman_so no way to salvage this thing?22:03
rm_worki mean22:04
rm_workyeah22:04
johnsomHad me worried I missed something, but I was pretty sure I had that.  (plus I just restacked this set of patches)22:04
rm_workyou can checkout that patch22:04
rm_workand then copy the correct file22:04
rm_workbut i'm more worried about the automation22:04
johnsomYeah, remove the 1481 file22:04
johnsomcopy over 148222:05
johnsomrm -rf /opt/stack/horizon/static22:05
johnsompip install22:05
johnsomre-run manage scripts22:05
johnsomrestarte apache22:05
johnsomIn theory that will get you where you want to be22:06
rm_workxgerman_: do you not stack cleanly every time? T_T22:06
xgerman_I stack cleanly every time22:07
xgerman_but I am tired of waiting22:07
johnsomQuestion for you folks, I'm building a list of dashboard bugs (joy, joy)...  Should we show "created at" and "updated at" on the details pages?  Or is that too much detail for the GUI?22:08
rm_workeh22:08
rm_workdetails probably?22:08
rm_workthough ...22:08
rm_workit'd be nice to sort by that maybe22:08
johnsomThis would be on the object details page, so no sorting.22:09
johnsomIt's too much for the columns IMO.  They are pretty wide already22:09
johnsomxgerman_ Success?22:13
xgerman_yep, installed the one with 148222:14
xgerman_uh, oh22:15
xgerman_https://usercontent.irccloud-cdn.com/file/fxz0UIeh/Screen%20Shot%202017-08-29%20at%203.15.05%20PM.png22:15
xgerman_I guess we have our work cutout22:15
johnsom???22:15
johnsomDid you *not* install neutron-lbaas?22:15
xgerman_no, I did not — after all it’s called OCTAVIA dashboard22:16
johnsomYeah, but we haven't converted that over yet.....22:16
xgerman_oh, ok22:16
*** sshank has quit IRC22:18
rm_worklol22:20
rm_workyeah i don't bother with neutron-lbaas22:20
rm_workso i guess that's also required T_T22:20
rm_workman, i don't even remember how to enable that22:21
johnsomThe conversion is next on my feature agenda....22:21
xgerman_ok, I am happy to enable more stuff for testing but not sure if it’s worth it at that point22:22
johnsomWell, I really want those two merged first, those are large-ish changes....22:23
johnsomIt will be a long chain otherwise22:23
*** tongl has quit IRC22:29
*** sshank has joined #openstack-lbaas22:33
rm_worktesting it out now with neutron-lbaas too >_>22:41
rm_workerugh22:41
xgerman_I think I will wait until it is more comprehensive22:42
johnsomThank you!22:42
johnsomxgerman_ Modifying 2,605 lines of code isn't a big enough change to review?  grin22:44
johnsomNoted...  hahaha22:44
xgerman_well, apparently that patch doesn’t work without the slug patc?22:44
johnsomI am busy opening bugs against dashboard so we have a queue if someone magically volunteers to help out22:45
xgerman_did we miss the summer of code?22:45
johnsomThe first patch works on it's own, slug is a child patch22:45
xgerman_so I don’t need the other filw?22:46
johnsomIf you want to test the first one alone, it will still use the 1481 file22:46
xgerman_oh, ok, so all I need is lbaasv2 and the second one to test everyhting ?22:49
johnsomYep22:50
xgerman_ok, I will have a look tonight —22:51
rm_worklol johnsom22:59
rm_work2017-08-29 15:58:15.206 18638 DEBUG octavia.amphorae.drivers.health.flip_utils [-] VIP re-association for 7222:59
rm_workbf071c-72a9-455e-bce7-a42c9ae2e282 ignored due to: mark_amphora_busy() got an unexpected keyword argument 'b22:59
rm_workusy' trigger_vip_reassociation /home/centos/octavia/octavia/amphorae/drivers/health/flip_utils.py:5822:59
rm_workwhoops22:59
johnsom????22:59
rm_worki mean, not whoops at pasting23:00
rm_worki mean whoops at "that arg"23:00
johnsomYeah, bug in your code23:00
rm_work"my code" :P23:02
rm_work(I do appreciate the help, just ribbing you)23:02
johnsomI see "flip_util" and T_T23:02
johnsomgrin23:02
johnsomAh, that was in the code I added?23:02
rm_workyeah heh23:03
rm_workhttps://review.openstack.org/#/c/435612/45..46/octavia/amphorae/drivers/health/flip_utils.py23:03
johnsomYeah, opps23:03
openstackgerritAdam Harwell proposed openstack/octavia master: WIP: Floating IP Network Driver (spans L3s)  https://review.openstack.org/43561223:03
rm_workhmmm looks like this doesn't work23:09
rm_workboth seem to have just gone ahead and gotten the lock and finished somehow T_T23:10
rm_workahhh i wonder if this is my same issue with the multi-master, did i forget to switch the HMs to a single one23:10
rm_work>_<23:10
rm_workhmm, no23:11
rm_workweird23:13
johnsomWell, if the first one finishes before the second one times out it's lock attempt it will run.  I thought we discussed that.23:14
johnsomOk, there we go, 23 dashboard bugs.  That is a start....23:14
johnsomI created a dashboard tag23:15
*** ianychoi has quit IRC23:17
*** ianychoi has joined #openstack-lbaas23:17
openstackgerritAdam Harwell proposed openstack/octavia master: WIP: Floating IP Network Driver (spans L3s)  https://review.openstack.org/43561223:18
rm_workoh hmm23:20
rm_workapparently if the HM fails to create an AMP once, like because of a keystone timeout error (what just happened to me), that whole thread dies :/23:20
rm_workerr sorry, not HM... HouseKeeping23:20
rm_workmy spares pool just randomly stopped filling23:21
rm_workand just found out why23:21
johnsomSame issue was we had in HM?23:22
rm_workessentially, yeah23:22
*** yamamoto has joined #openstack-lbaas23:28
rm_workyeah hmmm, this is not locking23:28
rm_workone is waiting for the other to finish i think >_>23:28
rm_workactually no23:29
rm_workthey both get past the lock part instantly O_o23:29
rm_workis this mark_amphora_busy correct?23:29
rm_worki think because it's not committing it automatically maybe? or something?23:30
rm_workthey can both "get_for_update" and both show the amp as not busy? dunno23:30
rm_workjohnsom: http://www.diff-online.com/view/59a5f95c9ee0623:31
johnsomSo, rewind for me here.  What was HK doing that caused this?  Building a spare and keystone failed?23:31
rm_workyeah23:31
rm_workkeystone just had a timeout23:32
rm_workfor whatever reason23:32
rm_workit happens sometimes in this env23:32
johnsomSo, the diff is HM, is it HK or HM that we are working on here?23:32
rm_worksorry, two things23:32
rm_workI mentioned the HK issue offhand23:32
rm_workthe diff is for the HM locking thing that you helped with23:33
rm_workwhich ... just doesn't seem to be locking23:33
johnsomYeah, ok, so HK is not handling exceptions and isn't using a thread dispatcher, so yeah, one dies, it's done.  Did we switch that from a thread pool?23:34
johnsomUgh23:34
johnsomEasy enough to fix however23:34
rm_worki can look at that later today or tomorrow23:35
johnsomrm_work are you fixing HK or should I open a bug for that?23:36
rm_workbut you can open a bug if you want23:36
johnsomhttps://bugs.launchpad.net/octavia/+bug/171387323:38
openstackLaunchpad bug 1713873 in octavia "Housekeeping is not handling exceptions" [Critical,Triaged]23:38
johnsomOk, looking at the HM thing now23:38
*** yamamoto has quit IRC23:46
johnsomrm_work So, yep, not working....  Want me to try a different approach with sqlalchemy?23:47
*** rstarmer has joined #openstack-lbaas23:47
johnsomI think this: https://review.openstack.org/#/c/435612/48/octavia/db/repositories.py isn't explicit enough for sqla23:48
rm_workhmm23:50
rm_worki mean, i am always open to help here23:50
rm_workthis whole thing seems wonky >_>23:50
johnsomOk, give me a couple of minutes and I will take another approach on it23:51
rm_workok cool23:51
rm_workxgerman_: ok current version of script works23:51
rm_workif you redownload that one i linked you (the oct_dash branch)23:51
rm_workinstalls neutron-lbaas and the dashboard correctly the first time :)23:51
rm_worktesting a LB create with this dashboard23:53
rm_worklooks good23:59
rm_workI'm +223:59
johnsomThanks23:59
johnsomHow easy is it for you to test this flip case?23:59
rm_workeasy23:59
johnsomsweet23:59
rm_workmaybe ... 60s to update the HMs with new code and then trigger a test23:59
« Monday, 2017-08-28 Index Wednesday, 2017-08-30 »

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!