Tuesday, 2022-04-12

frickleroh, interesting, so octavia is doing their own BGP setup instead of integrating with neutron?07:08
grami[m] Would that mean anycast could be an option in octavia to span over data centers? that would be a huge bonus for octavia. 07:31
opendevreviewGregory Thiemonge proposed openstack/octavia stable/xena: Restart rsyslog from cloud-init  https://review.opendev.org/c/openstack/octavia/+/83754710:49
opendevreviewGregory Thiemonge proposed openstack/octavia stable/wallaby: Restart rsyslog from cloud-init  https://review.opendev.org/c/openstack/octavia/+/83754810:50
opendevreviewMerged openstack/octavia stable/wallaby: Add fips jobs  https://review.opendev.org/c/openstack/octavia/+/82790011:25
opendevreviewMerged openstack/octavia stable/wallaby: Use centos amphora image in the FIPS jobs  https://review.opendev.org/c/openstack/octavia/+/83176711:25
johnsomfrickler there will still be integration points with neutron for ports and such, but the idea is to make the amphorae BGP participants13:22
johnsomgrami[m] yes, it helps solve the cross AZ/datacenter HA issue if there is no shared L2 for the VIP.13:24
opendevreviewGregory Thiemonge proposed openstack/octavia-tempest-plugin master: Testing c9s job with new ssh key type  https://review.opendev.org/c/openstack/octavia-tempest-plugin/+/83759716:38
opendevreviewGregory Thiemonge proposed openstack/octavia-tempest-plugin master: Testing c9s job with new ssh key type  https://review.opendev.org/c/openstack/octavia-tempest-plugin/+/83759716:46
opendevreviewGregory Thiemonge proposed openstack/octavia-tempest-plugin master: Testing c9s job with new ssh key type  https://review.opendev.org/c/openstack/octavia-tempest-plugin/+/83759717:35
rm_workjohnsom: ever noticed a DB-side issue with members where they seem to be "fine" but the driver code can't convert the model to the provider dict?19:22
rm_work  File "/opt/openstack/venv/octavia/lib/python3.7/site-packages/octavia/api/drivers/utils.py", line 445, in db_member_to_provider_member19:22
rm_work    new_member_dict = member_dict_to_provider_dict(db_member.to_dict())19:22
rm_workAttributeError: 'NoneType' object has no attribute 'to_dict'19:22
rm_workvery interesting, looking for root cause now19:23
rm_workimplies the db_member never came back from the DB but it is obviously THERE19:23
johnsomNo, doesn't ring a bell for me. What action was going on?19:24
rm_worklooks like just a pool GET O_o19:30
rm_workworking on repro19:30
opendevreviewMerged openstack/octavia stable/xena: Fix nft command line with negative priority values  https://review.opendev.org/c/openstack/octavia/+/82053619:31
opendevreviewMerged openstack/octavia stable/xena: Add keepalive for redis-based taskflow boards  https://review.opendev.org/c/openstack/octavia/+/82594419:31
johnsomThat doesn't make any sense an GET call won't call into the drivers, so wouldn't convert19:32
rm_workyeah looking at fully verifying, had ops folks forwarding me logs19:33
gthiemongerm_work: we have this open story: https://storyboard.openstack.org/#!/story/200912819:44
rm_workahhh interesting19:53
rm_workthat could be what's happening19:53
rm_workthat is the exact stack trace19:54
rm_workthanks gthiemonge I am much less confused now19:54
rm_workmaybe can help you on the repro/debugging?19:54
rm_workthough we're running... a shamefully old version I thinik19:55
johnsomA patch would be even more help. grin19:55
rm_work*I think, maybe ... Xena or something 😕19:55
rm_workwell can definitely replicate, will try to work out the bug20:22
rm_worki wonder if it's related to SQLAlchemy changes20:27
rm_workbecause it seems like the  duplication must  not be causing an exception now?20:27
rm_workoh nm i see20:41
rm_workde.columns is: ['member.uq_member_pool_id_address_protocol_port']20:43
rm_workoslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'e046177a-13b3-4321-810b-5f8fdda1194a-' for key 'member.uq_member_pool_id_address_protocol_port'")20:43
rm_workconstraint_list = ['uq_member_pool_id_address_protocol_port']20:44
rm_workde.columns = ['member.uq_member_pool_id_address_protocol_port']20:44
rm_workI think that's a change in how SQLAlchemy returns the error20:44
rm_workit's including "member."20:44
rm_workupdating that fixes the problem20:45
rm_workI wonder how many other places we're going to run into this --  and if the solution is to change the contraints string, or if there's some SQLAlchemy tweak20:46
rm_workjohnsom / gthiemonge ^^20:46
johnsomrm_work code link?20:47
johnsomThanks, context switching is hard. grin20:48
rm_workso we check if the de.columns that comes back is either column_list or constraint_list20:48
rm_workwhat comes back is 'member.uq_member_pool_id_address_protocol_port'20:49
rm_workbut we are looking for 'member.uq_member_pool_id_address_protocol_port'20:49
rm_workboth mean the same thing but I think SQLAlchemy changed how they return that error20:49
johnsomWhy do we even care? Why not just catch the DBDuplicateEntry and bounce a DuplicateMemberEntry20:50
rm_workI remember us making changes previously to deal with this, possibly in the other direction20:50
rm_workI think there could be some other cause?20:50
johnsomrm_work It seems like we are trying to be too smart with little to no value here20:52
rm_workI SWEAR I remember a CR up for changing this stuff20:55
rm_workwtf was it20:55
rm_workwas it removing these from other places?20:55
johnsomYeah, this rings a bell for me too20:55
johnsomThose are the only two places I see currently20:56
johnsomlol, so, yeah20:57
rm_workMight have been an attempt to return a more useful error message? as it's not ALWAYS the ID20:58
johnsomrm_work I can push a patch for this if you are not already doing it20:58
rm_worksee the error we return is more useful20:58
rm_work`Another member on this pool is already using ip on protocol_port 4443 (HTTP 409) (Request-ID: req-67c1aba7-9ada-45e3-b1b7-b5202fdab498)`20:58
rm_workinstead it will return ... ??? something worse20:58
johnsomYeah, but we can still return that info without trying to look into the exception raised from sqlalchemy20:58
rm_workdifferentiating from ID?20:59
rm_workcould leave the "id" comparison and just else it20:59
rm_workrather than elif20:59
johnsomWho cares if it's ID or what column, it's still member with IP x and Port Y20:59
rm_workhmm well20:59
rm_workok yeah I was expecting we use `IDAlreadyExists` for checking21:00
rm_workbut we don't21:00
johnsomPlus, users can't specify their IDs, so....21:00
rm_workit must only bubble up21:00
johnsomIt would only matter if we added the backup/restore and allowed preserving the IDs21:01
johnsomAn nothing catches that IDAlreadyExists21:01
johnsomJust bounces it to the user21:01
opendevreviewMichael Johnson proposed openstack/octavia master: Fix duplicate object error messages  https://review.opendev.org/c/openstack/octavia/+/83762721:13
johnsomrm_work ^^^21:13
johnsomGuess I should have run the tests to see if we were testing that stuff21:14
opendevreviewMichael Johnson proposed openstack/octavia master: Fix duplicate object error messages  https://review.opendev.org/c/openstack/octavia/+/83762721:22
johnsomNo test coverage, but an un-used variable pep8, so fixed that.21:22
rm_workyeah ok that's ... fair I think. the user error would need to be that anyway21:26
johnsomSeems to be some devstack borkage21:43
johnsomOr keystone. devstack is blowing up setting up keystone21:44
johnsomHmm, no, pbr maybe. the other job failed installing bbq-client21:45
opendevreviewMichael Johnson proposed openstack/octavia master: Fix duplicate object error messages  https://review.opendev.org/c/openstack/octavia/+/83762721:50
johnsomYeah, everything is borked. lots-o-red on the zuul board21:51

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!