| -opendevstatus- NOTICE: zuul was stuck processing jobs and has been restarted. pending jobs will be re-enqueued | 10:01 | |
| *** akahat is now known as akahat|afk | 11:54 | |
| opendevreview | Soniya Murlidhar Vyas proposed openstack/openstack-ansible-os_tempest master: [WIP] Need to add netstat logging in the background https://review.opendev.org/c/openstack/openstack-ansible-os_tempest/+/813990 | 12:20 |
|---|---|---|
| *** akahat|afk is now known as akahat | 13:10 | |
| spatel | jamesdenton morning! | 13:33 |
| jamesdenton | good morning | 13:33 |
| spatel | sorry yesterday i soaked up with prod issue | 13:33 |
| jamesdenton | naw, it's ok. get it worked out? | 13:33 |
| spatel | all good :) damn memory issue causing strange behavior | 13:34 |
| jamesdenton | i bet | 13:34 |
| jamesdenton | i kinda answered my own question yesterday regarding OVN clustering. I realized that your changes came in after Wallaby, and I didnt have em. | 13:35 |
| spatel | oh! | 13:37 |
| spatel | yes OVN clustering is part of wallaby release | 13:37 |
| spatel | is it working ok for you? | 13:37 |
| spatel | please find bug or any kind of issue because i don't who else using clustering can provide feedback | 13:38 |
| jamesdenton | it's in master but not wallaby | 13:39 |
| spatel | really? | 13:39 |
| jamesdenton | yeah | 13:39 |
| spatel | i am surprised why its not in wallaby. i thought its part of wallaby | 13:40 |
| spatel | so are we going to target it for Xena ? | 13:40 |
| spatel | noonedeadpunk ^ | 13:40 |
| jamesdenton | it should be there for xena, i would think | 13:41 |
| mgariepy | i guess the clustering part can be backported.. | 13:41 |
| jamesdenton | or that | 13:41 |
| spatel | mgariepy +1 | 13:41 |
| noonedeadpunk | yeah, I guess it can | 13:41 |
| mgariepy | wans't really supported before. so no upgrade issue there.. | 13:41 |
| spatel | yes its independent component and not going to hurt anyone else | 13:41 |
| jamesdenton | anyway, it seems to work OK so far but haven't really tested "failover" persay | 13:41 |
| jamesdenton | i did kill the HAproxy VIPs since they aren;t needed | 13:41 |
| jamesdenton | and didn't quite work anyway, it seems | 13:42 |
| spatel | we don't need haproxy vip for OVN | 13:42 |
| jamesdenton | right | 13:42 |
| mgariepy | it would be awesome to have some docs on the ops that needs/could happen with it. | 13:42 |
| jamesdenton | they were in there pre-clustering support | 13:42 |
| spatel | ovn-controller talk to central node and they auto figure out who is master instead of haproxy blindly send request to slave node and then redirect happened | 13:43 |
| spatel | jamesdenton yes they were | 13:43 |
| spatel | i am planning to do small doc around that related how to recover cluster or add new and remove dirty node | 13:44 |
| jamesdenton | spatel mgariepy if you so please: https://review.opendev.org/c/openstack/openstack-ansible/+/813858 | 13:46 |
| jamesdenton | actually - i do need to add a release note | 13:46 |
| spatel | yes | 13:46 |
| mgariepy | yep ;) | 13:47 |
| mgariepy | haha | 13:47 |
| mgariepy | spatel, also how to validate everything is healthy. | 13:48 |
| jamesdenton | openstack server create. ping successful. ... profit | 13:48 |
| spatel | cluster health right? | 13:48 |
| spatel | https://paste.opendev.org/show/810010/ | 13:50 |
| spatel | may be - Entries not yet committed: | 13:50 |
| mgariepy | nice. that with a wrapper text on how to read/parse it :D | 13:51 |
| spatel | i am running in lab so not enough data | 13:52 |
| spatel | i have plan to deploy in prod and waiting for downtime to deploy. i may rebuild whole cluster | 13:53 |
| opendevreview | James Denton proposed openstack/openstack-ansible master: Remove OVN-related haproxy configuration https://review.opendev.org/c/openstack/openstack-ansible/+/813858 | 13:55 |
| jamesdenton | spatel it's lab here, too. but it's something | 13:56 |
| mgariepy | jamesdenton, did you swtich from lxb > ovn directly | 13:58 |
| jamesdenton | yes | 13:58 |
| mgariepy | you kept the router and stuff around ? | 13:58 |
| jamesdenton | but, i killed all of my VMs first. simplified things | 13:58 |
| jamesdenton | no, killed the routers, too. Kept the networks | 13:58 |
| mgariepy | ok | 13:58 |
| jamesdenton | routers probably would have been OK, especially if admin down | 13:59 |
| mgariepy | jamesdenton, dont you do the swap like indiana jones? | 14:07 |
| spatel | hehe | 14:11 |
| jamesdenton | when i'm not being lazy | 14:50 |
| mgariepy | haha | 14:52 |
| spatel | jamesdenton we need to figure out how to setup SSL with ovn using OSA pki | 15:46 |
| *** dpawlik5 is now known as dpawlik | 16:37 | |
| spatel | folk, any good way i can add metadata of host hardware info to guest machine? | 17:02 |
| spatel | we want to identify on which hardware my guest VM running, like DELL or HP or more .. | 17:03 |
| spatel | I am looking at dynamic metdata but not sure how to consume it | 17:03 |
| mgariepy | arg .. i need some idea now .. upgrading my testbed worked flawlessly.. doing prod now network os broken. | 17:33 |
| mgariepy | some tap interface are state LOWERLAYERDOWN | 17:33 |
| jamesdenton | what kind of upgrade? | 17:38 |
| mgariepy | Rocky to stein | 17:43 |
| mgariepy | well.. the LAYERDOWN was some ha router interface. | 18:43 |
| mgariepy | tl;dr; rabbitmq ... neutron went crazy. all network moved to 1 node ... and you know how it is with lxb and 400 router.. | 19:32 |
| jamesdenton | glad you worked it out. sorry i missed it | 20:48 |
| jamesdenton | are these considered zero-downtime upgrades for you? | 20:49 |
| mgariepy | well.. sometimes it does happens... i guess | 21:02 |
| mgariepy | on my testbed it did work perfectly. | 21:02 |
| mgariepy | i was kinda amazed that it did work correcly on the testbed.. | 21:06 |
| mgariepy | technically the server are still up.. | 21:06 |
| mgariepy | hahah | 21:06 |
| mgariepy | anyway when it's too late it's too late. | 21:07 |
| mgariepy | with ovn the network realigmemnt would be a lot faster anyway. | 21:08 |
| mgariepy | with lxb it's kinda a bunch of `ip` `sysctl` and `arping` command glued together with dolorama duct tape | 21:10 |
| *** fresta is now known as Guest2919 | 22:04 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!