| hgy | Good morning, Ironic! | 02:20 |
|---|---|---|
| opendevreview | Steve Baker proposed openstack/bifrost master: Use distro conf loader_file_paths to copy ipxe/grub files https://review.opendev.org/c/openstack/bifrost/+/818709 | 03:59 |
| opendevreview | Steve Baker proposed openstack/bifrost master: WIP support grub network boot https://review.opendev.org/c/openstack/bifrost/+/807220 | 03:59 |
| rpittau | good morning ironic! o/ | 06:40 |
| opendevreview | Riccardo Pittau proposed openstack/bifrost master: Remove unused integration job https://review.opendev.org/c/openstack/bifrost/+/819057 | 06:52 |
| opendevreview | Riccardo Pittau proposed openstack/bifrost master: [WIP] Run bifrost on centos 9 stream https://review.opendev.org/c/openstack/bifrost/+/819058 | 06:57 |
| arne_wiebalck | Good morning hgy rpittau and Ironic! | 07:16 |
| opendevreview | Merged openstack/sushy master: Migrate Manager constants to enums https://review.opendev.org/c/openstack/sushy/+/818806 | 07:30 |
| rpittau | hey arne_wiebalck :) | 07:34 |
| opendevreview | Merged openstack/sushy master: Migrate System Storage constants to enums https://review.opendev.org/c/openstack/sushy/+/818396 | 07:38 |
| iurygregory | good morning arne_wiebalck rpittau and Ironic o/ | 07:45 |
| rpittau | hey iurygregory :) | 07:45 |
| arne_wiebalck | hey iurygregory o/ | 07:46 |
| rpittau | if anyone has a moment please review https://review.opendev.org/c/openstack/python-ironic-inspector-client/+/818523 so we can add it to the milestone-1 yoga release | 08:08 |
| iurygregory | looking now | 08:12 |
| iurygregory | lgtm | 08:14 |
| rpittau | thanks | 08:15 |
| opendevreview | Merged openstack/sushy master: Add support for additional network resources. https://review.opendev.org/c/openstack/sushy/+/813850 | 08:16 |
| hgy | Good morning arne_wiebalck and rpittau! | 08:20 |
| rpittau | hey hgy :) | 08:20 |
| opendevreview | Dmitry Tantsur proposed openstack/ironic master: Add a unit test job with Sushy from source https://review.opendev.org/c/openstack/ironic/+/818553 | 09:05 |
| ajya | dtantsur, TheJulia - I updated https://storyboard.openstack.org/#!/story/2008723 after RFE review feedback in a meeting. Removed zip support and added download_source. Is this OK to proceed with implementation? | 09:08 |
| dtantsur | good morning folks | 09:13 |
| dtantsur | ajya: will check in a few. ENEEDMORECOFFEE. | 09:13 |
| iurygregory | good morning dtantsur | 09:13 |
| rpittau | mmm I thought we already had centos-9-stream nodesets | 09:21 |
| hgy | Hello, Could I ask a question? | 09:24 |
| hgy | When I validate node use "openstack baremetal node validate $NODE_UUID" | 09:25 |
| hgy | I got "| network | False | Unexpected exception, traceback saved into log by ironic conductor service that is running on controller: 'ServiceTokenAuthWrapper' object has no attribute '_discovery_cache'" | 09:25 |
| hgy | Does anybody know why that is | 09:25 |
| dtantsur | hgy: you need to check the logs for a full traceback. but it seems to be coming from somewhere in the keystoneauth library, maybe an old version? | 09:31 |
| hgy | love you, dtantsur | 09:34 |
| opendevreview | Merged openstack/bifrost master: Upgrade from Xena in the upgrade job https://review.opendev.org/c/openstack/bifrost/+/818878 | 09:52 |
| opendevreview | Dmitry Tantsur proposed openstack/sushy master: Support for creating and deleting virtual media TLS certificates https://review.opendev.org/c/openstack/sushy/+/817792 | 10:02 |
| dtantsur | TheJulia: re https://review.opendev.org/c/openstack/releases/+/817593, there are quite a few unreleased ussuri changes. do we care? | 10:27 |
| opendevreview | Merged openstack/sushy master: Migrate CompositionService, EventService and TaskService to enums https://review.opendev.org/c/openstack/sushy/+/818797 | 10:50 |
| opendevreview | Dmitry Tantsur proposed openstack/sushy master: Migrate Chassis to enums https://review.opendev.org/c/openstack/sushy/+/818941 | 10:51 |
| opendevreview | Merged openstack/ironic master: Clarify driver load error message https://review.opendev.org/c/openstack/ironic/+/818973 | 11:00 |
| opendevreview | Dmitry Tantsur proposed openstack/sushy master: Migrate Fabric to enums https://review.opendev.org/c/openstack/sushy/+/819097 | 11:14 |
| opendevreview | Merged openstack/bifrost stable/wallaby: Fix the DHCP port in iptables (68 vs 67 and TCP vs UDP) https://review.opendev.org/c/openstack/bifrost/+/818747 | 11:22 |
| opendevreview | Dmitry Tantsur proposed openstack/sushy master: Migrate System Network to enums https://review.opendev.org/c/openstack/sushy/+/819099 | 11:27 |
| opendevreview | Merged openstack/python-ironic-inspector-client master: Improve testing https://review.opendev.org/c/openstack/python-ironic-inspector-client/+/818523 | 12:26 |
| opendevreview | Merged openstack/sushy master: Migrate Chassis to enums https://review.opendev.org/c/openstack/sushy/+/818941 | 12:49 |
| TheJulia | dtantsur: I guess we ought to actual have a real final release | 13:53 |
| TheJulia | the release team is supposed to do that not just tag the last release... | 13:53 |
| iurygregory | ++ for a final release in ussuri before we put to EM | 14:08 |
| iurygregory | TheJulia, dtantsur rpittau since most of the themes in Yoga we are the primary contacts please take a look https://review.opendev.org/c/openstack/ironic-specs/+/815308 =) | 14:14 |
| TheJulia | I can likely lookup the hashes and do it later today | 14:15 |
| TheJulia | the release that is. I'm in the middle of my nova patch of doom | 14:15 |
| rpittau | iurygregory: I'll have a look before EOD today | 14:15 |
| TheJulia | rpittau: thanks | 14:15 |
| * TheJulia writes words because apparently we never wrote doc strings | 14:15 | |
| iurygregory | yeah no worries =) the idea is that we can merge till friday =) | 14:15 |
| TheJulia | and.. honestly, we could drag our feet if we have to | 14:19 |
| TheJulia | but I don't think anyone is actually really using ussuri | 14:19 |
| opendevreview | Riccardo Pittau proposed openstack/ironic master: Remove default option from create iso image https://review.opendev.org/c/openstack/ironic/+/818874 | 15:06 |
| opendevreview | Riccardo Pittau proposed openstack/ironic master: [WIP] replace mkisofs with pycdlib https://review.opendev.org/c/openstack/ironic/+/819121 | 15:06 |
| opendevreview | Iury Gregory Melo Ferreira proposed openstack/ironic-specs master: Yoga Themes https://review.opendev.org/c/openstack/ironic-specs/+/815308 | 15:07 |
| opendevreview | Riccardo Pittau proposed openstack/ironic master: [WIP] replace mkisofs with pycdlib https://review.opendev.org/c/openstack/ironic/+/819121 | 15:08 |
| TheJulia | I'm finding myself researching, finding "no, that is actually the right thing to do" and adding more comments this morning | 15:11 |
| TheJulia | arne_wiebalck: you around? | 15:15 |
| arne_wiebalck | TheJulia: yes o/ | 15:15 |
| TheJulia | arne_wiebalck: I remember a while back, you restarted all your nova-computes or something like that | 15:15 |
| TheJulia | was that all in just rapid succession? or was there time between the restarts? | 15:16 |
| TheJulia | if rapid succession, I'm guessing cache warming was the primary problem and then resulting instability from it | 15:17 |
| arne_wiebalck | TheJulia: the context is thundering herd/startup issues? | 15:17 |
| TheJulia | yeah | 15:17 |
| arne_wiebalck | hmm | 15:18 |
| * arne_wiebalck tries to remember | 15:18 | |
| arne_wiebalck | usually, we do not restart the nova-computes, but only the ironic controllers, and this we do in a staged manner | 15:19 |
| TheJulia | there was some reason that was out of the ordinary | 15:19 |
| arne_wiebalck | tbh, I do not remember atm if restarting all of nova-compute would also cause issues ... | 15:19 |
| TheJulia | I guess what I'm getting at is if it would be good or if I need to account for something else because I'm literally in the middle of that code | 15:19 |
| arne_wiebalck | let me check with belmiro, he may remember ... brb | 15:20 |
| TheJulia | k | 15:20 |
| arne_wiebalck | belmiro is not available atm, I will let you know later, sorry | 15:22 |
| TheJulia | no worries | 15:22 |
| arne_wiebalck | does nova get all nodes on startup, or only the ones in the CG? | 15:23 |
| TheJulia | I think it is a bit of both | 15:24 |
| arne_wiebalck | heh | 15:24 |
| TheJulia | mainly focused on the cg | 15:24 |
| arne_wiebalck | right | 15:24 |
| arne_wiebalck | but will that result in a full table scan or is there an index? | 15:24 |
| TheJulia | each compute process tries to build a cache of all of the nodes based upon instances presently along with what matches the hash ring | 15:25 |
| TheJulia | so we're trying to navigate "fixing" that | 15:25 |
| TheJulia | and the related failure cases | 15:25 |
| TheJulia | it *is* column based for the cache, so there is at least that | 15:25 |
| TheJulia | so maybe nova-computes starting up only added fuel to the fire in that case | 15:26 |
| dtantsur | o/ | 16:12 |
| *** akahat|rover is now known as akahat | 16:33 | |
| NobodyCam | Good Morning Ironic'ers and OFC... | 16:45 |
| NobodyCam | Happy Virtual Friday | 16:45 |
| rpittau | hey NobodyCam :) | 16:45 |
| NobodyCam | ;p | 16:45 |
| NobodyCam | morning rpittau | 16:45 |
| rpittau | happy US virtual friday :P | 16:45 |
| NobodyCam | hehehehe | 16:45 |
| rpittau | good night! o/ | 16:49 |
| NobodyCam | night rpittau | 16:50 |
| NobodyCam | see you next week | 16:50 |
| iurygregory | good morning NobodyCam o/ | 16:57 |
| iurygregory | for me is almost virtual friday also :D (tomorrow PTO \o/) | 16:57 |
| NobodyCam | :) top of the morning iurygregory | 16:57 |
| NobodyCam | nice! | 16:58 |
| arne_wiebalck | TheJulia: Belmiro does not recall it being related to a nova restart. | 17:30 |
| TheJulia | Good morning NobodyCam | 17:35 |
| TheJulia | arne_wiebalck: ack | 17:36 |
| NobodyCam | Morning TheJulia :) | 17:40 |
| NobodyCam | o/ arne_wiebalck | 17:41 |
| TheJulia | brraaaaaaains | 19:39 |
| stevebaker[m] | morning | 19:56 |
| arne_wiebalck | Hey NobodyCam o/ | 20:22 |
| arne_wiebalck | TheJulia: I should probably spent some time to get numbers on the background noise the Ironic/Nova control plane is creating on the database ... it is quite substantial I think. | 20:23 |
| arne_wiebalck | Getting numbers is probably the easy part. Understanding them may take some time. | 20:25 |
| arne_wiebalck | Questions like: with X thousand nodes in Y conductor groups and a power sync interval of Z seconds, we see a select rate N select ops/s on the database. | 20:31 |
| arne_wiebalck | The DB monitoring tells me 30k rows are read per second with X=8.5, Y=23 (well, 25 controllers in 23CGs) and Z=600. | 20:39 |
| arne_wiebalck | That seems way too much ... | 20:40 |
| opendevreview | Steve Baker proposed openstack/ironic master: Capture [pxe]loader_file_paths for distros https://review.opendev.org/c/openstack/ironic/+/815392 | 20:53 |
| opendevreview | Steve Baker proposed openstack/ironic master: Write master grub config on startup https://review.opendev.org/c/openstack/ironic/+/815580 | 20:56 |
| opendevreview | Steve Baker proposed openstack/ironic master: Capture [pxe]loader_file_paths for distros https://review.opendev.org/c/openstack/ironic/+/815392 | 20:56 |
| opendevreview | Julia Kreger proposed openstack/ironic-tempest-plugin master: Move API base out of admin test location https://review.opendev.org/c/openstack/ironic-tempest-plugin/+/819164 | 21:16 |
| opendevreview | Julia Kreger proposed openstack/ironic-tempest-plugin master: WIP: An idea for rbac positive/negative testing https://review.opendev.org/c/openstack/ironic-tempest-plugin/+/819165 | 21:16 |
| TheJulia | dtantsur: if you have some braincells w/r/t tempest, I'd appreciate a glance to see if you think I'm heading in a bonkers direction with ^^^. The idea would be to always try to call everything and well go from there. Providing a list of what *should* work allows us to raise an exception upon failure or unexpected success. Naturally a ton of work ahead :( | 21:18 |
| TheJulia | stevebaker[m]: also ^^ | 21:18 |
| TheJulia | it might be totally bonkers though | 21:19 |
| stevebaker[m] | oki | 21:19 |
| TheJulia | arne_wiebalck: ++ although just rps may not be an acruate measurement with a larger pool of machines, I think it comes down to making sure we query smartly and when we need to. Surely there are places where we likely don't need to | 21:20 |
| opendevreview | Julia Kreger proposed openstack/ironic master: Test a cross repo fix https://review.opendev.org/c/openstack/ironic/+/813264 | 21:31 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!