opendevreview | ZhouHao proposed openstack/ironic master: [iRMC] Convert the type of irmc_port to int https://review.opendev.org/c/openstack/ironic/+/817154 | 01:08 |
---|---|---|
opendevreview | Jacob Anders proposed openstack/sushy master: Changing boot device string for vmedia on SuperMicro https://review.opendev.org/c/openstack/sushy/+/817137 | 06:17 |
opendevreview | ZhouHao proposed openstack/ironic master: [iRMC] Convert the type of irmc_port to int https://review.opendev.org/c/openstack/ironic/+/817154 | 06:51 |
iurygregory | good morning Ironic o/ | 06:53 |
arne_wiebalck | Good morning iurygregory and Ironic! | 07:04 |
iurygregory | morning arne_wiebalck o/ | 07:04 |
* iurygregory needs more coffee | 07:05 | |
arne_wiebalck | +1 | 07:05 |
rpittau | good morning ironic! o/ | 08:13 |
iurygregory | good morning rpittau o/ | 08:14 |
rpittau | hey iurygregory :) | 08:14 |
opendevreview | Riccardo Pittau proposed openstack/ironic-python-agent master: Move rescan device function to general utils https://review.opendev.org/c/openstack/ironic-python-agent/+/816916 | 08:29 |
*** akahat|rover is now known as akahat|lunch | 08:44 | |
dtantsur | morning ironic | 08:53 |
iurygregory | morning dtantsur o/ | 08:55 |
*** akahat|lunch is now known as akahat|rover | 09:13 | |
*** melwitt is now known as Guest5508 | 10:12 | |
opendevreview | Jacob Anders proposed openstack/sushy master: Changing boot device string for vmedia on SuperMicro https://review.opendev.org/c/openstack/sushy/+/817137 | 10:32 |
opendevreview | Jacob Anders proposed openstack/sushy master: Changing boot device string for vmedia on SuperMicro https://review.opendev.org/c/openstack/sushy/+/817137 | 10:40 |
*** dviroel|out is now known as dviroel | 11:16 | |
opendevreview | Dmitry Tantsur proposed openstack/ironic master: Stop relying on explicit values of Redfish constants https://review.opendev.org/c/openstack/ironic/+/817386 | 11:28 |
dtantsur | could I get a 2nd +2 on https://review.opendev.org/c/openstack/sushy-tools/+/806253 please? | 11:39 |
iurygregory | dtantsur, I will take a look later today =) | 13:25 |
dtantsur | thx! | 13:32 |
opendevreview | Bernd Mueller proposed openstack/ironic-python-agent-builder master: add element-deps for element burn-in https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/817408 | 13:42 |
opendevreview | Bernd Mueller proposed openstack/ironic-python-agent-builder master: more details where to find the iso-image-create script https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/817430 | 13:49 |
opendevreview | Riccardo Pittau proposed openstack/ironic-python-agent master: Move rescan device function to general utils https://review.opendev.org/c/openstack/ironic-python-agent/+/816916 | 14:34 |
opendevreview | Dmitry Tantsur proposed openstack/sushy master: WIP certificate collection https://review.opendev.org/c/openstack/sushy/+/815479 | 14:43 |
rpittau | ehm... is https://zuul.openstack.org/status working for anyone ? | 14:58 |
rpittau | ooook looking at the opendev channel seems there is an actual issue, all good | 15:02 |
opendevreview | Bernd Mueller proposed openstack/ironic-python-agent-builder master: add element-deps for element burn-in https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/817408 | 15:21 |
iurygregory | I really don't get why gerrit wants to set attention for the owner of the patch when we add +2 and/or +W LOL | 15:38 |
dtantsur | yeah, I'm puzzled just as well | 15:57 |
dtantsur | see you tomorrow folks | 15:57 |
iurygregory | bye dtantsur o/ | 15:57 |
*** akahat|rover is now known as akahat|lunch | 16:03 | |
*** akahat|lunch is now known as akahat|dinner | 16:03 | |
rpittau | bye everyone, see you on monday! o/ | 16:31 |
opendevreview | Merged openstack/sushy-tools master: Support uploading certificates for virtual media https://review.opendev.org/c/openstack/sushy-tools/+/806253 | 16:45 |
*** akahat|dinner is now known as akahat|rover | 16:52 | |
arne_wiebalck | bye everyone, see you tomorrow o/ | 17:05 |
TheJulia | NobodyCam: you around? | 17:09 |
NobodyCam | no ;p | 17:09 |
NobodyCam | hehehehe | 17:09 |
NobodyCam | Good Morning Ironic Folks! | 17:09 |
NobodyCam | I feel I am more rectangle then round | 17:10 |
NobodyCam | I am actually here | 17:15 |
TheJulia | I mean, I guess one can be square | 17:15 |
TheJulia | we don't judge people here ;) | 17:15 |
NobodyCam | LOL | 17:15 |
NobodyCam | how are you this morning? | 17:15 |
TheJulia | Next, we'll find out NobodyCam is alternating between being a greaser and then a square | 17:16 |
NobodyCam | hehehehe | 17:16 |
TheJulia | So, I'm working on this nova patch | 17:17 |
TheJulia | ... and you mentioned to me you had compute nodes just disappear rippling out from a rebalance right? | 17:18 |
NobodyCam | yes! | 17:19 |
NobodyCam | oh I see https://review.opendev.org/c/openstack/nova/+/813263 merged :happy_dance: | 17:19 |
NobodyCam | say one has three conductors (1,2,3) each with it's own nova-compute running on the conductor. restart nova-compute on say 2 and see what nodes are missing | 17:21 |
TheJulia | just a restart, and... oh | 17:25 |
NobodyCam | only way I know ti ensure all nodes make it back, is to shut down all the ironic nova-compute services and restart one by one letting each instance reach active state before moving on to the next | 17:25 |
TheJulia | oh, ouch | 17:26 |
NobodyCam | s/ti/to/ | 17:26 |
TheJulia | yeah | 17:26 |
TheJulia | because the hash ring rebalance says otherwise | 17:26 |
TheJulia | so yeah. ouch | 17:26 |
NobodyCam | 813263 will make a massive improvement in amount of time it takes | 17:27 |
NobodyCam | if there was a node reregister function that would force recreate the missing hypervisor record that would help a bunch, as the only other option I know of to recover the hypervisor record is to delete and reenroll the node | 17:29 |
TheJulia | NobodyCam: https://github.com/openstack/nova/blob/fded762f4df26ff5706438a66da33ff966f833c6/nova/compute/manager.py#L10024 right? | 17:32 |
TheJulia | oh gawd | 17:35 |
NobodyCam | I see 12 occurrences of that in the last 24 hours | 17:36 |
TheJulia | if the hash ring calculates it out but the record in the db it should be on the host | 17:36 |
TheJulia | it deletes it | 17:36 |
TheJulia | so basically, node x gets moved to compute y | 17:36 |
NobodyCam | this is what I see: | 17:40 |
NobodyCam | https://www.irccloud.com/pastebin/6AY0a17B/ | 17:40 |
TheJulia | any evidence of a rebalance in the logs? | 17:42 |
NobodyCam | that region listed 4 occurrences of Deleting orphan compute node in the last 24 hours | 17:43 |
TheJulia | bingo | 17:43 |
TheJulia | well | 17:44 |
TheJulia | semi-bingo | 17:44 |
TheJulia | NobodyCam: is there already a bug filed for this that your aware of? | 17:56 |
NobodyCam | I thought so | 17:58 |
NobodyCam | https://bugs.launchpad.net/nova/+bug/1825876 | 18:01 |
NobodyCam | maybe | 18:01 |
TheJulia | kind of yeah | 18:05 |
TheJulia | instance goes invisible in the list, I think they meant nova-compute service in their thing | 18:05 |
TheJulia | anyway, yeah | 18:05 |
TheJulia | that would trigger that case | 18:05 |
* TheJulia awaits git review to return | 18:13 | |
TheJulia | NobodyCam: https://review.opendev.org/c/openstack/nova/+/813897 | 18:18 |
TheJulia | arne_wiebalck: ^ | 18:18 |
* NobodyCam clicks the link | 18:18 | |
TheJulia | NobodyCam: https://review.opendev.org/c/openstack/nova/+/817467 might be what you really need in the end | 18:43 |
TheJulia | which should fully prevent orphaned nodes, and the logic changes... I think... should prevent the need for all of the computes to be bounced | 18:44 |
opendevreview | Julia Kreger proposed openstack/ironic master: Test nova-compute fix https://review.opendev.org/c/openstack/ironic/+/813264 | 18:59 |
opendevreview | Julia Kreger proposed openstack/ironic-python-agent bugfix/8.1: Output verbose info from efibootmgr https://review.opendev.org/c/openstack/ironic-python-agent/+/817468 | 19:06 |
opendevreview | Julia Kreger proposed openstack/ironic-python-agent bugfix/8.1: Delete EFI boot entry duplicate labels first https://review.opendev.org/c/openstack/ironic-python-agent/+/817469 | 19:06 |
opendevreview | Julia Kreger proposed openstack/ironic-python-agent bugfix/8.1: Fix UEFI record regex https://review.opendev.org/c/openstack/ironic-python-agent/+/817470 | 19:06 |
opendevreview | Merged openstack/ironic-python-agent master: Re-read the partition table with partx -a, part 2 https://review.opendev.org/c/openstack/ironic-python-agent/+/817197 | 19:19 |
opendevreview | Julia Kreger proposed openstack/ironic-python-agent master: Remove metalsmith centos7 job https://review.opendev.org/c/openstack/ironic-python-agent/+/817472 | 19:30 |
opendevreview | Julia Kreger proposed openstack/metalsmith master: CI: remove old centos7 job https://review.opendev.org/c/openstack/metalsmith/+/813249 | 19:36 |
opendevreview | Riccardo Pittau proposed openstack/ironic-python-agent stable/xena: Re-read the partition table with partx -a, part 2 https://review.opendev.org/c/openstack/ironic-python-agent/+/817419 | 19:53 |
opendevreview | Riccardo Pittau proposed openstack/ironic-python-agent bugfix/8.1: Re-read the partition table with partx -a, part 2 https://review.opendev.org/c/openstack/ironic-python-agent/+/817420 | 19:53 |
opendevreview | Riccardo Pittau proposed openstack/ironic-python-agent stable/wallaby: Re-read the partition table with partx -a, part 2 https://review.opendev.org/c/openstack/ironic-python-agent/+/817421 | 19:53 |
TheJulia | rpittau: should those partition table read changes be back ported further? | 20:19 |
opendevreview | Verification of a change to openstack/ironic bugfix/18.1 failed: Fix idrac-wsman deploy with existing non-BIOS jobs https://review.opendev.org/c/openstack/ironic/+/817107 | 20:53 |
*** dviroel is now known as dviroel|out | 21:15 | |
janders | good morning Ironic o/ | 22:17 |
TheJulia | good morning janders | 22:44 |
TheJulia | so quesiton for folks, are we aware older builds of tinyipa are broken? | 22:44 |
janders | hey TheJulia | 22:53 |
opendevreview | Jacob Anders proposed openstack/sushy master: Changing boot device string for vmedia on SuperMicro https://review.opendev.org/c/openstack/sushy/+/817137 | 23:59 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!