*** tosky has quit IRC | 00:06 | |
janders | TheJulia ACK regarding the NVMe work | 00:10 |
---|---|---|
*** anuradha1904 has quit IRC | 00:10 | |
TheJulia | good morning | 00:10 |
janders | TheJulia stevebaker dtantsur|afk regarding the API JSON sessions, I will join the second one, thank you for factoring in my timezone. | 00:10 |
janders | TheJulia stevebaker I have a question if you have time. I noticed something that seems weird behaviour by inspector+ironic while attempting to introspect a KVM-VM fake-baremetal that doesn't have a port defined. Inspector does have a go at doing it as it thinks it can lookup via IPMI address but it ends up failing the inspection and creating | 00:14 |
janders | another node in the process. In the CLI output the "flow" looks like this: http://paste.openstack.org/show/800044/ | 00:14 |
janders | I first suspected it's a bug introduced by my patches aiming to implement https://storyboard.openstack.org/#!/story/2008038 but after a day and a bit of testing and digging I tried this on an unpatched install and it did exactly the same thing. | 00:15 |
janders | have I found a bug? | 00:15 |
janders | I'd expect that if there isn't enough info to proceed successfully, the inspection should fail as opposed to resulting in ^^ | 00:16 |
janders | my (potentially naive) understanding is that the assumption that nodes can be looked up via IPMI address doesn't really work with VMs, hence the issue | 00:17 |
*** k_mouza has joined #openstack-ironic | 00:45 | |
*** k_mouza has quit IRC | 00:49 | |
stevebaker | janders: | 00:57 |
stevebaker | janders: thanks | 00:57 |
stevebaker | janders: inspector ends up creating another node!? | 00:57 |
janders | stevebaker yes! | 00:59 |
janders | I think it's auto-discovery being enabled by default in bifrost | 01:00 |
TheJulia | janders: so, I don't think that is a bug | 01:00 |
stevebaker | ah | 01:01 |
TheJulia | I think it is a side effect of auto-discovery + not being able to get the BMC address | 01:01 |
TheJulia | the human or software driving *needs* to provide something to align the node, mac being best, BMC address not always reliable | 01:01 |
*** Goneri has quit IRC | 01:01 | |
janders | TheJulia in this case, shouldn't it fail? | 01:01 |
TheJulia | how could it? | 01:01 |
TheJulia | well, inspection should fail yes | 01:01 |
TheJulia | but auto-discovery occured | 01:02 |
TheJulia | I don't think there is a way around that, tbh | 01:02 |
janders | TheJulia good points! :) | 01:03 |
* TheJulia cries at unit tests for _grub2_install | 01:04 | |
janders | to the operator it's obvious what's going on because of the timing of requesting inspection and the non-existent extra node popping up, but I don't have any good ideas how to build logic around that | 01:04 |
janders | the only thing I can think of is - is it useful to have Ironic create nodes with no IPMI address defined? | 01:05 |
janders | I'd think they can't do anything anyway | 01:05 |
* janders is testing if this is really the case | 01:05 | |
*** zzzeek has quit IRC | 01:06 | |
TheJulia | there are other bmc types | 01:07 |
TheJulia | and people actually use fake power | 01:07 |
janders | I can't power on, I can't provide, I think this auto-created node is unusable | 01:08 |
janders | I think this is a corner case with VMs | 01:08 |
*** zzzeek has joined #openstack-ironic | 01:09 | |
TheJulia | you can only not provide it if the hardware type results in power or management being ipmitool | 01:10 |
janders | if it were a real baremetal, what would IPA do? | 01:12 |
janders | getting BMC address is easy, how about the creds? | 01:12 |
* janders is looking which parts of inspector/ironic code do this | 01:13 | |
*** xinliang has joined #openstack-ironic | 01:15 | |
janders | hmm I have no enroll_node_driver set in ironic.conf and default='fake-hardware', wonder where ipmi comes from in this equation | 01:17 |
janders | barhttp://paste.openstack.org/show/800077/ | 01:18 |
janders | oops | 01:19 |
janders | http://paste.openstack.org/show/800077/ | 01:19 |
* janders is looking into inspector code further | 01:20 | |
*** xinliang has quit IRC | 01:31 | |
*** iurygregory has quit IRC | 01:32 | |
openstackgerrit | Julia Kreger proposed openstack/ironic-python-agent master: Support partition image efi contents https://review.opendev.org/748049 | 01:34 |
TheJulia | janders: you can't always trust the bmc address, like cisco gear and gear configured for dhcp will return 0.0.0.0 | 01:36 |
TheJulia | ipa extracts the addresses | 01:36 |
TheJulia | or at least tries | 01:36 |
TheJulia | and ships it over to inspector | 01:37 |
TheJulia | so that partition uefi code shoudl work... but I'm calling it a day | 01:38 |
janders | TheJulia thank you. Have a good evening! | 01:46 |
*** stevebaker has quit IRC | 02:39 | |
*** rcernin has quit IRC | 02:44 | |
*** Qianbiao has joined #openstack-ironic | 02:53 | |
*** rcernin has joined #openstack-ironic | 03:06 | |
*** rloo has quit IRC | 03:13 | |
*** rcernin has quit IRC | 03:13 | |
*** mkrai has joined #openstack-ironic | 03:16 | |
*** adrianc has quit IRC | 03:22 | |
*** adrianc has joined #openstack-ironic | 03:38 | |
openstackgerrit | Merged openstack/ironic-inspector stable/train: Fix database migrations and disable the non-standalone job https://review.opendev.org/759827 | 03:40 |
*** rcernin has joined #openstack-ironic | 03:43 | |
*** stevebaker has joined #openstack-ironic | 03:47 | |
*** adrianc has quit IRC | 03:47 | |
*** rcernin has quit IRC | 03:48 | |
*** rcernin has joined #openstack-ironic | 03:53 | |
*** rcernin has quit IRC | 03:54 | |
*** rcernin has joined #openstack-ironic | 03:54 | |
*** donnyd has quit IRC | 04:07 | |
*** sri_ has quit IRC | 04:07 | |
*** donnyd has joined #openstack-ironic | 04:08 | |
*** stevebaker has quit IRC | 04:11 | |
*** sri_ has joined #openstack-ironic | 04:11 | |
*** vkmc has quit IRC | 04:16 | |
*** vkmc has joined #openstack-ironic | 04:16 | |
*** adrianc has joined #openstack-ironic | 04:17 | |
*** adrianc has quit IRC | 04:23 | |
*** adrianc has joined #openstack-ironic | 04:41 | |
*** k_mouza has joined #openstack-ironic | 04:46 | |
*** johnsom has quit IRC | 04:46 | |
*** Qianbiao has quit IRC | 04:47 | |
*** johnsom has joined #openstack-ironic | 04:49 | |
*** k_mouza has quit IRC | 04:50 | |
*** rcernin has quit IRC | 04:56 | |
*** rcernin has joined #openstack-ironic | 04:57 | |
*** adrianc has quit IRC | 05:07 | |
*** adrianc has joined #openstack-ironic | 05:08 | |
*** adrianc_ has joined #openstack-ironic | 05:14 | |
*** adrianc has quit IRC | 05:16 | |
*** johnsom has quit IRC | 05:18 | |
*** adrianc_ has quit IRC | 05:18 | |
*** johnsom has joined #openstack-ironic | 05:19 | |
*** buhman has quit IRC | 05:19 | |
*** adrianc has joined #openstack-ironic | 05:20 | |
*** buhman has joined #openstack-ironic | 05:21 | |
*** zzzeek has quit IRC | 05:25 | |
*** zzzeek has joined #openstack-ironic | 05:27 | |
*** adrianc has quit IRC | 05:30 | |
*** adrianc has joined #openstack-ironic | 05:33 | |
*** adrianc has quit IRC | 05:38 | |
*** adrianc has joined #openstack-ironic | 05:40 | |
*** tzumainn has quit IRC | 05:49 | |
*** zzzeek has quit IRC | 05:50 | |
*** zzzeek has joined #openstack-ironic | 05:51 | |
*** JamesBenson has quit IRC | 05:53 | |
*** zzzeek has quit IRC | 06:10 | |
*** zzzeek has joined #openstack-ironic | 06:11 | |
*** johnsom has quit IRC | 06:27 | |
*** johnsom has joined #openstack-ironic | 06:27 | |
*** moshiur has joined #openstack-ironic | 06:52 | |
openstackgerrit | Verification of a change to openstack/ironic failed: Update `cleaning_error_handler` https://review.opendev.org/756258 | 06:54 |
*** moshiur has quit IRC | 06:54 | |
*** moshiur has joined #openstack-ironic | 06:55 | |
*** Qianbiao has joined #openstack-ironic | 06:55 | |
*** rcernin has quit IRC | 07:10 | |
*** rcernin has joined #openstack-ironic | 07:13 | |
*** zzzeek has quit IRC | 07:14 | |
*** zzzeek has joined #openstack-ironic | 07:16 | |
openstackgerrit | Yogesh Ramachandra proposed openstack/ironic master: Fix idrac-wsman bios factory-reset setting https://review.opendev.org/748696 | 07:30 |
*** zzzeek has quit IRC | 07:37 | |
*** nandita has quit IRC | 07:38 | |
*** nandita has joined #openstack-ironic | 07:38 | |
*** zzzeek has joined #openstack-ironic | 07:38 | |
*** ociuhandu has joined #openstack-ironic | 07:40 | |
*** ociuhandu has quit IRC | 07:42 | |
*** ociuhandu has joined #openstack-ironic | 07:43 | |
*** ociuhandu has quit IRC | 07:43 | |
*** rcernin has quit IRC | 07:44 | |
*** fmuyassarov has joined #openstack-ironic | 07:46 | |
*** rcernin has joined #openstack-ironic | 07:47 | |
arne_wiebalck | Good morning, ironic! | 07:57 |
*** rpittau|afk is now known as rpittau | 08:05 | |
rpittau | good morning ironic! o/ | 08:05 |
*** rcernin has quit IRC | 08:20 | |
*** ociuhandu has joined #openstack-ironic | 08:22 | |
*** uzumaki has quit IRC | 08:24 | |
*** zzzeek has quit IRC | 08:26 | |
*** zzzeek has joined #openstack-ironic | 08:30 | |
*** rcernin has joined #openstack-ironic | 08:33 | |
*** ociuhandu has quit IRC | 08:38 | |
*** zzzeek has quit IRC | 08:38 | |
*** zzzeek has joined #openstack-ironic | 08:40 | |
*** rcernin has quit IRC | 08:40 | |
arne_wiebalck | hey rpittau, all set for a nice coffee I figure? | 08:44 |
rpittau | ahah yeah! :) | 08:45 |
rpittau | still getting used to the grinder though, I "had to" try 6 coffees this morning with different grind size | 08:45 |
*** zzzeek has quit IRC | 08:48 | |
*** zzzeek has joined #openstack-ironic | 08:51 | |
*** mgoddard has joined #openstack-ironic | 08:51 | |
*** ociuhandu has joined #openstack-ironic | 08:52 | |
*** iurygregory has joined #openstack-ironic | 08:52 | |
*** tosky has joined #openstack-ironic | 08:56 | |
arne_wiebalck | yeah, I tried with a grinder as well ... but found some good ground coffee now | 08:59 |
arne_wiebalck | grinding coffee is not that obvious | 08:59 |
*** k_mouza has joined #openstack-ironic | 09:00 | |
rpittau | arne_wiebalck: yep, and it's been a while since I did it | 09:01 |
*** lucasagomes has joined #openstack-ironic | 09:03 | |
*** dtantsur|afk is now known as dtantsur | 09:04 | |
*** k_mouza has quit IRC | 09:05 | |
dtantsur | morning ironic | 09:05 |
dtantsur | janders: enroll_node_driver is in inspector.conf not ironic.conf | 09:06 |
*** mgoddard has quit IRC | 09:29 | |
*** rcernin has joined #openstack-ironic | 09:32 | |
*** rcernin has quit IRC | 09:40 | |
*** ociuhandu has quit IRC | 09:42 | |
*** ociuhandu has joined #openstack-ironic | 09:51 | |
*** ociuhandu has quit IRC | 09:56 | |
*** zzzeek has quit IRC | 09:57 | |
*** ociuhandu has joined #openstack-ironic | 09:58 | |
*** zzzeek has joined #openstack-ironic | 10:02 | |
*** rcernin has joined #openstack-ironic | 10:04 | |
*** Qianbiao has quit IRC | 10:06 | |
*** rcernin has quit IRC | 10:09 | |
*** jamesdenton has quit IRC | 10:10 | |
*** jamesdenton has joined #openstack-ironic | 10:10 | |
*** mgoddard has joined #openstack-ironic | 10:19 | |
janders | good omrning arne_wiebalck rpittau dtantsur | 10:30 |
rpittau | hey janders :) | 10:30 |
janders | dtantsur thank you! :) | 10:30 |
* janders is looking at inspector.conf | 10:30 | |
janders | rpittau I'm a total coffee nerd so if you're after coffee chat I can spend any amount of time on that | 10:31 |
janders | just finished customising my grinder :) | 10:31 |
janders | surplus test-cups of espresso are surprisingly nice chilled or in homemade gelato :) | 10:32 |
rpittau | janders: I'm definitely not at the level of grinder customization! But I already checked some interesting videos on how to modify mine, so getting there :D | 10:32 |
janders | two most important instruments: stopwatch and a precise scale | 10:33 |
janders | if the beans are fres, grinder can grind fine enough and you've got these two it's just a matter of time to get a perfect shot every time | 10:34 |
janders | s/fres/fresh | 10:34 |
janders | vacuum packed Illy is also awesome for testing new kit, as it's a really decent bean and 10/10 consistent in my experience | 10:34 |
rpittau | I see, to be honest I'm more focused on the coffee maker at the moment, you know, temperature, water type and quantity for each coffee shot | 10:34 |
rpittau | janders: we use Illy usually :) | 10:35 |
janders | I'm 93C, tap water, ~22grams of coffee and ~33grams of ristretto extracted over 42 seconds (12 sec pre-infusion + 30 sec extraction) | 10:37 |
janders | or... 10g of coffee and ~30grams of espresso in a more classic setup (for later in the day) | 10:37 |
janders | the coffee machine has its own funky water filter though so the water might not be exactly tap water when it gets into the pumps | 10:38 |
*** zzzeek has quit IRC | 10:38 | |
rpittau | tap water here is heavy, so we need filters, probably going to install one directly in the machine as well | 10:38 |
janders | good idea | 10:38 |
janders | ..or a good old Brita filters | 10:38 |
rpittau | janders: that one :) | 10:39 |
janders | my previous machine didnt have one, so I used brita for years and it worked very well | 10:39 |
janders | easier to get replacement filters compared to the machine-proprietary format | 10:39 |
*** zzzeek has joined #openstack-ironic | 10:40 | |
janders | if you try boutique roastery beans, they can have more complex flavour than Illy, but nearly each time you get a new batch it's new setup all over again, especially if you switch blends | 10:40 |
janders | and Illy is real good coffee and works well with bonsoy (boutiques often cause curdling or need lots of fine tuning to avoid it) | 10:41 |
*** Qianbiao has joined #openstack-ironic | 10:50 | |
*** k_mouza has joined #openstack-ironic | 10:51 | |
arne_wiebalck | ops lore: we had several mgmt switches being taken out, 1000+ nodes went into power state "error" and maintenance "True" (the nodes were subsequently also excluded from power sync); once the switches came back, Ironic updated the power state due to "power_failure_recovery_interval" and lifted the maintenance on these nodes, very nice! | 10:55 |
* arne_wiebalck goes and deletes the ticket he had created to clean things up once the switches are back | 10:56 | |
dtantsur | yay, happy to hear! | 10:56 |
*** rcernin has joined #openstack-ironic | 10:58 | |
janders | arne_wiebalck awesome! | 11:01 |
*** ociuhandu has quit IRC | 11:07 | |
*** ociuhandu has joined #openstack-ironic | 11:07 | |
*** zzzeek has quit IRC | 11:17 | |
*** zzzeek has joined #openstack-ironic | 11:20 | |
*** thiagop has joined #openstack-ironic | 11:25 | |
*** zzzeek has quit IRC | 11:27 | |
*** zzzeek has joined #openstack-ironic | 11:28 | |
*** mkrai has quit IRC | 11:29 | |
openstackgerrit | Verification of a change to openstack/ironic failed: Limit the default value of [api]api_workers to 4 https://review.opendev.org/761185 | 11:31 |
*** ociuhandu has quit IRC | 11:35 | |
openstackgerrit | Yogesh Ramachandra proposed openstack/ironic master: Fix idrac-wsman bios factory-reset setting https://review.opendev.org/748696 | 11:35 |
*** ociuhandu has joined #openstack-ironic | 11:36 | |
*** ociuhandu has quit IRC | 11:41 | |
*** rcernin has quit IRC | 11:45 | |
openstackgerrit | Jacob Anders proposed openstack/bifrost master: Disable inspector discovery by default https://review.opendev.org/762998 | 11:49 |
*** ociuhandu has joined #openstack-ironic | 11:53 | |
*** rcernin has joined #openstack-ironic | 11:58 | |
*** mgoddard has quit IRC | 12:01 | |
openstackgerrit | Merged openstack/ironic master: Update `cleaning_error_handler` https://review.opendev.org/756258 | 12:06 |
*** ianychoi_ has quit IRC | 12:10 | |
*** JamesBenson has joined #openstack-ironic | 12:12 | |
janders | see you tomorrow Ironic o/ | 12:13 |
*** rcernin has quit IRC | 12:14 | |
*** mgoddard has joined #openstack-ironic | 12:15 | |
*** ociuhandu has quit IRC | 12:28 | |
*** zzzeek has quit IRC | 12:39 | |
*** zzzeek has joined #openstack-ironic | 12:41 | |
arne_wiebalck | TheJulia: dtantsur: for https://review.opendev.org/#/c/762537/ we said we disable it by default (prio 0), but make it easy to enable via a flag ... isn't janders work on clean step priorities an easy way to enable it if desired? | 12:47 |
patchbot | patch 762537 - ironic-python-agent - Add clean step 'erase_pstore' - 3 patch sets | 12:47 |
*** ociuhandu has joined #openstack-ironic | 12:59 | |
*** dougsz has joined #openstack-ironic | 13:01 | |
*** rcernin has joined #openstack-ironic | 13:02 | |
*** rcernin has quit IRC | 13:02 | |
*** ociuhandu has quit IRC | 13:04 | |
*** ociuhandu has joined #openstack-ironic | 13:04 | |
*** ociuhandu has quit IRC | 13:15 | |
*** fmuyassarov has quit IRC | 13:20 | |
openstackgerrit | Arne Wiebalck proposed openstack/ironic-python-agent master: Add clean step 'erase_pstore' https://review.opendev.org/762537 | 13:22 |
openstackgerrit | Bob Fournier proposed openstack/ironic master: [WIP] Add vendor_passthru methods for virtual media https://review.opendev.org/763007 | 13:24 |
*** fmuyassarov has joined #openstack-ironic | 13:28 | |
*** fmuyassarov has quit IRC | 13:32 | |
*** Goneri has joined #openstack-ironic | 13:40 | |
*** k_mouza has quit IRC | 13:41 | |
*** ociuhandu has joined #openstack-ironic | 13:45 | |
*** dougsz has quit IRC | 13:48 | |
*** ociuhandu has quit IRC | 13:50 | |
rpittau | grep from_dict * -R | 13:52 |
rpittau | wrong window :P | 13:52 |
iurygregory | not found | 13:52 |
rpittau | lol | 13:53 |
*** fmuyassarov has joined #openstack-ironic | 13:53 | |
*** moshiur has quit IRC | 13:56 | |
*** ociuhandu has joined #openstack-ironic | 13:57 | |
*** paras333 has joined #openstack-ironic | 13:59 | |
TheJulia | good morning | 14:05 |
TheJulia | dtantsur: https://review.opendev.org/#/c/748049 works *insert evil laugh here* | 14:10 |
patchbot | patch 748049 - ironic-python-agent - Support partition image efi contents - 27 patch sets | 14:10 |
*** rloo has joined #openstack-ironic | 14:16 | |
*** dougsz has joined #openstack-ironic | 14:17 | |
*** dougsz has quit IRC | 14:17 | |
*** dougsz has joined #openstack-ironic | 14:18 | |
TheJulia | good morning | 14:18 |
paras333 | TheJulia: GM:) | 14:19 |
openstackgerrit | Riccardo Pittau proposed openstack/ironic master: Remove from_dict function from context https://review.opendev.org/763016 | 14:21 |
openstackgerrit | Merged openstack/ironic-python-agent stable/victoria: Fix physical memory calculation with new lshw https://review.opendev.org/762841 | 14:22 |
*** k_mouza has joined #openstack-ironic | 14:24 | |
*** k_mouza has quit IRC | 14:24 | |
*** k_mouza has joined #openstack-ironic | 14:24 | |
fmuyassarov | Hello Ironic Community. Sorry for asking it again. But can I get a review on https://review.opendev.org/#/c/762343/ please whenever you have time. I asked yesterday about a couple of failed CI checks, but I got know from rpittau that those are know issues in focal. And TBH, I don't know how to fix them | 14:26 |
patchbot | patch 762343 - ironic - Allow disabling automated_clean per node - 9 patch sets | 14:26 |
rpittau | fmuyassarov: hi! the issue wiht ironic-tempest-ipxe-ipv6 is something that we encountered when switching CI to focal, we're still investigating, it seems a change in behavior in ubuntu, not related to your patch, so no worries about that | 14:27 |
rpittau | first time I see the rabbvitmq issue in ironic-inspector-tempest-partition-bios-redfish-vmedia to be honest oO | 14:28 |
fmuyassarov | Hi rpittau , good to know. Thank you :) | 14:28 |
rpittau | still, not related to your change it seems | 14:28 |
dtantsur | morning TheJulia | 14:29 |
rpittau | mmmm "ERROR: epmd error for host ubuntu-focal-limestone-regionone-0021788829: address (cannot connect to host/port)" | 14:29 |
dtantsur | arne_wiebalck: he is, yes | 14:29 |
*** ociuhandu has quit IRC | 14:30 | |
rpittau | fmuyassarov: yep, not an issue on your patch as well. Could you please just add the task number to the commit message ? | 14:31 |
fmuyassarov | rpittau, sorry, what is the task number? I added story number. Is it the same thing? | 14:33 |
dtantsur | fmuyassarov: nope, any story has tasks. a patch is attached to a task in a story, so you need both numbers. | 14:34 |
rpittau | fmuyassarov: story refers to the story in storyboard, the task number is associated to the task in the story itself | 14:34 |
rpittau | or what dtantsur wrote :D | 14:34 |
arne_wiebalck | A sneak preview from my upcoming book "Make your ops life hard ... yourself!": Our downstream h/w manager deregisters the node from our PXE infra in evaluate_hardware_support(). The corresponding call can take up to several minutes. Anyone seeing why this is really bad idea? | 14:35 |
arne_wiebalck | Hint: I was seeing evaluate_hardware_support() being called surprisingly often ... | 14:35 |
fmuyassarov | Got it. Thanks rpittau and dtantsur | 14:35 |
TheJulia | arne_wiebalck: because it always wants to wait for disks? | 14:35 |
arne_wiebalck | TheJulia: no | 14:35 |
TheJulia | hmm | 14:35 |
rpittau | mmmm | 14:35 |
* arne_wiebalck hopes his explanation is actually correct | 14:36 | |
openstackgerrit | Feruzjon Muyassarov proposed openstack/ironic master: Allow disabling automated_clean per node https://review.opendev.org/762343 | 14:36 |
arne_wiebalck | there were several calls to evaluate_hardware_support() of our h/w manager | 14:36 |
TheJulia | oh, from within | 14:36 |
TheJulia | "fun!" | 14:37 |
arne_wiebalck | every time it took some minutes | 14:37 |
arne_wiebalck | eventually the cleaning timeout was reached | 14:37 |
arne_wiebalck | but why was it called multiple times?> | 14:37 |
arne_wiebalck | since the managers need to be sorted, and I guess sorted() calls the function several times | 14:38 |
TheJulia | wait_for_disks dispatches to all hardware managers | 14:38 |
TheJulia | I believe that does trigger evaluate as well | 14:38 |
arne_wiebalck | multiple times per hardware manager? | 14:39 |
TheJulia | no, in the main code path for various actions | 14:39 |
arne_wiebalck | I see evaluate_hardware_support() being called 3 times for our h/w manager | 14:39 |
TheJulia | diretly from... hmm | 14:40 |
arne_wiebalck | I obviously expected it to be called once | 14:40 |
TheJulia | are there any command dispatches taking place? | 14:40 |
arne_wiebalck | don't you think it could be sorted() invoking evaluate_hardware_support() to know how to sort the managers? | 14:41 |
fmuyassarov | dtantsur, rpittau related to 762343, I want the Ironic client to be able to pass "false" value for node.automated_clean. Is it okay If I work on that? and I think I need a story for that too right? | 14:41 |
TheJulia | zer0c00l: you going to be around today? | 14:41 |
TheJulia | arne_wiebalck: I'd have to look at that part of the code, give me a moment | 14:42 |
*** tzumainn has joined #openstack-ironic | 14:43 | |
arne_wiebalck | TheJulia: https://github.com/openstack/ironic-python-agent/blob/master/ironic_python_agent/hardware.py#L2199 | 14:43 |
arne_wiebalck | TheJulia: and https://github.com/openstack/ironic-python-agent/blob/master/ironic_python_agent/hardware.py#L2224 | 14:44 |
TheJulia | eww | 14:44 |
TheJulia | why did we do that?!? | 14:44 |
arne_wiebalck | it makes sense if that function returns immediately | 14:44 |
TheJulia | I was just about paste basically the same second link | 14:44 |
arne_wiebalck | :-D | 14:44 |
rpittau | fmuyassarov: I believe another task in the same story should be enough | 14:45 |
arne_wiebalck | our function has a 12 min timeout before it returns ... ouch | 14:45 |
TheJulia | the first link is definitely it though :( | 14:45 |
fmuyassarov | got it rpittau | 14:45 |
TheJulia | I was starting to wonder why the IPA logs were feeling much more noisy | 14:45 |
arne_wiebalck | I think it might better to call this once for all and sort with the values. | 14:46 |
arne_wiebalck | Depending on the algorithm that function is called often. | 14:46 |
arne_wiebalck | I guess none are as long as ours, but still :) | 14:46 |
TheJulia | I guess I'm not groking in my half coffeed state why we're evaluating then and there... | 14:47 |
TheJulia | looks like we only call get_managers twice, but even that first seems like it could derail you | 14:47 |
arne_wiebalck | sorted() will call it, then we loop once more over them and call it again | 14:52 |
arne_wiebalck | all fine, clearly a problem of our h/w manager :) | 14:53 |
TheJulia | well, inspection and then the main ipa code path if the inspection path is triggered | 14:53 |
arne_wiebalck | this is all on cleaning | 14:53 |
TheJulia | ahh, yeah and every step will resort the list too | 14:55 |
TheJulia | because latest clean steps need to be obtained | 14:55 |
arne_wiebalck | I don't get there: by the time I identify the hardware manager Ironic timed out :-D | 14:57 |
TheJulia | nice | 14:57 |
*** ociuhandu has joined #openstack-ironic | 15:01 | |
*** alexmcleod has quit IRC | 15:04 | |
*** tosin has joined #openstack-ironic | 15:07 | |
*** tosin has quit IRC | 15:08 | |
*** ociuhandu has quit IRC | 15:10 | |
*** ociuhandu has joined #openstack-ironic | 15:10 | |
*** dougsz has quit IRC | 15:11 | |
*** dougsz has joined #openstack-ironic | 15:17 | |
*** ociuhandu has quit IRC | 15:21 | |
*** ociuhandu has joined #openstack-ironic | 15:27 | |
dtantsur | TheJulia: we've had a discussion downstream that boiled down to the fact that enabling auto-discovery by default may be a source of confusing problems. | 15:28 |
dtantsur | because of that janders proposed https://review.opendev.org/#/c/762998/ I wonder what you think | 15:28 |
patchbot | patch 762998 - bifrost - Disable inspector discovery by default - 1 patch set | 15:28 |
dtantsur | (since it was you who introduced the discovery support initially IIRC) | 15:29 |
TheJulia | I have really mixed feelings at the moment on this | 15:37 |
*** ociuhandu has quit IRC | 15:38 | |
TheJulia | Wouldn't it make more sense to highlight the disparity in how inspection works documentation? | 15:38 |
TheJulia | I mean, I'm not saying no, don't disable it, I'm just confused by why this is suddenly a problem | 15:39 |
*** alexmcleod has joined #openstack-ironic | 15:40 | |
Qianbiao | arne_wiebalck are CERN using auto-discovery in your env? | 15:42 |
Qianbiao | I am courious about whether this feature will cause maintenance/debug issues in big cloud env | 15:43 |
*** fmuyassarov has quit IRC | 15:45 | |
*** mkrai has joined #openstack-ironic | 15:50 | |
TheJulia | The issue seems to be is that somehow this perception that one should be able to introspect without providing a physical port to match the identity of the machine, end up with all of the ports, and I guess I find that frustrating | 15:50 |
*** trandles has joined #openstack-ironic | 15:51 | |
TheJulia | Qianbiao: I think the problem is more about virutal and some cases where one can't just discover the IP address of the bmc | 15:51 |
Qianbiao | TheJulia think about a server has two port connected. | 15:53 |
Qianbiao | pxe port mac identify is not strong enough too. | 15:54 |
TheJulia | two port depends on inspection configuration, if it creates additional ports or not | 15:55 |
arne_wiebalck | Qianbiao: no, not yet | 15:55 |
arne_wiebalck | Qianbiao: we are planning to do this, though | 15:55 |
TheJulia | I agree it is not strong enough, but it is consitent and always available to the running OS. | 15:55 |
Qianbiao | arne_wiebalck I am afraid, if there are a server has some problem when inspect. | 15:55 |
Qianbiao | then we will lose the server. | 15:56 |
Qianbiao | and can not find it back. | 15:56 |
Qianbiao | TheJulia agree. | 15:56 |
TheJulia | this is also why allowing inspector to add all mac addresses is a good idea... And inspection should only be under a fairly controlled circumstance as part of their process, similar to those that use auto-discovery to wheel a rack into the data center and power it up. | 15:57 |
Qianbiao | I am courious whether auto-discovery will cause "duplicate discoveryed" or "miss discoveryed" | 15:58 |
Qianbiao | TheJulia yeah, it seem add_port all will fix the issue. | 15:59 |
arne_wiebalck | Qianbiao: TheJulia: are auto-discovery and inspection somehow tied together? I was thinking the former only creates an entry in the Ironic DB, no? | 15:59 |
Qianbiao | am I wrong: auto-discovery will ends up with auto inspect? | 16:00 |
Qianbiao | did not try that XD | 16:00 |
*** ociuhandu has joined #openstack-ironic | 16:02 | |
TheJulia | They are basically the same code path | 16:02 |
TheJulia | If the node is not found, then if discovery is enabled, then inspector creates the node | 16:02 |
TheJulia | often this is if the BMC IP can't be determined | 16:02 |
arne_wiebalck | so auto-discovery implies running an inspection? | 16:03 |
dtantsur | I would even say that auto-discovery is a special case of inspection | 16:05 |
Qianbiao | arne_wiebalck i think, yes. | 16:06 |
arne_wiebalck | dtantsur: Qianbiao: thanks! | 16:07 |
*** k_mouza has quit IRC | 16:07 | |
TheJulia | what dtantsur said :) | 16:10 |
Qianbiao | Is there a method that updating node's port while the node is in deploying status ? | 16:11 |
*** ebagakis has joined #openstack-ironic | 16:11 | |
Qianbiao | I need to store some data to port's extra field when provisioning the node. | 16:11 |
dtantsur | in deploying state a conductor is holding a lock, so likely no | 16:12 |
Qianbiao | sad :( | 16:12 |
Qianbiao | seems i need to find other way to roma | 16:12 |
TheJulia | deploy wait it shouldn't be locked, fwiw | 16:13 |
Qianbiao | TheJulia not deploy wait, when NOVA preparing the deploy, i need to bind-port and add extra data | 16:13 |
Qianbiao | and node's status is exactly deploying.. | 16:14 |
*** bnemec has quit IRC | 16:16 | |
TheJulia | oh, the deploy hasn't bneen started yet | 16:21 |
*** mkrai has quit IRC | 16:21 | |
openstackgerrit | Arne Wiebalck proposed openstack/ironic-python-agent master: Add clean step 'erase_pstore' https://review.opendev.org/762537 | 16:22 |
TheJulia | the ironic nova virt driver is still preparing as you said, so it hasn't called the deploy call yet. | 16:22 |
*** k_mouza has joined #openstack-ironic | 16:23 | |
Qianbiao | yes | 16:27 |
*** iurygregory has quit IRC | 16:29 | |
*** Qianbiao has quit IRC | 16:30 | |
openstackgerrit | Dmitry Tantsur proposed openstack/bifrost master: Support UEFI in bifrost-cli and document it https://review.opendev.org/762555 | 16:35 |
rpioso | Good morning, ironic | 16:44 |
*** bnemec has joined #openstack-ironic | 16:50 | |
JayF | arne_wiebalck: small issue with the erase_pstore, if you wanna clear up that comment I can +2 | 16:53 |
*** ociuhandu has quit IRC | 16:55 | |
*** ociuhandu has joined #openstack-ironic | 16:55 | |
openstackgerrit | Julia Kreger proposed openstack/ironic master: Minor follow-up doc change https://review.opendev.org/763045 | 16:57 |
*** ociuhandu has quit IRC | 17:00 | |
openstackgerrit | Arne Wiebalck proposed openstack/ironic-python-agent master: Add clean step 'erase_pstore' https://review.opendev.org/762537 | 17:02 |
arne_wiebalck | JayF: thanks, uploaded a new patch, oversight from enabled to disabled, good catch, thanks! | 17:03 |
JayF | It's already got my +2 :D | 17:03 |
* JayF loves how you can diff patchsets in gerrit | 17:03 | |
arne_wiebalck | JayF: great thanks | 17:04 |
arne_wiebalck | JayF: did you notice when diffing that I needed to change the iterator for the open context in 3.9 ? | 17:04 |
JayF | if that was in patchset 6, it wasn't | 17:05 |
JayF | the only change in patchset 6 is that comment I had commented on | 17:05 |
JayF | arne_wiebalck: if you think that change has iterator fixes in it, you missed a `git add` somewhere I think | 17:06 |
arne_wiebalck | JayF: no, it was earlier | 17:07 |
arne_wiebalck | JayF: https://review.opendev.org/#/c/762537/3..5/ironic_python_agent/tests/unit/test_hardware.py | 17:07 |
patchbot | patch 762537 - ironic-python-agent - Add clean step 'erase_pstore' - 6 patch sets | 17:07 |
JayF | nice | 17:08 |
*** lucasagomes has quit IRC | 17:08 | |
arne_wiebalck | realising iter was missing took me a while, but the initial version did not work on 3.9 | 17:08 |
arne_wiebalck | time spent: 10% code, 90% on the test :-S | 17:09 |
JayF | that's not super abnormal for me :D | 17:09 |
arne_wiebalck | it is probably even good ... I am just annoy... surprised every time :) | 17:10 |
*** ociuhandu has joined #openstack-ironic | 17:11 | |
openstackgerrit | Verification of a change to openstack/ironic failed: Update .rst files https://review.opendev.org/759236 | 17:18 |
*** iurygregory has joined #openstack-ironic | 17:19 | |
rpittau | good night! o/ | 17:27 |
*** rpittau is now known as rpittau|afk | 17:27 | |
*** ebagakis has quit IRC | 17:29 | |
*** k_mouza has quit IRC | 17:30 | |
*** dougsz has quit IRC | 17:32 | |
*** ociuhandu_ has joined #openstack-ironic | 17:43 | |
*** ociuhandu has quit IRC | 17:46 | |
*** ociuhandu_ has quit IRC | 17:47 | |
*** k_mouza has joined #openstack-ironic | 17:50 | |
*** fmuyassarov has joined #openstack-ironic | 17:53 | |
openstackgerrit | Julia Kreger proposed openstack/ironic stable/train: Sync boot mode when changing the boot device via Redfish https://review.opendev.org/762439 | 17:57 |
*** fmuyassarov has quit IRC | 17:58 | |
*** mgoddard has quit IRC | 18:02 | |
*** k_mouza has quit IRC | 18:10 | |
*** fmuyassarov has joined #openstack-ironic | 18:12 | |
openstackgerrit | Merged openstack/bifrost master: Expand testenv documentation https://review.opendev.org/762553 | 18:17 |
*** k_mouza has joined #openstack-ironic | 18:19 | |
openstackgerrit | Riccardo Pittau proposed openstack/sushy-tools master: Allow setting debug mode from command line https://review.opendev.org/763068 | 18:30 |
rpioso | dtantsur: I'm looking at your comment about https://review.opendev.org/#/c/763007: https://review.opendev.org/#/c/763007/1/ironic/drivers/modules/drac/vendor_passthru.py@199 | 18:31 |
patchbot | patch 763007 - ironic - [WIP] Add vendor_passthru methods for virtual media - 1 patch set | 18:31 |
patchbot | patch 763007 - ironic - [WIP] Add vendor_passthru methods for virtual media - 1 patch set | 18:31 |
openstackgerrit | Riccardo Pittau proposed openstack/sushy-tools master: Allow setting debug mode from command line https://review.opendev.org/763068 | 18:31 |
rpioso | dtantsur: Are you suggesting class DracRedfishVendorPassthru be a simple derivation of vendor-independent Redfish's VendorPassthru? | 18:33 |
openstackgerrit | Riccardo Pittau proposed openstack/sushy-tools master: Allow setting debug mode from command line https://review.opendev.org/763068 | 18:35 |
*** k_mouza has quit IRC | 18:56 | |
dtantsur | rpioso: honestly, we may not even need the Drac version | 18:57 |
*** ayoung has quit IRC | 18:57 | |
TheJulia | good morning | 18:58 |
* TheJulia guesses stevebaker will appear | 18:58 | |
* dtantsur hopes | 18:59 | |
rpioso | dtantsur: Are you saying there may not be a need for the idrac driver to support that Redfish vendor passthru vmedia stuffs? | 18:59 |
rpioso | TheJulia: Good morning :-) | 19:00 |
dtantsur | rpioso: no, for having a separate vendor passthru class | 19:00 |
TheJulia | This may sound a bit strong or firm, but ideally nothing should ever require new vendor passthru methods unless a vendor needs a highly specific thing | 19:01 |
TheJulia | and at that point, it is their driver only and there would be no reason to ever implement the same elsewhere... because if we have done that then we've not headed down the common/mutual/abstracted path | 19:01 |
dtantsur | TheJulia: even without Steve, should we jump on a call and start reviewing? | 19:02 |
rpioso | dtantsur, TheJulia: I'm unfamiliar with why that change has been proposed. There's no story. And that's understandable, since it's a WIP. | 19:03 |
TheJulia | dtantsur: I think that is a good idea to likely jump on a call | 19:03 |
TheJulia | maybe let me get some water first | 19:03 |
dtantsur | rpioso: that's a fair point. there is a bit of a context, I think it should be in the commit message. | 19:03 |
dtantsur | very tl;dr to support 3rd party installers via ironic | 19:04 |
TheJulia | so, meetpad.opendev.org/ironic good? | 19:06 |
dtantsur | chromium crashed, let's try firefox | 19:06 |
tosky | dtantsur: re chromium crashes: https://bugzilla.redhat.com/show_bug.cgi?id=1895920 | 19:12 |
openstack | bugzilla.redhat.com bug 1895920 in chromium "Chromium 86 crashes on WebRTC videos when switching window" [Urgent,New] - Assigned to spotrh | 19:12 |
*** fmuyassarov has quit IRC | 19:15 | |
TheJulia | wheeee | 19:15 |
*** gyee has joined #openstack-ironic | 19:18 | |
clarkb | note that firefox's webrtc system isn't very good compared to chrom* | 19:19 |
clarkb | it may work or it may not, we've seen it go both ways with jitsi meet | 19:19 |
dtantsur | clarkb: it tends to work for me | 19:20 |
dtantsur | they have improved it recently | 19:20 |
*** mgoddard has joined #openstack-ironic | 19:21 | |
*** stevebaker has joined #openstack-ironic | 19:22 | |
stevebaker | hey, late for my own review stab | 19:25 |
dtantsur | stevebaker: https://meetpad.opendev.org/ironic | 19:26 |
TheJulia | \o/ | 19:26 |
TheJulia | wait, stab?!? | 19:26 |
tzumainn | TheJulia, hi! thanks for the updated documentation. there's one thing that still confuses me and naved001 which is - if we use ipxe, how does the node get the ipxe file? | 19:27 |
dtantsur | we're on a review meeting, she'll likely answer later | 19:27 |
tzumainn | gotcha, no rush and thanks! | 19:27 |
*** jawad_axd has joined #openstack-ironic | 19:36 | |
*** fmuyassarov has joined #openstack-ironic | 19:37 | |
stevebaker | TheJulia, dtantsur: https://review.opendev.org/#/c/750120/20/ironic/api/controllers/v1/port.py | 19:46 |
patchbot | patch 750120 - ironic - Convert ports endpoint to plain JSON - 20 patch sets | 19:46 |
*** k_mouza has joined #openstack-ironic | 19:57 | |
*** k_mouza has quit IRC | 20:01 | |
*** gryf has quit IRC | 20:09 | |
*** gryf has joined #openstack-ironic | 20:09 | |
bfournie | rpioso: just catching up on above, yes I'll add a story for node passthru, note that it was referred to in https://etherpad.opendev.org/p/IronicWhiteBoard (line 327) | 20:09 |
bfournie | rpioso: actually 331 | 20:11 |
rpioso | bfournie: Thank you! Will it be booted from one-time? | 20:15 |
* rpioso believes one-time or continuous may not matter. | 20:16 | |
*** k_mouza has joined #openstack-ironic | 20:16 | |
*** dtantsur is now known as dtantsur|afk | 20:17 | |
dtantsur|afk | good night folks! | 20:18 |
rpioso | bfournie: At least for one-time, the iDRAC treats the inserted vmedia as a *virtual* CD/DVD/floppy. AFAIU, there's no such ironic boot device. Vendor-independent Redfish also doesn't grok those. | 20:19 |
*** k_mouza has quit IRC | 20:21 | |
bfournie | rpioso: yes I don't think we need to set that as not setting boot device, basically it is exposing the _insert_vmedia method as api https://github.com/openstack/ironic/blob/master/ironic/drivers/modules/redfish/boot.py#L144. The main reason for this is to expose eject_vmedia so that a user can set boot mode to DISK and disconnect the ISO that it originally booted with | 20:24 |
*** mgoddard has quit IRC | 20:31 | |
rpioso | bfournie: I'm familiar with that method. Are you trying to boot an ISO that has been inserted that way? | 20:31 |
*** fmuyassarov has quit IRC | 20:32 | |
* rpioso is unclear on how the system was instructed to connect the ISO and boot from it. | 20:32 | |
*** fmuyassarov has joined #openstack-ironic | 20:39 | |
*** fmuyassarov has quit IRC | 20:47 | |
bfournie | rpioso: the ISO insert and boot is done via an assisted installer outside of ironic, so ironic is responsible for ejecting it in order to boot from disk. Realisitically we can remove the insert_vmedia support from this patch if that is a concern, the main requirement is the eject | 20:51 |
rpioso | bfournie: That helps. Thanks! Could class DracRedfishVendorPassthru simply be derived from the vendor-independent Redfish class? We've done that for other iDRAC Redfish interface implementations, e.g., https://github.com/openstack/ironic/blob/172b76f0f949c7f927f62b915d730d5d0364432a/ironic/drivers/modules/drac/bios.py#L42-L49. | 20:59 |
janders | good morning Ironic o/ | 20:59 |
*** rcernin has joined #openstack-ironic | 21:02 | |
rpioso | Good morning, janders :) | 21:05 |
*** hjensas_ has joined #openstack-ironic | 21:06 | |
bfournie | rpioso: https://storyboard.openstack.org/#!/story/2008363, yes I will look at deriving this class | 21:07 |
bfournie | rpioso: thanks | 21:09 |
*** hjensas has quit IRC | 21:10 | |
rpioso | bfournie: Cool! You're most welcome. While I'm a big fan of symmetry, for iDRAC, the usefulness of an insert vmedia API is questionable, because it can't be booted from via typical ironic nor Redfish means. | 21:10 |
bfournie | rpioso: yeah, I tend to agree with that | 21:11 |
openstackgerrit | Jacob Anders proposed openstack/bifrost master: Disable inspector discovery by default https://review.opendev.org/762998 | 21:11 |
bfournie | rpioso: will strip that out | 21:12 |
iurygregory | morning janders o/ | 21:12 |
rpioso | bfournie: Go Pats! | 21:13 |
bfournie | rpioso: Ha! Not this year. eh | 21:13 |
bfournie | too bad | 21:13 |
TheJulia | Added a new agenda item for next week's meeting in discussion... converged adapters that do the wrong thing with the UEFI nvram table | 21:18 |
*** shadower has quit IRC | 21:20 | |
*** shadower has joined #openstack-ironic | 21:22 | |
janders | TheJulia dtantsur|afk just reading through the discussion about https://review.opendev.org/#/c/762998/ (disabling auto-discovery by default) - thank you for looking into this. In the end, do you think this is a good idea, bad idea, or further discussion is needed? It seems like the discussion drifted away to other topics without reaching | 21:28 |
janders | conclusion. From my perspective it would be useful, as bifrost is largely used in VM context for development and testing and that's where auto-discovery can be having problems - but ultimately it's your decision. Thanks! :) | 21:28 |
patchbot | patch 762998 - bifrost - Disable inspector discovery by default - 2 patch sets | 21:28 |
openstackgerrit | Verification of a change to openstack/ironic failed: New argument validate decorator https://review.opendev.org/742795 | 21:40 |
openstackgerrit | Verification of a change to openstack/ironic failed: Add expose body decorator, status_code argument https://review.opendev.org/743677 | 21:40 |
TheJulia | janders: From talking with physical operators who use bifrost for machine staging/testing/deployment, autodetect may be used, we simply don't have the right information to make an informed decision. | 21:51 |
TheJulia | janders: maybe sending a mailing list post might help and maybe reaching out to the operators group? | 21:51 |
janders | TheJulia ACK, will do, thank you. Regarding the operators group - do I just mention in the email that I'm particularly after feedback from them, or is there a particular tag or alias I should use for that? | 21:53 |
TheJulia | janders: they are most active on twitter, I don't have the group handy but they have an etherpad with the link somepalce | 21:53 |
iurygregory | https://twitter.com/osopsmeetup | 21:56 |
iurygregory | the etherpad they had for W https://etherpad.opendev.org/p/wallaby-ptg-ops-meetup | 21:56 |
openstackgerrit | Verification of a change to openstack/ironic failed: Limit the default value of [api]api_workers to 4 https://review.opendev.org/761185 | 21:59 |
janders | Thank you iurygregory! | 22:00 |
janders | (and go get some rest! :) | 22:00 |
iurygregory | 23h here =) | 22:00 |
iurygregory | 1hr to go sleep =) | 22:00 |
janders | :) | 22:03 |
TheJulia | iurygregory: thanks | 22:13 |
iurygregory | np =) | 22:14 |
*** shadower has quit IRC | 22:16 | |
*** shadower has joined #openstack-ironic | 22:19 | |
*** thiagop has quit IRC | 22:34 | |
*** stevebaker has quit IRC | 22:37 | |
*** stevebaker has joined #openstack-ironic | 22:38 | |
*** hjensas__ has joined #openstack-ironic | 22:47 | |
*** hjensas_ has quit IRC | 22:51 | |
*** thiagop has joined #openstack-ironic | 23:19 | |
*** thiagop has quit IRC | 23:24 | |
openstackgerrit | Julia Kreger proposed openstack/ironic stable/train: Sync boot mode when changing the boot device via Redfish https://review.opendev.org/762439 | 23:32 |
TheJulia | tzumainn: if you use ipxe, there still needs to be someplace avaialble that the machines on the ironic nodes can reach through the neutron router so neutron dhcp can tell it where to go and the machine can make that request. | 23:37 |
TheJulia | tzumainn: maybe there is a case where it would make sense wrap extra layers around that, but that path still needs to also tie back to conductors too, on a plus side, there can be numerous ones | 23:39 |
tzumainn | TheJulia, okay, so this is probably where my confusion lies; when one uses, say, metalsmith and ansible networking to do this, neutron will just attach the provisioning network for you; if I understand correctly, that won't happen in the boot-from-volume case? | 23:48 |
TheJulia | No | 23:49 |
TheJulia | in the case of booting from a volume, the end network for the machine is attached | 23:49 |
TheJulia | the dhcp information supplied includes where to get the ipxe loader | 23:49 |
tzumainn | okay, so it's the responsibility of the deployer to ensure that the machine can reach the ipxe file? | 23:50 |
TheJulia | so kind of like how provisioning works, except not the inherent risk nor need of an isolated island network to keep the machines isolated from the world for provisioning/cleaning operations | 23:50 |
*** tosky has quit IRC | 23:58 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!