Tuesday, 2020-11-17

*** tosky has quit IRC00:06
jandersTheJulia ACK regarding the NVMe work00:10
*** anuradha1904 has quit IRC00:10
TheJuliagood morning00:10
jandersTheJulia stevebaker dtantsur|afk regarding the API JSON sessions, I will join the second one, thank you for factoring in my timezone.00:10
jandersTheJulia stevebaker I have a question if you have time. I noticed something that seems weird behaviour by inspector+ironic while attempting to introspect a KVM-VM fake-baremetal that doesn't have a port defined. Inspector does have a go at doing it as it thinks it can lookup via IPMI address but it ends up failing the inspection and creating00:14
jandersanother node in the process. In the CLI output the "flow" looks like this: http://paste.openstack.org/show/800044/00:14
jandersI first suspected it's a bug introduced by my patches aiming to implement https://storyboard.openstack.org/#!/story/2008038 but after a day and a bit of testing and digging I tried this on an unpatched install and it did exactly the same thing.00:15
jandershave I found a bug?00:15
jandersI'd expect that if there isn't enough info to proceed successfully, the inspection should fail as opposed to resulting in ^^00:16
jandersmy (potentially naive) understanding is that the assumption that nodes can be looked up via IPMI address doesn't really work with VMs, hence the issue00:17
*** k_mouza has joined #openstack-ironic00:45
*** k_mouza has quit IRC00:49
stevebakerjanders:00:57
stevebakerjanders: thanks00:57
stevebakerjanders: inspector ends up creating another node!?00:57
jandersstevebaker yes!00:59
jandersI think it's auto-discovery being enabled by default in bifrost01:00
TheJuliajanders:  so, I don't think that is a bug01:00
stevebakerah01:01
TheJuliaI think it is a side effect of auto-discovery + not being able to get the BMC address01:01
TheJuliathe human or software driving *needs* to provide something to align the node, mac being best, BMC address not always reliable01:01
*** Goneri has quit IRC01:01
jandersTheJulia in this case, shouldn't it fail?01:01
TheJuliahow could it?01:01
TheJuliawell, inspection should fail yes01:01
TheJuliabut auto-discovery occured01:02
TheJuliaI don't think there is a way around that, tbh01:02
jandersTheJulia good points! :)01:03
* TheJulia cries at unit tests for _grub2_install01:04
jandersto the operator it's obvious what's going on because of the timing of requesting inspection and the non-existent extra node popping up, but I don't have any good ideas how to build logic around that01:04
jandersthe only thing I can think of is - is it useful to have Ironic create nodes with no IPMI address defined?01:05
jandersI'd think they can't do anything anyway01:05
* janders is testing if this is really the case 01:05
*** zzzeek has quit IRC01:06
TheJuliathere are other bmc types01:07
TheJuliaand people actually use fake power01:07
jandersI can't power on, I can't provide, I think this auto-created node is unusable01:08
jandersI think this is a corner case with VMs01:08
*** zzzeek has joined #openstack-ironic01:09
TheJuliayou can only not provide it if the hardware type results in power or management being ipmitool01:10
jandersif it were a real baremetal, what would IPA do?01:12
jandersgetting BMC address is easy, how about the creds?01:12
* janders is looking which parts of inspector/ironic code do this01:13
*** xinliang has joined #openstack-ironic01:15
jandershmm I have no enroll_node_driver set in ironic.conf and default='fake-hardware', wonder where ipmi comes from in this equation01:17
jandersbarhttp://paste.openstack.org/show/800077/01:18
jandersoops01:19
jandershttp://paste.openstack.org/show/800077/01:19
* janders is looking into inspector code further01:20
*** xinliang has quit IRC01:31
*** iurygregory has quit IRC01:32
openstackgerritJulia Kreger proposed openstack/ironic-python-agent master: Support partition image efi contents  https://review.opendev.org/74804901:34
TheJuliajanders: you can't always trust the bmc address, like cisco gear and gear configured for dhcp will return 0.0.0.001:36
TheJuliaipa extracts the addresses01:36
TheJuliaor at least tries01:36
TheJuliaand ships it over to inspector01:37
TheJuliaso that partition uefi code shoudl work... but I'm calling it a day01:38
jandersTheJulia thank you. Have a good evening!01:46
*** stevebaker has quit IRC02:39
*** rcernin has quit IRC02:44
*** Qianbiao has joined #openstack-ironic02:53
*** rcernin has joined #openstack-ironic03:06
*** rloo has quit IRC03:13
*** rcernin has quit IRC03:13
*** mkrai has joined #openstack-ironic03:16
*** adrianc has quit IRC03:22
*** adrianc has joined #openstack-ironic03:38
openstackgerritMerged openstack/ironic-inspector stable/train: Fix database migrations and disable the non-standalone job  https://review.opendev.org/75982703:40
*** rcernin has joined #openstack-ironic03:43
*** stevebaker has joined #openstack-ironic03:47
*** adrianc has quit IRC03:47
*** rcernin has quit IRC03:48
*** rcernin has joined #openstack-ironic03:53
*** rcernin has quit IRC03:54
*** rcernin has joined #openstack-ironic03:54
*** donnyd has quit IRC04:07
*** sri_ has quit IRC04:07
*** donnyd has joined #openstack-ironic04:08
*** stevebaker has quit IRC04:11
*** sri_ has joined #openstack-ironic04:11
*** vkmc has quit IRC04:16
*** vkmc has joined #openstack-ironic04:16
*** adrianc has joined #openstack-ironic04:17
*** adrianc has quit IRC04:23
*** adrianc has joined #openstack-ironic04:41
*** k_mouza has joined #openstack-ironic04:46
*** johnsom has quit IRC04:46
*** Qianbiao has quit IRC04:47
*** johnsom has joined #openstack-ironic04:49
*** k_mouza has quit IRC04:50
*** rcernin has quit IRC04:56
*** rcernin has joined #openstack-ironic04:57
*** adrianc has quit IRC05:07
*** adrianc has joined #openstack-ironic05:08
*** adrianc_ has joined #openstack-ironic05:14
*** adrianc has quit IRC05:16
*** johnsom has quit IRC05:18
*** adrianc_ has quit IRC05:18
*** johnsom has joined #openstack-ironic05:19
*** buhman has quit IRC05:19
*** adrianc has joined #openstack-ironic05:20
*** buhman has joined #openstack-ironic05:21
*** zzzeek has quit IRC05:25
*** zzzeek has joined #openstack-ironic05:27
*** adrianc has quit IRC05:30
*** adrianc has joined #openstack-ironic05:33
*** adrianc has quit IRC05:38
*** adrianc has joined #openstack-ironic05:40
*** tzumainn has quit IRC05:49
*** zzzeek has quit IRC05:50
*** zzzeek has joined #openstack-ironic05:51
*** JamesBenson has quit IRC05:53
*** zzzeek has quit IRC06:10
*** zzzeek has joined #openstack-ironic06:11
*** johnsom has quit IRC06:27
*** johnsom has joined #openstack-ironic06:27
*** moshiur has joined #openstack-ironic06:52
openstackgerritVerification of a change to openstack/ironic failed: Update `cleaning_error_handler`  https://review.opendev.org/75625806:54
*** moshiur has quit IRC06:54
*** moshiur has joined #openstack-ironic06:55
*** Qianbiao has joined #openstack-ironic06:55
*** rcernin has quit IRC07:10
*** rcernin has joined #openstack-ironic07:13
*** zzzeek has quit IRC07:14
*** zzzeek has joined #openstack-ironic07:16
openstackgerritYogesh Ramachandra proposed openstack/ironic master: Fix idrac-wsman bios factory-reset setting  https://review.opendev.org/74869607:30
*** zzzeek has quit IRC07:37
*** nandita has quit IRC07:38
*** nandita has joined #openstack-ironic07:38
*** zzzeek has joined #openstack-ironic07:38
*** ociuhandu has joined #openstack-ironic07:40
*** ociuhandu has quit IRC07:42
*** ociuhandu has joined #openstack-ironic07:43
*** ociuhandu has quit IRC07:43
*** rcernin has quit IRC07:44
*** fmuyassarov has joined #openstack-ironic07:46
*** rcernin has joined #openstack-ironic07:47
arne_wiebalckGood morning, ironic!07:57
*** rpittau|afk is now known as rpittau08:05
rpittaugood morning ironic! o/08:05
*** rcernin has quit IRC08:20
*** ociuhandu has joined #openstack-ironic08:22
*** uzumaki has quit IRC08:24
*** zzzeek has quit IRC08:26
*** zzzeek has joined #openstack-ironic08:30
*** rcernin has joined #openstack-ironic08:33
*** ociuhandu has quit IRC08:38
*** zzzeek has quit IRC08:38
*** zzzeek has joined #openstack-ironic08:40
*** rcernin has quit IRC08:40
arne_wiebalckhey rpittau, all set for a nice coffee I figure?08:44
rpittauahah yeah! :)08:45
rpittaustill getting used to the grinder though, I "had to" try 6 coffees this morning with different grind size08:45
*** zzzeek has quit IRC08:48
*** zzzeek has joined #openstack-ironic08:51
*** mgoddard has joined #openstack-ironic08:51
*** ociuhandu has joined #openstack-ironic08:52
*** iurygregory has joined #openstack-ironic08:52
*** tosky has joined #openstack-ironic08:56
arne_wiebalckyeah, I tried with a grinder as well ... but found some good ground coffee now08:59
arne_wiebalckgrinding coffee is not that obvious08:59
*** k_mouza has joined #openstack-ironic09:00
rpittauarne_wiebalck: yep, and it's been a while since I did it09:01
*** lucasagomes has joined #openstack-ironic09:03
*** dtantsur|afk is now known as dtantsur09:04
*** k_mouza has quit IRC09:05
dtantsurmorning ironic09:05
dtantsurjanders: enroll_node_driver is in inspector.conf not ironic.conf09:06
*** mgoddard has quit IRC09:29
*** rcernin has joined #openstack-ironic09:32
*** rcernin has quit IRC09:40
*** ociuhandu has quit IRC09:42
*** ociuhandu has joined #openstack-ironic09:51
*** ociuhandu has quit IRC09:56
*** zzzeek has quit IRC09:57
*** ociuhandu has joined #openstack-ironic09:58
*** zzzeek has joined #openstack-ironic10:02
*** rcernin has joined #openstack-ironic10:04
*** Qianbiao has quit IRC10:06
*** rcernin has quit IRC10:09
*** jamesdenton has quit IRC10:10
*** jamesdenton has joined #openstack-ironic10:10
*** mgoddard has joined #openstack-ironic10:19
jandersgood omrning arne_wiebalck rpittau dtantsur10:30
rpittauhey janders :)10:30
jandersdtantsur thank you! :)10:30
* janders is looking at inspector.conf10:30
jandersrpittau I'm a total coffee nerd so if you're after coffee chat I can spend any amount of time on that10:31
jandersjust finished customising my grinder :)10:31
janderssurplus test-cups of espresso are surprisingly nice chilled or in homemade gelato :)10:32
rpittaujanders: I'm definitely not at the level of grinder customization! But I already checked some interesting videos on how to modify mine, so getting there :D10:32
janderstwo most important instruments: stopwatch and a precise scale10:33
jandersif the beans are fres, grinder can grind fine enough and you've got these two it's just a matter of time to get a perfect shot every time10:34
janderss/fres/fresh10:34
jandersvacuum packed Illy is also awesome for testing new kit, as it's a really decent bean and 10/10 consistent in my experience10:34
rpittauI see, to be honest I'm more focused on the coffee maker at the moment, you know, temperature, water type and quantity for each coffee shot10:34
rpittaujanders: we use Illy usually :)10:35
jandersI'm 93C, tap water, ~22grams of coffee and ~33grams of ristretto extracted over 42 seconds (12 sec pre-infusion + 30 sec extraction)10:37
jandersor... 10g of coffee and ~30grams of espresso in a more classic setup (for later in the day)10:37
jandersthe coffee machine has its own funky water filter though so the water might not be exactly tap water when it gets into the pumps10:38
*** zzzeek has quit IRC10:38
rpittautap water here is heavy, so we need filters, probably going to install one directly in the machine as well10:38
jandersgood idea10:38
janders..or a good old Brita filters10:38
rpittaujanders: that one :)10:39
jandersmy previous machine didnt have one, so I used brita for years and it worked very well10:39
janderseasier to get replacement filters compared to the machine-proprietary format10:39
*** zzzeek has joined #openstack-ironic10:40
jandersif you try boutique roastery beans, they can have more complex flavour than Illy, but nearly each time you get a new batch it's new setup all over again, especially if you switch blends10:40
jandersand Illy is real good coffee and works well with bonsoy (boutiques often cause curdling or need lots of fine tuning to avoid it)10:41
*** Qianbiao has joined #openstack-ironic10:50
*** k_mouza has joined #openstack-ironic10:51
arne_wiebalckops lore: we had several mgmt switches being taken out, 1000+ nodes went into power state "error" and maintenance "True" (the nodes were subsequently also excluded from power sync); once the switches came back, Ironic updated the power state due to "power_failure_recovery_interval" and lifted the maintenance on these nodes, very nice!10:55
* arne_wiebalck goes and deletes the ticket he had created to clean things up once the switches are back10:56
dtantsuryay, happy to hear!10:56
*** rcernin has joined #openstack-ironic10:58
jandersarne_wiebalck awesome!11:01
*** ociuhandu has quit IRC11:07
*** ociuhandu has joined #openstack-ironic11:07
*** zzzeek has quit IRC11:17
*** zzzeek has joined #openstack-ironic11:20
*** thiagop has joined #openstack-ironic11:25
*** zzzeek has quit IRC11:27
*** zzzeek has joined #openstack-ironic11:28
*** mkrai has quit IRC11:29
openstackgerritVerification of a change to openstack/ironic failed: Limit the default value of [api]api_workers to 4  https://review.opendev.org/76118511:31
*** ociuhandu has quit IRC11:35
openstackgerritYogesh Ramachandra proposed openstack/ironic master: Fix idrac-wsman bios factory-reset setting  https://review.opendev.org/74869611:35
*** ociuhandu has joined #openstack-ironic11:36
*** ociuhandu has quit IRC11:41
*** rcernin has quit IRC11:45
openstackgerritJacob Anders proposed openstack/bifrost master: Disable inspector discovery by default  https://review.opendev.org/76299811:49
*** ociuhandu has joined #openstack-ironic11:53
*** rcernin has joined #openstack-ironic11:58
*** mgoddard has quit IRC12:01
openstackgerritMerged openstack/ironic master: Update `cleaning_error_handler`  https://review.opendev.org/75625812:06
*** ianychoi_ has quit IRC12:10
*** JamesBenson has joined #openstack-ironic12:12
janderssee you tomorrow Ironic o/12:13
*** rcernin has quit IRC12:14
*** mgoddard has joined #openstack-ironic12:15
*** ociuhandu has quit IRC12:28
*** zzzeek has quit IRC12:39
*** zzzeek has joined #openstack-ironic12:41
arne_wiebalckTheJulia: dtantsur: for https://review.opendev.org/#/c/762537/ we said we disable it by default (prio 0), but make it easy to enable via a flag ... isn't janders work on clean step priorities an easy way to enable it if desired?12:47
patchbotpatch 762537 - ironic-python-agent - Add clean step 'erase_pstore' - 3 patch sets12:47
*** ociuhandu has joined #openstack-ironic12:59
*** dougsz has joined #openstack-ironic13:01
*** rcernin has joined #openstack-ironic13:02
*** rcernin has quit IRC13:02
*** ociuhandu has quit IRC13:04
*** ociuhandu has joined #openstack-ironic13:04
*** ociuhandu has quit IRC13:15
*** fmuyassarov has quit IRC13:20
openstackgerritArne Wiebalck proposed openstack/ironic-python-agent master: Add clean step 'erase_pstore'  https://review.opendev.org/76253713:22
openstackgerritBob Fournier proposed openstack/ironic master: [WIP] Add vendor_passthru methods for virtual media  https://review.opendev.org/76300713:24
*** fmuyassarov has joined #openstack-ironic13:28
*** fmuyassarov has quit IRC13:32
*** Goneri has joined #openstack-ironic13:40
*** k_mouza has quit IRC13:41
*** ociuhandu has joined #openstack-ironic13:45
*** dougsz has quit IRC13:48
*** ociuhandu has quit IRC13:50
rpittaugrep from_dict * -R13:52
rpittauwrong window :P13:52
iurygregorynot found13:52
rpittaulol13:53
*** fmuyassarov has joined #openstack-ironic13:53
*** moshiur has quit IRC13:56
*** ociuhandu has joined #openstack-ironic13:57
*** paras333 has joined #openstack-ironic13:59
TheJuliagood morning14:05
TheJuliadtantsur: https://review.opendev.org/#/c/748049 works *insert evil laugh here*14:10
patchbotpatch 748049 - ironic-python-agent - Support partition image efi contents - 27 patch sets14:10
*** rloo has joined #openstack-ironic14:16
*** dougsz has joined #openstack-ironic14:17
*** dougsz has quit IRC14:17
*** dougsz has joined #openstack-ironic14:18
TheJuliagood morning14:18
paras333TheJulia: GM:)14:19
openstackgerritRiccardo Pittau proposed openstack/ironic master: Remove from_dict function from context  https://review.opendev.org/76301614:21
openstackgerritMerged openstack/ironic-python-agent stable/victoria: Fix physical memory calculation with new lshw  https://review.opendev.org/76284114:22
*** k_mouza has joined #openstack-ironic14:24
*** k_mouza has quit IRC14:24
*** k_mouza has joined #openstack-ironic14:24
fmuyassarovHello Ironic Community. Sorry for asking it again. But can I get a review on https://review.opendev.org/#/c/762343/ please whenever you have time. I asked yesterday about a couple of failed CI checks, but I got know from rpittau that those are know issues in focal. And TBH, I don't know how to fix them14:26
patchbotpatch 762343 - ironic - Allow disabling automated_clean per node - 9 patch sets14:26
rpittaufmuyassarov: hi! the issue wiht ironic-tempest-ipxe-ipv6 is something that we encountered when switching CI to focal, we're still investigating, it seems a change in behavior in ubuntu, not related to your patch, so no worries about that14:27
rpittaufirst time I see the rabbvitmq issue in ironic-inspector-tempest-partition-bios-redfish-vmedia to be honest oO14:28
fmuyassarovHi rpittau , good to know. Thank you :)14:28
rpittaustill, not related to your change it seems14:28
dtantsurmorning TheJulia14:29
rpittaummmm "ERROR: epmd error for host ubuntu-focal-limestone-regionone-0021788829: address (cannot connect to host/port)"14:29
dtantsurarne_wiebalck: he is, yes14:29
*** ociuhandu has quit IRC14:30
rpittaufmuyassarov: yep, not an issue on your patch as well. Could you please just add the task number to the commit message ?14:31
fmuyassarovrpittau, sorry, what is the task number? I added story number. Is it the same thing?14:33
dtantsurfmuyassarov: nope, any story has tasks. a patch is attached to a task in a story, so you need both numbers.14:34
rpittaufmuyassarov: story refers to the story in storyboard, the task number is associated to the task in the story itself14:34
rpittauor what dtantsur wrote :D14:34
arne_wiebalckA sneak preview from my upcoming book "Make your ops life hard ... yourself!": Our downstream h/w manager deregisters the node from our PXE infra in evaluate_hardware_support(). The corresponding call can take up to several minutes. Anyone seeing why this is really bad idea?14:35
arne_wiebalckHint: I was seeing evaluate_hardware_support() being called surprisingly often ...14:35
fmuyassarovGot it. Thanks rpittau and dtantsur14:35
TheJuliaarne_wiebalck: because it always wants to wait for disks?14:35
arne_wiebalckTheJulia: no14:35
TheJuliahmm14:35
rpittaummmm14:35
* arne_wiebalck hopes his explanation is actually correct14:36
openstackgerritFeruzjon Muyassarov proposed openstack/ironic master: Allow disabling automated_clean per node  https://review.opendev.org/76234314:36
arne_wiebalckthere were several calls to evaluate_hardware_support() of our h/w manager14:36
TheJuliaoh, from within14:36
TheJulia"fun!"14:37
arne_wiebalckevery time it took some minutes14:37
arne_wiebalckeventually the cleaning timeout was reached14:37
arne_wiebalckbut why was it called multiple times?>14:37
arne_wiebalcksince the managers need to be sorted, and I guess sorted() calls the function several times14:38
TheJuliawait_for_disks dispatches to all hardware managers14:38
TheJuliaI believe that does trigger evaluate as well14:38
arne_wiebalckmultiple times per hardware manager?14:39
TheJuliano, in the main code path for various actions14:39
arne_wiebalckI see evaluate_hardware_support() being called 3 times for our h/w manager14:39
TheJuliadiretly from... hmm14:40
arne_wiebalckI obviously expected it to be called once14:40
TheJuliaare there any command dispatches taking place?14:40
arne_wiebalckdon't you think it could be sorted() invoking evaluate_hardware_support() to know how to sort the managers?14:41
fmuyassarovdtantsur, rpittau related to 762343, I want the Ironic client to be able to pass "false" value for node.automated_clean. Is it okay If I work on that? and I think I need a story for that too right?14:41
TheJuliazer0c00l: you going to be around today?14:41
TheJuliaarne_wiebalck: I'd have to look at that part of the code, give me a moment14:42
*** tzumainn has joined #openstack-ironic14:43
arne_wiebalckTheJulia: https://github.com/openstack/ironic-python-agent/blob/master/ironic_python_agent/hardware.py#L219914:43
arne_wiebalckTheJulia: and https://github.com/openstack/ironic-python-agent/blob/master/ironic_python_agent/hardware.py#L222414:44
TheJuliaeww14:44
TheJuliawhy did we do that?!?14:44
arne_wiebalckit makes sense if that function returns immediately14:44
TheJuliaI was just about paste basically the same second link14:44
arne_wiebalck:-D14:44
rpittaufmuyassarov: I believe another task in the same story should be enough14:45
arne_wiebalckour function has a 12 min timeout before it returns ... ouch14:45
TheJuliathe first link is definitely it though :(14:45
fmuyassarovgot it rpittau14:45
TheJuliaI was starting to wonder why the IPA logs were feeling much more noisy14:45
arne_wiebalckI think it might better to call this once for all and sort with the values.14:46
arne_wiebalckDepending on the algorithm that function is called often.14:46
arne_wiebalckI guess none are as long as ours, but still :)14:46
TheJuliaI guess I'm not groking in my half coffeed state why we're evaluating then and there...14:47
TheJulialooks like we only call get_managers twice, but even that first seems like it could derail you14:47
arne_wiebalcksorted() will call it, then we loop once more over them and call it again14:52
arne_wiebalckall fine, clearly a problem of our h/w manager :)14:53
TheJuliawell, inspection and then the main ipa code path if the inspection path is triggered14:53
arne_wiebalckthis is all on cleaning14:53
TheJuliaahh, yeah and every step will resort the list too14:55
TheJuliabecause latest clean steps need to be obtained14:55
arne_wiebalckI don't get there: by the time I identify the hardware manager Ironic timed out :-D14:57
TheJulianice14:57
*** ociuhandu has joined #openstack-ironic15:01
*** alexmcleod has quit IRC15:04
*** tosin has joined #openstack-ironic15:07
*** tosin has quit IRC15:08
*** ociuhandu has quit IRC15:10
*** ociuhandu has joined #openstack-ironic15:10
*** dougsz has quit IRC15:11
*** dougsz has joined #openstack-ironic15:17
*** ociuhandu has quit IRC15:21
*** ociuhandu has joined #openstack-ironic15:27
dtantsurTheJulia: we've had a discussion downstream that boiled down to the fact that enabling auto-discovery by default may be a source of confusing problems.15:28
dtantsurbecause of that janders proposed https://review.opendev.org/#/c/762998/ I wonder what you think15:28
patchbotpatch 762998 - bifrost - Disable inspector discovery by default - 1 patch set15:28
dtantsur(since it was you who introduced the discovery support initially IIRC)15:29
TheJuliaI have really mixed feelings at the moment on this15:37
*** ociuhandu has quit IRC15:38
TheJuliaWouldn't it make more sense to highlight the disparity in how inspection works documentation?15:38
TheJuliaI mean, I'm not saying no, don't disable it, I'm just confused by why this is suddenly a problem15:39
*** alexmcleod has joined #openstack-ironic15:40
Qianbiaoarne_wiebalck are CERN using auto-discovery in your env?15:42
QianbiaoI am courious about whether this feature will cause maintenance/debug issues in big cloud env15:43
*** fmuyassarov has quit IRC15:45
*** mkrai has joined #openstack-ironic15:50
TheJuliaThe issue seems to be is that somehow this perception that one should be able to introspect without providing a physical port to match the identity of the machine, end up with all of the ports, and I guess I find that frustrating15:50
*** trandles has joined #openstack-ironic15:51
TheJuliaQianbiao: I think the problem is more about virutal and some cases where one can't just discover the IP address of the bmc15:51
QianbiaoTheJulia think about a server has two port connected.15:53
Qianbiaopxe port mac identify is not strong enough too.15:54
TheJuliatwo port depends on inspection configuration, if it creates additional ports or not15:55
arne_wiebalckQianbiao: no, not yet15:55
arne_wiebalckQianbiao: we are planning to do this, though15:55
TheJuliaI agree it is not strong enough, but it is consitent and always available to the running OS.15:55
Qianbiaoarne_wiebalck I am afraid, if there are a server has some problem when inspect.15:55
Qianbiaothen we will lose the server.15:56
Qianbiaoand can not find it back.15:56
QianbiaoTheJulia agree.15:56
TheJuliathis is also why allowing inspector to add all mac addresses is a good idea... And inspection should only be under a fairly controlled circumstance  as part of their process, similar to those that use auto-discovery to wheel a rack into the data center and power it up.15:57
QianbiaoI am courious whether auto-discovery will cause "duplicate discoveryed" or "miss discoveryed"15:58
QianbiaoTheJulia yeah, it seem add_port all will fix the issue.15:59
arne_wiebalckQianbiao: TheJulia: are auto-discovery and inspection somehow tied together? I was thinking the former only creates an entry in the Ironic DB, no?15:59
Qianbiaoam I wrong: auto-discovery will ends up with auto inspect?16:00
Qianbiaodid not try that XD16:00
*** ociuhandu has joined #openstack-ironic16:02
TheJuliaThey are basically the same code path16:02
TheJuliaIf the node is not found, then if discovery is enabled, then inspector creates the node16:02
TheJuliaoften this is if the BMC IP can't be determined16:02
arne_wiebalckso auto-discovery implies running an inspection?16:03
dtantsurI would even say that auto-discovery is a special case of inspection16:05
Qianbiaoarne_wiebalck i think, yes.16:06
arne_wiebalckdtantsur: Qianbiao: thanks!16:07
*** k_mouza has quit IRC16:07
TheJuliawhat dtantsur said :)16:10
QianbiaoIs there a method that updating node's port while the node is in deploying status ?16:11
*** ebagakis has joined #openstack-ironic16:11
QianbiaoI need to store some data to port's extra field when provisioning the node.16:11
dtantsurin deploying state a conductor is holding a lock, so likely no16:12
Qianbiaosad :(16:12
Qianbiaoseems i need to find other way to roma16:12
TheJuliadeploy wait it shouldn't be locked, fwiw16:13
QianbiaoTheJulia not deploy wait, when NOVA preparing the deploy, i need to bind-port and add extra data16:13
Qianbiaoand node's status is exactly deploying..16:14
*** bnemec has quit IRC16:16
TheJuliaoh, the deploy hasn't bneen started yet16:21
*** mkrai has quit IRC16:21
openstackgerritArne Wiebalck proposed openstack/ironic-python-agent master: Add clean step 'erase_pstore'  https://review.opendev.org/76253716:22
TheJuliathe ironic nova virt driver is still preparing as you said, so it hasn't called the deploy call yet.16:22
*** k_mouza has joined #openstack-ironic16:23
Qianbiaoyes16:27
*** iurygregory has quit IRC16:29
*** Qianbiao has quit IRC16:30
openstackgerritDmitry Tantsur proposed openstack/bifrost master: Support UEFI in bifrost-cli and document it  https://review.opendev.org/76255516:35
rpiosoGood morning, ironic16:44
*** bnemec has joined #openstack-ironic16:50
JayFarne_wiebalck: small issue with the erase_pstore, if you wanna clear up that comment I can +216:53
*** ociuhandu has quit IRC16:55
*** ociuhandu has joined #openstack-ironic16:55
openstackgerritJulia Kreger proposed openstack/ironic master: Minor follow-up doc change  https://review.opendev.org/76304516:57
*** ociuhandu has quit IRC17:00
openstackgerritArne Wiebalck proposed openstack/ironic-python-agent master: Add clean step 'erase_pstore'  https://review.opendev.org/76253717:02
arne_wiebalckJayF: thanks, uploaded a new patch, oversight from enabled to disabled, good catch, thanks!17:03
JayFIt's already got my +2 :D17:03
* JayF loves how you can diff patchsets in gerrit17:03
arne_wiebalckJayF: great thanks17:04
arne_wiebalckJayF: did you notice when diffing that I needed to change the iterator for the open context in 3.9 ?17:04
JayFif that was in patchset 6, it wasn't17:05
JayFthe only change in patchset 6 is that comment I had commented on17:05
JayFarne_wiebalck: if you think that change has iterator fixes in it, you missed a `git add` somewhere I think17:06
arne_wiebalckJayF: no, it was earlier17:07
arne_wiebalckJayF: https://review.opendev.org/#/c/762537/3..5/ironic_python_agent/tests/unit/test_hardware.py17:07
patchbotpatch 762537 - ironic-python-agent - Add clean step 'erase_pstore' - 6 patch sets17:07
JayFnice17:08
*** lucasagomes has quit IRC17:08
arne_wiebalckrealising iter was missing took me a while, but the initial version did not work on 3.917:08
arne_wiebalcktime spent: 10% code, 90% on the test :-S17:09
JayFthat's not super abnormal for me :D17:09
arne_wiebalckit is probably even good ... I am just annoy... surprised every time :)17:10
*** ociuhandu has joined #openstack-ironic17:11
openstackgerritVerification of a change to openstack/ironic failed: Update .rst files  https://review.opendev.org/75923617:18
*** iurygregory has joined #openstack-ironic17:19
rpittaugood night! o/17:27
*** rpittau is now known as rpittau|afk17:27
*** ebagakis has quit IRC17:29
*** k_mouza has quit IRC17:30
*** dougsz has quit IRC17:32
*** ociuhandu_ has joined #openstack-ironic17:43
*** ociuhandu has quit IRC17:46
*** ociuhandu_ has quit IRC17:47
*** k_mouza has joined #openstack-ironic17:50
*** fmuyassarov has joined #openstack-ironic17:53
openstackgerritJulia Kreger proposed openstack/ironic stable/train: Sync boot mode when changing the boot device via Redfish  https://review.opendev.org/76243917:57
*** fmuyassarov has quit IRC17:58
*** mgoddard has quit IRC18:02
*** k_mouza has quit IRC18:10
*** fmuyassarov has joined #openstack-ironic18:12
openstackgerritMerged openstack/bifrost master: Expand testenv documentation  https://review.opendev.org/76255318:17
*** k_mouza has joined #openstack-ironic18:19
openstackgerritRiccardo Pittau proposed openstack/sushy-tools master: Allow setting debug mode from command line  https://review.opendev.org/76306818:30
rpiosodtantsur: I'm looking at your comment about https://review.opendev.org/#/c/763007: https://review.opendev.org/#/c/763007/1/ironic/drivers/modules/drac/vendor_passthru.py@19918:31
patchbotpatch 763007 - ironic - [WIP] Add vendor_passthru methods for virtual media - 1 patch set18:31
patchbotpatch 763007 - ironic - [WIP] Add vendor_passthru methods for virtual media - 1 patch set18:31
openstackgerritRiccardo Pittau proposed openstack/sushy-tools master: Allow setting debug mode from command line  https://review.opendev.org/76306818:31
rpiosodtantsur: Are you suggesting class DracRedfishVendorPassthru be a simple derivation of vendor-independent Redfish's VendorPassthru?18:33
openstackgerritRiccardo Pittau proposed openstack/sushy-tools master: Allow setting debug mode from command line  https://review.opendev.org/76306818:35
*** k_mouza has quit IRC18:56
dtantsurrpioso: honestly, we may not even need the Drac version18:57
*** ayoung has quit IRC18:57
TheJuliagood morning18:58
* TheJulia guesses stevebaker will appear18:58
* dtantsur hopes18:59
rpiosodtantsur: Are you saying there may not be a need for the  idrac driver to support that Redfish vendor passthru vmedia stuffs?18:59
rpiosoTheJulia: Good morning :-)19:00
dtantsurrpioso: no, for having a separate vendor passthru class19:00
TheJuliaThis may sound a bit strong or firm, but ideally nothing should ever require new vendor passthru methods unless a vendor needs a highly specific thing19:01
TheJuliaand at that point, it is their driver only and there would be no reason to ever implement the same elsewhere... because if we have done that then we've not headed down the common/mutual/abstracted path19:01
dtantsurTheJulia: even without Steve, should we jump on a call and start reviewing?19:02
rpiosodtantsur, TheJulia: I'm unfamiliar with why that change has been proposed. There's no story. And that's understandable, since it's a WIP.19:03
TheJuliadtantsur: I think that is a good idea to likely jump on a call19:03
TheJuliamaybe let me get some water first19:03
dtantsurrpioso: that's a fair point. there is a bit of a context, I think it should be in the commit message.19:03
dtantsurvery tl;dr to support 3rd party installers via ironic19:04
TheJuliaso, meetpad.opendev.org/ironic good?19:06
dtantsurchromium crashed, let's try firefox19:06
toskydtantsur: re chromium crashes: https://bugzilla.redhat.com/show_bug.cgi?id=189592019:12
openstackbugzilla.redhat.com bug 1895920 in chromium "Chromium 86 crashes on WebRTC videos when switching window" [Urgent,New] - Assigned to spotrh19:12
*** fmuyassarov has quit IRC19:15
TheJuliawheeee19:15
*** gyee has joined #openstack-ironic19:18
clarkbnote that firefox's webrtc system isn't very good compared to chrom*19:19
clarkbit may work or it may not, we've seen it go both ways with jitsi meet19:19
dtantsurclarkb: it tends to work for me19:20
dtantsurthey have improved it recently19:20
*** mgoddard has joined #openstack-ironic19:21
*** stevebaker has joined #openstack-ironic19:22
stevebakerhey, late for my own review stab19:25
dtantsurstevebaker: https://meetpad.opendev.org/ironic19:26
TheJulia\o/19:26
TheJuliawait, stab?!?19:26
tzumainnTheJulia, hi! thanks for the updated documentation. there's one thing that still confuses me and naved001 which is - if we use ipxe, how does the node get the ipxe file?19:27
dtantsurwe're on a review meeting, she'll likely answer later19:27
tzumainngotcha, no rush and thanks!19:27
*** jawad_axd has joined #openstack-ironic19:36
*** fmuyassarov has joined #openstack-ironic19:37
stevebakerTheJulia, dtantsur: https://review.opendev.org/#/c/750120/20/ironic/api/controllers/v1/port.py19:46
patchbotpatch 750120 - ironic - Convert ports endpoint to plain JSON - 20 patch sets19:46
*** k_mouza has joined #openstack-ironic19:57
*** k_mouza has quit IRC20:01
*** gryf has quit IRC20:09
*** gryf has joined #openstack-ironic20:09
bfournierpioso: just catching up on above, yes I'll add a story for node passthru, note that it was referred to in https://etherpad.opendev.org/p/IronicWhiteBoard (line 327)20:09
bfournierpioso: actually 33120:11
rpiosobfournie: Thank you! Will it be booted from one-time?20:15
* rpioso believes one-time or continuous may not matter.20:16
*** k_mouza has joined #openstack-ironic20:16
*** dtantsur is now known as dtantsur|afk20:17
dtantsur|afkgood night folks!20:18
rpiosobfournie: At least for one-time, the iDRAC treats the inserted vmedia as a *virtual* CD/DVD/floppy. AFAIU, there's no such ironic boot device. Vendor-independent Redfish also doesn't grok those.20:19
*** k_mouza has quit IRC20:21
bfournierpioso: yes I don't think we need to set that as not setting boot device, basically it is exposing the _insert_vmedia method as api https://github.com/openstack/ironic/blob/master/ironic/drivers/modules/redfish/boot.py#L144. The main reason for this is to expose eject_vmedia so that a user can set boot mode to DISK and disconnect the ISO that it originally booted with20:24
*** mgoddard has quit IRC20:31
rpiosobfournie: I'm familiar with that method. Are you trying to boot an ISO that has been inserted that way?20:31
*** fmuyassarov has quit IRC20:32
* rpioso is unclear on how the system was instructed to connect the ISO and boot from it.20:32
*** fmuyassarov has joined #openstack-ironic20:39
*** fmuyassarov has quit IRC20:47
bfournierpioso: the ISO insert and boot is done via an assisted installer outside of ironic, so ironic is responsible for ejecting it in order to boot from disk. Realisitically we can remove the insert_vmedia support from this patch if that is a concern, the main requirement is the eject20:51
rpiosobfournie: That helps. Thanks! Could class DracRedfishVendorPassthru simply be derived from the vendor-independent Redfish class? We've done that for other iDRAC Redfish interface implementations, e.g., https://github.com/openstack/ironic/blob/172b76f0f949c7f927f62b915d730d5d0364432a/ironic/drivers/modules/drac/bios.py#L42-L49.20:59
jandersgood morning Ironic o/20:59
*** rcernin has joined #openstack-ironic21:02
rpiosoGood morning, janders :)21:05
*** hjensas_ has joined #openstack-ironic21:06
bfournierpioso: https://storyboard.openstack.org/#!/story/2008363, yes I will look at deriving this class21:07
bfournierpioso: thanks21:09
*** hjensas has quit IRC21:10
rpiosobfournie: Cool! You're most welcome. While I'm a big fan of symmetry, for iDRAC, the usefulness of an insert vmedia API is questionable, because it can't be booted from via typical ironic nor Redfish means.21:10
bfournierpioso: yeah, I tend to agree with that21:11
openstackgerritJacob Anders proposed openstack/bifrost master: Disable inspector discovery by default  https://review.opendev.org/76299821:11
bfournierpioso: will strip that out21:12
iurygregorymorning janders o/21:12
rpiosobfournie: Go Pats!21:13
bfournierpioso: Ha! Not this year. eh21:13
bfournietoo bad21:13
TheJuliaAdded a new agenda item for next week's meeting in discussion... converged adapters that do the wrong thing with the UEFI nvram table21:18
*** shadower has quit IRC21:20
*** shadower has joined #openstack-ironic21:22
jandersTheJulia dtantsur|afk just reading through the discussion about https://review.opendev.org/#/c/762998/ (disabling auto-discovery by default) - thank you for looking into this. In the end, do you think this is a good idea, bad idea, or further discussion is needed? It seems like the discussion drifted away to other topics without reaching21:28
jandersconclusion. From my perspective it would be useful, as bifrost is largely used in VM context for development and testing and that's where auto-discovery can be having problems - but ultimately it's your decision. Thanks! :)21:28
patchbotpatch 762998 - bifrost - Disable inspector discovery by default - 2 patch sets21:28
openstackgerritVerification of a change to openstack/ironic failed: New argument validate decorator  https://review.opendev.org/74279521:40
openstackgerritVerification of a change to openstack/ironic failed: Add expose body decorator, status_code argument  https://review.opendev.org/74367721:40
TheJuliajanders: From talking with physical operators who use bifrost for machine staging/testing/deployment, autodetect may be used, we simply don't have the right information to make an informed decision.21:51
TheJuliajanders: maybe sending a mailing list post might help and maybe reaching out to the operators group?21:51
jandersTheJulia ACK, will do, thank you. Regarding the operators group - do I just mention in the email that I'm particularly after feedback from them, or is there a particular tag or alias I should use for that?21:53
TheJuliajanders: they are most active on twitter, I don't have the group handy but they have an etherpad with the link somepalce21:53
iurygregoryhttps://twitter.com/osopsmeetup21:56
iurygregorythe etherpad they had for W https://etherpad.opendev.org/p/wallaby-ptg-ops-meetup21:56
openstackgerritVerification of a change to openstack/ironic failed: Limit the default value of [api]api_workers to 4  https://review.opendev.org/76118521:59
jandersThank you iurygregory!22:00
janders(and go get some rest! :)22:00
iurygregory23h here =)22:00
iurygregory1hr to go sleep =)22:00
janders:)22:03
TheJuliaiurygregory: thanks22:13
iurygregorynp =)22:14
*** shadower has quit IRC22:16
*** shadower has joined #openstack-ironic22:19
*** thiagop has quit IRC22:34
*** stevebaker has quit IRC22:37
*** stevebaker has joined #openstack-ironic22:38
*** hjensas__ has joined #openstack-ironic22:47
*** hjensas_ has quit IRC22:51
*** thiagop has joined #openstack-ironic23:19
*** thiagop has quit IRC23:24
openstackgerritJulia Kreger proposed openstack/ironic stable/train: Sync boot mode when changing the boot device via Redfish  https://review.opendev.org/76243923:32
TheJuliatzumainn: if you use ipxe, there still needs to be someplace avaialble that the machines on the ironic nodes can reach through the neutron router so neutron dhcp can tell it where to go and the machine can make that request.23:37
TheJuliatzumainn: maybe there is a case where it would make sense wrap extra layers around that, but that path still needs to also tie back to conductors too, on a plus side, there can be numerous ones23:39
tzumainnTheJulia, okay, so this is probably where my confusion lies; when one uses, say, metalsmith and ansible networking to do this, neutron will just attach the provisioning network for you; if I understand correctly, that won't happen in the boot-from-volume case?23:48
TheJuliaNo23:49
TheJuliain the case of booting from a volume, the end network for the machine is attached23:49
TheJuliathe dhcp information supplied includes where to get the ipxe loader23:49
tzumainnokay, so it's the responsibility of the deployer to ensure that the machine can reach the ipxe file?23:50
TheJuliaso kind of like how provisioning works, except not the inherent risk nor need of an isolated island network to keep the machines isolated from the world for provisioning/cleaning operations23:50
*** tosky has quit IRC23:58

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!