Wednesday, 2020-07-29

*** rcernin has quit IRC00:05
*** alexmcleod has quit IRC00:06
*** alexmcleod has joined #openstack-ironic00:07
*** k_mouza has joined #openstack-ironic00:15
jandersI have a question regarding https://storyboard.openstack.org/#!/story/161801400:17
jandersthere is a gerrit for it, it's been abandoned:00:17
jandershttps://review.opendev.org/#/c/442383/00:17
patchbotpatch 442383 - ironic - [WIP]Generic way to configure clean step priorites... (ABANDONED) - 1 patch set00:17
jandersI'm picking it back up and getting to a point where I want to upload my own WIP revision00:18
jandershow do I do that?00:18
jandersam I suppose to continue in the same gerrit, or create a new one?00:18
jandersif I submit with the same Change-ID will Gerrit know it's a continuation of the old work?00:18
*** k_mouza has quit IRC00:20
*** rcernin has joined #openstack-ironic00:31
*** weshay|ruck has quit IRC00:53
*** weshay_ has joined #openstack-ironic00:53
*** andrein has quit IRC00:57
*** zzzeek has quit IRC00:57
*** dtantsur|afk has quit IRC00:57
*** andrein has joined #openstack-ironic01:01
*** zzzeek has joined #openstack-ironic01:01
*** dtantsur|afk has joined #openstack-ironic01:01
*** gyee has quit IRC01:02
*** mkowalski has quit IRC01:03
*** tkajinam has quit IRC01:06
*** tkajinam has joined #openstack-ironic01:06
*** mkowalski has joined #openstack-ironic01:10
stevebakerjanders: yes, the Change-ID is gerrit's identifier for a change. If you re-use a change-id in the same branch (master) then it will show up as a new revision in the existing review01:12
stevebakerjanders: HOWEVER01:14
stevebakerjanders: if the original change is abandoned, you'll need a core or the original author to unabandon it before you can push a new review, and it might be easier to generate a new change-id in that case01:15
*** ociuhandu has joined #openstack-ironic01:17
*** ociuhandu has quit IRC01:21
*** Goneri has quit IRC01:25
*** k_mouza has joined #openstack-ironic01:31
*** tkajinam has quit IRC01:33
jandersnoted! thank you stevebaker01:33
*** tkajinam has joined #openstack-ironic01:34
*** k_mouza has quit IRC01:36
*** ociuhandu has joined #openstack-ironic01:43
*** ociuhandu has quit IRC01:48
*** ricolin has quit IRC02:15
*** yaawang has quit IRC02:21
*** ricolin has joined #openstack-ironic02:21
*** yaawang has joined #openstack-ironic02:22
*** k_mouza has joined #openstack-ironic02:45
*** k_mouza has quit IRC02:50
*** ociuhandu has joined #openstack-ironic02:53
*** ociuhandu has quit IRC02:57
*** diurnalist has joined #openstack-ironic02:59
*** rloo has quit IRC03:08
*** stevebaker has quit IRC03:18
*** diurnalist has quit IRC03:20
*** k_mouza has joined #openstack-ironic03:27
*** k_mouza has quit IRC03:31
*** mkrai has joined #openstack-ironic03:44
*** rh-jelabarre has quit IRC04:32
*** rh-jlabarre has joined #openstack-ironic04:32
*** rh-jlabarre has quit IRC04:33
*** rh-jlabarre has joined #openstack-ironic04:33
*** Lucas_Gray has quit IRC04:44
*** diurnalist has joined #openstack-ironic05:01
*** janders has quit IRC05:02
*** tzumainn has quit IRC05:04
*** diurnalist has quit IRC05:06
*** waleedm has joined #openstack-ironic05:07
*** janders has joined #openstack-ironic05:08
*** waleedm_ has joined #openstack-ironic05:12
*** waleedm has quit IRC05:15
*** waleedm_ has quit IRC05:16
*** waleedm__ has joined #openstack-ironic05:16
*** stevebaker has joined #openstack-ironic05:16
*** diurnalist has joined #openstack-ironic05:16
*** diurnalist has quit IRC05:22
openstackgerritSteve Baker proposed openstack/ironic master: Convert root controller to plain controller  https://review.opendev.org/74106405:23
openstackgerritSteve Baker proposed openstack/ironic master: Convert v1 controller to plain, return JSON  https://review.opendev.org/74106505:23
openstackgerritSteve Baker proposed openstack/ironic master: New argument validate decorator  https://review.opendev.org/74279505:23
openstackgerritSteve Baker proposed openstack/ironic master: Convert drivers endpoint to plain JSON  https://review.opendev.org/74186505:23
openstackgerritSteve Baker proposed openstack/ironic master: Break out collection functions for json usage  https://review.opendev.org/74367605:23
openstackgerritSteve Baker proposed openstack/ironic master: Add status_code argument to @method.expose  https://review.opendev.org/74367705:23
openstackgerritSteve Baker proposed openstack/ironic master: WIP Add @method.body decorator  https://review.opendev.org/74367805:23
openstackgerritSteve Baker proposed openstack/ironic master: WIP Convert allocation endpoint to plain JSON  https://review.opendev.org/74367905:23
stevebakerboom, soz05:24
*** yaawang has quit IRC05:52
*** yaawang has joined #openstack-ironic05:53
*** ociuhandu has joined #openstack-ironic05:59
*** diurnalist has joined #openstack-ironic06:08
*** abdysn has joined #openstack-ironic06:08
*** mkrai has quit IRC06:10
iurygregorygood morning Ironic!06:12
*** diurnalist has quit IRC06:12
*** mkrai has joined #openstack-ironic06:15
*** brtknr has quit IRC06:17
jandersgood morning iurygregory06:20
jandershow are you?06:21
iurygregoryhey janders, doing good =) and you?06:29
jandersI'm doing allright, hitting a bit of a wall with https://review.opendev.org/#/c/442383/ specifically around handing how MultiOpt configs work06:31
patchbotpatch 442383 - ironic - [WIP]Generic way to configure clean step priorites... (ABANDONED) - 1 patch set06:31
janderswould you have some time to have a look at it with me?06:31
iurygregoryin a few, let met finish a few things here and I will take a look =)06:32
jandersthat would be great, thank you :)06:32
jandersin the meantime I will gather some debug info in a sharable format06:33
iurygregoryack06:33
iurygregory=)06:33
*** mkrai has quit IRC06:40
*** yaawang has quit IRC06:42
*** yaawang has joined #openstack-ironic06:42
arne_wiebalckGood morning iurygregory janders and ironic!06:51
iurygregorymorning arne_wiebalck o/06:51
jandersgood morning arne_wiebalck o/06:51
*** mkrai has joined #openstack-ironic06:55
*** diurnalist has joined #openstack-ironic06:59
*** diurnalist has quit IRC07:04
arne_wiebalckrpittau: rpioso: I added a summary of our redfish meeting to the whiteboard, l. 418ff; feel free to add or correct07:12
*** mkrai has quit IRC07:15
*** mkrai has joined #openstack-ironic07:26
*** dougsz has joined #openstack-ironic07:32
*** yolanda has quit IRC07:40
*** diurnalist has joined #openstack-ironic07:50
*** dtantsur|afk is now known as dtantsur07:53
dtantsurmorning ironic07:53
*** diurnalist has quit IRC07:55
*** mkrai has quit IRC07:55
jandersgood morning dtantsur07:55
iurygregorygood morning dtantsur07:59
openstackgerritAija Jaunteva proposed openstack/ironic-specs master: [WIP] Operationally efficient & fast-track provisioning  https://review.opendev.org/74072108:05
rpittaugood morning ironic! o/08:06
*** lucasagomes has joined #openstack-ironic08:08
rpittauarne_wiebalck: thanks! :)08:08
openstackgerritDmitry Tantsur proposed openstack/ironic-python-agent master: Return the final RAID configuration from apply_configuration  https://review.opendev.org/74368908:11
dtantsuryet another sw RAID fix ^^^08:11
*** mkrai has joined #openstack-ironic08:12
jandersI'm hitting this issue pretty hard and was wondering if you have any pointers: http://paste.openstack.org/show/796399/08:13
iurygregorymorning rpittau o/08:14
rpittauhey iurygregory :)08:14
jandersI think my main problem is that the config in lines 4-5 of the paste gets mangled in the form visible in line 1208:14
janders*something* is adding double quotes which very effectively break any attempts of parsing this config option in a reasonable manner08:15
jandersdo you have any thoughts on what's happening and why?08:15
rpittaujanders: what's item in config ? it seems you're interpreting it like a dict but looks like it's a string instead08:16
jandersrpittau it should be a dict (MultiOpt should merge the lines into a list of dicts) but you are right, it is not08:17
*** yolanda has joined #openstack-ironic08:17
rpittauoh I see you're printing the type there, read to fast :D08:17
rpittautoo*08:17
* rpittau insert more coffee to continue08:18
jandersyeah I was trying to narrow it down08:18
openstackgerritDmitry Tantsur proposed openstack/ironic master: AgentRAID: Account for empty results in post-configuration checks  https://review.opendev.org/74369008:20
dtantsurand another ^^08:21
rpittau\o/08:21
*** derekh has joined #openstack-ironic08:23
rpittaujanders: wondering if MultiOpt doesn't do the merge correctly for some reason08:24
rpittauoh wait08:24
openstackgerritDmitry Tantsur proposed openstack/ironic master: Trivial: fix a minor issue in standalone docs and improve formatting  https://review.opendev.org/74369208:24
rpittaujanders: that is technically correct, it is a list of dictionaries indeed, with just one dict08:25
iurygregorydtantsur, it's [json_rpc] auth_type and not auth_strategy?08:27
dtantsuriurygregory: strictly speaking, auth_type is for a client, auth_strategy is for a server08:27
dtantsurto make matters easier, my previous patch defaults auth_type to auth_strategy08:27
dtantsurbut if you want to be strictly correct, you need to provide both08:27
iurygregorywas asking because of https://github.com/metal3-io/ironic-image/blob/master/configure-ironic.sh#L74 =)08:27
jandersrpittau correct - however it injects the quotes in a weird way, leaving the opening curly brace in and the other one out, e.g. 'erase_devices': "{'shred_random_overwrite_iterations':1"08:28
rpittaujanders: mmm I see08:29
jandersmy understanding is the CONF.conductor.clean_step_add_params should be a list of dicts, but because of the double-quote weirdness those dicts can't be parsed easily08:29
rpittaujanders: what happen if you put a space between the : and the 1 ?08:29
jandersin ironic.conf?08:29
rpittauyeah, how's your ironic.conf ?08:30
janderslines 4-5 in the paste have the relevant lines08:30
rpittaucause it looks like this is happening only for that integer08:30
rpittauinteresting, that might be a bug08:31
jandersI since tried switching to DictOpt for troubleshooting, let me undo it and retest08:31
janderswell spotted with the integer, maybe spacing or quoting it will make a difference08:31
janders(I had same issues with DictOpt BTW)08:32
rpittauI wonder if the parser make some confusion with integers, cuase the strings look ok08:32
gudrutis0Have you guys used devstack with ubuntu2004?08:37
rpittaugudrutis0: yes08:37
gudrutis0Ok, so it is in favor of ubuntu1804 :)08:38
rpittaugudrutis0: mmm the contrary, ubuntu2004 replaces ubuntu180408:39
gudrutis0that is what I actually ment...08:40
rpittauok08:40
rpittauwe're using it in CI for some jobs, for example ironic-tempest-ipa-partition-uefi-pxe-grub208:40
jandersrpittau I tried single-quoting all the values, no change: http://paste.openstack.org/show/796409/08:40
rpittaufantastic.....08:41
*** diurnalist has joined #openstack-ironic08:41
jandersI will try a double-quote for completeness but I think I tried it and it makes an backslash-escaped mess which is equally broken08:42
jandersdo you have a guess what's inserting those quotes exactly? oslo_config I suppose?08:43
iurygregoryoslo_config would be the one08:43
iurygregoryI would raise a question on #openstack-oslo to see if they have more ideas .-.08:43
jandersyeah I think this is a great recommendation08:43
jandersI will 1) re-test with double quotes 2) get some dinner and  3) drop into their channel08:44
rpittauyep, at this point it really looks like something possibly weird on oslo.config08:44
jandersthanks guys, I really appreciate your help08:44
jandersthis issue has taken a couple days away from me already :(08:45
jandersgreat to have some good pointers! :)08:45
*** brtknr has joined #openstack-ironic08:46
*** diurnalist has quit IRC08:46
gudrutis0rpittau: we stopped supporting python3.6, rigth?08:47
rpittaugudrutis0: no, we still supprot it08:48
openstackgerritAija Jaunteva proposed openstack/ironic-specs master: [WIP] Operationally efficient & fast-track provisioning  https://review.opendev.org/74072108:48
rpittauthe base os in CI is still ubuntu180408:48
rpittauand it has python3.6 by default08:48
jandersfor completeness using double-quotes just makes oslo use single quotes for the disruptive auto-quoting exercise with the same disruptive result08:48
gudrutis0interesting... when I was trying to settup devstack, it complained that I am not using supported py version (py37 or py38)08:49
rpittaugudrutis0: setting up devstack on ubuntu204 ?08:49
rpittau2004*08:49
gudrutis0no, 180408:50
gudrutis0granted, it had devstack settuped before, so I cleaned it and tried to setup again and started having issues, so now I am starting over :D08:51
iurygregoryI would try a fresh installed OS08:51
*** ociuhandu has quit IRC08:56
*** dtantsur is now known as dtantsur|brb08:58
*** diurnalist has joined #openstack-ironic09:07
openstackgerritzhufl proposed openstack/ironic master: Fix invalid assertTrue which should be assertEqual  https://review.opendev.org/74370009:10
*** diurnalist has quit IRC09:12
*** rcernin has quit IRC09:12
*** Lucas_Gray has joined #openstack-ironic09:23
*** Lucas_Gray has quit IRC09:27
*** k_mouza has joined #openstack-ironic09:29
*** Lucas_Gray has joined #openstack-ironic09:30
rpittaujanders: can you please try to run that again but implementing item_type as types.Dict(value_type=Dict()) ?09:30
janderssure, will do! :)09:30
jandersactually, I think that was the original config09:31
jandershttp://paste.openstack.org/show/796410/09:31
jandersthat's what we're after, correct?09:31
jandersI take it back, sorry, missed the last part09:32
rpittauno, I mean, you see the line with item_type=types.Dict() ? Change that to item_type=types.Dict(value_type=Dict())09:33
rpittauok, no worries :)09:33
*** ociuhandu has joined #openstack-ironic09:33
* iurygregory pinged a friend on oslo09:33
*** ociuhandu has quit IRC09:37
jandersrpittau conductor doesn't seem to like it09:41
jandershttp://paste.openstack.org/show/796412/09:41
*** ociuhandu has joined #openstack-ironic09:42
rpittaujanders: yeah, my bad09:42
rpittaujanders: change that to types.Dict()09:43
rpittauitem_type=types.Dict(value_type=types.Dict())09:43
janderstrying now09:44
jandersoslo guys hinted the same thing09:45
jandersthere has to be something in it if you both thought that :)09:45
iurygregorymagic09:45
rpittau:)09:49
*** diurnalist has joined #openstack-ironic09:58
jandersunfortunately I had no luck with item_type=types.Dict(value_type=types.Dict())10:00
*** tkajinam has quit IRC10:01
jandersI will need to look into this further tomorrow - just hit 2000hrs here and I'm planning to make it to the 0500hrs meeting tomorrow, so time to wrap up10:01
*** diurnalist has quit IRC10:03
jandersgood news is it is a different error10:03
openstackgerritJohn Garbutt proposed openstack/networking-generic-switch master: WIP: Attempt to batch up cmds  https://review.opendev.org/74328310:04
rpittaujanders: sure, hopefully some steps in the right direction :)10:05
*** yaawang has quit IRC10:06
*** yaawang has joined #openstack-ironic10:07
jandersdefinitely! :) thank you for your help iurygregory and rpittau10:09
jandershttp://paste.openstack.org/show/796417/10:09
jandersthat's where I got to10:09
jandersthe latter run might be repairable... will look tomorrow in more detail10:09
rpittaulooks better, yeah10:09
rpittaujanders: take care, talk later :)10:10
iurygregoryjanders, yw o/ good luck10:12
janderscheers, have a good one10:12
jandershopefully we'll speak in just under 9hrs or so. Apologies if I am late - please don't wait for me and leave my update for the end.10:13
*** ntt__ has joined #openstack-ironic10:17
ntt__Hi, I'm using ussuri with Centos8 and "openstack overcloud node introspect" doesn't recognize disks (local_gb = 0). It seems that the problem is related to missing megaraid drivers in initrd. I can install manually centos 8 using Disk Update Driver iso, but how can I solve using ironic? Thank you10:19
rpittauntt__: if you're using the ipa ramdisk directly from ironic-python-agent-builder, you will have to rebuild it including the correct drivers10:26
ntt__rpittau, ok. But I don't know how to include an iso file. Actually my DUD driver is an iso file10:27
ntt__I'm following this: https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/16.0/html/partner_integration/overcloud_images#initrd_modifying_the_initial_ramdisks10:27
ntt__the problem is that rpm2cpio accepts an rpm file10:28
ntt__is there a way to extract rpm from iso dud driver?10:28
rpittauntt__: probably mounting the iso locally will permit to explore the content10:29
ntt__Another solution is to add "inst.dd=location" to the cmdline of the initrd. Is there an easy way to do this?10:29
ntt__I can expose a web server containing the iso dud driver if I'm going to add inst.dd to the cmdline10:29
*** diurnalist has joined #openstack-ironic10:49
*** ricolin has quit IRC10:52
*** diurnalist has quit IRC10:54
ntt__rpittau, I've created a new initramfs extracting rpm from iso dud driver (I used kmod-megaraid_sas-07.710.50.00-1.el8_2.elrepo.x86_64.rpm) but I have local_gb=0 after node introspect. Actually there is one RAID1 volume on the server that should be used as main disk. Have you some hint? Thank you11:03
*** ntt__ has quit IRC11:12
*** uzumaki has joined #openstack-ironic11:20
*** dtantsur|brb is now known as dtantsur11:26
dtantsurfolks, a really easy patch for review please: https://review.opendev.org/#/c/743689/11:27
patchbotpatch 743689 - ironic-python-agent - Return the final RAID configuration from apply_con... - 1 patch set11:27
dtantsurblocks deploy-time software RAID11:27
uzumakilemmi check11:34
dtantsurI'd also be thankful if somebody could help me reduce the number of open bifrost patches: https://review.opendev.org/#/q/owner:dtantsur%2540protonmail.com+status:open+project:openstack/bifrost11:34
dtantsurmore of them are rather trivial11:34
* dtantsur has 31 open patch atm and should likely stop producing more11:37
uzumakihow can I help? I don't have +2 but can provide an extra set of eyes if dtantsur wants11:38
dtantsuruzumaki: an extra pair of eyes never hurt, especially on larger patches11:39
dtantsuras a bonus, you'll learn more about our code :)11:39
uzumakisounds awesome, i'll see how much of these patches I can stab11:39
dtantsuruzumaki: thanks, I don't hesitate to ask questions if you don't understand some bit11:40
uzumakiwill do, for sure11:40
*** diurnalist has joined #openstack-ironic11:40
*** diurnalist has quit IRC11:45
rpittaudtantsur: I was testing bifrost and I ended up with another completely different issue :D12:13
dtantsur\o/12:13
rpittauI hope it's just my environment that is a bit messy12:13
rpittauoh mmm ntt left :/12:15
uzumakihttps://review.opendev.org/#/c/742888/12:17
patchbotpatch 742888 - bifrost - Store inspector ramdisk logs by default - 1 patch set12:17
uzumakihttps://review.opendev.org/#/c/742854/12:17
patchbotpatch 742854 - bifrost - Make ironic logging more in line with other services. - 6 patch sets12:17
uzumakiThese look too similar to me, what's the different? dtantsur12:17
dtantsuruzumaki: one is more about ironic, the other - about ironic-inspector12:31
*** diurnalist has joined #openstack-ironic12:31
uzumakiI see, so we're doing the same thing for both guys, got it12:32
openstackgerritDmitry Tantsur proposed openstack/bifrost master: Make ironic logging more in line with other services.  https://review.opendev.org/74285412:32
*** diurnalist has quit IRC12:36
dtantsuruzumaki: updated the release note ^^12:39
dtantsurand hopefully answered all your questions, lemme know if I missed something12:39
uzumakicool, lemmi check12:41
openstackgerritRiccardo Pittau proposed openstack/bifrost master: Remove upstart templates  https://review.opendev.org/74374612:42
dtantsuroh lol, we still have them?12:44
rpittauyeah :)12:44
* dtantsur gets a torch12:45
rpittauthere's also a command that refers to upstart, but I think it's safe to keep it, it kills dnsmasq at some point12:45
dtantsuryou may like or hate systemd, but at least it more or less consolidated all major distros12:45
uzumakidtantsur, true that12:45
rpittauit fails consistenly12:45
dtantsurrpittau: I *think* you can kill that command too12:45
* uzumaki wonders what time travel cave like the one from "dark" dtantsur is going to enter with that torch12:45
rpittaudtantsur: ok, I'll remove it and send a patch, let's see how it behaves12:46
uzumakibtw, dark was such an amazing series12:46
openstackgerritRiccardo Pittau proposed openstack/bifrost master: Remove old kill dnsmasq command  https://review.opendev.org/74374912:47
rpittauuzumaki: dark IS an mazing series12:47
rpittaustill havent watched season 3 though12:47
uzumakioh I thought it ended? I need to watch season 3 as well..12:48
* iurygregory didn't like it12:49
*** rloo has joined #openstack-ironic12:51
*** weshay_ is now known as weshay|ruck12:54
*** noama has joined #openstack-ironic12:58
noamaHi12:59
noamacan we get another +2 for https://review.opendev.org/#/c/743498/12:59
patchbotpatch 743498 - ironic-inspector - Relax extra_hardware data validation by default - 2 patch sets12:59
* iurygregory looks12:59
*** diurnalist has joined #openstack-ironic13:01
*** Goneri has joined #openstack-ironic13:01
openstackgerritMerged openstack/bifrost master: Store inspector ramdisk logs by default  https://review.opendev.org/74288813:02
*** ntt__ has joined #openstack-ironic13:04
*** diurnalist has quit IRC13:06
rpittaudtantsur: wondering if we should change the 'testenv' command and the '--testenv' install option in bifrost-cli, kind of confusing :)13:07
TheJuliagood morning13:07
rpittaugood morning TheJulia :)13:07
iurygregorygood morning TheJulia =)13:08
TheJuliaJust so folks are aware, I'm going to basically take Friday off13:11
iurygregorylong weekend \o/13:11
TheJuliaI have a 2.5-3 hour errand so it just kind of made sense to rest/relax a little13:12
iurygregory++13:12
ntt__Hi, it seems the error I'm having is "version of block device name is unavailable cause: [Errno 2] No such file or directory: '/dev/disk/by-path': FileNotFoundError: [Errno 2] No such file or directory: '/dev/disk/by-path'"13:26
*** abdysn has quit IRC13:26
ntt__At this point I don't know if the problem is related to raid controller driver13:27
rpittauntt__: do you have the all ipa ramdisk logs ?13:29
ntt__rpittau, no... how can I collect ipa ramdisk logs?13:30
rpittauntt__: you could add 'logs' to the ironic-inspector collectors13:31
ntt__please, can you show how to do this? Is there a configuration file for this?13:32
rpittauyou should find the config here: https://docs.openstack.org/ironic-inspector/latest/install/index.html13:33
dtantsurmorning TheJulia13:37
dtantsurfolks, does anyone know how universal /etc/defaults/grub is?13:38
dtantsurI'd like to add an opt-in deploy step that modifies kernel parameters13:38
ntt__rpittau, It seems I have the logs in /var/log/containers/ironic-inspector/ramdisk/605d6e1c-d53a-4dae-a85e-72d18cbd8609_20200729-132903.724102.tar.gz Can I share ?13:38
rpittaudtantsur: should be very universal13:38
rpittauntt__: please share in an external paste13:39
ntt__ok... just one moment please13:39
rpittauntt__: like paste.openstack.org/13:39
rpittaudtantsur: /etc/default/grub is the default in grub: https://www.gnu.org/software/grub/manual/grub/html_node/Simple-configuration.html13:40
dtantsurokay, makes sense13:40
* dtantsur prototypes13:41
ntt__rpittau, http://paste.openstack.org/show/796427/ --> lshw13:42
ntt__http://paste.openstack.org/show/796428/ --> journal13:42
ntt__lsblk is empty e mdstat doesn't give any relevant information13:43
rpittauI see the raid controller there13:43
ntt__ok... but "openstack baremetal node show controller2" gives this: "properties             | {'local_gb': '0', 'cpus': '24', 'cpu_arch': 'x86_64', 'memory_mb': '32768', 'capabilities': 'cpu_vt:true,cpu_aes:true,cpu_hugepages:true,cpu_hugepages_1g:true,cpu_txt:true'}"13:44
ntt__and local_gb = 0. Is this a problem?13:44
ntt__how can I create a new volume13:45
ntt__?13:45
openstackgerritMerged openstack/bifrost master: bifrost_inventory: use stderr for logging  https://review.opendev.org/74286913:47
*** waleedm__ has quit IRC13:47
*** tzumainn has joined #openstack-ironic13:51
*** rloo has quit IRC13:52
rpittauntt__: you can find some info on raid here https://docs.openstack.org/ironic/latest/admin/raid.html13:55
openstackgerritMerged openstack/ironic-python-agent master: Return the final RAID configuration from apply_configuration  https://review.opendev.org/74368914:00
rpittauooook looks like we still need the killall dnsmasq command :D14:04
*** yaawang has quit IRC14:09
TheJuliaheh14:10
*** yaawang has joined #openstack-ironic14:10
TheJuliarpioso: Is there anything off the top of your head that might cause an idrac controller to take an exceptionally long time to return power state data?14:10
*** sshnaidm is now known as sshnaidm|bbl14:17
*** rloo has joined #openstack-ironic14:18
openstackgerritDmitry Tantsur proposed openstack/ironic-python-agent master: [PoC] A deploy step to update /etc/default/grub  https://review.opendev.org/74377114:24
dtantsurjust an idea ^^^14:24
openstackgerritMerged openstack/ironic-inspector master: Relax extra_hardware data validation by default  https://review.opendev.org/74349814:25
TheJuliaupdate just that or update that and update the placed grub config?14:26
* dtantsur reboots, thank you intel for your DSP drivers14:28
dtantsurTheJulia: this runs before we install bootloader.. although we don't install it for whole disk images, hmmm14:29
TheJuliaWhole disk, we try to update the nvram.. now14:29
TheJuliaand we leave it at that14:29
TheJuliaand that whole dance with software raid takes a long long time :(14:29
TheJuliabecasue of rescans mounts/unmounts14:30
openstackgerritDmitry Tantsur proposed openstack/ironic-python-agent stable/ussuri: Return the final RAID configuration from apply_configuration  https://review.opendev.org/74377314:35
openstackgerritDmitry Tantsur proposed openstack/bifrost stable/ussuri: bifrost_inventory: use stderr for logging  https://review.opendev.org/74377414:36
dtantsurTheJulia: yeah, this PoC is becoming more complex that I hoped for :(14:37
dtantsur* than14:37
TheJuliasorry :(14:37
TheJuliaI mean, if there is a single case where it can be helpful and then we expand on that as time goes on...14:38
dtantsurI'm mostly exploring the power of deploy steps14:38
dtantsurI hoped this could be a good example of how to write a more or less isolated deploy steps14:38
dtantsurbut then, looking for a root disk is quite a task of its own14:38
*** ricolin has joined #openstack-ironic14:40
openstackgerritMerged openstack/ironic-inspector master: Add support for retrieving unprocessed data  https://review.opendev.org/74350414:40
openstackgerritMerged openstack/ironic stable/train: Remove iDRAC Redfish boot device known issue  https://review.opendev.org/73212214:40
*** rcernin has joined #openstack-ironic14:41
TheJuliaYeah, I've been thinking about that in a background process in my brain the last few days because I want to try and hack on kexec this weekend14:41
TheJuliaSince largely I've been focused on bugs14:41
TheJuliaSpeaking of.... https://review.opendev.org/#/c/740360 could use another reviewer14:41
patchbotpatch 740360 - ironic - Remove locks before RPC bus is started - 3 patch sets14:41
*** Lucas_Gray has quit IRC14:41
dtantsurbugs, tons of them :(14:41
TheJulia(Likely why when upgrading nova might go "rutro, ironic node in error!"14:42
TheJulia)14:42
TheJuliawell, ironic and nova in concert at the same time14:42
TheJuliawith multiple restarts and automagic vif plugging14:42
TheJuliabecause reasons14:42
* dtantsur needs to go and bribe someone from keystone14:42
TheJulia++14:43
*** sdanni has joined #openstack-ironic14:44
dtantsurTheJulia: it's implicit in my patch, but I've never asked you directly: wdyt about eventually making bifrost authenticated by default?14:44
dtantsuri.e. make noauth_mode=false and use HTTP basic auth or keystone depending on enable_keystone?14:44
TheJuliadtantsur: I was actually thinking of asking you to make that change14:45
TheJuliaand then decided maybe ease it in14:46
dtantsurI'm doing it via a deprecation14:46
TheJuliaThe noauth use case was good to provide/force consideration of the non-openstack usecase14:46
dtantsur(and switching bifrost-cli straight away since it's new)14:46
TheJuliaIt is still kind of valid, just not a great idea and we let explicit noauth live too long14:46
TheJuliapersonally, so I'm for it14:46
dtantsurTheJulia: https://review.opendev.org/#/c/741964/22/releasenotes/notes/http-basic-40df399ea63956aa.yaml14:46
patchbotpatch 741964 - bifrost - [WIP] Support HTTP basic auth and switch bifrost-c... - 22 patch sets14:46
TheJuliano objections :)14:48
*** Lucas_Gray has joined #openstack-ironic14:49
*** rcernin_ has joined #openstack-ironic14:51
sdanniHi there! I'm currently working on provisioning an ironic node and have issue with node cleaning. The issue is that during cleaning, the node is able to pick up an IP address for pxe boot and download the agent images, but somehow loses the ability to communicate with whatever-endpoint. Does anyone has similar issue before? Thanks!14:52
*** irclogbot_0 has quit IRC14:52
TheJuliasdanni: silly question, do you have LACP enabled on your switch on your provisioning network interface?14:52
*** early` has joined #openstack-ironic14:52
TheJuliasdanni: it sounds a lot like an issue I lost some time to recently, which is why I ask14:53
*** uzumaki has quit IRC14:53
dtantsurTheJulia: we need to add it after https://docs.openstack.org/ironic/latest/admin/troubleshooting.html#dhcp-during-pxe-or-ipxe-is-inconsistent-or-unreliable14:54
*** fdegir5 has joined #openstack-ironic14:54
TheJuliadtantsur: I'll go get more coffee and add it14:55
dtantsurI'm also looking for ideas for a simple deploy step that we could use an example14:56
dtantsurI'd like to add an example hardware manager to the IPA repo (but not IPA itself)14:56
*** rcernin_ has quit IRC14:56
TheJuliadtantsur: deploy is actually super simple it just doesn't really indicate "This is the starting point and several paths exist"14:57
TheJuliafwiw14:57
dtantsurdeploy.deploy is not on the IPA side though14:57
TheJuliadidn't JayF have an example on github14:57
TheJuliayeah, its not14:57
dtantsuror did I misunderstand you?14:57
TheJuliano, i misunderstood you14:57
TheJuliadelay between the two lines14:58
*** rcernin has quit IRC14:58
*** fdegir has quit IRC14:58
*** early has quit IRC14:58
*** zer0c00l has quit IRC14:58
*** irclogbot_1 has joined #openstack-ironic14:59
*** diurnalist has joined #openstack-ironic15:01
sdanniTheJulia, hi! How to check if LACP is enabled? Sorry I have no idea about it.15:06
TheJuliasdanni: I'd look for port-channel config in the switch15:06
TheJuliatypically that is the term used15:06
sdannicool15:07
TheJuliadtantsur: have you ever, by chance, heard of a show from the US called "dragnet"15:08
TheJulia?15:08
*** hoonetorg has quit IRC15:09
JayFTheJulia: dtantsur https://opendev.org/x/ipa-example-hardware-managers/15:15
dtantsurI think we should move this to openstack/ironic-python-agent/examples if you don't mind15:17
JayFNo objection; please indicate me as the author if you can15:17
JayF(I think gerrit reads Co-Authored-By these days?)15:18
TheJuliathe reporting seems to15:18
JayFthat's my go-to interview "look at how good I am at documenting things" portfolio item lol15:18
JayFone thing to consider: I know multiple folks have done IPA managers by just taking that repo and forking and editing it15:19
JayFmoving this into examples/ means you lose the "cookiecutter" behavior, and the obvious/simple setup.cfg examples15:20
openstackgerritArne Wiebalck proposed openstack/ironic master: [doc] Set the default_{management,power}_interface for redfish  https://review.opendev.org/74378215:21
*** hoonetorg has joined #openstack-ironic15:21
jrollJayF: you could still make it a standalone package in that examples directory15:22
*** janders has quit IRC15:26
dtantsurI wanted this ^^^15:29
JayF++15:34
dtantsurnow I need to invent something that is small and looks more or less as a real deploy step (not just log "hello world")15:35
JayFwhy not some kind of hardware validation?15:35
JayFe.g. "is the machine in a good state to be deployed" as a step15:35
JayFthat's a use case mentioned internally for deploy steps15:36
TheJuliaarne_wiebalck: hey, benchmarking? ^^^ or load test?15:36
JayFthe existing examples don'tdo anything15:36
dtantsurJayF: mm, verify that we don't have less memory than expected?15:36
JayFjust describe a use case and lay out a design for you to fill in with your own ideas15:36
JayFso maybe take that same approach; be more generic15:37
dtantsurmaybe that, yeah15:37
jrollI like the memory check, that's an easy one15:37
TheJulia++15:37
*** ntt__ has quit IRC15:37
JayFif you write one that actually checks for something; e.g. memory; it should actually be shipped instead of being an example IMO15:37
TheJuliabut don't fail it on the lack of the setting being there, since it hasn't been required for a while15:37
JayFwhy not, if it provides actual value?15:37
arne_wiebalckTheJulia: hey o/15:37
* arne_wiebalck is reading the backlog ...15:38
dtantsurJayF: actually.. yes, it can be a built-in deploy step!15:38
JayFyou should still have an example15:38
dtantsurdefinitely easier to write than bootloader params...15:38
dtantsuryup15:38
JayFjust if the example does a thing, you should not make it an example :D15:38
openstackgerritJulia Kreger proposed openstack/ironic master: Detail iPXE + LACP troubleshooting information  https://review.opendev.org/74378715:40
TheJuliadtantsur: ^^15:40
TheJuliasdanni: if you have a port channel/LACP configuration, this might provide insight ^^15:41
openstackgerritMerged openstack/python-ironic-inspector-client master: Support retrieving unprocessed data  https://review.opendev.org/74351815:42
arne_wiebalckchecking if the system is in good shape should be done right before moving to available, not when the machine is deployed, no?15:43
JayFTheJulia: that's extra-screwy, because if you are chainloading into ipxe, that LACP support is useless, because the base pxe ron won't15:43
JayFthat support only provides value if flashed into a network card15:43
*** mkrai has quit IRC15:43
TheJuliaJayF: doesn't even really provide value then because as soon as the OS takes control it stops15:43
*** mkrai_ has joined #openstack-ironic15:43
JayFTheJulia: I'm thinking if you wanted lacp in the agent, and you can pxe boot into it, you just need the OS configuration to do LACP (and I've done LACP + DHCP before in an OS)15:44
JayF(in theory, I agree with the doc that it's a bad idea)15:45
TheJuliayou have to bond the interfaces together and we don't know which nor should we... and some people run public networks15:45
TheJuliajust... it would be a nightmare15:45
* TheJulia has insufficent tasty beverages to support the ramdisk doing such15:45
sdanniTheJulia, thanks!15:49
rpittaubtw, not sure if it can be useful in this case, but LCAP support in ipxe is configurable at build time15:54
JayFIf you're building your own iPXE, you probably aren't looking to Ironic docs to help you troubleshoot it ¯\_(ツ)_/¯15:59
*** lucasagomes has quit IRC16:02
rpittauprobably not :)16:03
rpittaumaybe we can mention it? it seems like it was added to bypass this kind of issue16:04
*** k_mouza has quit IRC16:09
openstackgerritDmitry Tantsur proposed openstack/ironic-python-agent master: Import example hardware managers from ipa-example-hardware-managers  https://review.opendev.org/74379416:09
openstackgerritDmitry Tantsur proposed openstack/ironic-python-agent master: Import example hardware managers from ipa-example-hardware-managers  https://review.opendev.org/74379416:11
dtantsurJayF: ^^16:11
JayFakready looking16:11
JayFbut you patch set ninja'd me, refreshing lol16:11
dtantsurI'm also going to update irrelevant-files for other jobs to add 'examples', so another update is expected16:13
openstackgerritDmitry Tantsur proposed openstack/ironic-python-agent master: Import example hardware managers from ipa-example-hardware-managers  https://review.opendev.org/74379416:14
openstackgerritDmitry Tantsur proposed openstack/ironic-python-agent master: Import example hardware managers from ipa-example-hardware-managers  https://review.opendev.org/74379416:17
openstackgerritAija Jaunteva proposed openstack/ironic master: Fix idrac-wsman RAID apply_configuration  https://review.opendev.org/74379616:20
*** ociuhandu_ has joined #openstack-ironic16:21
*** ociuhandu has quit IRC16:23
*** ociuhandu_ has quit IRC16:27
*** sshnaidm|bbl is now known as sshnaidm16:28
*** gyee has joined #openstack-ironic16:28
*** k_mouza has joined #openstack-ironic16:28
*** dougsz has quit IRC16:31
*** k_mouza has quit IRC16:33
*** ociuhandu has joined #openstack-ironic16:35
*** ociuhandu has quit IRC16:41
*** trandles has joined #openstack-ironic16:46
*** mkrai_ has quit IRC16:46
*** k_mouza has joined #openstack-ironic16:46
openstackgerritMerged openstack/ironic-python-agent-builder master: Log IPv6 configuration status for tinycore  https://review.opendev.org/74353616:46
sdanniHi TheJulia, we checked the switch and LACP is not enabled, the problem must be something else I guess.16:47
trandlesTime for my stupid question of the day: Is it possible to run ironic-inspector standalone without rabbitmq (i.e., is there something like ironic standalone's rpc_transport)?16:48
dtantsurtrandles: I'm afraid not (yet)16:48
trandlesdtantsur, thanks for the quick reply16:48
dtantsurthere was a spec approved on that, but I don't think the code has been finished16:48
* trandles goes off to start up rabbitmq ;)16:48
dtantsurtrandles: sorry, lemme clarify16:49
dtantsuryou CAN run ironic-inspector all-in-one process (API + worker) with transport_url=fake://16:49
dtantsurif you want to go serious scale and split API and worker processes, that's where you're stuck with rabbit for now16:49
dtantsurtrandles: now this ^^ is more correct16:49
trandlesAh ok16:50
*** k_mouza has quit IRC16:50
trandlesI won't need scaling for my testbed but I want to test in something close to my production use case. We have a production rabbitmq we use for all kinds of things, so it's not a problem to require it.16:51
trandlesIt's good to know that at scale we'll want rabbitmq anyway, so I'll get it running.16:52
*** Lucas_Gray has quit IRC16:54
*** k_mouza has joined #openstack-ironic16:57
*** fdegir5 is now known as fdegir16:57
*** derekh has quit IRC17:00
*** ricolin has quit IRC17:01
*** k_mouza has quit IRC17:01
*** sshnaidm is now known as sshnaidm|afk17:15
*** k_mouza has joined #openstack-ironic17:15
*** k_mouza has quit IRC17:19
*** dtantsur is now known as dtantsur|afk17:21
dtantsur|afko/17:21
openstackgerritMerged openstack/ironic master: Remove locks before RPC bus is started  https://review.opendev.org/74036017:29
openstackgerritMerged openstack/ironic master: Fix invalid assertTrue which should be assertEqual  https://review.opendev.org/74370017:29
TheJuliaoh yay, the lock removal merged17:31
* TheJulia dances17:31
TheJuliasdanni: ipv4, ipv6? it sounds like cleaning at least starts?17:32
openstackgerritJulia Kreger proposed openstack/ironic stable/ussuri: Remove locks before RPC bus is started  https://review.opendev.org/74380817:33
openstackgerritJulia Kreger proposed openstack/ironic stable/train: Remove locks before RPC bus is started  https://review.opendev.org/74380917:33
*** k_mouza has joined #openstack-ironic17:35
openstackgerritJulia Kreger proposed openstack/ironic stable/stein: Remove locks before RPC bus is started  https://review.opendev.org/74381017:37
*** k_mouza has quit IRC17:39
openstackgerritJulia Kreger proposed openstack/ironic stable/rocky: Remove locks before RPC bus is started  https://review.opendev.org/74381117:40
openstackgerritJulia Kreger proposed openstack/ironic stable/queens: Remove locks before RPC bus is started  https://review.opendev.org/74381217:40
TheJuliawheeeeeee17:40
iurygregorywheeeeeeeeeeee17:40
sdanniTheJulia, ipv4. Yes, during ipxe boot, it can download the agent at 192.168.1.2:8088, but after that the ironic-python-agent can't connect to 192.168.1.2:6385, 'network is unreachable'17:52
*** k_mouza has joined #openstack-ironic17:54
*** k_mouza has quit IRC17:58
TheJuliasdanni: what ramdisk are you using?18:05
TheJuliait sounds like your ramdisk lacks hardware support18:05
*** k_mouza has joined #openstack-ironic18:10
JayFRe those stable patches ^ is it normal for us to fix bugs just based on a RH bugzilla number without a story upstream for it?18:13
JayFI was hoping to see the details for that in the story but there wasn't one18:14
TheJuliaThat is an odd one and we're getting a bit more lax on it from a lack of spoon/bandwidth standpoint18:14
*** k_mouza has quit IRC18:14
JayFMakes sense, but being unable to see rh bugs without an account is a significant regression in transparency, although I understand not having enough time in the day18:15
iurygregoryonly people (except me voted so far), for the meeting to talk about the ironic grenade multinode, I was planning in close the doodle today .-.18:17
iurygregorytwo people*18:17
TheJuliaYeah, I tried to be super verbose int he commit message as a result18:17
TheJuliaI honestly haven't had a chance to look at upstream email all week18:18
* TheJulia sighs18:18
iurygregory=S18:19
iurygregorymaybe I should consider to let open till friday =)18:19
JayFTheJulia: yeah, it is what it is, just raising a hrm :)18:19
JayFTheJulia: I did appreciate the detailed commit message, just hard to evaluate if it's worth a downstream port without a better idea of the impact18:19
TheJuliaJayF: sorry :(18:19
JayFTheJulia: you don't have the time, you don't have the time :) I don't have the time to volunteer to help, so I can't complain that hard :D18:20
*** k_mouza has joined #openstack-ironic18:31
openstackgerritRiccardo Pittau proposed openstack/ironic master: Enforce autospec in test_iscsi_deploy  https://review.opendev.org/74382018:37
*** k_mouza has quit IRC18:39
sdanniTheJulia, I download the deploy ramdisk built from https://tarballs.openstack.org/ironic-python-agent/coreos/files/coreos_production_pxe_image-oem.cpio.gz18:45
TheJuliasdanni: wow, that is very out of date. What version of ironic are you trying to use?18:47
TheJuliaAlso, we don't advise CoreOS use anymore18:47
TheJuliaAnd CoreOS lacked a lot of drivers for some hardware.18:47
sdanniTheJulia: we are now using ussuri. Any suggestions where to build or download a deploy ramdisk image?18:50
TheJuliasdanni: we have a handy tool for that18:56
TheJuliatake a look at the ironic-python-agent-image-builder18:56
TheJuliaI don't have the link handy, trying to get my notes together for my weekly meeting18:56
sdanniTheJulia, found it. Thanks a lot! I'm going to build to ramdisk to see if it works.18:57
*** janders has joined #openstack-ironic19:01
*** zer0c00l has joined #openstack-ironic19:05
JayFsdanni: TheJulia: https://tarballs.opendev.org/openstack/ironic-python-agent/dib/ appears to have reccently-made prepublished IPA DIB images19:06
JayFincluding stable/ussuri version19:06
*** zer0c00l has quit IRC19:06
sdannicool!19:06
JayFBuilding your own is trivial, and allows you to customize it somewhat. I do suggest going that route long-term if you're using IPA+Ironic.19:07
*** zer0c00l has joined #openstack-ironic19:13
sdanniThank you for the advice, JayF!19:15
*** k_mouza has joined #openstack-ironic19:22
*** k_mouza has quit IRC19:27
*** k_mouza has joined #openstack-ironic19:33
*** cp- has quit IRC19:33
*** k_mouza has quit IRC19:37
*** k_mouza has joined #openstack-ironic19:42
*** k_mouza has quit IRC19:47
*** k_mouza has joined #openstack-ironic20:05
*** yaawang has quit IRC20:07
*** yaawang has joined #openstack-ironic20:07
*** k_mouza has quit IRC20:09
*** noama has quit IRC20:12
*** waleedm__ has joined #openstack-ironic20:28
*** ociuhandu has joined #openstack-ironic20:39
*** waleedm_ has joined #openstack-ironic20:42
*** waleedm_ has quit IRC20:42
*** ociuhandu has quit IRC20:43
*** waleedm__ has quit IRC20:45
openstackgerritJulia Kreger proposed openstack/ironic master: Add cirros-specific FAQ item for troubleshooting  https://review.opendev.org/74270321:01
rpiosoTheJulia: I'm not entirely clear on what you asked about the iDRAC taking an exceptionally long time to return the system's power state. Does this mean you made a request and it took a long time for the response to be returned? Did it seem to hold onto the request for some time before responding?21:29
openstackgerritMerged openstack/bifrost master: Remove upstart templates  https://review.opendev.org/74374621:37
*** sdanni has quit IRC21:42
NobodyCamGood afternoon ironic’ers21:50
TheJuliarpioso: I was looking at a downstream bug and I was seeing requests out to the idrac just taking an increadibly long time for configuration jobs... then it hit me.. the person who filed the bug likely has memory testing enabled or something21:58
TheJuliagood morning NobodyCam  :)21:58
rpiosoTheJulia: Ahhhh ... That makes sense. Following a reboot, it takes some time before the iDRAC is ready to report stuffs. It takes an inventory. Memory testing might prolong the amount of time before it is able to do that.22:02
rpiosoNobodyCam: o/22:03
TheJuliaYeah, and I know depending on the job it can make the time window variable so22:03
* TheJulia told them to check their memory testing setting22:03
rpiosoTheJulia: Lots of things can, including how decked out the system is. The more things to inventory, ...22:04
NobodyCamGood morning TheJulia and rpioso22:04
JayFNobodyCam: good day to you sir22:10
TheJuliarpioso: yup :\22:18
NobodyCamSir where’s my dad22:25
NobodyCam;p22:25
NobodyCamHowdy JayF22:25
JayFlol22:26
TheJuliaNobodyCam: Uhh, do you mean that badcub fellow? ;)22:30
NobodyCam😂22:31
*** rcernin_ has joined #openstack-ironic22:35
*** trandles has left #openstack-ironic22:41
*** rcernin_ has quit IRC22:48
*** rcernin has joined #openstack-ironic22:48
*** tkajinam has joined #openstack-ironic22:53
*** ociuhandu has joined #openstack-ironic23:01
openstackgerritVerification of a change to openstack/ironic failed: Convert root controller to plain controller  https://review.opendev.org/74106423:01
openstackgerritVerification of a change to openstack/ironic failed: Convert v1 controller to plain, return JSON  https://review.opendev.org/74106523:01
*** ociuhandu has quit IRC23:06
*** diurnalist has quit IRC23:59

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!