Thursday, 2020-07-30

*** diurnalist has joined #openstack-ironic00:00
*** diurnalist has quit IRC00:09
*** yaawang has quit IRC00:14
*** yaawang has joined #openstack-ironic00:14
*** diurnalist has joined #openstack-ironic00:36
*** diurnalist has quit IRC00:59
openstackgerritSteve Baker proposed openstack/ironic master: Convert root controller to plain controller  https://review.opendev.org/74106401:02
openstackgerritSteve Baker proposed openstack/ironic master: Convert v1 controller to plain, return JSON  https://review.opendev.org/74106501:02
openstackgerritSteve Baker proposed openstack/ironic master: Break out collection functions for json usage  https://review.opendev.org/74367601:02
openstackgerritSteve Baker proposed openstack/ironic master: New argument validate decorator  https://review.opendev.org/74279501:02
openstackgerritSteve Baker proposed openstack/ironic master: Convert drivers endpoint to plain JSON  https://review.opendev.org/74186501:02
openstackgerritSteve Baker proposed openstack/ironic master: Add status_code argument to @method.expose  https://review.opendev.org/74367701:02
openstackgerritSteve Baker proposed openstack/ironic master: Add @method.body decorator  https://review.opendev.org/74367801:02
openstackgerritSteve Baker proposed openstack/ironic master: Convert allocation endpoint to plain JSON  https://review.opendev.org/74367901:02
*** diurnalist has joined #openstack-ironic01:11
*** ricolin has joined #openstack-ironic01:12
*** diurnalist has quit IRC01:16
*** k_mouza has joined #openstack-ironic01:19
*** janders has quit IRC01:20
*** k_mouza has quit IRC01:24
*** yaawang has quit IRC01:31
*** yaawang has joined #openstack-ironic01:32
*** diurnalist has joined #openstack-ironic01:43
*** k_mouza has joined #openstack-ironic01:49
*** diurnalist has quit IRC01:50
*** diurnalist has joined #openstack-ironic01:51
*** k_mouza has quit IRC01:53
*** rh-jlabarre has quit IRC01:55
*** janders has joined #openstack-ironic02:02
*** diurnalist has quit IRC02:09
*** diurnalist has joined #openstack-ironic02:17
*** diurnalist has quit IRC02:25
*** rloo has quit IRC02:25
*** diurnalist has joined #openstack-ironic02:27
*** gyee has quit IRC02:34
*** diurnalist has quit IRC02:37
*** yaawang has quit IRC02:41
*** yaawang has joined #openstack-ironic02:42
*** k_mouza has joined #openstack-ironic02:42
*** tzumainn has quit IRC02:42
*** Goneri has quit IRC02:46
*** k_mouza has quit IRC02:47
*** SuiongNg|ITRI_TW has joined #openstack-ironic03:00
*** yaawang has quit IRC03:14
*** yaawang has joined #openstack-ironic03:15
*** k_mouza has joined #openstack-ironic03:20
*** diurnalist has joined #openstack-ironic03:23
*** k_mouza has quit IRC03:25
*** diurnalist has quit IRC03:29
*** yaawang has quit IRC03:47
*** yaawang has joined #openstack-ironic03:48
*** diurnalist has joined #openstack-ironic03:59
*** sziviani has quit IRC04:04
*** ociuhandu has joined #openstack-ironic04:22
*** ociuhandu has quit IRC04:26
*** diurnalist has quit IRC04:32
openstackgerritparesh sao proposed openstack/ironic master: [WIP] OOB one button secure erase for iLO5 based HPE Proliant servers.  https://review.opendev.org/74386805:12
*** diurnalist has joined #openstack-ironic05:19
*** yaawang has quit IRC05:23
*** yaawang has joined #openstack-ironic05:24
*** diurnalist has quit IRC05:24
*** diurnalist has joined #openstack-ironic06:10
openstackgerritFelix Maurer proposed openstack/ironic master: Use TLS for json_rpc when configured  https://review.opendev.org/74391206:11
*** diurnalist has quit IRC06:15
*** akahat is now known as akahat|PTO06:30
*** ociuhandu has joined #openstack-ironic06:30
*** yaawang has quit IRC06:32
*** yaawang has joined #openstack-ironic06:34
*** ociuhandu has quit IRC06:35
*** ricolin has quit IRC06:43
*** ricolin has joined #openstack-ironic06:43
*** ricolin has quit IRC06:44
*** uzumaki has joined #openstack-ironic06:45
uzumakimorning ironic o/06:45
jandersgood morning uzumaki06:46
uzumakiyou's it going janders ? o/06:46
uzumakihow's*06:46
jandersdoing well, thank you :)06:46
jandershow are you?06:46
*** ricolin has joined #openstack-ironic06:46
uzumakiI'm fine, thanks :) what's up?06:46
janderstackling some interesting MultiOpt + list-of-dicts issues06:47
jandersI've been fighting this for the last couple days06:47
uzumakiOhh my06:47
jandersI got some pointers yesterday from the guys here as well as oslo guys06:47
jandersre-testing now06:47
janderswas working on something else earlier in the day06:47
uzumakiI like how you say "interesting issues" meaning you perception of the problem is not frustration, you're focused ;)06:47
jandersthere is an element of frustration but as long as there are knowledgeable people happy to give a hand, it's more of a positive challenge06:48
uzumakithat's very true indeed!06:48
jandersthat's the great thing about the community we're in06:48
uzumakiikr? It's only been a while for me here, but I love coming here, just to help out, talk to people and stuff, I love this community06:49
*** diurnalist has joined #openstack-ironic07:01
iurygregorygood morning uzumaki janders and Ironic!07:01
uzumakimorning iurygregory ! how's it going?07:02
jandersgood morning iurygregory07:02
iurygregoryuzumaki, doing good =)07:02
uzumakiahan? CI no longer giving you a hard time?07:02
iurygregorywell I try to not include CI when people ask how's it going =)07:03
uzumakihaha! =D07:04
uzumakiI was naive enough to think that Zuul had started behaving again x(07:04
uzumakiwell, all things considered, you must be doing pretty well in anticipation of the long weekend ;)07:05
iurygregorywell zuul has being a good boy and we gave a cookie07:05
uzumakiiurygregory, i thought I was the cookie ambassador =O how can he get a cookie without it getting through me?07:05
openstackgerritMerged openstack/ironic master: Trivial: fix a minor issue in standalone docs and improve formatting  https://review.opendev.org/74369207:05
iurygregoryuzumaki, everybody can give cookies to zuul07:06
iurygregoryhow can we get +1 from it?!07:06
uzumakiOh, I see!07:06
*** diurnalist has quit IRC07:06
iurygregoryhehehehe07:06
uzumakixD07:07
uzumakiany other special holidays coming up in cz? iurygregory07:07
iurygregoryuzumaki, only on Sep07:07
uzumakiindependence day?07:08
iurygregorySt. Wenceslas Day07:08
iurygregoryIndependence is on Oct07:08
uzumakiInteresting.. I'd been meaning to ask, if you sat on the porch with a basket of cookies/candies on the children's day handing it out to kids? :D07:09
uzumakiuncle iurygregory07:09
iurygregoryLMAO07:09
rpittaugood morning ironic! o/07:09
iurygregorygood morning rpittau o/07:10
uzumakimorning rpittau o/07:10
rpittauhey iurygregory :)07:10
rpittauhey uzumaki :)07:10
uzumakiUncle iurygregory was telling us something, go on iurygregory xD07:10
iurygregoryI have the feeling today is Friday lol07:10
uzumakiiurygregory, c'mon now, stop rubbing our noses into your long weekend xD07:10
iurygregorylol07:11
iurygregoryI still need to work tomorrow =)07:11
uzumakiwooot? :O07:11
rpittauisn't it Friday?!?!?07:11
rpittau:/07:11
iurygregory=(07:12
* iurygregory blames uzumaki 07:12
* uzumaki is wondering what he did..07:12
iurygregoryonly because it's not friday yet =)07:12
* iurygregory grabs more coffee07:12
* uzumaki thinks iurygregory needs a LOT more coffee xD07:13
* rpittau looks around and doesn't know where to start07:13
* uzumaki holds a coffee to rpittau here you go, start here07:14
openstackgerritVerification of a change to openstack/ironic failed: Fix idrac-wsman RAID apply_configuration  https://review.opendev.org/74379607:14
rpittauuzumaki: that sounds like a great start :)07:14
uzumakiit sure does!07:14
*** yaawang has quit IRC07:15
*** yaawang has joined #openstack-ironic07:18
*** yaawang has quit IRC07:26
iurygregorylol07:26
*** yaawang has joined #openstack-ironic07:26
* janders noticed that coffee time in Europe is beer time in Australia07:27
jandershmm07:27
*** diurnalist has joined #openstack-ironic07:27
jandersdoesn't seem to work the other way round though07:27
jandersoh well07:28
jandersiurygregory rpittau I've done more testing on the oslo/multiopt issue07:28
jandershttp://paste.openstack.org/show/796449/07:28
iurygregoryWell in Brno Beer time is 24h I would say lol07:29
jandersitem_type=types.Dict(value_type=types.Dict()) seems to turn the old problem into another problem but the nested dicts are still unparsable AFAIK07:29
iurygregorydamm =(07:30
jandersiurygregory haha then it does work the other way around, too, at least when it comes to (aus)coffee => (cz)beer07:30
iurygregoryhehehe07:30
jandersbefore I call it quits and start re-thinking the data structure for the config, can you have a look at my testing and cross-check if I come to the right conclusions?07:31
iurygregoryso basically oslo is crazy and it add the double quotes to make things worst for us right?07:31
jandersyeah looks like it07:31
* janders is digging out an example07:31
janders'{"shred_random_overwrite_iterations"'07:31
jandersthat's a sample key07:32
*** rcernin has quit IRC07:32
iurygregoryOGW07:32
janders'{"shred_random_overwrite_iterations"': '"1"'07:32
jandersand key:value pair07:32
* iurygregory blames oslo07:32
*** diurnalist has quit IRC07:32
jandersI don't think we can work with this07:33
uzumakiiurygregory, what's OGW?07:33
janders+107:33
iurygregoryOh God Why07:33
janders:)07:33
uzumakiiurygregory, LOL xD07:33
iurygregory=)07:33
jandershaha! :) noted07:34
iurygregoryʕノ•ᴥ•ʔノ ︵ ┻━┻07:34
iurygregorydamm the copy didn't went perfect07:34
jandersa friend of mine used to say, while asked this question in regards to systemd issues07:34
iurygregoryʕノ•ᴥ•ʔノ ︵ ┻━┻07:34
janders"but why?"07:34
uzumakiwell, it does show a bear throwing a table over07:35
janders"because f*** you that's why"07:35
rpittaulooks like someone with giant ears07:35
uzumakijanders, LOL xD07:35
iurygregoryjanders, that totally makes sense07:35
jandershaha that's pretty good ASCII art iurygregory07:35
iurygregoryhttp://asciimoji.com/ ++07:35
uzumakiI'd say it's unicode07:35
uzumakiwhat appear to be the arms, are a japanese letter, if i'm not mistaken, which would make this unicode07:36
*** dougsz has joined #openstack-ironic07:38
rpittaujanders: just out of curiosity, have you tried passing the values as comma-separated instead of an actual dictionary ?07:38
jandersrpittau I was thinking exactly this07:38
jandersI think if there's only [A-Z][a-z][0-9] and [,] in the config, oslo won't freak07:39
rpittauyeah07:39
rpittaumy thought exactly looking at the parser07:39
jandersI think braces and quotes make it freak out07:39
*** ociuhandu has joined #openstack-ironic07:39
rpittaushould also make users life a bit easier :)07:39
janders+107:40
jandersI will write something up and test it07:40
rpittauorrr..... crazy thought, maybe you can pass a json file for each option ?07:40
jandersinclude steps.d/*.json style?07:41
rpittausomething like that, yeah07:41
iurygregoryjson file would make things easier I would say07:41
rpittauwell, let's try with the simple one first :D07:41
jandersyeah that wouldn't go through oslo right?07:41
jandersfile name goes through oslo07:41
janderscontent does not07:41
iurygregorysince oslo wouldn't touch the file lol07:41
rpittauyeah07:41
iurygregoryexactly07:42
rpittauyou pass teh file, we read the file and extract the values07:42
iurygregoryand we take over the world07:43
* rpittau just compared itself to a json parser07:43
jandersthat's right! :)07:43
jandersOSI summit lightning talk07:43
rpittau:)07:43
jandersI fought oslo... and I won07:43
iurygregory++07:44
SuiongNg|ITRI_TWHi, I have a question about rescue mode and agent token.07:44
SuiongNg|ITRI_TWIt appears that I cannot rescue the node more than on time.07:44
SuiongNg|ITRI_TWIf I rescue a node for the second times, the conductor would refuse to accept the heartbeat because07:44
SuiongNg|ITRI_TWthe agent token has already been generated in the first rescue attempt.07:44
SuiongNg|ITRI_TWThe workflow would look like this : rescue (success) > unrescue (success) > rescue (fail)07:44
SuiongNg|ITRI_TWIs this normal ?07:44
rpittauSuiongNg|ITRI_TW: interesting, do you have maybe the logs of the ipa ramdisk and ironic conductor?07:45
iurygregorycan you share some logs ?07:45
rpittauSuiongNg|ITRI_TW: also which version are you on?07:45
SuiongNg|ITRI_TWironic-python-agent 6.2.0, the conductor is 15.0.0 (ubuntu07:48
SuiongNg|ITRI_TW"message": "The agent token has already been retrieved. IPA may not operate as intended and the deployment may fail depending on settings in the ironic deployment."07:50
*** diurnalist has joined #openstack-ironic07:53
SuiongNg|ITRI_TW2020-07-30 07:50:29.096 2007618 INFO eventlet.wsgi.server [req-3527099a-a551-42fa-91c4-18667a69a917 - - - - -] ::ffff:127.0.0.1 "POST / HTTP/1.1" status: 200  len: 337 time: 0.022988307:53
SuiongNg|ITRI_TW2020-07-30 07:53:03.908 2007618 WARNING ironic.conductor.manager [req-993a6201-a569-4a70-a820-f4afa26bddbc - - - - -] An agent token generation request is being refused as one is already present for node 6a838a47-6683-477b-b036-699e34f6eedd07:53
SuiongNg|ITRI_TW2020-07-30 07:53:03.911 2007618 INFO eventlet.wsgi.server [req-993a6201-a569-4a70-a820-f4afa26bddbc - - - - -] ::ffff:127.0.0.1 "POST / HTTP/1.1" status: 200  len: 2911 time: 0.029104507:53
SuiongNg|ITRI_TW2020-07-30 07:53:04.006 2007618 ERROR ironic.conductor.manager [req-3c7e2a47-af88-4f7e-936c-a1f541731dff - - - - -] Suspicious activity detected for node 6a838a47-6683-477b-b036-699e34f6eedd when attempting to heartbeat. Heartbeat request has been rejected as the version of ironic-python-agent indicated in the heartbeat operation should support agent token functionality.07:53
rpittauSuiongNg|ITRI_TW: please don't post long logs here, use an external service07:54
*** diurnalist has quit IRC07:58
SuiongNg|ITRI_TWhttp://paste.openstack.org/show/796451/07:58
jandersiurygregory rpittau victory http://paste.openstack.org/show/796452/08:01
jandersit's very interesttting how it works08:01
rpittaujanders: nice :)08:02
jandersthis: clean_step_add_params=erase_devices.shred_random_overwrite_iterations:1,erase_devices.shred_final_overwrite_with_zeros:True,erase_devices.agent_continue_if_ata_erase_failed:False08:02
jandersautomagically turns into this with oslo: {'erase_devices.shred_random_overwrite_iterations': '1', 'erase_devices.shred_final_overwrite_with_zeros': 'True', 'erase_devices.agent_continue_if_ata_erase_failed': 'False'}08:02
rpittauyeah, I was expecting that to work with : instead of .08:02
iurygregoryjanders, \o/08:03
jandersI suppose because MultiOpt item_type is dict, it handles the dicts itself and doesn't like being fed dicts08:03
jandersin a way what I was doing was feeding chicken with chicken08:03
janderspeople do it but please dont08:04
rpittaulol08:04
jandersI wonder if I specified item_type as string or list and fed it dicts whether that would work too08:04
jandersI guess there is one way to find out...08:04
iurygregoryyeah08:08
*** lucasagomes has joined #openstack-ironic08:08
*** Lucas_Gray has joined #openstack-ironic08:11
jandersit also doesn't inject any quotes08:12
jandersbut the resulting structure is harder to work with08:12
jandersas it is a list of strings not dicts08:12
jandersdict-like strings I should say but they are missing outer braces, so would need pre-processing08:12
jandersthe comma separated key:value pairs which are then used by oslo to build the dict seems to be the winning combination08:13
jandersI suppose it's worth letting openstack-oslo guys know we solved this right?08:13
jandersthis way next time someone asks they have a solution ready08:14
rpittauSuiongNg|ITRI_TW: I believe you will have to try and update your ironic version to 15.1.008:14
rpittauSuiongNg|ITRI_TW: looks like a bug that was fixed in that version08:14
rpittaujanders: yep, that would be nice :)08:14
SuiongNg|ITRI_TWok, thank you very much. I will update my ironic installation.08:15
*** Lucas_Gray has quit IRC08:21
openstackgerritAija Jaunteva proposed openstack/ironic master: Fix idrac-wsman RAID apply_configuration  https://review.opendev.org/74379608:22
*** Lucas_Gray has joined #openstack-ironic08:27
*** rcernin has joined #openstack-ironic08:28
*** rcernin has quit IRC08:34
*** derekh has joined #openstack-ironic08:39
*** diurnalist has joined #openstack-ironic08:44
*** diurnalist has quit IRC08:48
*** diurnalist has joined #openstack-ironic08:56
*** yaawang has quit IRC08:58
*** yaawang has joined #openstack-ironic08:59
*** diurnalist has quit IRC09:01
*** dtantsur|afk is now known as dtantsur09:03
dtantsurmorning ironic09:03
dtantsurjanders: clean_step_add_params is a cool idea, but I'd suggest delaying it until you sort out the priorities09:06
dtantsurjanders: also, dicts in oslo.config don't look like python dicts09:06
dtantsurtheir format is key1:value1,key2:value209:06
dtantsurquotes are not supported by default, so to be able to do key1:"value1,still value1" you need to opt-in09:07
dtantsurmmm, I see you've discovered it later in the scrollback09:08
dtantsurarne_wiebalck: thinking about the apply_configuration deploy step.. what will happen if we call create_configuration on the already existing configuration?09:09
openstackgerritDmitry Tantsur proposed openstack/ironic-tempest-plugin master: POC deploy-time RAID  https://review.opendev.org/74122709:11
*** diurnalist has joined #openstack-ironic09:12
dtantsurarne_wiebalck: the background of my question: should we default delete_existing to True for software RAID?09:14
*** diurnalist has quit IRC09:17
openstackgerritDmitry Tantsur proposed openstack/ironic master: Add cirros-specific FAQ item for troubleshooting  https://review.opendev.org/74270309:17
*** sshnaidm|afk is now known as sshnaidm09:20
jandersdtantsur thank you for your insights09:25
jandersso - I "split" this work and get priorities working first, correct?09:25
dtantsuryep09:25
janderscool!09:25
jandersand just confirming what we figured out earlier - the original change is abandoned so I submit the work for review as a new change, correct?09:26
dtantsurjanders: correct09:26
dtantsurif you needed just a few fixes, we could restore the old change. but the current RFE is quite different09:27
jandersACK09:27
dtantsuriurygregory: what's the state https://review.opendev.org/#/c/741886/ and the tempest plugin patch? do they work as intended?09:27
patchbotpatch 741886 - ironic - Add tempest default_boot_option and altflavor - 5 patch sets09:27
dtantsurrpioso, ajya, hi! we have a person using metal3 and hitting http://paste.openstack.org/raw/796454/ any ideas?09:30
iurygregorydtantsur, hey09:31
iurygregorylooking at the logs they seem to be testing with both images09:32
iurygregoryso I think it's working09:32
dtantsurokay, so ready for review?09:32
iurygregoryyup09:32
rpittauiurygregory: I was looking at that, I think there's something missing on L297 in devstack/lib/ironic :)09:34
rpittauL2997 sorry09:34
iurygregoryrpittau, ack will look in a few min09:35
iurygregorymeeting09:35
*** k_mouza has joined #openstack-ironic09:36
ajyadtantsur: haven't seen this. Is there anywhere response logged? Smth that starts with "Received response from". Otherwise it looks that there was no response and can't decode what's not received (hence 0-1 position)09:39
openstackgerritDmitry Tantsur proposed openstack/ironic master: Log the traceback of unexpected errors when verifying power credentials  https://review.opendev.org/74394109:39
dtantsura trivial improvement ^^^09:39
dtantsurajya: I don't see a response. and we see the same problem with the redfish driver.09:39
openstackgerritRiccardo Pittau proposed openstack/ironic-lib master: Update version of hacking  https://review.opendev.org/74394209:44
*** ntt__ has joined #openstack-ironic09:47
ajyadtantsur: what state is the server? That GetRemoteServicesAPIStatus is checking if idrac is ready and it looks like it can't get event that. If server looks OK, could try to reset idrac09:48
dtantsurajya: I can only relay your request to the person09:48
dtantsurbut I suspect they've somehow put wrong credentials..09:48
*** dougsz has quit IRC09:49
ajyadtantsur: I think then there is error saying that, but could be. There are cases when server is in e.g. BIOS setup, then iDRAC is down, though in those cases it times out.09:52
openstackgerritvinay kumar muddu proposed openstack/ironic master: Allow HttpImageService to accept custom certificate  https://review.opendev.org/74293609:53
*** diurnalist has joined #openstack-ironic10:03
*** mkrai has joined #openstack-ironic10:03
*** dougsz has joined #openstack-ironic10:04
openstackgerritMerged openstack/ironic master: AgentRAID: Account for empty results in post-configuration checks  https://review.opendev.org/74369010:06
*** diurnalist has quit IRC10:08
*** k_mouza has quit IRC10:13
janderswrapping up for the night, thank you for all your help, see you tomorrow10:16
jandershave a good one everyone10:16
*** ociuhandu_ has joined #openstack-ironic10:17
*** ociuhandu has quit IRC10:19
*** ociuhandu_ has quit IRC10:21
*** ociuhandu has joined #openstack-ironic10:22
*** dtantsur is now known as dtantsur|brb10:23
*** ociuhandu has quit IRC10:27
*** k_mouza has joined #openstack-ironic10:35
*** rcernin has joined #openstack-ironic10:37
openstackgerritMerged openstack/ironic stable/train: Remove locks before RPC bus is started  https://review.opendev.org/74380910:37
*** uzumaki has quit IRC10:50
openstackgerritFelix Maurer proposed openstack/ironic master: Use TLS for json_rpc when configured  https://review.opendev.org/74391210:53
*** rcernin has quit IRC10:54
*** diurnalist has joined #openstack-ironic10:55
*** diurnalist has quit IRC11:00
*** ociuhandu has joined #openstack-ironic11:00
openstackgerritMerged openstack/ironic master: Reset power state upon adoption failure  https://review.opendev.org/74332111:21
openstackgerritMerged openstack/ironic master: Add cirros-specific FAQ item for troubleshooting  https://review.opendev.org/74270311:21
*** sshnaidm_ has joined #openstack-ironic11:27
*** sshnaidm has quit IRC11:27
*** sshnaidm_ is now known as sshnaidm11:30
*** mkrai has quit IRC11:32
*** mkrai_ has joined #openstack-ironic11:32
*** mkrai_ has quit IRC11:38
*** uzumaki has joined #openstack-ironic11:40
*** ociuhandu has quit IRC11:44
*** ociuhandu has joined #openstack-ironic11:57
*** rh-jelabarre has joined #openstack-ironic12:01
*** ociuhandu_ has joined #openstack-ironic12:15
*** ociuhandu has quit IRC12:18
*** derekh has quit IRC12:23
*** k_mouza has quit IRC12:24
*** k_mouza has joined #openstack-ironic12:39
*** k_mouza has quit IRC12:43
*** Wryhder has joined #openstack-ironic12:44
*** Lucas_Gray has quit IRC12:44
*** Wryhder is now known as Lucas_Gray12:44
*** tkajinam has quit IRC12:47
openstackgerritRiccardo Pittau proposed openstack/sushy-tools master: Update version of hacking  https://review.opendev.org/74396212:49
openstackgerritRiccardo Pittau proposed openstack/sushy-cli master: Update version of hacking  https://review.opendev.org/74396312:50
*** yaawang has quit IRC12:51
*** rloo has joined #openstack-ironic12:52
*** yaawang has joined #openstack-ironic12:53
openstackgerritFelix Maurer proposed openstack/ironic master: Use TLS for json_rpc when configured  https://review.opendev.org/74391212:53
*** dtantsur|brb is now known as dtantsur12:53
dtantsurdeploy-time RAID working \o/ https://zuul.opendev.org/t/openstack/build/96f015c91b9b4d929a0f6ab996309486/log/controller/logs/screen-ir-cond.txt?severity=2#688212:54
dtantsurI only need to understand if it's a good idea to default delete_existing to True12:55
*** diurnalist has joined #openstack-ironic12:57
openstackgerritIury Gregory Melo Ferreira proposed openstack/ironic master: Add tempest default_boot_option and altflavor  https://review.opendev.org/74188612:59
*** diurnalist has quit IRC13:06
*** sshnaidm is now known as sshnaidm|mtg13:08
*** derekh has joined #openstack-ironic13:11
*** k_mouza has joined #openstack-ironic13:16
*** k_mouza has quit IRC13:20
TheJuliagood morning13:20
rpittaugood morning TheJulia :)13:21
TheJuliadtantsur: a pre-coffee response to 743606, please take a quick look because if I do what you want we'll have to likely double catch typeerror to try and just log the original exception.13:26
* TheJulia goes and gets coffee13:26
*** k_mouza has joined #openstack-ironic13:26
dtantsurmorning TheJulia13:27
TheJuliaactually maybe we wouldn't need to double catch13:28
TheJuliathe original exception is what generates the typeerror13:28
iurygregorygood morning TheJulia =)13:28
TheJuliawhen it tries to go through the logging code, which is why I did the explicit wrap of str()13:29
TheJuliabecause otherwise we would risk the exact same thing again most likely13:29
* TheJulia gets coffee13:29
iurygregoryping reminder TheJulia should have breakfast and do some slides13:29
*** k_mouza has quit IRC13:30
TheJuliagaaaah13:32
iurygregorysorry for the reminder13:34
TheJuliaits okay13:34
iurygregorymaybe wasn't the best time =X13:35
dtantsurTheJulia: %s is strictly equivalent to calling str()13:37
dtantsurthere is never a case for using them together13:37
dtantsur(it's a nit though)13:37
TheJuliadtantsur: I believe it is up to the actual driver writing the logs, and in the case possible to hit it explodes13:38
*** diurnalist has joined #openstack-ironic13:38
dtantsurnonetheless, it's about moving the str() call around within the same try..catch13:38
* dtantsur does not understand this patch in the end13:40
TheJuliaYeah, I'm not sure what is actually causing the original exception13:40
TheJuliaI'm just 95% sure I know where it is occuring at13:40
*** k_mouza has joined #openstack-ironic13:41
TheJuliaThe whole thing really makes no sense, but maybe I should just checkout some supermicro gear out of one of the labs and see how many times I can get it to crash on inspection13:41
TheJuliaI bet it is something in or about the underlying config thoug13:41
TheJuliatough13:42
*** k_mouza has quit IRC13:45
*** tzumainn has joined #openstack-ironic13:47
*** sdanni has joined #openstack-ironic13:49
*** k_mouza has joined #openstack-ironic13:50
*** Goneri has joined #openstack-ironic13:52
*** k_mouza has quit IRC13:54
*** k_mouza has joined #openstack-ironic13:59
openstackgerritDmitry Tantsur proposed openstack/ironic-tempest-plugin master: [WIP] Test software RAID in deploy-time on Victoria and newer  https://review.opendev.org/74122714:03
*** k_mouza has quit IRC14:04
*** diurnalist has quit IRC14:05
*** k_mouza has joined #openstack-ironic14:10
*** k_mouza has quit IRC14:14
*** k_mouza has joined #openstack-ironic14:15
*** k_mouza has quit IRC14:19
openstackgerritFelix Maurer proposed openstack/ironic master: Use TLS for json_rpc when configured  https://review.opendev.org/74391214:20
*** k_mouza has joined #openstack-ironic14:22
*** k_mouza has quit IRC14:27
*** diurnalist has joined #openstack-ironic14:27
openstackgerritDmitry Tantsur proposed openstack/bifrost master: Install openstacksdk from pypi by default  https://review.opendev.org/74398914:29
sdanniHi TheJulia, I downloaded the ussuri ramdisk from https://tarballs.opendev.org/openstack/ironic-python-agent/dib/, directly uploaded the *.kernel and *.initramfs files as images, and after pxe boot, it failed to start all dhcp interfaces and Network Manager Online. Have you faced the similar issue?14:30
openstackgerritMerged openstack/ironic master: Fix idrac-wsman RAID apply_configuration  https://review.opendev.org/74379614:31
*** diurnalist has quit IRC14:31
JayFsdanni: I sorta renew what was said yesterday; it sounds like you're missing a driver in the ramdisk. Do you know if any OS of that version (CentOS 7 or 8) boots and works on that machine? I know RHEL/CentOS 8 removed support for some pretty common hardware (older dells).14:35
JayFsdanni: getting that working at this point is basically just troubleshooting the OS on the ramdisk; there are ways to get access .. one is to build with the DIB dynamic-login element and providing login credentials, then using them on the console (https://docs.openstack.org/diskimage-builder/latest/elements/dynamic-login/README.html)p14:36
TheJuliasdanni: exactly what JayF said. Basically we see these sorts of issues when the base OS has no idea what to do about networking because of a lack of drivers or understanding how to bring the interfaces up. By default they do attempt to bring up all interfaces found, fwiw.14:40
sdanniJayF, TheJulia: CentOS 8 worked on that machine several days before. And now it doesn't work anymore. After updating the ramdisk, I don't see yesterday's issue anymore.14:41
sdanniYeah, a lack of driver makes sense for me.14:42
JayFThe nice thing is, you should be able to be pretty confident the images you downloaded worked because they're used in CI/integration tests -- I'm sorry there's not much more to give you, but it's gotta be something specific to how the image is booting on the server. Checking logs/etc on the server is the path to go.14:42
JayFAnother helpful element to add in for troubleshooting is https://docs.openstack.org/diskimage-builder/latest/elements/journal-to-console/README.html14:42
TheJuliaOh, that is a good one14:43
TheJuliaI always forget because in CI the console is ttyS014:43
JayFTheJulia: we have logged consoles here for troubleshooting. journal-to-console is just good, good stuff14:43
sdanniThanks JayF, a lot of helpful information!14:43
TheJulia++14:43
openstackgerritDmitry Tantsur proposed openstack/bifrost master: requirements: remove paramiko and simplify setuptools version  https://review.opendev.org/74399114:44
dtantsurJayF: TIL! does it make sense to use it by default?14:47
TheJuliadtantsur: I sure think so!14:47
JayFdtantsur: I don't have any idea who the users of our curated images are, so I don't have a strong opinion.14:47
JayFThere can be some downsides of putting journal to console, such as on machines with very slow consoles, printing messages to console can actually slow down what happens.14:48
TheJulianumerous operators without special driver needs14:48
TheJuliait ca14:48
TheJuliacan14:48
TheJuliaBut we already do it in CI14:48
rpiosoGood morning, ironic :-)14:48
TheJuliagood morning rpioso14:48
JayFSounds like it's a pretty easy choice then :)14:48
rpiosoTheJulia: o/14:48
openstackgerritDmitry Tantsur proposed openstack/ironic stable/ussuri: Fix idrac-wsman RAID apply_configuration  https://review.opendev.org/74399214:49
dtantsurTheJulia: have I mentioned that I've got deploy-time software RAID working in the CI?14:50
rpiosodtantsur: ty for the backport.14:50
dtantsurnp14:50
*** diurnalist has joined #openstack-ironic14:50
rpiosodtantsur: re: ^^^^^^^, that's an interesting error -- "Error: 'latin-1' codec can't encode characters in position 0-1: ordinal not in range(256)". More context would be helpful. Also, which version of the iDRAC FW is installed?14:54
dtantsurrpioso: asking. if you're on kubernetes slack, it's in #cluster-api-baremetal14:55
TheJuliadtantsur: I saw the comment14:57
* rpioso is effectively not on slack, but appreciates the pointer14:59
dtantsurgood for you :)14:59
openstackgerritMerged openstack/ironic-lib master: Update version of hacking  https://review.opendev.org/74394215:01
openstackgerritMerged openstack/ironic master: Enforce autospec in test_iscsi_deploy  https://review.opendev.org/74382015:01
rpioso:)15:01
*** k_mouza has joined #openstack-ironic15:04
*** k_mouza has quit IRC15:08
*** k_mouza has joined #openstack-ironic15:12
*** k_mouza has quit IRC15:16
*** k_mouza has joined #openstack-ironic15:21
iurygregoryhttps://review.opendev.org/741886  https://review.opendev.org/74144515:23
patchbotpatch 741886 - ironic - Add tempest default_boot_option and altflavor - 6 patch sets15:23
patchbotpatch 741445 - ironic-tempest-plugin - Update basic_ops - 13 patch sets15:23
*** gyee has joined #openstack-ironic15:25
*** k_mouza has quit IRC15:25
openstackgerritJulia Kreger proposed openstack/ironic-python-agent master: set EVENTLET_NO_GREENDNS to 'yes'  https://review.opendev.org/74201115:26
TheJuliadtantsur: https://review.opendev.org/#/c/742011/ <-- should at least make one of the inspector jobs happier. Just changed it to noqa the os import so hopefully it excludes everything else. Fun thing it passed pep8 on my desktop. :(15:27
patchbotpatch 742011 - ironic-python-agent - set EVENTLET_NO_GREENDNS to 'yes' - 3 patch sets15:27
*** k_mouza has joined #openstack-ironic15:28
*** k_mouza has quit IRC15:32
*** k_mouza has joined #openstack-ironic15:33
*** k_mouza has quit IRC15:37
*** sshnaidm|mtg is now known as sshnaidm15:39
openstackgerritVerification of a change to openstack/bifrost failed: Add non-voting jobs with the openstack ansible collection from source  https://review.opendev.org/74350815:42
*** mkrai has joined #openstack-ironic15:42
dtantsurrpioso: iDRAC Firmware Version: 4.00.00.0015:45
*** livelace has joined #openstack-ironic15:52
*** lucasagomes has quit IRC16:09
openstackgerritDmitry Tantsur proposed openstack/ironic-tempest-plugin master: Test software RAID in deploy-time on Victoria and newer  https://review.opendev.org/74122716:12
openstackgerritDmitry Tantsur proposed openstack/ironic master: Enable deploy-time software RAID in standalone jobs  https://review.opendev.org/74400916:14
dtantsurhere it goes ^^^16:14
*** ntt__ has quit IRC16:17
*** trandles has joined #openstack-ironic16:19
openstackgerritDmitry Tantsur proposed openstack/bifrost master: Clean up requirements  https://review.opendev.org/74399116:20
dtantsurfolks, would really appreciate reviews on https://review.opendev.org/#/c/741894/ to close this topic for now16:24
patchbotpatch 741894 - bifrost - Support configuring credential-less deploy - 5 patch sets16:24
*** dtantsur is now known as dtantsur|afk16:25
dtantsur|afko/16:25
*** dougsz has quit IRC16:30
dkingI have a probably unusual request. I'm interested in trying to run containerized Ironic/Bifrost. Essentially, I want to be able to launch a Docker service which gives me just whatever I need to run Ironic, such as ironic-conductor, ironic-inspector, ironic-api, and maybe the other services such as rabbitmq, and maybe dnsmasq. I don't necessarily need keystone or DIB. Would anybody be able to recommend the best route to take for16:42
dking that?16:42
*** derekh has quit IRC16:50
openstackgerritVerification of a change to openstack/ironic failed: Log the traceback of unexpected errors when verifying power credentials  https://review.opendev.org/74394116:57
*** ociuhandu_ has quit IRC17:02
*** ociuhandu has joined #openstack-ironic17:09
*** uzumaki has quit IRC17:12
*** mkrai has quit IRC17:14
*** ociuhandu has quit IRC17:14
sdanniHi JayF, TheJulia, I tried dynamic-login when building ramdisk image. The doc says 'append rootpwd=”$ENCRYPTED_PASSWORD” to the kernel command line on boot', but I'm not sure where to append the password argument.17:21
JayFfrom memory, I think it's pxe_append_params in ironic.conf?17:21
JayFcheck ironic.conf.sample to confirm17:21
TheJuliadking: so, as a single container or multiple containers, because I think all services and components... feels like we should already have a docker file for that17:21
JayFsdanni: https://docs.openstack.org/ironic/ussuri/configuration/sample-config.html appears to agree17:22
TheJuliayeah, it is pxe_append_params17:22
TheJuliain the [pxe] section17:22
sdanniCool! Thanks!17:23
dkingTheJulia: I could go either way. The ones I've seen so far seem like they need configuration after they start up, but I can't see why they don't just have everything they need and just mount the config directories.17:23
TheJuliaiurygregory: you around?17:24
openstackgerritMerged openstack/bifrost master: Add non-voting jobs with the openstack ansible collection from source  https://review.opendev.org/74350817:26
iurygregoryTheJulia, finishing some exercises, in about 30 min is ok?17:29
TheJuliaiurygregory: sure, I'm just wondering if there is two metal3 changes you can look at17:29
TheJuliaAnd exercise actually sounds like a REALLY good idea17:29
iurygregorythe PRs on the images?17:30
*** sdanni has quit IRC17:30
TheJuliayeah17:31
TheJuliaironic-image and ironic-inspector-image17:31
iurygregoryack I will do, I think i looked at the ironic-image yesterday17:31
TheJuliaiurygregory: looks like zaneb is going to also propose the openshift namespace cherry-picks17:32
*** diurnalist has quit IRC17:33
*** Lucas_Gray has quit IRC17:44
*** diurnalist has joined #openstack-ironic17:44
TheJuliadking: looks like we killed the dockerfile at some point17:46
TheJuliadking: you can just run install in a container, that should work just fine.17:46
*** sdanni has joined #openstack-ironic17:47
*** diurnalist has quit IRC17:49
dkingTheJulia: Well, I'd like to, as the DevOps folks say, treat them like cattle, not pets. So, I'd like to kill and spawn a new one easily if that's possible, rather than having to bring it up and then run external scripts. If that's possible, of course.17:52
TheJuliadking: well, installer to generate the base container and then just start the services on container start17:53
clarkbmnaser was working on simple container images for openstack services. I'm not sure if he got to ironic, but if he did those may be useable17:53
mnaserclarkb: the infrastructure is all there, we've gotten through a few, ironic should happen when we can get to it..17:54
mnaseronce we integrate it'll be part of our operator so it'll actually be an image that's tested using tempest17:54
clarkbmnaser: you've moved away from adding the dockerfiles in project through right? where is the work happening now?17:57
mnaserclarkb: well, not moved away as much as i haven't gotten traction so the goal once we have all (major) services inside containers and tested, we'd propose those into projects.  https://opendev.org/vexxhost/openstack-operator/src/branch/master/images17:58
*** livelace has quit IRC18:11
*** uzumaki has joined #openstack-ironic18:12
TheJuliafwiw, we afaik have no objection to putting dockerfile in ironic projets if it makes sense. I honestly think the resistance was really short signted18:20
TheJuliasighted18:20
dkingI think that would be nice. Ironic is different from a lot of the other services in that it has a lot of use cases where a full stack isn't being run.18:23
*** diurnalist has joined #openstack-ironic18:35
*** livelace has joined #openstack-ironic18:36
*** diurnalist has quit IRC18:40
openstackgerritAmy Marrich (spotz) proposed openstack/virtualbmc master: Correct vmbc add command  https://review.opendev.org/74402518:42
clarkbhttps://review.opendev.org/#/c/713975/4/Dockerfile is the in repo POC for keystone18:42
patchbotpatch 713975 - keystone - DNM: container images poc - 4 patch sets18:42
clarkbironic would likely be slightly more complicated as you'll probably want different images for each entrypoint but otherwise pretty similar18:43
openstackgerritJulia Kreger proposed openstack/ironic-python-agent master: Capture TypeError when the logger fails  https://review.opendev.org/74360618:51
*** diurnalist has joined #openstack-ironic19:26
sdanniJayF, TheJulia: after we login with dynamic-login element, we add the ip address (192.168.1.2) that the machine previously cannot connect with, and we can successfully curl this ip. After running ironic-python-agent.py, we again lose the connectivity with 192.168.1.2. The issue may related to ironic-python-agent I guess?19:41
mnaserTheJulia: i can submit a general patch but i cant guarantee that it actually works... until we implement it inside our operator and consume that image19:48
*** uzumaki has quit IRC20:14
*** ianychoi has joined #openstack-ironic20:41
dkingclarkb: Thanks for the information. I tried that with keystone and ironic, and I get "Package mysql-client is not available, but is referred to by another package."20:43
*** ociuhandu has joined #openstack-ironic20:46
clarkbdking: I'm guessing that means debian has switched to mariadb20:47
dkingProbably. I think everything else has.20:49
dkingYeah. It looks like if I change it to mariadb-*, it can get started.20:50
*** uzumaki has joined #openstack-ironic20:52
dkingWell, for ironic, anyway, and the it fails for gcc not being installed. For keystone, it complains about python3-all-dev.20:55
openstackgerritJulia Kreger proposed openstack/ironic-python-agent stable/stein: improve error messages during node lookup failures  https://review.opendev.org/74405220:57
openstackgerritJulia Kreger proposed openstack/ironic-python-agent stable/rocky: improve error messages during node lookup failures  https://review.opendev.org/74405320:57
openstackgerritJulia Kreger proposed openstack/ironic-python-agent stable/queens: improve error messages during node lookup failures  https://review.opendev.org/74405420:57
openstackgerritIury Gregory Melo Ferreira proposed openstack/ironic master: Add tempest default_boot_option and altflavor  https://review.opendev.org/74188620:58
iurygregorysmall update to fix the postgress for wholedisk+partition testing (I didn't noticed before since the job was non-voting)20:58
*** ociuhandu_ has joined #openstack-ironic21:04
*** ociuhandu has quit IRC21:06
*** uzumaki has quit IRC21:07
*** ociuhandu_ has quit IRC21:10
*** ociuhandu has joined #openstack-ironic21:11
*** livelace has quit IRC21:14
*** sshnaidm is now known as sshnaidm|afk21:42
rpiosodtantsur: We support 4.10.10.10 in the most recently released version of our downstream ironic idrac driver solution. Don't know if it would help, but they may want to consider upgrading to that version. Again, more Conductor log context would be helpful.22:03
rpiosodtantsur|afk: ^^^22:03
*** tkajinam has joined #openstack-ironic22:05
rpiosoTheJulia: Is there a story for making DeployInterface.prepare should probably become a step?22:13
* rpioso takes a do over22:14
rpiosoTheJulia: Is there a story for creating a deploy step from  DeployInterface.prepare?22:16
TheJuliarpioso: I'm not entirely sure it totally CAN22:20
TheJuliait might be that it needs to in part, but discussion is needed22:20
TheJuliaso no, afaik there is no story22:20
rpiosoTheJulia: How do you suggest we move forward?22:24
rpiosoTheJulia: I believe you and dtantsur understand the issue we're facing with executing an OOB deploy step which needs to perform a reboot to apply configuration. That causes the one-time PXE boot to be consumed and deployment to fail.22:27
*** sdanni has quit IRC22:37
*** rloo has quit IRC22:58
JayFrpioso: that's a situation that's handled (I hope/presume) for in-band IPA clean steps with reboot_requested=true... perhaps that could be a model to follow?22:59
rpiosoJayF: Do you have any insights why that model hasn't been applied to out-of-band? And please confirm for me the location of reboot_requested.23:01
*** rcernin has joined #openstack-ironic23:13
*** rcernin has quit IRC23:14
*** rcernin has joined #openstack-ironic23:14
*** diurnalist has quit IRC23:19
openstackgerritJulia Kreger proposed openstack/ironic-python-agent master: Fix TypeError on agent lookup failure  https://review.opendev.org/74407723:19
TheJuliadtantsur|afk: ^^^^ You'll want a beverage before reviewing that23:20
TheJuliaor at least before looking at the story and then looking at it23:20
TheJuliarpioso: The consensus was to make that part of the decorator so it could be declared  and then acted upon appropriately23:21
TheJuliayeah,, we discussed doing it that way because otherwise we expect to be able to at least ask the agent what IS possible23:21
TheJuliawhich I believe we wrote up in the shanghai notes and the story23:22
TheJuliadtantsur|afk: wf-2'ed my original change23:26
*** bdodd_ has quit IRC23:38
*** bdodd has joined #openstack-ironic23:42
*** diurnalist has joined #openstack-ironic23:53

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!