Wednesday, 2020-07-22

*** Lucas_Gray has joined #openstack-ironic00:12
*** gyee has quit IRC00:20
openstackgerritVerification of a change to openstack/ironic failed: Auto extend the timeout for RAX hosts  https://review.opendev.org/74185300:46
*** k_mouza has joined #openstack-ironic01:12
*** k_mouza has quit IRC01:16
* TheJulia sighs01:27
openstackgerritJulia Kreger proposed openstack/ironic master: Add some tuning documentation  https://review.opendev.org/73978301:30
openstackgerritVerification of a change to openstack/ironic-inspector failed: Make inspector base CI job use 2 CPUs  https://review.opendev.org/74074701:39
*** yaawang has quit IRC01:57
*** yaawang has joined #openstack-ironic01:58
*** ricolin has joined #openstack-ironic02:01
*** rloo has quit IRC02:15
openstackgerritJacob Anders proposed openstack/bifrost master: Auto-set ansible_python_interpreter when in venv  https://review.opendev.org/74233503:07
jandersstevebaker would you have a couple minutes to discuss this change? I need some guidance on the direction I'm taking hance workflow-103:08
stevebakerjanders: yep but I'm about to do a school run, I'll ping you when I'm online03:09
jandersok no worries :) I may take a break soon, if I'm quiet I'll be back later03:10
stevebakerjanders: but also you could ask mwhahaha and cloudnull in #tripleo because they've dealt with every kind of ansible/python interaction03:12
janderscool! thanks stevebaker willlllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll do03:13
jandersoopss keyboard going nuts again03:13
janderstime for a new one :)03:13
*** stevebaker has quit IRC03:16
*** stevebaker has joined #openstack-ironic03:50
stevebakerjanders: I'm back03:51
jandersI was just chatting to mwhahaha, got some pointers03:53
jandersbut very interested in hearing your thoughts as well if you have time. I'm not sure if the playbook I'm tweaking is supported to run outside VENV (I don't have a way of making it run unless I activate a VENV)03:54
stevebakerjanders: my hunch is that the playbook should be left alone, and the thing calling the playbook should set ansible_python_interpreter if it isn't the default03:57
stevebakerjanders: or maybe there is a local directory ansible.cfg way of doing it03:58
stevebakerjanders: ah, like this https://docs.ansible.com/ansible/latest/reference_appendices/interpreter_discovery.html03:59
jandersnice find stevebaker thank you!03:59
jandersI did try the ansible.cfg approach but without success hence moving on, but from this doco I see what might have been wrong04:00
janderswill try this04:00
jandersdo you think it's cool to have scripts/env-setup.sh generate ansible.cfg with the appropriate path matching VENV?04:01
jandersVENV is being stood up in this step so that would be my first idea on where to put config generation04:02
janderstaking a break, will be back a bit later04:23
*** rh-jelabarre has quit IRC04:25
*** diurnalist has quit IRC04:35
*** Lucas_Gray has quit IRC04:36
stevebakerjanders: that could be worth looking into, but I don't know where the ansible.cfg should be written to. The current directory? A known location which is referenced each time ansible-playbook is run? dtantsur|afk may have some thoughts on that04:41
*** stevebaker has quit IRC04:52
openstackgerritMerged openstack/ironic master: Iso booting via redfish virtual media  https://review.opendev.org/72748004:55
*** abdysn has joined #openstack-ironic04:58
*** tzumainn has quit IRC05:08
*** diurnalist has joined #openstack-ironic05:32
*** diurnalist has quit IRC05:36
*** iurygregory has joined #openstack-ironic06:25
iurygregorygood morning Ironic!06:26
jandersgood morning iurygregory o/06:49
jandershow are you going?06:49
rpittaugood morning ironic! o/06:58
jandersg'day rpittau! o/07:00
rpittauhey janders :)07:01
janderswith a little help from stevebaker I think I'm getting on top of ansible+venv challenges07:01
rpittauI was reading the log, yeah07:01
jandersI think my second patch is irrelevant but I'm curious what are your thoughts on the first one:07:02
jandershttps://review.opendev.org/#/c/742192/07:02
patchbotpatch 742192 - bifrost - Replace ANSIBLE_COLLECTIONS_PATHS var with symlink - 1 patch set07:02
iurygregoryhey janders and rpittau o/07:02
iurygregorydoing good =)07:02
rpittauhey iurygregory :)07:02
jandersis this an acceptable approach to configuring things up? If so I will add another snippet which will autogenerate ansible config for the playbooks which  will force the right python interpreter and we should be closee to done07:03
jandersiurygregory good to hear! :)07:03
rpittaujanders: the base check is ok, it will work07:06
iurygregoryI'm a bit puzzled by " This change replaces the variable based approach..." you need the variable to use the symb link approach no?07:06
jandersmaybe I need to be more clear on which variaable07:06
iurygregoryin the commit message, seems like we are getting rid off the variable..07:06
jandersVENV is all good07:06
janderswhere things were falling apart for me is ANSIBLE_COLLECTIONS_PATHS07:07
jandersit almost never gets set and this causes the issue where ansible can't find collections that the playbook relies on07:07
jandersI should have said "exported" not "set"07:07
rpittauyou're accounting only for the presence or not of the symlink, but that will fail if you have an actual file called collections already in place07:07
rpittauwhich is a remote possibility07:08
iurygregorymaybe we should create the ansible.cfg and set collections_paths....07:08
jandersiurygregory this is a very timely comment cause I'm just working on auto-generating ansible.cfggggggggggggggggg07:09
janderssorry sticky keys07:09
jandersneed new keyboardd07:09
iurygregoryhttps://docs.ansible.com/ansible/latest/reference_appendices/config.html#collections-paths07:09
janderstoo many trips in a backpack07:09
iurygregoryI think ansible won't like to many g's =)07:09
jandersLOL07:10
rpittauonly limitation is that ansible.cfg must be generated before running ansible :)07:10
janderslet me add the other snippet and upload the new revision07:10
*** ociuhandu has quit IRC07:10
jandersand I can rewrite the collections stuff in the same fashion if you like this apporach07:10
rpittauit's probably more clean, wondering if it might conflict with any other config already in place by the final user07:11
jandersgood point rpittau07:11
jandersmy "defence" against that is I'm writing ansible.cfg in the "playbooks" subdirectory07:11
jandersso it should override global and user config07:11
rpittaummm ok so pointing to that specifically07:11
openstackgerritJacob Anders proposed openstack/bifrost master: Replace ANSIBLE_COLLECTIONS_PATHS var with symlink  https://review.opendev.org/74219207:14
jandershow do you find this? happy to rewrite the collections bit along the same lines07:15
jandersthe commit message will need total rewrite (or I might abandon and resubmit :) ) but I think this captures what I'm trying to do07:16
iurygregorywhen you do git commit --amend you can rewrite the whole commit message =)07:17
iurygregoryjust keep the Change-Id07:17
jandersok! can do :)07:18
iurygregoryotherwise it will generate a new change on gerrit =)07:18
rpittauI'm ok for the symlink approach, although I would not limit the check to just symlinks07:20
rpittauthe python interpreter proably deserves a separate patch07:20
rpittauI'm still not sure bypassing other ansible config files is the right approach07:20
jandershttps://review.opendev.org/#/c/742335/07:21
patchbotpatch 742335 - bifrost - Auto-set ansible_python_interpreter when in venv - 1 patch set07:21
jandersthat was my other idea07:21
jandersor I should say the other working idea07:22
janderscause I was hoping to address this by setting a custom fact (or an addition to inventory) but in both of those cases the variable seems to be ignored07:22
jandersit seems to be one of those weird ansible things where a simple fix is surprisingly difficult to make for some reason07:23
jandersI don't quite understand why it needs  interpreter override in the first place instead of just running off VENV interpreter07:24
rpittauit's also not how it's explained in the docs :)07:24
*** rcernin has quit IRC07:32
rpittauthinking a bit more about it, I'm slightly more inclined to the vars approach, just because it's a temporary override07:32
jandersdo we support (or intend to support) running these playbooks outside bifrost VENV context?07:34
jandersthat was my only concern about the vars bit - if VENV wasn't set it would fail07:34
jandersbut having said that I haven't managed to get this playbook to start doing anything without VENV being active07:34
jandersso maybe this is not a concern?07:35
rpittauwe currently support only VENV runs07:36
*** ociuhandu has joined #openstack-ironic07:36
jandersgood! :)07:37
rpittauslightly fewer headaches :D07:37
jandersagreed07:37
jandersso - to move forward - would you be happy with two separate fixes: 1) collections/symlink (provided it checks for existence of files, not just symlinks) and 2) var-override based fix for the interpreter issue?07:39
rpittaujanders: yeah, to me it looks like the best approach at the moment07:39
rpittauI guess dtantsur|afk will have his saying :)07:39
janderssounds good!07:40
jandersin the meantime I will carry on with testing and see if these fixes allow me to actually deploy something07:40
rpittausounds good07:41
janderswithout these I couldn't get my C8 node to get to enrolment step, ansible would err out before that07:41
janderscool! thanks for your help rpittau and iurygregory much appreciated! :)07:41
janderscheers guys07:41
rpittauanytime :)07:41
iurygregorynp =)07:41
janderswhat's the best way to snoop on the VMs when they sit in wait call-back? virt-manager connecting to the beaker node?07:42
jandershttp://paste.openstack.org/show/796198/07:43
jandersvirsh console doesn't seem to be doing much07:43
iurygregorystrange .-.07:45
iurygregorymaybe you can edit the xml?07:47
jandersI virt-man'ed in, I see splash screens of tinycorelinux07:48
jandersbut the instances are still in "wait call-back" as far as ironic is concerned07:49
* janders is reading the playbook to learn what tinycorelinux is used for in bifrost07:49
rpittaujanders: we use tinycore 1007:50
rpittaujanders: if the node is in 'wait call-back' should mean that it's being deployed07:51
rpittaujanders: a useful reference -> https://docs.openstack.org/ironic/latest/contributor/api/ironic.common.states.html07:52
jandersagreed, but in this case it's been in this state for quite long and I see the OS login screen07:52
rpittauah07:53
jandersI had quite a few failures on the way here due to ansible bugs so I suspect something isn't cool. I might need to clean up a bit and recreate the VMs I suspect07:53
rpittauthe OS login screen of tinycore ?07:53
janderscorrect07:53
rpittauok that's weird07:53
jandersbut if I see that and the instances are in wait-callback I am leaning to think these were deployed by one of the previous installs07:53
jandersor install attempts I should say07:54
jandersI did virsh destroy/undefine prior to re-deploying bifrost but perhaps the qcows were spared or something07:54
jandersI think I might break for dinner now and when I'm back dtantsur|afk will likely be online. Then maybe we can confirm how we want to go about my fixes and I can clean up the test environment and test new patches.07:58
rpittaummmmm so what you're seeing, the login screen, is normal if you used virt-manager, you should check the console instead07:58
rpittaujanders: enjoy :)07:58
jandersbifrost doesn't have "uninstall" playbook right?07:59
jandersokay... so broken virsh console is likely indicating some problem. Good to know!07:59
jandersrpittau thanks :)07:59
rpittauno "uninstall" playbook :/08:00
rpittaunp08:00
*** lucasagomes has joined #openstack-ironic08:07
openstackgerritRiccardo Pittau proposed openstack/sushy master: Update system schema to 1.10  https://review.opendev.org/74223608:10
*** yaawang has quit IRC08:13
*** yaawang has joined #openstack-ironic08:13
*** dtantsur|afk is now known as dtantsur08:52
dtantsurmorning ironic08:52
iurygregoryI'm wondering if we need to set something else for the partition tests .-.08:52
iurygregorymorning dtantsur08:53
dtantsurjanders: "uninstall" is something we should have eventually08:53
dtantsurjanders: also bifrost saves log output from nodes to somewhere like /var/log/libvirt/baremetal_nodes08:54
openstackgerritDmitry Tantsur proposed openstack/bifrost master: Support HTTP basic auth and switch bifrost-cli to it  https://review.opendev.org/74196409:03
iurygregorydtantsur, seems like the partition test is trying to install grub2 and it fails O.o09:03
dtantsuriurygregory: then boot_option:netboot is not applied09:04
dtantsurI wonder if nova overwrites instance_info completely09:04
iurygregorybad nova =(09:04
dtantsuriurygregory: it overwrites capabilities: https://opendev.org/openstack/nova/src/branch/master/nova/virt/ironic/patcher.py#L11509:06
iurygregoryenr .-.09:06
dtantsuriurygregory: if we cover local boot tests in the standalone job well enough, we can just use netboot in all "large" jobs09:07
dtantsurstandalone tests control their nodes, there is no nova to interfere09:07
iurygregoryyeah09:07
iurygregoryother approach could also be a new parameter "capabilities" ?09:09
iurygregoryin the nova code maybe09:09
jandersdtantsur thanks for your comments. I will take notes as I clean up my beaker node - maybe I can turn these into uninstall playbook one day :)09:09
*** rcernin has joined #openstack-ironic09:10
iurygregoryjanders++09:10
jandersdtantsur we were chatting about my ansible-venv fixes earlier, it would be great to have your thoughts on the possible approaches. Would you have time to chat about this now?09:11
dtantsuriurygregory: capabilities are coming from the flavor, you'd have to modify it09:11
dtantsurjanders: absolutely; I'd appreciate a tl;dr from the scrollback though :)09:11
iurygregorywe can create a second flavor I assume?09:12
* iurygregory wondering what would be better09:12
jandersI have some proposed patches to choose from that take different approaches:09:12
jandershttps://review.opendev.org/#/c/742192/2/scripts/env-setup.sh09:13
patchbotpatch 742192 - bifrost - Replace ANSIBLE_COLLECTIONS_PATHS var with symlink - 2 patch sets09:13
jandersthis handles collections issue by symlinking them to the playbook and the interpreter issue by creating a local ansible.cfg override in the playbook dir09:13
jandersalternative solution to the interpreter issue is this:09:14
jandershttps://review.opendev.org/#/c/742335/1/playbooks/enroll-dynamic.yaml09:14
patchbotpatch 742335 - bifrost - Auto-set ansible_python_interpreter when in venv - 1 patch set09:14
janders(it's based on a top-level var)09:14
jandersI tried injecting things into dynamic inventory or setting custom facts but didn't have any luck. The ansible.cfg override and vars based approach both fix the interpreter issue.09:15
jandersquestion for you is what do you think of these options and do you have preference for a specific one (especially for the interpreter issue)09:15
dtantsurjanders: does https://review.opendev.org/#/c/742335/ also solve the collections location problem?09:16
patchbotpatch 742335 - bifrost - Auto-set ansible_python_interpreter when in venv - 1 patch set09:16
dtantsur(just to make sure I understand it right)09:17
jandersno, this only solves the interpreter problem09:17
jandersto my best knowledge09:17
jandersbut it is a good question that justifies a quick test09:17
dtantsuryes please09:17
* janders is removing the symlink from the former fix and re-testing09:18
iurygregorydtantsur, wdyt if we add a new flavor in https://github.com/openstack/ironic/blob/d867488e90dfe0ff09c35223a842f2407dad1a75/devstack/lib/ironic and we can add a config in the tempest-plugin that would have the id and we can set the flavor for the test?09:19
jandersthe test failed, dropping the output into pastebin09:19
jandershttp://paste.openstack.org/show/796201/09:20
jandersI think this is due to a different mechanism in which the collections work - collections rely on an environment variable (which I am having issues with in the current codebase) or the symlink09:21
jandersthere is no desire to go looking for collections in venv if one of those isn't specified AFAIK09:21
iurygregoryjanders, if you have the path in the ansible.cfg does it works without problems?09:21
dtantsuriurygregory: there even some flavor_ref_alt: https://github.com/openstack/ironic/blob/d867488e90dfe0ff09c35223a842f2407dad1a75/devstack/lib/ironic#L2993 (not sure what it does)09:22
* janders testing this now09:22
iurygregorydtantsur, WUT09:22
dtantsurjanders: does the symlink approach work without 'cd playbooks' (e.g. if you call them from somewhere else)?09:22
iurygregoryflavor_alt_ref lol09:22
iurygregoryaccording to tempest `flavor_ref_alt is only used in tests where two different-sized servers are required (for example, a resize test).`09:23
iurygregorybut I do think we can workaround this by setting our new flavor as flavor_ref_alt and change the config09:24
jandersiurygregory  if collections_paths=/opt/stack/bifrost/collections/ is set in local ansible.cfg override it does work without the symlink09:26
jandersdtantsur testing running from another working dir09:26
iurygregoryI guess we can say that it's necessary to set on the ansible.cfg, we can probably generate the ansible.cfg with this config in the env setup (I think)09:27
jandersdtantsur ansible.cfg based approach does not work if CWD isn't playbooks. Combination of symlink + vars tweak DOES WORK.09:29
dtantsurjanders: I'd go for the latter then (+ documentation for those who build custom venv)09:30
dtantsurthe only thing about the vars tweak: I think it should go to each role that relies on the collection09:30
dtantsurbecause people can use custom playbooks based on our roles09:30
jandersok! I can do that.09:31
jandersI will just make sure-sure it still works (ansible gave me a lot of grief today silently ignoring interpreter overrides from a lot of places in code) but I think it will09:32
dtantsurjanders: as a bonus, you'll be able to use our standard defaults (bifrost_venv_dir and so on)09:33
*** dsneddon has quit IRC09:33
jandersnice! :)09:33
jandersI'm glad all the years of hacking on ansible didn't go to waste :)09:33
dtantsurheh, good indeed :)09:34
janderslooks like 7 roles out of 20 would need changes09:35
*** rcernin has quit IRC09:35
jandersis recursive grepping for openstack.cloud in the roles dir a sufficient check what roles need collections? I suppose so...09:36
iurygregorydtantsur, I will go with the flavor_ref_alt approach, to get some results =)09:37
dtantsurjanders: we haven't fully switched to collections yet, but everything that uses os_* will also need this hack eventually09:37
dtantsurif iurygregory does not object, you can pick up the task of switching them ;)09:38
dtantsuriurygregory: let's give it a try indeed09:38
iurygregoryI have no problem, my plan was to do next week =)09:39
*** hjensas has quit IRC09:39
jandersok! :) I will just check priorities with shadower cause I know he wanted me to look aat deploying OCP/IPI on some nodes. Having said that I'm more than happy to pick this up! :)09:44
dtantsurmaybe you can split the load? :) I'm not trying to pick on you, but ansible 2.10 will not have the os_* modules and they're deprecated aliases in the collection09:45
dtantsurso we're not in a big hurry, but we also cannot delay the switch indefinitely09:45
iurygregoryyeah09:45
dtantsur(mostly setting up context for shadower ^^^)09:45
shadowerI appreciate it09:46
iurygregoryshadower is back \o/09:46
shadowero/09:46
iurygregory`openstack flavor set altbaremetal --property "capabilities:boot_option"="netboot"` this is correct right dtantsur ?09:48
openstackgerritIury Gregory Melo Ferreira proposed openstack/ironic master: Create altbaremetal flavor  https://review.opendev.org/74238109:51
*** k_mouza has joined #openstack-ironic09:53
jandersdtantsur I moved the interpreter to the role vars as you suggested, it works fine (and I like it, it's cleaner). Before I start applying this more widely, can you confirm if this is how you wanted it to be done?09:56
jandershttp://paste.openstack.org/show/796204/09:56
openstackgerritIury Gregory Melo Ferreira proposed openstack/ironic-tempest-plugin master: Update basic_ops  https://review.opendev.org/74144510:00
* iurygregory fingers crossed10:00
iurygregorywe will probably need to backport the alt flavor to stable branches otherwise CI will *BOOM*10:02
dtantsuriurygregory: 1. I think so, 2. Let's somehow skip the partition test if the flavor is not provided and default_boot_option is local?10:03
iurygregorymakes sense10:04
iurygregoryI was thinking in removing the "default_boot_option", but maybe will be useful for other cases in the future so I will re add that, and add the skip10:05
dtantsurjanders: I think to account for all variables we need to do something like http://paste.openstack.org/show/796205/10:06
jandersACK! thank you dtantsur10:07
jandersI will aim to progress further on the fixes tomorrow - wrapping up for tonight as I'm aiming to have a go at our 5am (my time) meeting10:09
jandersthank you for your help and guidance today dtantsur rpittau iurygregory - invaluable!10:10
jandershave a good one and we'll speak tomorrow10:10
iurygregoryjanders, good night! you are welcome =)10:10
*** yaawang has quit IRC10:11
dtantsurg'night!10:13
*** yaawang has joined #openstack-ironic10:13
jandersapologies if I am slightly late tomorrow :)10:13
dtantsurno worries :)10:13
*** yaawang has quit IRC10:20
*** yaawang has joined #openstack-ironic10:21
openstackgerritDmitry Tantsur proposed openstack/bifrost master: Use random passwords by default  https://review.opendev.org/74220310:31
*** priteau has joined #openstack-ironic10:33
*** Lucas_Gray has joined #openstack-ironic10:41
openstackgerritVerification of a change to openstack/ironic-lib failed: Support more bcrypt prefixes  https://review.opendev.org/73871810:42
*** ricolin has quit IRC10:48
*** tkajinam has quit IRC10:59
*** sshnaidm is now known as sshnaidm|afk11:14
openstackgerritDmitry Tantsur proposed openstack/bifrost master: Use random passwords by default  https://review.opendev.org/74220311:14
*** uzumaki has joined #openstack-ironic11:31
*** rcernin has joined #openstack-ironic11:49
*** rh-jelabarre has joined #openstack-ironic12:00
*** ociuhandu has quit IRC12:03
*** ociuhandu_ has joined #openstack-ironic12:03
openstackgerritIury Gregory Melo Ferreira proposed openstack/ironic-tempest-plugin master: Update basic_ops  https://review.opendev.org/74144512:08
*** Goneri has joined #openstack-ironic12:25
*** rloo has joined #openstack-ironic12:28
*** m1p_ has joined #openstack-ironic12:31
iurygregoryseems like test_baremetal_vm_multitenancy didn't like my approach, maybe we can just use flavor_ref in the multitenancy test, thoughts?12:32
openstackgerritIury Gregory Melo Ferreira proposed openstack/ironic-tempest-plugin master: Update basic_ops  https://review.opendev.org/74144512:34
*** m1p has quit IRC12:38
*** seba has quit IRC12:38
*** m1p_ is now known as m1p12:38
*** mkowalski has quit IRC12:40
*** seba has joined #openstack-ironic12:42
*** mkowalski has joined #openstack-ironic12:49
openstackgerritDmitry Tantsur proposed openstack/bifrost master: Use random passwords by default  https://review.opendev.org/74220312:50
dtantsuriurygregory: not familiar with this test12:50
iurygregorydtantsur, I gave a try, since the VM that is create was using flavor_ref_alt (and before it was the same as flavor_ref)12:50
iurygregoryso let's see =)12:51
dtantsurmakes sense12:51
TheJuliagood morning12:51
dtantsurmorning TheJulia12:51
iurygregorygood morning TheJulia =)12:51
dtantsuriurygregory: you can ask dradez or mchapman (e.g. downstream on #rhos-netansible) why the test was written this way12:51
iurygregorydtantsur, gotcha will ask12:52
iurygregorymaybe in their setup the flavor_alt_ref is different...12:52
iurygregorybut in our CI we were using the same..12:53
* iurygregory doesn't want to break things12:53
openstackgerritVerification of a change to openstack/ironic failed: Extend base build timeouts  https://review.opendev.org/74181812:56
*** tzumainn has joined #openstack-ironic12:57
dtantsuroh, nice music on opendev today :)12:58
*** sshnaidm|afk is now known as sshnaidm13:29
*** bnemec has joined #openstack-ironic13:30
*** rcernin has quit IRC13:32
*** bfournie has joined #openstack-ironic13:34
openstackgerritRiccardo Pittau proposed openstack/ironic-python-agent master: Fix versions in release notes  https://review.opendev.org/74243113:42
*** ricolin has joined #openstack-ironic13:43
openstackgerritDmitry Tantsur proposed openstack/bifrost master: Use random passwords by default  https://review.opendev.org/74220313:46
openstackgerritDmitry Tantsur proposed openstack/bifrost master: Use random passwords by default  https://review.opendev.org/74220313:50
openstackgerritDmitry Tantsur proposed openstack/bifrost master: [WIP] Support HTTP basic auth and switch bifrost-cli to it  https://review.opendev.org/74196413:59
*** uzumaki has quit IRC14:09
*** yolanda has quit IRC14:10
openstackgerritDmitry Tantsur proposed openstack/bifrost master: Use random passwords by default  https://review.opendev.org/74220314:11
*** abdysn has quit IRC14:11
*** yolanda has joined #openstack-ironic14:12
*** dsneddon has joined #openstack-ironic14:19
*** dsneddon has quit IRC14:26
openstackgerritDmitry Tantsur proposed openstack/bifrost master: Use random passwords by default  https://review.opendev.org/74220314:32
dtantsurkeystone is going to drive me nuts14:34
iurygregorywhat happen?14:37
iurygregoryhappened*14:38
iurygregoryinteresting, the flavor_ref_alt is set but tempest didn't use O.o14:39
*** bfournie has quit IRC14:45
*** bfournie has joined #openstack-ironic14:48
dtantsuriurygregory: keystone has a "feature" that locks you out of your account after N login attempts14:48
dtantsursounds okay, until you realize it's a service supposed to be used on public clouds14:49
iurygregoryyou can't override the N?14:49
dtantsuryou can, but what's the point?14:49
dtantsuras long as this feature is enabled, anybody on the network can lock out anyone by their user name + domain ID14:49
dtantsurwhich is exactly what I've just done to myself :)14:50
openstackgerritIury Gregory Melo Ferreira proposed openstack/ironic-tempest-plugin master: Update basic_ops  https://review.opendev.org/74144514:52
iurygregorydtantsur, ouch!14:52
iurygregory=X14:52
dtantsurokay, it's not forever by default, it's half an hour (which is eternity for a public cloud)14:53
openstackgerritDmitry Tantsur proposed openstack/bifrost master: Disable keystone account locking by default  https://review.opendev.org/74245614:55
dtantsuriurygregory: ^^^14:57
*** diurnalist has joined #openstack-ironic14:58
openstackgerritDmitry Tantsur proposed openstack/bifrost master: Use random passwords by default  https://review.opendev.org/74220314:58
iurygregorylooking14:58
openstackgerritDmitry Tantsur proposed openstack/bifrost master: Support configuring credential-less deploy  https://review.opendev.org/74189414:59
*** k_mouza has quit IRC15:00
*** k_mouza has joined #openstack-ironic15:01
openstackgerritDmitry Tantsur proposed openstack/bifrost master: [WIP] Support HTTP basic auth and switch bifrost-cli to it  https://review.opendev.org/74196415:01
*** rcernin has joined #openstack-ironic15:25
openstackgerritDmitry Tantsur proposed openstack/bifrost master: [WIP] Support HTTP basic auth and switch bifrost-cli to it  https://review.opendev.org/74196415:25
*** rcernin has quit IRC15:29
*** k_mouza has quit IRC15:41
openstackgerritMerged openstack/ironic stable/train: Stop running test_schedule_to_all_nodes in the multinode job  https://review.opendev.org/74221515:43
*** k_mouza has joined #openstack-ironic15:44
*** gyee has joined #openstack-ironic15:48
iurygregorydtantsur, https://docs.openstack.org/ironic/latest/configuration/config.html#json-rpc .-.15:54
rpiosopas-ha: Can https://review.opendev.org/#/c/732158/ be abandoned?15:54
patchbotpatch 732158 - ironic (stable/train) - Revert "redfish: handle hardware that is unable to... - 2 patch sets15:54
iurygregoryhttp_basic_username and http_basic_password .-.15:54
dtantsuriurygregory: yeah, dunno how we missed that15:54
dtantsurI'm fixing it now (with deprecation, SIGH)15:54
rpiosopas-ha: I believe the fix merged on stable/train: https://review.opendev.org/#/c/736941/15:54
patchbotpatch 736941 - ironic (stable/train) - Fix Redfish handle no continuous override boot src (MERGED) - 1 patch set15:54
iurygregoryoh we need to deprecated because we have a release right?15:55
pas-harpioso: seems like can be abandoned indeed15:55
*** k_mouza has quit IRC15:55
dtantsuriurygregory: yeaaah :(15:56
rpiosopas-ha: Cool! Thank you.15:56
* iurygregory cries =( 15:56
*** lucasagomes has quit IRC15:57
rpiosopas-ha: Also, it seems like https://review.opendev.org/#/c/732122/ should be all set now. Please consider removing the Workflow-1.15:58
patchbotpatch 732122 - ironic (stable/train) - Remove iDRAC Redfish boot device known issue - 1 patch set15:58
openstackgerritMerged openstack/ironic master: Use native oslo.concurrency execution timeout in ipmitool  https://review.opendev.org/73859115:58
rpiosopas-ha: Errr Code-Review-115:58
pas-hayep, done too15:59
rpiosopas-ha: :-)15:59
openstackgerritVerification of a change to openstack/ironic failed: Stop running test_schedule_to_all_nodes in the multinode job  https://review.opendev.org/74221415:59
openstackgerritDmitry Tantsur proposed openstack/bifrost master: [WIP] Support HTTP basic auth and switch bifrost-cli to it  https://review.opendev.org/74196416:06
openstackgerritMerged openstack/bifrost master: Disable keystone account locking by default  https://review.opendev.org/74245616:08
openstackgerritDmitry Tantsur proposed openstack/ironic master: Deprecate http_basic_username and http_basic_password in [json_rpc]  https://review.opendev.org/74247416:10
dtantsuriurygregory: ^^16:13
dtantsurI got hit by it in my bifrost patch16:13
iurygregorydtantsur, ack16:14
iurygregoryat least on the bright side we have username/password for inspector =)16:17
TheJuliayay opendev is over!16:18
*** ricolin has quit IRC16:19
*** bfournie has quit IRC16:20
dtantsur:)16:20
TheJuliaWould it help folks if I were to update the review list on the etherpad because we have a few fixes that could use some review traffic16:21
dtantsurTheJulia: once you get enough rest, I'd be curious to hear your opinion on https://review.opendev.org/74220316:21
patchbotpatch 742203 - bifrost - Use random passwords by default - 11 patch sets16:21
dtantsurTheJulia++16:21
* TheJulia goes and makes a fresh pot of coffee before updating the etherpad16:21
iurygregoryI'm still doing something wrong since partition test are still failing even with using the other flavor .-.16:23
*** Lucas_Gray has quit IRC16:24
*** jadonn has joined #openstack-ironic16:24
openstackgerritDmitry Tantsur proposed openstack/ironic master: Add an option to choose the hash ring algorithm  https://review.opendev.org/73977916:28
dtantsurfixes ironic in FIPS mode, reviews appreciated ^^^16:28
dtantsurwell, it doesn't fix automatically, but makes it possible to fix16:28
iurygregoryI thought the fix was on tooz only16:30
dtantsurunfortunately, no16:30
dtantsurwe cannot just switch tooz to sha256, it would be breaking for rolling upgrades16:31
*** Lucas_Gray has joined #openstack-ironic16:31
*** dtantsur is now known as dtantsur|afk16:35
dtantsur|afko/16:35
TheJulia:(16:35
TheJuliaYeah, I was suspecting that would be the case16:35
openstackgerritZygimantas Matonis proposed openstack/ironic-inspector master: WIP: adding condition_join to rules.  https://review.opendev.org/73179416:38
TheJuliacrazy question: What if on rax hosts, we change a default on jobs where we don't build IPA to use tinycore?16:53
TheJuliahttps://8fedc9098215014c7687-39f5f8dab729bdab706fbbfdb548a0f1.ssl.cf1.rackcdn.com/741818/5/gate/ironic-tempest-ipa-wholedisk-bios-agent_ipmitool-indirect/695d93c/controller/logs/ironic-bm-logs/ <-- because the dib images running in RAX just seem like we're never going to balance things if we don't16:58
*** sshnaidm is now known as sshnaidm|afk17:09
*** ociuhandu has joined #openstack-ironic17:11
*** ociuhandu_ has quit IRC17:14
*** ociuhandu has quit IRC17:16
*** dsneddon has joined #openstack-ironic17:16
openstackgerritJulia Kreger proposed openstack/ironic-inspector master: Respond so Apache HTTPd doesn't think the request failed  https://review.opendev.org/74034417:18
*** ociuhandu has joined #openstack-ironic17:22
*** Nisha_Agarwal has joined #openstack-ironic17:22
*** Lucas_Gray has quit IRC17:23
*** sshnaidm|afk has quit IRC17:23
*** ociuhandu has quit IRC17:26
*** sshnaidm has joined #openstack-ironic17:27
*** sshnaidm is now known as sshnaidm|afk17:27
openstackgerritJulia Kreger proposed openstack/ironic master: Extend PXE boot retry timeout for RAX hosts  https://review.opendev.org/74250117:38
openstackgerritJulia Kreger proposed openstack/ironic master: Extend PXE boot retry timeout for RAX hosts  https://review.opendev.org/74250117:41
*** paraplan has joined #openstack-ironic17:43
openstackgerritJulia Kreger proposed openstack/ironic master: Force RAX hosts to run tinyipa  https://review.opendev.org/74250217:50
*** k_mouza has joined #openstack-ironic17:55
*** k_mouza has quit IRC17:59
TheJuliaOn the list of review priorities there are 6 changes which should address most of our current sporatic CI failures or major issues that are problematic17:59
gudrutis0very knew this kind of status code existed https://httpstatuses.com/41818:10
gudrutis0newer*18:10
TheJuliaplease let me guess without looking... it is a teapot?18:10
TheJuliaAnyone know if Doug Szumski frequents irc?18:11
TheJulialol, it is a teapot18:12
TheJuliaIronic should respond with that at some point18:12
TheJuliaMaybe out of memory + auth failed18:12
openstackgerritJulia Kreger proposed openstack/ironic-python-agent master: Fix bootloader install issue with MDRAID  https://review.opendev.org/74033118:17
openstackgerritMerged openstack/ironic-lib master: Support more bcrypt prefixes  https://review.opendev.org/73871818:19
TheJuliaAnyone remember what james denton's irc nick is?18:24
*** jj123 has joined #openstack-ironic18:54
openstackgerritJulia Kreger proposed openstack/ironic stable/stein: Pin the python-ibmcclient upper bounds due to six usage  https://review.opendev.org/74251219:01
*** bnemec has quit IRC19:03
*** stevebaker has joined #openstack-ironic19:05
*** Nisha_Agarwal has quit IRC19:27
jandersg'day Ironic!  o/19:28
stevebakerjanders: you're up early19:28
jandershaha just a little19:28
jandersI'm more of a morning person so this is not too bad19:29
jandersjust need a doubleshot flat white and she'll be right19:30
janders:)19:30
stevebakerlol19:30
*** ociuhandu has joined #openstack-ironic19:31
jandersthat's something I love about visiting Wellington -being able to get a nice takeaway cofee any time of the day19:31
janders(or night)19:31
*** ociuhandu_ has joined #openstack-ironic19:34
*** ociuhand_ has joined #openstack-ironic19:35
*** paraplan has quit IRC19:35
*** ociuhandu has quit IRC19:36
*** ociuhandu_ has quit IRC19:38
*** diurnalist has quit IRC19:38
* TheJulia could use a flat white at the moment19:43
*** adrianc_ has joined #openstack-ironic19:44
*** ociuhand_ has quit IRC19:45
*** adrianc has quit IRC19:47
*** jj123 has quit IRC19:50
openstackgerritJacob Anders proposed openstack/bifrost master: Create symlink to ansible collections on install  https://review.opendev.org/74219219:53
openstackgerritMerged openstack/ironic-python-agent master: Adds poll mode deployment support  https://review.opendev.org/69446119:53
*** jj123 has joined #openstack-ironic19:53
jandersRegarding https://review.opendev.org/742192  - do we need to update the doco here: https://docs.openstack.org/bifrost/latest/install/index.html#dependencies and mention the creation of the symlink?19:57
patchbotpatch 742192 - bifrost - Create symlink to ansible collections on install - 3 patch sets19:57
jandersif so, should the doco update be a part of the same change proposal, or would that be separate?19:57
*** yolanda has quit IRC20:05
*** yolanda has joined #openstack-ironic20:08
*** diurnalist has joined #openstack-ironic20:13
*** tzumainn has quit IRC20:22
openstackgerritVerification of a change to openstack/ironic failed: Stop running test_schedule_to_all_nodes in the multinode job  https://review.opendev.org/74221420:44
*** priteau has quit IRC20:47
* arne_wiebalck realises it is probably time to call it a day when he sees janders leave and come back :)20:48
arne_wiebalckbye everyone, cu tmrw o/20:48
jandershave a good one arne_wiebalck20:53
*** bnemec has joined #openstack-ironic21:04
jj123I could use some guidance on identifying a known-good recipe for the example DevStack ironic build at https://docs.openstack.org/ironic/latest/contributor/dev-quickstart.html. After following the recipe on an Ubuntu 18.04.4 VirtualBox VM, My node is available in `openstack baremetal node list`, but is shown as 'ERROR None NOSTATE' in `openstack21:05
jj123server list --long`. From a fresh unstack.sh/clean.sh/stack.sh, tempest ironic tests fail: test_baremetal_server_ops_partition_image, test_baremetal_server_ops_wholedisk_image. I've made no changes from the example, except to checkout the devstack:stable/ussuri branch and a few minor workarounds to complete the stash.sh build.21:05
jj123I want to achieve a functional starting point that allows me to create and access baremetal instances. Is there a reference environment + script and/or CI recipe that is known to work? I'd prefer a local VM, but am flexible to use a different hypervisor, a cloud instance or physical server.21:06
*** bnemec has quit IRC21:11
jandersjj123 do you need full devstack, or would ironic-standalone suffice for your needs?21:11
jj123I want the whole devstack (or similar set of capabilities) to exercise all of the ironic / device features as they would work in an OpenStack deployment.21:13
jandersI'm focusing specifically on ironic, so bifrost works pretty well https://docs.openstack.org/bifrost/latest/ I don't have recent experience with devstack but perhaps others will be able to help21:14
jj123The scope of my development will likely be creating a redfish-compatible BMC for an in-house server appliance, a custom hardware type in ironic so I can manage it, and an application that wraps access to an OpenStack deployment that includes the devices, but also uses other OpenStack services.21:18
*** bnemec has joined #openstack-ironic21:18
openstackgerritJacob Anders proposed openstack/bifrost master: Auto-set ansible_python_interpreter when in venv  https://review.opendev.org/74233521:24
*** cdearborn has joined #openstack-ironic21:25
jj123I've got devstack (without ironic) working ok, and ironic standalone working ok. Perhaps it makes sense to try to get a non-ironic devstack to talk to a standalone ironic...21:28
openstackgerritJulia Kreger proposed openstack/ironic master: Force RAX hosts to run tinyipa  https://review.opendev.org/74250221:40
openstackgerritMerged openstack/ironic stable/stein: Pin the python-ibmcclient upper bounds due to six usage  https://review.opendev.org/74251221:42
TheJuliajj123: our published devstack configs are typically reliable... although thtey typically don't trigger tempest21:43
TheJuliaalso openstack baremetal node list might be more insightful as to the state of youre test machine21:43
TheJuliaonly a little bit gets surfaced to nova.21:43
*** ociuhandu has joined #openstack-ironic21:46
iurygregoryjj123, if you want standalone you can try http://paste.openstack.org/show/796224/21:47
*** rh-jlabarre has joined #openstack-ironic21:47
iurygregorywhat I normaly do is get the local.conf from the jobs set ERROR_ON_CLONE to false, and remove HOST_IP and SERVICE_HOST21:48
*** ociuhandu has quit IRC21:50
*** rh-jelabarre has quit IRC21:51
openstackgerritMerged openstack/ironic-python-agent master: Fix bootloader install issue with MDRAID  https://review.opendev.org/74033122:13
openstackgerritZygimantas Matonis proposed openstack/ironic-inspector master: WIP: adding condition_join to rules.  https://review.opendev.org/73179422:28
*** Lucas_Gray has joined #openstack-ironic22:32
openstackgerritJulia Kreger proposed openstack/ironic stable/stein: Allow retrying PXE boot if it takes too long  https://review.opendev.org/74231322:34
*** jj123 has quit IRC22:50
*** rcernin has joined #openstack-ironic22:52
openstackgerritSteve Baker proposed openstack/ironic-inspector master: Add public api support to basic auth  https://review.opendev.org/74230622:53
*** rcernin has quit IRC22:58
*** rcernin has joined #openstack-ironic23:01
*** rcernin has quit IRC23:04
*** rcernin has joined #openstack-ironic23:05
*** Lucas_Gray has quit IRC23:09
janderswith the patches fixing up ansible-collection and python interpreter paths I was able to deploy Bifrost on C8 seamlessly, zero errors on the way23:12
jandersi did notice a couple things that seemed unexpected though23:13
jandersafter deploying instances23:13
jandersare the "baremetal" nodes expected to have the IP addresses listed in the json inventory?23:14
janders(mine came up with some random ones)23:14
jandersand second question: are we using tinycore linux as IPA image?23:15
*** jj123 has joined #openstack-ironic23:16
TheJuliaDepends on settings but yes, tinycore does get used as one of the two ipa images we test with23:17
jandersthank you TheJulia23:17
TheJuliaI think ip addresses in the inventory are optional because that is also driven fromtest case, if it is dhcp only or not23:17
jandersnoted!23:18
janderswith standalone, is there a way of  figuring out what IPs got allocated without consoling into the VMs?23:18
jandersit feels like libvirt allocates random IPs and ironic has done it's job by the time it comes to that so my guess is no...23:19
jj123TheJulia Thanks. I added and ran tempest myself following the DevStack Quickstart (https://docs.openstack.org/ironic/latest/contributor/dev-quickstart.html), to troubleshoot. For the emulated baremetal node, `openstack baremetal node list` looked fine. For some reason the corresponding nova entry didn't match the tutorial... nova Status was ERROR,23:20
jj123Power State was NOSTATE, and no network listed.23:20
jj123iurygregory Thanks. I'll try that config.23:20
*** yolanda has quit IRC23:23
*** yolanda has joined #openstack-ironic23:24
*** diurnalist has quit IRC23:26
openstackgerritMerged openstack/ironic stable/ussuri: Fix the error message when reached max number of traits  https://review.opendev.org/73947723:43
*** ociuhandu has joined #openstack-ironic23:47
*** ociuhandu has quit IRC23:52

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!