| *** openstack has joined #openstack-ironic | 07:31 | |
| *** ChanServ sets mode: +o openstack | 07:31 | |
| *** rcernin_ has quit IRC | 07:34 | |
| *** bbezak has joined #openstack-ironic | 07:50 | |
| *** dtantsur|afk is now known as dtantsur | 07:55 | |
| dtantsur | morning ironic | 07:55 |
|---|---|---|
| *** ociuhandu has joined #openstack-ironic | 08:08 | |
| *** rcernin_ has joined #openstack-ironic | 08:16 | |
| janders | hi dtantsur | 08:21 |
| janders | I've got a couple bifrost questions, if you guys have time | 08:22 |
| janders | (just getting snippets ready) | 08:23 |
| janders | http://paste.openstack.org/show/796146/ | 08:24 |
| *** priteau has joined #openstack-ironic | 08:24 | |
| janders | with help from iurygregory I managed to get past the issue I was having yesterday. It seems it's to do with ANSIBLE_COLLECTIONS_PATHS variable not being set by the venv | 08:24 |
| *** derekh has joined #openstack-ironic | 08:25 | |
| janders | I wonder if I'm doing something wrong, or am I hitting a bug. Have you guys seen this issue? | 08:25 |
| *** rcernin_ has quit IRC | 08:26 | |
| * dtantsur is on a German class till the top of the hour | 08:26 | |
| gudrutis0 | Hi folks | 08:31 |
| rpittau | good morning ironic! o/ | 08:36 |
| janders | hey rpittau | 08:36 |
| rpittau | hey janders :) | 08:36 |
| janders | how are you going? | 08:36 |
| *** aedc has joined #openstack-ironic | 08:37 | |
| rpittau | all right all considered, lot of things going on at the moment | 08:41 |
| rpittau | how's it going for you? | 08:41 |
| janders | good, thank you. Hitting some bumps getting bifrost set up for VM testing but making steady progress | 08:42 |
| janders | thank you for your feedback regarding my doco change proposal - I did get a few things wrong there, hopefully all fixed noww | 08:42 |
| rpittau | janders: no problem, thanks for putting up that :) | 08:45 |
| janders | no worries, you were on the money | 08:46 |
| janders | I seem to have particularly bad luck for venv issues | 08:46 |
| janders | and sometimes when I hit one it seems to me things are broken while in fact it's just my confusion | 08:46 |
| *** rcernin_ has joined #openstack-ironic | 08:46 | |
| janders | speaking of venv issues - I have a fresh one I wanted to ask for assistance with | 08:47 |
| janders | http://paste.openstack.org/show/796147/ | 08:47 |
| rpittau | eheh shoot :) | 08:47 |
| janders | I have openstacksdk installed in venv, however if I run a playbook it complains about openstacksdk missing | 08:47 |
| janders | task is become=no so I'd hope the venv one will work but it doesn't | 08:48 |
| rpittau | mmmm | 08:54 |
| janders | I enabled rdo ussuri repo and installed openstacksdk via dnf as a test, it did make the issue go away | 08:56 |
| janders | so it is what it claims it is, not some weird cascade error | 08:56 |
| rpittau | I was going to suggest to install that systemeide, ok, so it doesn't really detect the virtualenv | 08:56 |
| janders | but I still wonder why the venv openstacksdk doesn't suffice from playbook perspective | 08:57 |
| janders | I had the same issue with jsonpatch earlier (this is in the main C8 repo so was an easy fix) | 08:57 |
| janders | ansible-playbook does point to /opt/stack/bifrost/bin/ansible-playbook so venv does work other than that | 08:58 |
| dtantsur | janders, iurygregory, we need ANSIBLE_COLLECTIONS_PATHS handled automatically or documented | 09:00 |
| dtantsur | (ideally the former) | 09:00 |
| dtantsur | janders: re openstacksdk: did you source the venv? I've seen this problem when venv was not sourced | 09:01 |
| dtantsur | (i.e. it's not enough to call /opt/stack/bifrost/bin/ansible-playbook) | 09:01 |
| janders | dtantsur yes I did | 09:01 |
| rpittau | dtantsur: it seems it was sourced looking at the paste | 09:01 |
| rpittau | janders: | 09:02 |
| rpittau | janders: wait, you install openstacksdk using --user | 09:02 |
| rpittau | not in the venv | 09:02 |
| dtantsur | Requirement already satisfied: openstacksdk in /opt/stack/bifrost/lib/python3.6/site-packages | 09:02 |
| dtantsur | --user is incorrect, but it should be fine anyway (we install the sdk as part of bifrost) | 09:03 |
| rpittau | true | 09:03 |
| janders | correct, I thought that was the right thing. So if I'm venv, I don't add --user, ACK. | 09:03 |
| dtantsur | janders: what if you do `-e ansible_python_interpreter=/opt/stack/bifrost/bin/python3`? | 09:03 |
| janders | let me reproduce it, 1min (gotta remove systemwide rpm) | 09:04 |
| rpittau | maybe it's ansible not detecting the virtualenv | 09:04 |
| openstackgerrit | Merged openstack/ironic master: Add wsme core types, remove WSME https://review.opendev.org/704490 | 09:05 |
| janders | dtantsur it works if `-e ansible_python_interpreter=/opt/stack/bifrost/bin/python3` is added | 09:06 |
| janders | this rings a bell actually | 09:06 |
| rpittau | so yeah, it's ansible not getting the venv | 09:07 |
| *** rcernin_ has quit IRC | 09:07 | |
| dtantsur | ansible's handling of venv is a mess :( | 09:10 |
| janders | I remember having to do this while ansible-ising venv setup for ansible in the past (may sound crazy I know) | 09:10 |
| janders | can't find the code anymore though, doesn't matter | 09:11 |
| janders | I wonder if we need to add this to doco (thinking where) | 09:11 |
| janders | I reckon if I hit this the users might as well | 09:12 |
| janders | https://docs.openstack.org/bifrost/latest/user/howto.html#enroll | 09:12 |
| janders | doco claims it's a fixed problem, but I am not convinced based on my experience | 09:13 |
| dtantsur | janders: what if we modify our playbooks/roles to set ansible_python_interpreter if in a venv? | 09:14 |
| rpittau | ^ IMHO this is how it should be | 09:15 |
| * janders is checking if we have ansible config file inside the repo | 09:16 | |
| janders | nope | 09:17 |
| janders | I think your suggestion is the way to go dtantsur rpittau | 09:17 |
| janders | would you like me to have a go at fixing this as well as the missing ansible collection paths? | 09:17 |
| janders | happy to | 09:17 |
| rpittau | janders: if you got the time, I will gladly review :) | 09:18 |
| janders | cool! I've been looking for stuff to work on last week (and these issues blocked working on the cleaning step ordering RFE) so I don't see why not | 09:19 |
| openstackgerrit | Dmitry Tantsur proposed openstack/bifrost master: Support HTTP basic auth and switch bifrost-cli to it https://review.opendev.org/741964 | 09:19 |
| openstackgerrit | Dmitry Tantsur proposed openstack/bifrost master: Support HTTP basic auth and switch bifrost-cli to it https://review.opendev.org/741964 | 09:20 |
| *** k_mouza has joined #openstack-ironic | 09:21 | |
| openstackgerrit | Riccardo Pittau proposed openstack/ironic-python-agent master: More refactoring of the image module https://review.opendev.org/742110 | 09:27 |
| *** rcernin_ has joined #openstack-ironic | 09:41 | |
| openstackgerrit | Verification of a change to openstack/bifrost failed: Expand advanced install instructions (path info) https://review.opendev.org/741611 | 09:42 |
| *** k_mouza has quit IRC | 09:53 | |
| *** rcernin_ has quit IRC | 09:55 | |
| *** k_mouza has joined #openstack-ironic | 10:01 | |
| *** ricolin has quit IRC | 10:18 | |
| janders | what happened with https://review.opendev.org/741611 - does the Zuul fail mean the document changed in the meantime? | 10:20 |
| patchbot | patch 741611 - bifrost - Expand advanced install instructions (path info) - 7 patch sets | 10:20 |
| rpittau | janders: yeah, try a rebase on latest master | 10:20 |
| janders | ok | 10:20 |
| *** aedc_ has joined #openstack-ironic | 10:22 | |
| *** aedc has quit IRC | 10:22 | |
| *** rcernin_ has joined #openstack-ironic | 10:28 | |
| *** Lucas_Gray has joined #openstack-ironic | 10:33 | |
| *** vmud213 has joined #openstack-ironic | 10:34 | |
| openstackgerrit | Jacob Anders proposed openstack/bifrost master: Expand advanced install instructions https://review.opendev.org/741611 | 10:45 |
| vmud213 | Hello Ironic ! | 10:45 |
| vmud213 | Good morning and Good evening .. | 10:46 |
| openstackgerrit | Jacob Anders proposed openstack/bifrost master: Expand advanced install instructions https://review.opendev.org/741611 | 10:47 |
| janders | ok I think this revision should be okay | 10:48 |
| janders | now back to venv & ansible challenges :) | 10:49 |
| openstackgerrit | Merged openstack/ironic-python-agent-builder master: Add Tripleo IPA build image job in zuul layout https://review.opendev.org/741917 | 10:53 |
| janders | thinking how to add setting Ansible collections path to bifrost venv | 11:01 |
| janders | I see we do it here: | 11:01 |
| janders | https://github.com/openstack/bifrost/blob/master/scripts/install-deps.sh#L96 | 11:01 |
| janders | would it be okay if I just append the export to the end of activate script? | 11:01 |
| janders | kind of braindead but should just work... | 11:02 |
| janders | what do you think? | 11:02 |
| iurygregory | hello everyone o/ | 11:14 |
| janders | g'day iurygregory | 11:14 |
| janders | thanks heaps for your help with ansible collections yesterday, your hints got me out of a rut | 11:15 |
| janders | fixing a related bug now | 11:15 |
| iurygregory | np o/ | 11:15 |
| dtantsur | I'm not sure how I feel about updating the activate script(s) | 11:16 |
| janders | what would be an alternative? | 11:16 |
| iurygregory | dtantsur, it's documented and it was handle automatic O.o | 11:16 |
| dtantsur | Maybe we should set the variable for our roles ourself (inside the roles) and document what should be done | 11:17 |
| dtantsur | janders: the problem is, not everyone will necessary use 'activate' | 11:17 |
| janders | ok happy to try that | 11:17 |
| iurygregory | the CI has no problem with the collections and when I tested locally it was working | 11:17 |
| dtantsur | not everyone will even use a shell, imagine someone calling ansible-playbook from a python/go/whatever program | 11:17 |
| janders | true | 11:17 |
| dtantsur | we need to tell them what to do in the very base case | 11:17 |
| janders | iurygregory was the variable being automatically set in your setup or was it working using some different mechanism? | 11:19 |
| iurygregory | janders, for me it was automatically set | 11:19 |
| iurygregory | https://docs.openstack.org/bifrost/latest/install/index.html#dependencies | 11:19 |
| janders | right! I wonder if it has to do with ansible.cfg contents | 11:20 |
| * janders is checking his ansible.cfg | 11:21 | |
| dtantsur | stevebaker: did some local testing, the following are wrongly authenticated: / /v1 /v1/continue in ironic-inspector, /v1/heartbeat in ironic: http://paste.openstack.org/show/796150/ | 11:22 |
| openstackgerrit | Dmitry Tantsur proposed openstack/bifrost master: Support HTTP basic auth and switch bifrost-cli to it https://review.opendev.org/741964 | 11:24 |
| openstackgerrit | vinay kumar muddu proposed openstack/ironic master: Decouple the ISO creation logic from redfish https://review.opendev.org/739174 | 11:32 |
| *** vmud213 has quit IRC | 11:40 | |
| *** sshnaidm|afk is now known as sshnaidm | 11:41 | |
| *** Lucas_Gray has quit IRC | 11:41 | |
| dtantsur | if anyone has a minute: https://review.opendev.org/#/c/740246/ | 11:43 |
| patchbot | patch 740246 - bifrost - Support extra-vars in bifrost-cli - 6 patch sets | 11:43 |
| *** Lucas_Gray has joined #openstack-ironic | 11:44 | |
| *** iurygregory has quit IRC | 11:49 | |
| *** tkajinam has quit IRC | 11:56 | |
| *** iurygregory has joined #openstack-ironic | 11:59 | |
| *** ociuhandu_ has joined #openstack-ironic | 12:04 | |
| openstackgerrit | Merged openstack/bifrost master: Update to cirros 0.5.1 https://review.opendev.org/741923 | 12:05 |
| *** ociuhandu has quit IRC | 12:07 | |
| *** rh-jelabarre has joined #openstack-ironic | 12:09 | |
| rpittau | dtantsur: wondering if a release note is needed for the extra-vars ? | 12:11 |
| *** ricolin has joined #openstack-ironic | 12:15 | |
| *** ociuhandu_ has quit IRC | 12:15 | |
| *** ociuhandu has joined #openstack-ironic | 12:16 | |
| *** Goneri has joined #openstack-ironic | 12:17 | |
| *** k_mouza has quit IRC | 12:18 | |
| openstackgerrit | Verification of a change to openstack/ironic-inspector failed: Do not try to set local_gb to -1 when the matched root device size is 0 https://review.opendev.org/741937 | 12:26 |
| *** diurnalist has joined #openstack-ironic | 12:33 | |
| janders | regarding Ansible Collection I have one more idea for a potential fix | 12:34 |
| janders | symlinking the collections install directory to the bifrost playbooks directory can remove the need of setting environment variable | 12:35 |
| janders | I suppose that would be more cross-platform than tweaking activate script | 12:35 |
| janders | dtantsur what do you think about this approach? | 12:35 |
| janders | I'm still trying to get to the bottom of why the var isn't set right now but I wonder if this idea would be a fix material or not | 12:36 |
| *** diurnalist has quit IRC | 12:37 | |
| dtantsur | rpittau: likely, I can follow-up | 12:39 |
| rpittau | dtantsur: okay | 12:39 |
| dtantsur | janders: thinking.. | 12:39 |
| dtantsur | janders: it may work, just needs to be documented (and added to .gitignore :) | 12:39 |
| janders | I was thinking creating the symlink from env-setup.sh | 12:40 |
| janders | cause that's where collections are getting installed into the venv (if I understand correctly) | 12:41 |
| janders | does this make sense? | 12:41 |
| dtantsur | janders: yep, at least at first glance | 12:41 |
| janders | should I treat this and the ansible python interpreter as two separate change proposals, or should I combine the two? | 12:42 |
| janders | my gut feel = separate | 12:43 |
| rpittau | janders: separate sounds right, thanks | 12:43 |
| janders | cool, thank you | 12:43 |
| dtantsur | whatever works for you | 12:43 |
| openstackgerrit | Dmitry Tantsur proposed openstack/bifrost master: Support extra-vars in bifrost-cli https://review.opendev.org/740246 | 12:50 |
| dtantsur | rpittau: I've realized it was only the 1st +2 and the next patch won't pass ^^ | 12:50 |
| dtantsur | so added now | 12:50 |
| rpittau | ah yeah | 12:50 |
| openstackgerrit | Riccardo Pittau proposed openstack/bifrost master: Make ansible ask for sudo password https://review.opendev.org/742175 | 12:52 |
| TheJulia | brraaaaiiinnnssss | 12:53 |
| dtantsur | morning TheJulia | 12:53 |
| rpittau | thanks, I'd like one :) | 12:53 |
| TheJulia | good morning everyone | 12:53 |
| dtantsur | :) | 12:53 |
| rpittau | good morning TheJulia :) | 12:53 |
| TheJulia | is everyone ready for another exciting OpenDev discussion!? | 12:53 |
| janders | g'day TheJulia o/ | 12:53 |
| rpittau | I missed the one yesterday, how was it ? | 12:53 |
| dtantsur | TheJulia: I have a visit from the house management company, so will be randomly on and off | 12:53 |
| *** rcernin_ has quit IRC | 12:54 | |
| * arne_wiebalck prepared coffee and is hence totally ready | 12:54 | |
| dking | TheJulia: Actually, I never got my email for this one with the details, even though I registered. | 12:54 |
| rpittau | mmmmm brain + coffee | 12:54 |
| dking | Do you know where we could get that? | 12:54 |
| arne_wiebalck | the combo of success | 12:54 |
| TheJulia | dtantsur: ack | 12:55 |
| TheJulia | dking: eek | 12:55 |
| TheJulia | I can get you the link | 12:55 |
| TheJulia | https://zoom.us/j/97532742477?pwd=b2ZydG50eXBFekdFMGNaTVNzVUFYZz09#success | 12:56 |
| *** ricolin has quit IRC | 12:56 | |
| *** tzumainn has joined #openstack-ironic | 12:58 | |
| rpittau | the music is very relazing | 12:58 |
| rpittau | or relaxing | 12:58 |
| TheJulia | ++ | 13:00 |
| *** rloo has joined #openstack-ironic | 13:02 | |
| openstackgerrit | Jacob Anders proposed openstack/bifrost master: Replace ANSIBLE_COLLECTIONS_PATHS var with symlink https://review.opendev.org/742190 | 13:24 |
| openstackgerrit | Jacob Anders proposed openstack/bifrost master: Replace ANSIBLE_COLLECTIONS_PATHS var with symlink https://review.opendev.org/742192 | 13:37 |
| *** bnemec has joined #openstack-ironic | 13:38 | |
| janders | wrapping up for the night, see you tomorrow o/ | 13:39 |
| *** tenbrae has joined #openstack-ironic | 13:47 | |
| *** bfournie has joined #openstack-ironic | 13:48 | |
| *** bfournie has quit IRC | 14:03 | |
| *** yaawang has quit IRC | 14:05 | |
| *** yaawang has joined #openstack-ironic | 14:05 | |
| openstackgerrit | Riccardo Pittau proposed openstack/bifrost master: Consolidate env-setup https://review.opendev.org/742201 | 14:09 |
| openstackgerrit | Dmitry Tantsur proposed openstack/bifrost master: Use random passwords by default https://review.opendev.org/742203 | 14:15 |
| openstackgerrit | Dmitry Tantsur proposed openstack/bifrost master: Use random passwords by default https://review.opendev.org/742203 | 14:16 |
| openstackgerrit | Dmitry Tantsur proposed openstack/bifrost master: Support HTTP basic auth and switch bifrost-cli to it https://review.opendev.org/741964 | 14:17 |
| *** ricolin has joined #openstack-ironic | 14:21 | |
| *** markmcclain has quit IRC | 14:23 | |
| *** markmcclain has joined #openstack-ironic | 14:24 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/bifrost master: Use random passwords by default https://review.opendev.org/742203 | 14:30 |
| *** diurnalist has joined #openstack-ironic | 14:32 | |
| *** abdysn has quit IRC | 14:36 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/bifrost master: Support HTTP basic auth and switch bifrost-cli to it https://review.opendev.org/741964 | 14:39 |
| *** k_mouza has joined #openstack-ironic | 14:45 | |
| openstackgerrit | Merged openstack/ironic master: Remove File type https://review.opendev.org/741862 | 14:45 |
| openstackgerrit | Merged openstack/ironic master: Remove Link type https://review.opendev.org/741863 | 14:45 |
| openstackgerrit | Merged openstack/ironic master: Stop running test_schedule_to_all_nodes in the multinode job https://review.opendev.org/741884 | 14:45 |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic stable/ussuri: Stop running test_schedule_to_all_nodes in the multinode job https://review.opendev.org/742214 | 14:51 |
| *** rcernin_ has joined #openstack-ironic | 14:52 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic stable/train: Stop running test_schedule_to_all_nodes in the multinode job https://review.opendev.org/742215 | 14:53 |
| *** rcernin_ has quit IRC | 14:57 | |
| openstackgerrit | Shivanand Tendulker proposed openstack/ironic master: Adds raid validation for in-band AgentRAID deploy step https://review.opendev.org/742217 | 15:05 |
| *** stendulker has quit IRC | 15:09 | |
| openstackgerrit | Verification of a change to openstack/ironic failed: Auto extend the timeout for RAX hosts https://review.opendev.org/741853 | 15:16 |
| openstackgerrit | Shivanand Tendulker proposed openstack/ironic master: Adds raid validation for in-band AgentRAID deploy step https://review.opendev.org/742217 | 15:35 |
| *** aedc_ has quit IRC | 15:38 | |
| *** gyee has joined #openstack-ironic | 15:41 | |
| *** yolanda has quit IRC | 15:42 | |
| *** yolanda has joined #openstack-ironic | 15:44 | |
| *** stendulker_ has joined #openstack-ironic | 15:53 | |
| stendulker_ | Hi dtantsur | 15:54 |
| dtantsur | o/ | 15:54 |
| stendulker_ | This is regarding https://review.opendev.org/#/c/740893 | 15:54 |
| patchbot | patch 740893 - ironic - Adds support SUM based firmware update as deploy step - 1 patch set | 15:54 |
| stendulker_ | Apart from software raid this gets used in Ilo RAID as well | 15:55 |
| dtantsur | you probably mean another patch | 15:55 |
| stendulker_ | Do you suggest to remove volume related args in the AgentRAID / | 15:55 |
| dtantsur | stendulker_: okay, this makes sense, but then you need to use the arguments you add | 15:56 |
| stendulker_ | yes https://review.opendev.org/#/c/742217/2 | 15:56 |
| patchbot | patch 742217 - ironic - Adds raid validation for in-band AgentRAID deploy ... - 2 patch sets | 15:56 |
| dtantsur | yep, you don't pass the arguments anywhere in https://review.opendev.org/#/c/742217/2/ironic/drivers/modules/agent.py | 15:56 |
| stendulker_ | Yes, I understand that... It was a quick copy paste :( | 15:56 |
| patchbot | patch 742217 - ironic - Adds raid validation for in-band AgentRAID deploy ... - 2 patch sets | 15:56 |
| stendulker_ | Will add the argsinfo | 15:57 |
| dtantsur | you need something like raid.filter_target_raid_config I guess? | 15:57 |
| dtantsur | thinking of it, why are these arguments needed? | 15:57 |
| dtantsur | stendulker_: a user provides the raid_config, why cannot they just provide only the bits they need? | 15:58 |
| dtantsur | not a hard objection, just wondering | 15:58 |
| stendulker_ | yes I will update the filter_target_raid_config | 15:58 |
| stendulker_ | dtantsur: I agree with you. But in the base method we are supporting it and I think that is how implemented in DRAC as well, I suppose | 15:59 |
| TheJulia | Okay, Opendev day #2 is a wrap | 15:59 |
| *** ricolin has quit IRC | 16:02 | |
| stendulker_ | dtantsur: I think it makes sense to drop create_root_volume and create_nonroot_volumes | 16:03 |
| stendulker_ | Will post the patch shortly. | 16:04 |
| *** k_mouza has quit IRC | 16:04 | |
| *** k_mouza has joined #openstack-ironic | 16:05 | |
| *** strigazi has joined #openstack-ironic | 16:06 | |
| *** diurnalist has quit IRC | 16:10 | |
| *** bfournie has joined #openstack-ironic | 16:12 | |
| *** ociuhandu_ has joined #openstack-ironic | 16:17 | |
| openstackgerrit | Shivanand Tendulker proposed openstack/ironic master: Adds raid validation for in-band AgentRAID deploy step https://review.opendev.org/742217 | 16:18 |
| *** k_mouza has quit IRC | 16:20 | |
| *** ociuhandu has quit IRC | 16:20 | |
| openstackgerrit | Riccardo Pittau proposed openstack/sushy master: Update system schema to 1.10 https://review.opendev.org/742236 | 16:21 |
| *** ociuhandu_ has quit IRC | 16:21 | |
| TheJulia | Well this is a new one | 16:22 |
| TheJulia | https://www.irccloud.com/pastebin/pvmCchsR/ | 16:22 |
| rpittau | oO | 16:24 |
| dking | I have a couple questions about node cleaning. First, on https://docs.openstack.org/ironic/latest/admin/cleaning.html, it seems to list cleaning steps, but only lists the specific steps for the Management Interfaces. Is there a list of cleaning steps for Deploy, etc.? | 16:28 |
| *** priteau has quit IRC | 16:28 | |
| *** k_mouza has joined #openstack-ironic | 16:28 | |
| TheJulia | I wonder if we broke API version negotiation :\ | 16:35 |
| TheJulia | I have to take my cat to the vet shortly, but I can dig into it afterwards | 16:35 |
| TheJulia | https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_691/727480/7/gate/ironic-tempest-ipa-wholedisk-direct-tinyipa-multinode/691d51e/testr_results.html | 16:35 |
| *** diurnalist has joined #openstack-ironic | 16:38 | |
| TheJulia | dking: so, there are steps for deploy but we've also not ever been able to gain consensus as to how to programmatically present such data because of the dynamic nature combined with the chicken/egg problem that exists with such. Mainly they are part of the deployment process in newer ironic BUT! by default the clean steps available by default are erase_devices and erase_devices_metadata | 16:38 |
| *** bfournie has quit IRC | 16:42 | |
| dking | TheJulia: Thanks for the answer. So, right now, other than steps done by Management Interfaces, I should just count on erase_devices and erase_devices_metadata ? | 16:43 |
| TheJulia | dking: yes, now if you have other steps in a hardware manager, I believe they all become "deploy" interface backed | 16:45 |
| dking | Another question, is there a way to hook to an external script? For instance, I want to be able to update the BMC and BIOS firmware using redfish, which doesn't seem to be an option listed for the interface. But if I could somehow trigger an external script, it would make it easier for me to manually add that to the cleaning process. | 16:45 |
| TheJulia | the management ones are bmc/driver specific through the management interface in the hardware type | 16:45 |
| *** k_mouza has quit IRC | 16:45 | |
| TheJulia | an external script running on the conductor? | 16:46 |
| TheJulia | at present, no. I think the security risk there would really prevent it.... although we could likely come to agreement on local files only | 16:46 |
| TheJulia | and it really wouldn't be much code I think | 16:46 |
| *** JamesBenson has joined #openstack-ironic | 16:47 | |
| dking | I'm really looking for anything that I can do to trigger that sort of work. I'm assuming that it isn't included in the redfish driver because the option isn't universal enough, but that would be best. But other options would be things like having an option to run a shell script, or perhaps even send an API request. | 16:48 |
| TheJulia | Automated? every time cleaning is run? Generally we've recommended using a hardware manager in the agent because older BMCs were incapible of flashing all devices | 16:52 |
| TheJulia | I think disk backplanes are the only thing that are not supported by most vendors these days | 16:52 |
| TheJulia | Although some may have gotten that going | 16:52 |
| TheJulia | I guess it also depends on the vendor, because there are some differences between vendors even with redfish, aiui | 16:53 |
| dking | Yeah. Even currently, I'm having trouble with the implementation from our vendor. However, I have a ticket in for that. | 16:54 |
| TheJulia | so I guess kind of goes back to the manual or automatic | 16:56 |
| dking | The purpose for the update is the example that Jadon gave in the discussion earlier. If an attacker gains access to a server, they can modify the firmware, which then gets passed back into the pool when they are finished with it. So, to be clean, we need the firmware updated between each deployment. | 16:56 |
| *** stendulker_ has quit IRC | 16:56 | |
| *** tenbrae has quit IRC | 16:56 | |
| dking | Ideally, I'd prefer if that could all happen anytime Ironic cleaning is triggered (going into available). Otherwise, I have to write a system in between that takes calls to remove a system, and then have that manually do the cleaning, and maybe have to find a way to flag the servers in Ironic so that we know they're being manually cleaned. | 16:58 |
| *** derekh has quit IRC | 17:01 | |
| *** priteau has joined #openstack-ironic | 17:01 | |
| TheJulia | yeah, so automatic | 17:03 |
| TheJulia | hmm | 17:03 |
| TheJulia | Ideally this is the purpose of hardware managers | 17:03 |
| TheJulia | but doing it out of band is... different | 17:03 |
| * TheJulia wonders if driver_info could be populated with specific fields that could be read | 17:04 | |
| * TheJulia needs to take the elder cat to the vet, bbl | 17:04 | |
| *** yolanda has quit IRC | 17:10 | |
| *** yolanda has joined #openstack-ironic | 17:13 | |
| *** dtantsur is now known as dtantsur|afk | 17:15 | |
| dtantsur|afk | o/ | 17:15 |
| *** priteau has quit IRC | 17:21 | |
| *** klarin has joined #openstack-ironic | 17:27 | |
| *** Lucas_Gray has quit IRC | 17:40 | |
| klarin | Hi guys! We're on Train/CentOS7 and have an issue on BM instance deployment via nova: info in files under /var/lib/neutron/dhcp is missing for some nodes, and servers cannot PXE boot | 17:44 |
| klarin | For some nodes though /var/lib/neutron/dhcp becomes populated and nodes successfully boot. Working and non-working nodes may be even assigned to the same conductor | 17:44 |
| klarin | What might be the cause? | 17:45 |
| openstackgerrit | Merged openstack/ironic-inspector stable/train: Do not try to set local_gb to -1 when the matched root device size is 0 https://review.opendev.org/741937 | 17:50 |
| openstackgerrit | Verification of a change to openstack/ironic failed: Use native oslo.concurrency execution timeout in ipmitool https://review.opendev.org/738591 | 17:50 |
| arne_wiebalck | Bye everyone o/ | 17:51 |
| *** diurnalist has quit IRC | 18:24 | |
| *** diurnalist has joined #openstack-ironic | 18:35 | |
| *** diurnalist has quit IRC | 18:39 | |
| *** rloo has quit IRC | 18:42 | |
| *** rloo has joined #openstack-ironic | 18:43 | |
| *** ociuhandu has joined #openstack-ironic | 18:48 | |
| *** diurnalist has joined #openstack-ironic | 18:49 | |
| *** ociuhandu has quit IRC | 18:53 | |
| *** ociuhandu has joined #openstack-ironic | 18:56 | |
| TheJulia | klarin: anything in the neutron logs? I think there was a neutron bug around then that caused dhcp updates to fail under certian conditions | 19:02 |
| TheJulia | klarin: Which reminds me, there is also something on centos with dhcp.... | 19:05 |
| TheJulia | Maybe one of my fellow redhat folks remembers | 19:05 |
| TheJulia | unfortunately I don't remember | 19:05 |
| klarin | TheJulia: I apologize, I was wrong, DHCP must be fine, since manually booted image gets proper hostname and address | 19:06 |
| TheJulia | klarin: so funny thing about this bug | 19:07 |
| TheJulia | it is like a delay and dnsmasq doesn't pickup the configuration in time | 19:08 |
| TheJulia | or maybe not at all until another change occurs | 19:08 |
| klarin | if I boot from local disk with centos, node receives the address, but then if I reboot PXE interface does not get it, so must be something with firmware or missing dhcp opts | 19:12 |
| klarin | not sure why it works on other nodes, as they are all identical | 19:12 |
| TheJulia | klarin: are the dhcp ops present on the port configuration and/or the actual dnsmasq config files? | 19:13 |
| TheJulia | klarin: I ask because they are separate operations/interactions | 19:13 |
| klarin | tag:port-<...>,150,192.168.<...>tag:port-<...>,210,/tftpboot/tag:port-<...>,66,192.168.<...>tag:port-<...>,67,EFI/bootx64.efitag:port-<...>,option:server-ip-address,192.168.<...> | 19:14 |
| klarin | sorry, no line breaks | 19:14 |
| TheJulia | so the question is... if you hup dnsmasq or touch that file with another change... will the machine pxe boot with those options? | 19:15 |
| TheJulia | another thought, is the network interface in use actually enabled to pxe boot? | 19:15 |
| klarin | not sure what options to add there, but manually loading EFI/bootx64.efi with tftp command works | 19:17 |
| klarin | and yes, PXE prompt for specific interface shows up | 19:17 |
| klarin | I'll try firmware update or boot using another interface | 19:18 |
| klarin | TheJulia: thank you so much for your response! | 19:18 |
| TheJulia | klarin: has it bene over an hour? | 19:20 |
| TheJulia | in other words, has ironic purged the files from disk? | 19:21 |
| klarin | dhcp data gets removed properly, I was probably looking in the wrong direction, expecting to see entries show up in leases file | 19:22 |
| openstackgerrit | Julia Kreger proposed openstack/ironic master: Extend base build timeouts https://review.opendev.org/741818 | 19:25 |
| TheJulia | Ahh yeah, your basically saying when the bmc is asked to network boot, it just doesn't work | 19:25 |
| TheJulia | and I guess your manually asking from uefi shell? | 19:26 |
| klarin | yes, it's probably firmware that does not work | 19:29 |
| klarin | did not try anything from uefi shell | 19:30 |
| TheJulia | could the default pxe interface in the firmware config be wrong? | 19:30 |
| klarin | no, it actually tries all interfaces one after another, and also displays its MAC | 19:31 |
| TheJulia | so identically configured machines? | 19:33 |
| klarin | yes, there are 4 intel interfaces: 2 built-in 10G, 2 add-in 40G, and at least 2 nodes boot OK from 40G | 19:36 |
| klarin | bb thank you again | 19:51 |
| *** klarin has quit IRC | 19:51 | |
| openstackgerrit | Julia Kreger proposed openstack/ironic-python-agent-builder master: Strip some excess from the centos dib image https://review.opendev.org/741819 | 19:58 |
| openstackgerrit | Julia Kreger proposed openstack/ironic-inspector master: Respond so Apache HTTPd doesn't think the request failed https://review.opendev.org/740344 | 20:20 |
| openstackgerrit | Julia Kreger proposed openstack/ironic master: Guard conductor from consuming all of the ram https://review.opendev.org/726483 | 20:30 |
| *** lbotell has quit IRC | 20:49 | |
| *** JamesBenson has quit IRC | 21:12 | |
| openstackgerrit | Merged openstack/sushy-cli master: Set min version of tox to 3.2.1 https://review.opendev.org/741189 | 21:32 |
| *** tkaprol has joined #openstack-ironic | 21:37 | |
| openstackgerrit | Merged openstack/ironic-inspector master: Set min version of tox to 3.2.1 https://review.opendev.org/741186 | 21:42 |
| *** tkaprol has quit IRC | 21:45 | |
| openstackgerrit | Verification of a change to openstack/ironic failed: Stop running test_schedule_to_all_nodes in the multinode job https://review.opendev.org/742214 | 21:45 |
| *** Goneri has quit IRC | 21:50 | |
| openstackgerrit | Merged openstack/bifrost master: Support extra-vars in bifrost-cli https://review.opendev.org/740246 | 21:56 |
| *** rcernin_ has joined #openstack-ironic | 22:22 | |
| *** rcernin_ has quit IRC | 22:22 | |
| janders | good morning / evening Ironic! o/ | 22:26 |
| *** rcernin has joined #openstack-ironic | 22:34 | |
| *** bnemec has quit IRC | 22:47 | |
| openstackgerrit | Steve Baker proposed openstack/ironic-inspector master: Add public api support to basic auth https://review.opendev.org/742306 | 22:53 |
| *** tkajinam has joined #openstack-ironic | 22:55 | |
| *** Goneri has joined #openstack-ironic | 22:59 | |
| TheJulia | good morning janders | 23:12 |
| TheJulia | So question folks. Dmitry wrote support in Train to bounce the node if it had not checked-in after PXE for a while. I'm wondering if we should go through the process of backporting that feature as one of those "generally this will make lives better" sort of thing | 23:13 |
| janders | TheJulia just to make sure I'm undestanding this correctly: don't we have something similar in place already (or is it introspection only)? | 23:18 |
| openstackgerrit | Steve Baker proposed openstack/bifrost master: Support HTTP basic auth and switch bifrost-cli to it https://review.opendev.org/741964 | 23:18 |
| TheJulia | there are two levels | 23:18 |
| janders | I remember my nodes recovering from intermittent portfast issues on their own | 23:18 |
| TheJulia | there is the rendered template "please retry" | 23:18 |
| TheJulia | and then there is a "kick the node again" if it doesn't check-in with-in x amount of time | 23:19 |
| janders | right! | 23:19 |
| janders | in general this is a useful feature for sure. Is there a config option to match allowing the operator to disable it? | 23:21 |
| TheJulia | yes, and it is disabled by default | 23:21 |
| janders | 9/10+ I think it will help. Just thinking of that rare case where the operator is troubleshooting something so doesn't want the node to reboot in the middle - or if the environment has some rare hard to reproduce issue and the operator wants to know when things go wrong, as opposed to having the system auto-recover | 23:22 |
| janders | I think it is useful, might prevent a fair bit of support tickets from being raised. | 23:22 |
| janders | as much as pxe booting and building nodes should work 100/100 my experience is it rarely does and often this is due to hardware being non-deterministic | 23:23 |
| openstackgerrit | Julia Kreger proposed openstack/ironic stable/stein: Allow retrying PXE boot if it takes too long https://review.opendev.org/742313 | 23:23 |
| janders | this is a good way to relieve the pain caused by this and it will likely be appreciated by more conservative folk running older long-life releases | 23:24 |
| TheJulia | hmm. to go beyond rocky is going to require manually doing so | 23:24 |
| TheJulia | maybe something for tomorrow | 23:24 |
| TheJulia | Yeah, I'm actually looking at an internal ticket thinking "if this was present, it would have likely prevented a lot of this" | 23:25 |
| janders | this makes me appreciate more why we are putting effort into vmedia-based builds :) | 23:28 |
| TheJulia | ++ | 23:29 |
| *** Goneri has quit IRC | 23:36 | |
| *** iurygregory has quit IRC | 23:42 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!