| *** ildikov has joined #openstack-ironic | 00:01 | |
| *** pas-ha has joined #openstack-ironic | 00:02 | |
| *** sri_ has joined #openstack-ironic | 00:02 | |
| *** rpioso has quit IRC | 00:09 | |
| *** rpioso has joined #openstack-ironic | 00:11 | |
| *** sri_ has quit IRC | 00:16 | |
| *** sri_ has joined #openstack-ironic | 00:18 | |
| *** rpioso has quit IRC | 00:18 | |
| *** rpioso has joined #openstack-ironic | 00:19 | |
| *** rloo has quit IRC | 00:37 | |
| *** Goneri has joined #openstack-ironic | 01:01 | |
| *** cdearborn has quit IRC | 01:03 | |
| *** zbitter is now known as zaneb | 01:06 | |
| *** Goneri has quit IRC | 01:14 | |
| *** ijw has quit IRC | 01:45 | |
| *** rh-jelabarre has quit IRC | 01:54 | |
| *** gyee has quit IRC | 01:55 | |
| *** Qianbiao has joined #openstack-ironic | 02:33 | |
| *** rcernin has quit IRC | 02:37 | |
| *** mkrai has joined #openstack-ironic | 02:54 | |
| *** rcernin has joined #openstack-ironic | 02:58 | |
| *** rcernin has quit IRC | 03:04 | |
| *** rcernin has joined #openstack-ironic | 03:04 | |
| *** tzumainn has quit IRC | 03:18 | |
| *** Qianbiao has quit IRC | 04:11 | |
| *** hjensas has joined #openstack-ironic | 04:39 | |
| *** mkrai has quit IRC | 05:22 | |
| *** janders has quit IRC | 05:30 | |
| *** janders has joined #openstack-ironic | 05:30 | |
| *** Qianbiao has joined #openstack-ironic | 05:32 | |
| *** Qianbiao has joined #openstack-ironic | 05:32 | |
| *** mkrai has joined #openstack-ironic | 05:32 | |
| *** yolanda has quit IRC | 06:38 | |
| *** weshay|ruck has quit IRC | 06:38 | |
| *** bdodd has quit IRC | 06:38 | |
| *** akahat|rover has quit IRC | 06:38 | |
| *** ddurst has quit IRC | 06:38 | |
| *** rnoriega_ has quit IRC | 06:38 | |
| *** yolanda has joined #openstack-ironic | 06:43 | |
| *** weshay|ruck has joined #openstack-ironic | 06:43 | |
| *** bdodd has joined #openstack-ironic | 06:43 | |
| *** akahat|rover has joined #openstack-ironic | 06:43 | |
| *** ddurst has joined #openstack-ironic | 06:43 | |
| *** rnoriega_ has joined #openstack-ironic | 06:43 | |
| *** ociuhandu has quit IRC | 06:56 | |
| *** ociuhandu has joined #openstack-ironic | 06:57 | |
| *** belmoreira has joined #openstack-ironic | 06:57 | |
| *** jtomasek has joined #openstack-ironic | 06:59 | |
| *** ricolin has quit IRC | 06:59 | |
| *** ociuhandu has quit IRC | 07:01 | |
| *** mkrai has quit IRC | 07:03 | |
| iurygregory | good morning Ironic | 07:08 |
|---|---|---|
| iurygregory | happy friday | 07:08 |
| *** rcernin has quit IRC | 07:22 | |
| *** Lucas_Gray has joined #openstack-ironic | 07:25 | |
| *** dtantsur|afk is now known as dtantsur | 07:31 | |
| dtantsur | morning ironic | 07:31 |
| iurygregory | morning dtantsur | 07:33 |
| dtantsur | Could not find a version that satisfies the requirement bcrypt>=3.1.3 (from ironic-lib>=4.1.0->ironic_python_agent) (from versions: ) | 07:34 |
| dtantsur | rpittau: is it something your ipa-builder patches handles ^^? | 07:34 |
| *** dougsz has joined #openstack-ironic | 07:36 | |
| *** mgoddard has joined #openstack-ironic | 07:36 | |
| dtantsur | RAMDISK: gzip image found at block 0 | 07:36 |
| dtantsur | RAMDISK: EOF while reading compressed data | 07:37 |
| dtantsur | meh wut | 07:37 |
| *** priteau has joined #openstack-ironic | 07:40 | |
| iurygregory | dtantsur, I was thinking if we don't need some unit test to cover https://review.opendev.org/#/c/747144/ | 07:43 |
| patchbot | patch 747144 - ironic - Fix network_data path for dhcpless deployments - 1 patch set | 07:43 |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic-python-agent master: Eventlet should be monkey patched as early as possible https://review.opendev.org/746774 | 07:44 |
| rpittau | dtantsur: re bcrypt yes | 07:45 |
| dtantsur | iurygregory: it changes a constant, dunno. but it's suspicious that no tests had to be updated. | 07:45 |
| rpittau | good morning ironic! Happy Friday! o/ | 07:45 |
| dtantsur | hey-hey rpittau | 07:45 |
| dtantsur | rpittau: and the bcrypt patch is blocked by the ramdisk unpacking problem, sweet | 07:46 |
| dtantsur | any clues on that already? | 07:46 |
| rpittau | \o/ | 07:46 |
| rpittau | just saw it | 07:46 |
| iurygregory | morning rpittau o/ | 07:46 |
| rpittau | back from a painting session :) | 07:46 |
| iurygregory | I hope you enjoyed =) | 07:46 |
| rpittau | almost done, tomorrow last day :) | 07:47 |
| iurygregory | \o/ | 07:47 |
| janders | good morning / evening Ironic o/ | 07:48 |
| rpittau | interesting that everything was working perfectly until 3 days ago | 07:48 |
| rpittau | hey janders :) | 07:48 |
| iurygregory | morning janders o/ | 07:49 |
| rpittau | let's check some history | 07:49 |
| *** dougsz has quit IRC | 07:50 | |
| *** Lucas_Gray has quit IRC | 07:50 | |
| *** hjensas has quit IRC | 07:50 | |
| rpittau | the kernel is the same | 07:50 |
| rpittau | "There is insufficient memory available for loading the Initial Ramdisk, preventing Linux from accessing the disk." | 07:55 |
| rpittau | did the image grow in the last 3 days ? | 07:55 |
| openstackgerrit | Riccardo Pittau proposed openstack/ironic-python-agent master: [DNM] Increase memory for tinyipa jobs https://review.opendev.org/747355 | 07:59 |
| openstackgerrit | Riccardo Pittau proposed openstack/ironic-python-agent-builder master: Fix finalise tinyipa https://review.opendev.org/747198 | 08:00 |
| *** mkrai has joined #openstack-ironic | 08:02 | |
| dtantsur | rpittau: no idea :( | 08:02 |
| rpittau | let's see if increasing the size changes something, I don't see changes on tinycore that could've caused that | 08:03 |
| iurygregory | it's magic | 08:07 |
| *** rcernin has joined #openstack-ironic | 08:09 | |
| brtknr | thanks rpittau | 08:09 |
| brtknr | thanks rpioso* | 08:09 |
| * rpittau proudly stealing rpioso pings since 2010 | 08:09 | |
| dtantsur | :D | 08:10 |
| iurygregory | lol | 08:10 |
| *** lucasagomes has joined #openstack-ironic | 08:10 | |
| *** ricolin has joined #openstack-ironic | 08:12 | |
| rpittau | anyhow, the image does look a tiny bit bigger than 3 days ago | 08:12 |
| rpittau | oh well, let's wait and see | 08:13 |
| *** rcernin has quit IRC | 08:13 | |
| dtantsur | well, the image may increase with updated pip packages | 08:14 |
| dtantsur | that's the only explanation I have | 08:14 |
| rpittau | that is true, it would be kind of hilarious | 08:15 |
| iurygregory | anyone with some ideas for https://review.opendev.org/741886 the only jobs failling are the ones with uefi and the multinode one (for multinode I'm thinking in decreassing the tempest concurrency this idea came from the nova folks) | 08:18 |
| patchbot | patch 741886 - ironic - Add tempest default_boot_option and altflavor - 11 patch sets | 08:18 |
| *** dougsz has joined #openstack-ironic | 08:22 | |
| *** Lucas_Gray has joined #openstack-ironic | 08:22 | |
| *** hjensas has joined #openstack-ironic | 08:22 | |
| brtknr | rpioso: what is the uri for system event logs? | 08:26 |
| *** k_mouza has joined #openstack-ironic | 08:37 | |
| brtknr | rpioso: Oh I found it, its called DCIM_SELRecordLog | 08:38 |
| brtknr | rpioso: No, DCIM_SELLogEntry | 08:41 |
| *** arne_wiebalck has quit IRC | 08:44 | |
| *** arne_wiebalck has joined #openstack-ironic | 08:46 | |
| janders | wrapping up for the week - see you next week Ironic o/ | 08:49 |
| janders | have a great weekend everyone | 08:49 |
| janders | thanks for all your help this week, much appreciated | 08:49 |
| rpittau | janders: thanks, you too! | 08:49 |
| iurygregory | enjoy the weekend janders | 08:50 |
| dtantsur | have a good weekend janders | 08:54 |
| *** openstackgerrit has quit IRC | 08:54 | |
| janders | thank you rpittau iurygregory dtantsur, will do! | 08:56 |
| *** rcernin has joined #openstack-ironic | 09:07 | |
| *** openstackgerrit has joined #openstack-ironic | 09:26 | |
| openstackgerrit | Merged openstack/bifrost bugfix/8.3: Update .gitreview for bugfix/8.3 https://review.opendev.org/746178 | 09:26 |
| openstackgerrit | Merged openstack/bifrost bugfix/8.3: Update TOX_CONSTRAINTS_FILE for bugfix/8.3 https://review.opendev.org/746179 | 09:26 |
| *** k_mouza has quit IRC | 09:27 | |
| openstackgerrit | Merged openstack/bifrost bugfix/8.3: dynamic: always use clouds.yaml when it is available https://review.opendev.org/746978 | 09:37 |
| dtantsur | TheJulia: we may want to release 8.3.1 because of ^^^ | 09:39 |
| *** k_mouza has joined #openstack-ironic | 09:42 | |
| openstackgerrit | Merged openstack/ironic master: Fix network_data path for dhcpless deployments https://review.opendev.org/747144 | 09:47 |
| *** ociuhandu has joined #openstack-ironic | 09:53 | |
| *** ociuhandu has quit IRC | 09:53 | |
| *** ociuhandu has joined #openstack-ironic | 09:54 | |
| *** Lucas_Gray has quit IRC | 09:58 | |
| *** Lucas_Gray has joined #openstack-ironic | 10:03 | |
| *** belmoreira has quit IRC | 10:07 | |
| *** rcernin has quit IRC | 10:13 | |
| *** mkrai has quit IRC | 10:14 | |
| *** k_mouza has quit IRC | 10:31 | |
| *** mnasiadka has joined #openstack-ironic | 10:39 | |
| *** k_mouza has joined #openstack-ironic | 10:42 | |
| *** jhesketh has quit IRC | 10:53 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic master: Do not wipe cached deploy/clean steps on fast-track deployment https://review.opendev.org/747389 | 10:54 |
| dtantsur | and this is another candidate for a bugfix release ^^ | 10:54 |
| *** jhesketh has joined #openstack-ironic | 10:55 | |
| *** rcernin has joined #openstack-ironic | 10:59 | |
| mnasiadka | hello, just stumbled across an issue using stable/ussuri ironic (bifrost deployed) and ironic-python-agent (stable/ussuri as well), where IPA during deployment has a problem sending heartbeats, due to some agent token issue. I read through https://docs.openstack.org/ironic/latest/admin/agent-token.html - but it lacks a bit of a description what settings affect the behaviour (and how). | 11:06 |
| mnasiadka | The error I see in conductor logs is: Heartbeat request has been rejected as the version of ironic-python-agent indicated in the heartbeat operation should support agent token functionality. | 11:06 |
| mnasiadka | When [DEFAULT] require_agent_token is set to False - everything works, but I would rather want to understand how to get it to work :) | 11:07 |
| *** rcernin has quit IRC | 11:12 | |
| dtantsur | mnasiadka: you also need to check the agent logs on the node to see what was happening with the token there (did it receive a token, etc) | 11:14 |
| *** k_mouza has quit IRC | 11:34 | |
| *** Lucas_Gray has quit IRC | 11:40 | |
| *** rh-jelabarre has joined #openstack-ironic | 11:59 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic master: Ensure in-band deploy steps are present in time for fast-track deployments https://review.opendev.org/747389 | 12:01 |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic master: Ensure in-band deploy steps are present in time for fast-track deployments https://review.opendev.org/747389 | 12:02 |
| bbezak | dtantsur: these are the logs from agent. Warning look interesting, the similar was on conductor https://www.irccloud.com/pastebin/nx0MJfuk/ipa%20logs | 12:02 |
| bbezak | 2020-08-21 11:42:40.820 26 WARNING ironic.conductor.manager [req-3ff50857-31f3-4813-afff-fdcd20d54983 - - - - -] An agent token generation request is being refused as one is already present for node e3efc74c-11f2-4545-85f5-0d5ae8c94f61 | 12:02 |
| bbezak | hello btw :) | 12:02 |
| dtantsur | yep, this is bad | 12:02 |
| dtantsur | and hello :) | 12:02 |
| dtantsur | I wonder if ussuri lacks one of the fixes.. can you confirm you use the latest stable/ussuri? | 12:03 |
| dtantsur | (check /opt/stack/ironic against opendev.org) | 12:03 |
| bbezak | yeah, but I'll double check | 12:03 |
| bbezak | yeah, bifrost has latest ironic stable/ussuri | 12:04 |
| dtantsur | bbezak: what's the exact sequence of actions you attempted? | 12:05 |
| *** k_mouza has joined #openstack-ironic | 12:06 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic stable/ussuri: Wipe agent token during reboot or power off https://review.opendev.org/747394 | 12:07 |
| dtantsur | bbezak: can you try applying ^^^ and seeing if it helps? | 12:07 |
| bbezak | I was using bifrost playbooks deploy-dynamic.yml via Kayobe to deploy hosts. Ok I will try this fix, thx | 12:12 |
| *** k_mouza has quit IRC | 12:21 | |
| *** k_mouza has joined #openstack-ironic | 12:21 | |
| *** k_mouza has quit IRC | 12:26 | |
| hjensas | what's the trick to get DIB rootpwd in pxe_append_params to work? (https://docs.openstack.org/ironic-python-agent/ocata/troubleshooting.html#id8). Do I need to escape '$' chars in the encrypted password? | 12:27 |
| dtantsur | hjensas: definitely | 12:27 |
| bbezak | dtantsur: much better now thank you | 12:28 |
| hjensas | dtantsur: ok, trying with escapes then. | 12:28 |
| dtantsur | bbezak: great. Not sure why I forgot to backport it.. | 12:29 |
| rpittau | dtantsur: it was the memory of the image, increasing it worked, I'm going to propose a full patch for that | 12:31 |
| dtantsur | rpittau: bumping to 512? | 12:31 |
| rpittau | yeah, should be enough | 12:32 |
| dtantsur | don't forget any jobs on other projects that may use tinyipa: ironic, inspector, metalsmith.. | 12:32 |
| rpittau | yes, I'm going to check them all, starting with ironic-python-agent that is also broken | 12:32 |
| dtantsur | ++ thanks! | 12:32 |
| rpittau | np | 12:32 |
| iurygregory | if you change in ironic-base it should just work | 12:35 |
| dtantsur | at least metalsmith does not use ironic-base | 12:36 |
| rpittau | iurygregory: it's for tinyipa only, the ironic-base hase the ram for DIB iamges | 12:36 |
| iurygregory | oh =) | 12:36 |
| rpittau | which is 3072 | 12:36 |
| iurygregory | yeah | 12:36 |
| rpittau | bit too much :) | 12:36 |
| openstackgerrit | Riccardo Pittau proposed openstack/ironic master: Increase memory of tinyipa vms https://review.opendev.org/747397 | 12:41 |
| openstackgerrit | Riccardo Pittau proposed openstack/ironic-python-agent master: Increase memory for tinyipa jobs https://review.opendev.org/747355 | 12:42 |
| openstackgerrit | Riccardo Pittau proposed openstack/ironic-python-agent-builder master: Fix finalise tinyipa https://review.opendev.org/747198 | 12:43 |
| dtantsur | rpittau: one small issue in the IPA patch | 12:44 |
| rpittau | dtantsur: in ironic ? | 12:44 |
| dtantsur | rpittau: https://review.opendev.org/#/c/747355/2/zuul.d/ironic-python-agent-jobs.yaml | 12:44 |
| patchbot | patch 747355 - ironic-python-agent - Increase memory for tinyipa jobs - 2 patch sets | 12:44 |
| rpittau | oh | 12:45 |
| rpittau | yeah | 12:45 |
| rpittau | it's experimental, can we remove it after that patch lands ? | 12:46 |
| openstackgerrit | Riccardo Pittau proposed openstack/ironic-python-agent master: Increase memory for tinyipa jobs https://review.opendev.org/747355 | 12:46 |
| iurygregory | the experimental job I think we should just drop | 12:49 |
| rpittau | metalsmith is already at 1024 so should be ok | 12:49 |
| openstackgerrit | Riccardo Pittau proposed openstack/ironic-python-agent master: Remove experimental tinyipa job https://review.opendev.org/747400 | 12:51 |
| iurygregory | can't we drop all experimental? | 12:56 |
| iurygregory | I have the feeling we already test - ipa-tempest-dsvm-partition-ipmi-iscsi-tinyipa ... | 12:56 |
| rpittau | we peobably should, that is quite old also | 12:56 |
| iurygregory | yeah | 12:57 |
| iurygregory | the only thing is the dsvm in the name hehe | 12:57 |
| rpittau | ok, it feels old :) | 12:59 |
| dtantsur | I think we can drop all these jobs | 13:11 |
| rpittau | okey dokey | 13:11 |
| openstackgerrit | Riccardo Pittau proposed openstack/ironic-python-agent master: Remove experimental jobs https://review.opendev.org/747400 | 13:13 |
| *** Goneri has joined #openstack-ironic | 13:16 | |
| *** k_mouza has joined #openstack-ironic | 13:19 | |
| openstackgerrit | Riccardo Pittau proposed openstack/ironic-python-agent master: Increase memory for tinyipa jobs https://review.opendev.org/747355 | 13:30 |
| TheJulia | braaaains | 13:30 |
| openstackgerrit | Riccardo Pittau proposed openstack/ironic-python-agent-builder master: Fix finalise tinyipa https://review.opendev.org/747198 | 13:31 |
| rpittau | on the rocks please | 13:31 |
| dtantsur | morning TheJulia | 13:31 |
| openstackgerrit | Julia Kreger proposed openstack/ironic stable/queens: Fix for failure in cleaning https://review.opendev.org/747288 | 13:33 |
| TheJulia | https://www.bulleit.com/whiskeys/bulleit-rye/ <-- no, this on the rocks. | 13:34 |
| rpittau | :) | 13:34 |
| rpittau | they have a France dedicated site, wow | 13:35 |
| openstackgerrit | Julia Kreger proposed openstack/ironic stable/queens: Retries and timeout for IPA command https://review.opendev.org/747271 | 13:35 |
| TheJulia | I've managed to find Bulleit Rye multiple countries | 13:35 |
| dtantsur | I've found another reason to ditch the iscsi deploy: we neither clean up the iscsi target automatically nor re-read the partition table | 13:35 |
| dtantsur | sigh | 13:36 |
| TheJulia | Including .au and .cn | 13:36 |
| TheJulia | dtantsur: kill it with fire! | 13:36 |
| TheJulia | the reason I'm backporting the agent retry changes is beause I've been digging at logs of cases where basically it was communication to the agent that was breaking after an iscsi deploy was finished. They wanted the other patches we did for client retries, but they wouldn't quite help the situation | 13:37 |
| openstackgerrit | Dmitry Tantsur proposed openstack/metalsmith master: CI: use the direct deploy by default https://review.opendev.org/747413 | 13:39 |
| rpittau | mmm interesting | 13:39 |
| openstackgerrit | Riccardo Pittau proposed openstack/metalsmith master: [WIP] Use Centos 8 https://review.opendev.org/737278 | 13:39 |
| dtantsur | TheJulia: I guess I'll wait for the meetup before proposing a deprecation release note | 13:40 |
| *** rloo has joined #openstack-ironic | 13:40 | |
| dtantsur | but I did a quick poll among the operators I know and they were mostly "whatever" | 13:41 |
| rpittau | dtantsur: you can change most of the jobs definitions in metalsmith like that I guess | 13:41 |
| dtantsur | mmmm? | 13:41 |
| dtantsur | sorry, didn't quite get it | 13:42 |
| rpittau | sorry, you changed IRONIC_DEFAULT_DEPLOY_INTERFACE to direct, I guess you'll have to change the children jobs accordingly | 13:42 |
| TheJulia | 2020-08-20 21:27:05.488031 | controller | Could not find a version that satisfies the requirement bcrypt>=3.1.3 (from ironic-lib>=4.1.0->ironic_python_agent) (from versions: ) | 13:44 |
| TheJulia | 2 | 13:44 |
| TheJulia | *sigh* | 13:44 |
| rpittau | TheJulia: fix is in the oven, we also had to change size of tinyipa jobs | 13:45 |
| TheJulia | any reviews needed? | 13:45 |
| rpittau | https://review.opendev.org/747198 https://review.opendev.org/747397 https://review.opendev.org/747355 | 13:45 |
| patchbot | patch 747198 - ironic-python-agent-builder - Fix finalise tinyipa - 4 patch sets | 13:45 |
| patchbot | patch 747397 - ironic - Increase memory of tinyipa vms - 1 patch set | 13:45 |
| patchbot | patch 747355 - ironic-python-agent - Increase memory for tinyipa jobs - 4 patch sets | 13:45 |
| TheJulia | dragons will appear to set everything on fire | 13:46 |
| TheJulia | nice! | 13:46 |
| rpittau | based on a true story :) | 13:46 |
| rpittau | btw bcrypt faults "Shipped abi3 Windows wheels (requires pip >= 20)" | 13:47 |
| rpittau | need to drop, have an appointment in 10 minutes(!), but I'll check the patches later tonight, and feel free to update them if needed! :) | 13:53 |
| rpittau | have a great weekend everyone! o/ | 13:53 |
| *** jtomasek has quit IRC | 13:56 | |
| *** tzumainn has joined #openstack-ironic | 13:57 | |
| Qianbiao | hello ironic. | 14:03 |
| Qianbiao | good afternoon, (maybe?) | 14:03 |
| Qianbiao | is there any network architecture picture which desc how ironic + openstack neutron works? | 14:04 |
| dtantsur | I don't think we have a picture for that | 14:08 |
| Qianbiao | ok ~ | 14:09 |
| dtantsur | TheJulia: I'm pondering non-voting CI jobs that fail if one of our services have warnings (maybe with some exclusions) during a run | 14:09 |
| dtantsur | context: I've found a bug by finding a warning while looking at some logs for an unrelated reason | 14:09 |
| Qianbiao | dtantsur should bm management port be in same L2 network with openstack services | 14:10 |
| dtantsur | Qianbiao: you mean, BMC port? no, not necessary (and even undesired) | 14:10 |
| TheJulia | dtantsur: maybe such a job could be helpful, even experimental to start would be good | 14:10 |
| Qianbiao | dtantsur once ironic could reach BMC port, everything goes? | 14:12 |
| TheJulia | fyi, that ansible job failure on redfish is before the ramdisk job runs.... funny thing is it would have found the error I'm looking at now | 14:12 |
| dtantsur | Qianbiao: yep, only L3 connectivity between ironic-conductor and the BMC should be required | 14:13 |
| TheJulia | and FWIW, L2 just won't work with some BMCs | 14:13 |
| TheJulia | specifically supermicro doesn't let you talk to the bmc unless your the default gateway | 14:13 |
| dtantsur | wow | 14:13 |
| dtantsur | stable cores, please: https://review.opendev.org/#/c/747394/ | 14:13 |
| patchbot | patch 747394 - ironic (stable/ussuri) - Wipe agent token during reboot or power off - 1 patch set | 14:13 |
| Qianbiao | TheJulia dtantsur thanks for the information. | 14:14 |
| Qianbiao | and is there any ansible playbook or docker images which can simple help setting up openstack + ironic env | 14:15 |
| openstackgerrit | Aija Jaunteva proposed openstack/ironic-specs master: System configuration within whole clean or deploy step https://review.opendev.org/740721 | 14:15 |
| TheJulia | okay, this is bizzar | 14:18 |
| dtantsur | Qianbiao: depending on what exactly you need. devstack provides a testing environment with VMs. | 14:18 |
| TheJulia | somehow /dev/sr0 is getting chosen as the root device in the ansible drier | 14:19 |
| Qianbiao | dtantsur ok, i will start from devstack. | 14:20 |
| dtantsur | TheJulia: \o/ why not indeed | 14:20 |
| dtantsur | I think the ansible driver's root device handling is much more primitive than IPA's | 14:21 |
| TheJulia | far | 14:21 |
| TheJulia | far far | 14:21 |
| dtantsur | specifically, I'm not sure it has all our code for excluding CD's, floppies, etc | 14:21 |
| dtantsur | I wonder if we should move them to ironic-lib and use in 2 places | 14:21 |
| TheJulia | looks like it does its processing in a dedicated ansible module intree | 14:22 |
| * dtantsur is wondering what happens if somebody tries to use software RAID with a non-IPA deploy method | 14:22 | |
| TheJulia | hahahaha | 14:41 |
| TheJulia | I know why this happened | 14:41 |
| TheJulia | the change to enable the test actually set everything to use virtual media and when the tinycore ramdisk booted via cdrom, the ansible deploy module decided that it should be used! | 14:42 |
| openstackgerrit | Julia Kreger proposed openstack/ironic master: Ansible deploy - Ignore invalid devices https://review.opendev.org/747428 | 14:42 |
| dtantsur | TheJulia: sounds plausible | 14:44 |
| dtantsur | TheJulia: release note please ^^^ | 14:44 |
| TheJulia | k | 14:44 |
| TheJulia | https://github.com/openstack/ironic/blob/master/ironic/drivers/redfish.py#L56 | 14:45 |
| TheJulia | and that is why | 14:45 |
| TheJulia | the union of the supported and enabled interfaces, everything defaulted to it | 14:45 |
| dtantsur | yep. and I already regret it. | 14:45 |
| dtantsur | I mean, regret putting this first, given how well hardware supports it... | 14:45 |
| TheJulia | we can move it to the back? | 14:45 |
| dtantsur | TheJulia: potentially breaking, but we can | 14:46 |
| TheJulia | it is not widely used, we tell people to use defaults or to submit default interfaces | 14:46 |
| dtantsur | maybe we should, at least until the industry makes up its mind | 14:46 |
| TheJulia | We may want to do the cycle release as a major release then | 14:46 |
| TheJulia | which I'm fine with | 14:46 |
| dtantsur | yup. we can remove something deprecated while we're on it :) | 14:46 |
| dtantsur | wanna do a patch or should I? | 14:46 |
| TheJulia | up to you, I'm going to go write the reno now for the ansible change | 14:47 |
| dtantsur | TheJulia: okay. and please handle floppy devices too | 14:47 |
| TheJulia | oooh | 14:47 |
| TheJulia | yes | 14:47 |
| TheJulia | fd | 14:47 |
| dtantsur | I cannot for the live of me remember how they're called | 14:47 |
| dtantsur | right | 14:47 |
| * dtantsur changes redfish | 14:47 | |
| openstackgerrit | Julia Kreger proposed openstack/ironic master: Ansible deploy - Ignore invalid devices https://review.opendev.org/747428 | 14:52 |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic master: Move redfish-virtual-media to the back of supported_boot_interfaces https://review.opendev.org/747430 | 14:53 |
| dtantsur | TheJulia: something like ^^? | 14:54 |
| TheJulia | looks fairly good, mind if I do a quick edit to the reno? | 14:55 |
| dtantsur | not at all | 14:55 |
| dtantsur | TheJulia: another thing that justifies a major version.. making agent token required? | 14:56 |
| openstackgerrit | Julia Kreger proposed openstack/ironic master: Move redfish-virtual-media to the back of supported_boot_interfaces https://review.opendev.org/747430 | 14:56 |
| dtantsur | now, no, not - the same word, just different spelling | 14:56 |
| dtantsur | :D | 14:56 |
| dtantsur | JayF: yep, the IPA CI is broken, the patches are in the gate | 14:57 |
| JayF | dtantsur: ack; ty | 14:57 |
| JayF | dtantsur: How do you feel about TLS patch? Obviously it's failing because I accidentally a comma, but in theory is it likely to merge? | 14:57 |
| dtantsur | JayF: I think I fixed and +2'ed it | 14:58 |
| JayF | That was the eventlet patch IIRC, I'll check again | 14:58 |
| JayF | yeah https://review.opendev.org/#/c/747193/ is failing, syntax error (missing comma) | 14:59 |
| patchbot | patch 747193 - ironic-python-agent - If listen_ssl is true, enable TLS on wsgi server - 2 patch sets | 14:59 |
| dtantsur | ah, sorry | 14:59 |
| dtantsur | JayF: LGTM modulo the syntax error | 15:00 |
| dtantsur | I've set up my vim to run flake8 on save, helps avoid such situations | 15:00 |
| JayF | work machine is a mac, and things sometimes barely work on it :| | 15:00 |
| dtantsur | I see | 15:00 |
| JayF | Supposedly we're getting a linux option soon, which I think I'll avail myself of | 15:01 |
| JayF | especially upstream, it can be tough to get code between a test server and the local machine without causing massive gerrit spam | 15:01 |
| JayF | with a github-workflow, you can just push and pull branches | 15:01 |
| *** trandles has joined #openstack-ironic | 15:01 | |
| TheJulia | dtantsur: I tink we should got on it during the midcycle calls. I'd like to but I have long term upgradability concerns for commercial distributions. :\ | 15:02 |
| openstackgerrit | Jay Faulkner proposed openstack/ironic-python-agent master: If listen_ssl is true, enable TLS on wsgi server https://review.opendev.org/747193 | 15:02 |
| JayF | dtantsur: ^ should fix that syntax error, although I understand if you wanna let CI go before a review. Apparently doesn't matter since the CI is broken anyway :| | 15:02 |
| dtantsur | TheJulia: ah, that people that want upgrades to work across several versions but don't want to put much effort upstream? :( yep, I know them | 15:02 |
| dtantsur | JayF: as an aside, do you know that it's now preferred to use URLs in Depends-On? | 15:03 |
| JayF | dtantsur: like https://review.opendev.org/747193 style URLs? | 15:04 |
| patchbot | patch 747193 - ironic-python-agent - If listen_ssl is true, enable TLS on wsgi server - 3 patch sets | 15:04 |
| TheJulia | dtantsur: yeah..... I've put it on the topic list | 15:04 |
| dtantsur | JayF: exactly | 15:04 |
| JayF | ack; thanks for the heads up | 15:04 |
| dtantsur | TheJulia: we can keep IPA support for older ironic. I don't think old IPA + new ironic is going to work for many other reasons. | 15:06 |
| TheJulia | I'm actually kind of pondering if those vendors want seamless, that they just need to update IPA | 15:07 |
| TheJulia | in advance and maybe "tough" is the approach to take | 15:07 |
| dtantsur | this requires us to keep support for no agent token in IPA | 15:07 |
| dtantsur | which.. is probably fine. | 15:07 |
| TheJulia | yeah | 15:08 |
| JayF | That convieniently would allow me to follow latest-IPA for the most part as well... | 15:08 |
| dtantsur | given that JayF successfully uses Ussuri IPA with Ocata, it works already :) | 15:08 |
| JayF | *for cleaning only | 15:08 |
| dtantsur | I mean, agent tokens don't prevent that | 15:08 |
| JayF | some restrictions apply, read the fine print, etc | 15:08 |
| JayF | lol | 15:08 |
| TheJulia | we have put a lot of effort to keep IPA very compatible | 15:08 |
| JayF | I mean, and frankly, IPA just doesn't have a lot of code churn | 15:09 |
| TheJulia | yup | 15:09 |
| dtantsur | we're trying, but we don't ensure it in the CI | 15:09 |
| JayF | I pretty much did a full review of all code from ocata -> ussuri as part of deciding it was OK, there's not a lot there | 15:09 |
| dtantsur | JayF: wait for us to remove the iscsi deploy ;) | 15:09 |
| JayF | dtantsur: I consider it a personal victory that is being removed | 15:09 |
| dtantsur | :D | 15:09 |
| TheJulia | I think there has been one case where there was someone who had a breaking config but it was like... queens ipa with fairly recent master branch | 15:09 |
| JayF | dtantsur: The first ever Ironic mid-cycle we went to, in yahoo at Sunnyvale hilariously enough, and the iscsi deploy was explained to us, we basically spent the next two days WTF-ing at it. Agent wins the day \o/ | 15:10 |
| dtantsur | right :) well, it was good for start, I guess. remember the old bash ramdisk? | 15:10 |
| JayF | Some memories are better suppressed :} | 15:10 |
| dtantsur | it's still haunting me | 15:10 |
| JayF | TheJulia: I am ++ to the idea that keeping new-IPA -> old-Ironic is reasonable, I don't think newer ironic supporting older IPA holds any value whatsoever | 15:11 |
| dtantsur | at least you haven't seen the several-hundred-lines-of-code bash ramdisk we inherited downstream from one older product | 15:11 |
| * TheJulia radios into HQ to indicate that dtantsur is remembering the bash ramdisk | 15:11 | |
| JayF | TheJulia: is there a direct use case for "I can't run new IPA in my ramdisk" | 15:11 |
| TheJulia | age of images+drivers really | 15:11 |
| dtantsur | JayF: N+1 ironic with N ipa is useful for upgrades | 15:11 |
| JayF | Why? Why can't the process be "update IPA first" | 15:12 |
| JayF | TheJulia: AFAICT, there's no reason the IPA *code* can't be built into an older ramdisk/OS, right? | 15:12 |
| trandles | I want to confirm I didn't do something stupid (likely) before I claim something is broken (less likely). When debugging my bifrost install it doesn't look like BOOTP was allowed. firewall-cmd doesn't show it and when I stopped firewalld service then requests started reaching dnsmasq. I _assume_ this isn't expected and I missed something again. | 15:12 |
| JayF | I guess py27 vs py3x reasons | 15:12 |
| TheJulia | that is basically it | 15:12 |
| dtantsur | JayF: it can, but people appreciate flexibility | 15:12 |
| dtantsur | TheJulia, JayF, well, we support centos 7 in ipa-builder again.. | 15:13 |
| JayF | I mean, we have a downstream expectation that at some point, for older hardware, it's likely we'll have to have a CentOS7 version of the IPA ramdisk as well. | 15:15 |
| dtantsur | well, good news: it builds again | 15:15 |
| JayF | \o/ | 15:16 |
| TheJulia | https://thumbs.gfycat.com/HauntingDirtyDragon-size_restricted.gif | 15:17 |
| * dtantsur prepares to kill non-venv bifrost | 15:22 | |
| *** tkajinam has quit IRC | 15:26 | |
| *** gyee has joined #openstack-ironic | 15:34 | |
| rpioso | brtknr: You are most welcome. Glad you found the WS-Man class. | 15:34 |
| rpioso | rpittau: LOL | 15:34 |
| *** cp- has quit IRC | 15:42 | |
| *** cp- has joined #openstack-ironic | 15:43 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/bifrost master: Remove support for non-venv deployments https://review.opendev.org/747434 | 15:45 |
| dtantsur | boom! | 15:45 |
| TheJulia | not much code, but pervasive | 15:45 |
| dtantsur | yeah | 15:45 |
| TheJulia | looks riccardo's chagnes are good, but I'm guessing we'll need to backport them all | 15:46 |
| dtantsur | except for ipa-builder, yeah | 15:47 |
| dtantsur | the reason we need image_download_source=http https://zuul.opendev.org/t/openstack/build/74b24b98bde34c19a26aff67f7f0db43/log/controller/ironic-bm-logs/node-0_no_ansi_2020-08-21-15:23:12.log#3551 :) | 15:48 |
| openstackgerrit | Dmitry Tantsur proposed openstack/metalsmith master: CI: use the direct deploy by default https://review.opendev.org/747413 | 15:49 |
| *** mkrai has joined #openstack-ironic | 15:53 | |
| *** ociuhandu_ has joined #openstack-ironic | 15:53 | |
| *** ociuhandu has quit IRC | 15:56 | |
| dtantsur | trandles: it used to work at least, including with firewalld | 15:57 |
| *** ociuhandu_ has quit IRC | 15:57 | |
| bdodd | Good morning, ironic | 15:57 |
| JayF | o/ | 15:58 |
| dtantsur | trandles: http://paste.openstack.org/show/797042/ includes dhcp for me. or are you using it on bare metal? | 15:59 |
| trandles | bare metal | 15:59 |
| *** lucasagomes has quit IRC | 16:00 | |
| dtantsur | mmm, I wonder if we missed it | 16:00 |
| trandles | dtantsur, updated http://paste.openstack.org/show/797043/ | 16:01 |
| bdodd | I created a redfish devstack env. And I can access the SimpleStorage redfish resource on each System (node) in the emulator. But trying to access the Storage resource results in a 500 error. | 16:01 |
| dtantsur | trandles: lemme try something | 16:02 |
| bdodd | Anyone have pointers to what I need to do to get the Storage resources enabled? | 16:02 |
| bdodd | ajya ^^^^ (in case you may know about this) | 16:03 |
| openstackgerrit | Dmitry Tantsur proposed openstack/bifrost master: [WIP] Explicitly enable DHCP services on baremetal CentOS/RHEL https://review.opendev.org/747437 | 16:04 |
| dtantsur | trandles: could you try ^^^ | 16:04 |
| TheJulia | bdodd: the emulator? | 16:04 |
| dtantsur | bdodd: check sushy-tools logs, probably a bug | 16:04 |
| bdodd | dtantsur Thanks. Will check that. | 16:05 |
| bdodd | TheJulia Yes, the sushy-tools emulator | 16:06 |
| TheJulia | it may be that the emulator only supports the static entry return insteadd of the dynamic vm control | 16:06 |
| dtantsur | TheJulia: this is definitely true | 16:06 |
| dtantsur | maybe SimpleStorage is real, but most resources are static | 16:07 |
| *** ociuhandu has joined #openstack-ironic | 16:15 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/bifrost master: Explicitly enable DHCP services on baremetal CentOS/RHEL https://review.opendev.org/747437 | 16:15 |
| trandles | dtantsur, sorry, testing is taking a minute | 16:18 |
| *** ociuhandu has quit IRC | 16:19 | |
| dtantsur | rpittau: we might have a chicken-and-egg problem with your patches.. | 16:19 |
| dtantsur | https://review.opendev.org/#/c/747355/ requires the ipa-builder fix, which in turn requires it | 16:20 |
| patchbot | patch 747355 - ironic-python-agent - Increase memory for tinyipa jobs - 4 patch sets | 16:20 |
| JayF | perhaps temporarily make one of the broken jobs on one repo or the other non-voting long enough for it to merge in? | 16:24 |
| trandles | dtantsur, that worked...requests are getting through to dnsmasq | 16:24 |
| TheJulia | What JayF said is what I did a couple weeks ago instead of torture myself with any other path | 16:25 |
| TheJulia | and reverted the ci change right after the patchs to fix things merged | 16:25 |
| trandles | tftpd isn't set up to serve pxelinux.0 so that's the next thing to figure out | 16:26 |
| TheJulia | trandles: well... hmm | 16:26 |
| trandles | undionly.kpxe causes it to fail and move on | 16:27 |
| TheJulia | I think we nuked syslinux chain booting | 16:27 |
| trandles | oops | 16:27 |
| * TheJulia rasies an eyebrow | 16:27 | |
| TheJulia | wait, your machines are refusing to boot directly to the ipxe binary and want to go through syslinux first? | 16:27 |
| trandles | ipxe isn't supported from what I can tell | 16:27 |
| * TheJulia tries to wrap her head around this statement | 16:29 | |
| trandles | let me back up...it doesn't look like chainloading into ipxe is working | 16:30 |
| dtantsur | trandles: maybe your machines are in UEFI? | 16:30 |
| dtantsur | although.. bifrost should work anyway | 16:30 |
| *** k_mouza has quit IRC | 16:30 | |
| trandles | definitely not UEFI | 16:30 |
| trandles | that gets disabled as soon as we unbox | 16:31 |
| *** dougsz has quit IRC | 16:31 | |
| trandles | I'll try to capture the SOL output | 16:32 |
| dtantsur | trandles: what exactly happens with undionly? | 16:32 |
| dtantsur | yes please | 16:32 |
| trandles | immediately drops through to grub on the local drive after downloading undionly | 16:33 |
| dtantsur | Oo | 16:33 |
| dtantsur | do you have x86-64 machines? | 16:33 |
| trandles | yup | 16:33 |
| TheJulia | yeah, console output | 16:34 |
| TheJulia | It _might_ be that you need a different ipxe build | 16:34 |
| TheJulia | or specific drivers, or... somethign | 16:34 |
| dtantsur | or fallback to plain PXE, which we don't have in bifrost.. | 16:34 |
| trandles | http://paste.openstack.org/show/797045/ | 16:35 |
| trandles | got it before the screen refreshed with local grub | 16:35 |
| dtantsur | trandles: mmm, it doesn't seem to get to iPXE even | 16:35 |
| dtantsur | do the IP and netmask match expected? | 16:36 |
| * dtantsur thinks of a conflicting DHCP server | 16:36 | |
| *** Qianbiao has quit IRC | 16:37 | |
| dtantsur | my next suggestion was to try iPXE builds from their website (https://boot.ipxe.org/undionly.kpxe), but this doesn't look like it's getting to iPXE at all | 16:37 |
| trandles | oh, this time I found in the logs, permission problem with undionly.kpxe | 16:37 |
| TheJulia | oh... hmm | 16:37 |
| dtantsur | SIGH | 16:37 |
| trandles | the previous time I didn't have the permission denied...wtf, am I dropping log messages? | 16:38 |
| trandles | *it didn't | 16:38 |
| dtantsur | trandles: selinux or regular ACL? | 16:38 |
| dtantsur | on my machine it's readable only by root, but dnsmasq should also be run as root.. | 16:38 |
| trandles | in permissive mode, should be regular | 16:38 |
| dtantsur | btw bifrost does work in enforcing mode (if it does not - please report) | 16:39 |
| trandles | on CentOS 8 dnsmasq is running as dnsmasq | 16:39 |
| dtantsur | mmmmmmm... | 16:39 |
| dtantsur | and it works in the CI because libvirt has iPXE ROM | 16:39 |
| dtantsur | Work On DevStack (tm) | 16:39 |
| dtantsur | okay, should not be hard to fix. hold on. | 16:40 |
| trandles | I should quit my current job and just be a QA engineer. This entire week has gone exactly like this. On two different internal projects I've been finding corner cases and writing bug reports. | 16:42 |
| trandles | BTW - what git remote should I be pointed at to test your stuff dtantsur? | 16:46 |
| openstackgerrit | Dmitry Tantsur proposed openstack/bifrost master: Make the iPXE and PXE boot files world-readable https://review.opendev.org/747442 | 16:49 |
| dtantsur | trandles: next patch ^^^ | 16:49 |
| dtantsur | I don't have a single branch, just use the cherry-pick feature of the gerrit UI | 16:49 |
| dtantsur | now I have to go, have a great weekend everyone. feel free to ping me on Monday trandles if you still struggle | 16:49 |
| *** dtantsur is now known as dtantsur|afk | 16:50 | |
| JayF | o/ | 16:50 |
| clarkb | git review -d and git review -x will do it for you too | 16:51 |
| clarkb | (you don't need the web ui) | 16:51 |
| trandles | testing, thanks dtantsur|afk | 16:51 |
| openstackgerrit | Merged openstack/ironic master: Increase memory of tinyipa vms https://review.opendev.org/747397 | 16:55 |
| *** ijw has joined #openstack-ironic | 17:01 | |
| *** mkrai has quit IRC | 17:04 | |
| *** priteau has quit IRC | 17:07 | |
| trandles | \o/ everything seems to be working now :) | 17:09 |
| trandles | thanks for your patience with me #openstack-ironic | 17:09 |
| TheJulia | trandles: \o/ | 17:16 |
| JayF | congrats on getting it working! | 17:16 |
| TheJulia | trandles: thank you for pointing out the issue! | 17:16 |
| *** k_mouza has joined #openstack-ironic | 17:17 | |
| openstackgerrit | Julia Kreger proposed openstack/bifrost bugfix/8.3: Make the iPXE and PXE boot files world-readable https://review.opendev.org/747446 | 17:17 |
| openstackgerrit | Julia Kreger proposed openstack/bifrost bugfix/8.2: Make the iPXE and PXE boot files world-readable https://review.opendev.org/747447 | 17:17 |
| openstackgerrit | Julia Kreger proposed openstack/bifrost stable/ussuri: Make the iPXE and PXE boot files world-readable https://review.opendev.org/747448 | 17:17 |
| openstackgerrit | Julia Kreger proposed openstack/bifrost stable/train: Make the iPXE and PXE boot files world-readable https://review.opendev.org/747449 | 17:17 |
| TheJulia | That seems like it is enough... | 17:18 |
| *** chenhaw has joined #openstack-ironic | 17:22 | |
| openstackgerrit | Julia Kreger proposed openstack/ironic master: Ansible deploy - Ignore invalid devices https://review.opendev.org/747428 | 17:24 |
| trandles | I really should just bite the bullet and figure out how to actually contribute instead of whining in here :P | 17:28 |
| TheJulia | it is not hard! | 17:31 |
| * TheJulia wonders if they are goign to do a virtual upstream institute this year | 17:33 | |
| openstackgerrit | Julia Kreger proposed openstack/ironic stable/queens: Fix for failure in cleaning https://review.opendev.org/747288 | 17:37 |
| trandles | The node booted and the SOL console got reset in a way that only 4 lines at the top of my terminal were refreshing and scrolling. I saw some IPA stuff scroll by quickly. 'baremetal node list' and 'baremetal introspection list' and 'baremetal port list' are all empty and there's nothing in the logs on the ironic host. :( | 17:39 |
| trandles | and I can't ping the host | 17:39 |
| *** rloo has quit IRC | 17:40 | |
| *** rloo has joined #openstack-ironic | 17:40 | |
| * TheJulia tries to grok this | 17:44 | |
| TheJulia | is it still on? sounds like it may have hit discovery? | 17:45 |
| TheJulia | but the node is not present so I'm wondering if it was before hand? | 17:45 |
| *** rloo has quit IRC | 17:45 | |
| *** dustinc has quit IRC | 17:45 | |
| trandles | it's on | 17:45 |
| trandles | I'm wondering if it tried to callback to localhost instead of the actual IP of the ironic server | 17:45 |
| TheJulia | possibly | 17:46 |
| TheJulia | I'd reset your sol connection and see what is going on | 17:46 |
| TheJulia | typically IPA will keep scrolling and running | 17:46 |
| trandles | I'll reset the terminal and reconnect and let it sit to see if anything shows up | 17:46 |
| *** ijw has quit IRC | 17:47 | |
| trandles | I'm using iterm2 because it's a lot better than the mac builtin terminal, but I do miss the days of a real VT | 17:47 |
| *** ijw has joined #openstack-ironic | 17:48 | |
| *** rloo has joined #openstack-ironic | 17:49 | |
| TheJulia | yeah, I've had that happen to me some too. | 17:50 |
| trandles | how would I go about modifying the kernel cmdline in bifrost to add "console=ttyS0,115200" ? My testbed is actually a bunch of greenblade nodes so I have a chassis BMC that I can telnet to for a serial console instead of using IPMI's SOL support. | 17:52 |
| trandles | e.g., can I use imgargs? | 17:53 |
| trandles | oh, imgargs is deprecated...I need to get with the times | 17:54 |
| TheJulia | you can set pxe_append_something in ironic.conf and restart the conductor | 17:55 |
| TheJulia | it will append the attributes as longa s the node is being controlled | 17:55 |
| TheJulia | if not, you can just exit the ipxe boot script | 17:55 |
| trandles | ah, ok | 17:55 |
| trandles | network fails to come up | 18:05 |
| openstackgerrit | Julia Kreger proposed openstack/ironic bugfix/15.2: Increase memory of tinyipa vms https://review.opendev.org/747455 | 18:05 |
| trandles | it at least is trying to reach the correct IP for the ironic server | 18:07 |
| *** sradhakrishna has joined #openstack-ironic | 18:08 | |
| openstackgerrit | Julia Kreger proposed openstack/ironic-python-agent master: Temporary disable multiple jobs to land CI fixes. https://review.opendev.org/747457 | 18:08 |
| sradhakrishna | looking for help with debugging ironic agent. Is there a procedure to get the inspector image to boot a system outside openstack, so I can test my agent independently, before I integrate it with openstack? | 18:09 |
| openstackgerrit | Julia Kreger proposed openstack/ironic bugfix/15.2: Increase memory of tinyipa vms https://review.opendev.org/747455 | 18:10 |
| openstackgerrit | Julia Kreger proposed openstack/ironic stable/queens: Retries and timeout for IPA command https://review.opendev.org/747271 | 18:13 |
| TheJulia | no approving/rechecking patches on ipa right now please | 18:16 |
| TheJulia | or I'll become very sad :( | 18:16 |
| TheJulia | sradhakrishna: You should be able to boot it regardless, but I'm wondering what is actually driving the desire to test independently because they are tightly coupled components | 18:17 |
| TheJulia | an explict documented procedure, not really but you can just follow the same steps as if your doing discovery and have a network boot config to the kernel/ramdisk | 18:18 |
| sradhakrishna | TheJulia: We have a custom ironic agent that discovers a custom set of attributes as part of the inspection. Want to test that out | 18:19 |
| sradhakrishna | TheJulia: Was looking to boot out the image separately and test the agent from inside the system, once booted. Is there an easier way to test? | 18:20 |
| TheJulia | I mean you can, but you'd still need someplace to post that data to | 18:20 |
| TheJulia | which means you'd need an inspector instance running | 18:21 |
| TheJulia | if your doing a one-off test, you could always make an iso and attach it directly to a test machine and just let it publish that data to the api endpoint | 18:21 |
| sradhakrishna | @TheJulia: I've tried creating an iso and use it to run inspector on the node from horizon, but it failed - couldn't debug further, hence the thought of this one-off test | 18:23 |
| TheJulia | from horizon is not going to work | 18:24 |
| TheJulia | your going to have to do a custom one off iso image by hand | 18:24 |
| TheJulia | your off the path any of this gets tested | 18:24 |
| TheJulia | because your basically doing development of custom agent code | 18:24 |
| sradhakrishna | TheJulia: So, I've written a custom inspector plugin - a python class deriving from base.ProcessingHook (ironic_inspector.plugins) | 18:26 |
| sradhakrishna | And I've created the iso image. | 18:27 |
| TheJulia | wait, for ironic-inspector? | 18:27 |
| TheJulia | yeah, your going to need to somehow setup an ironic-inspector instance with your patches, and then have an agent boot (somehow) to post to that | 18:28 |
| TheJulia | typically upstream we would use devstack or bifrost | 18:28 |
| TheJulia | since all the tooling is there and we would just attach it to a network interface or use test virtual machines. | 18:28 |
| openstackgerrit | Riccardo Pittau proposed openstack/ironic-python-agent master: Increase memory for tinyipa jobs https://review.opendev.org/747355 | 18:34 |
| TheJulia | *sigh* | 18:34 |
| TheJulia | rpittau: should I kill my ci change? | 18:34 |
| trandles | confirmed IPA isn't bringing up all of my network interfaces...most importantly the one connected to my management network | 18:35 |
| TheJulia | oh, I see you rebased it | 18:35 |
| TheJulia | trandles: doh! | 18:35 |
| TheJulia | is it drivers? | 18:35 |
| trandles | shouldn't be, it's just an igb card | 18:35 |
| TheJulia | which ipa image are you using? | 18:35 |
| trandles | whatever bifrost downloaded or setup | 18:36 |
| trandles | I'm on the master branch | 18:36 |
| TheJulia | okay | 18:36 |
| TheJulia | so.... it likely downloaded tinyipa | 18:36 |
| openstackgerrit | Riccardo Pittau proposed openstack/metalsmith master: [WIP] Use Centos 8 https://review.opendev.org/737278 | 18:37 |
| trandles | if there's an easy way to check by looking at the images in /httpboot/ I'll let you know | 18:37 |
| TheJulia | what is the size of the files in /httpboot? | 18:40 |
| TheJulia | the kernel and initramfs | 18:40 |
| TheJulia | well really the initramfs | 18:40 |
| TheJulia | if it is like 40-50mb, it is tinyipa | 18:41 |
| trandles | 56MB | 18:41 |
| TheJulia | I'd replace those files with the centos8 equivelent files then | 18:41 |
| *** ijw has quit IRC | 18:41 | |
| TheJulia | trandles: let me get you a linik! | 18:41 |
| trandles | gotcha | 18:41 |
| trandles | oh, even better, I was gonna go searching ;) | 18:41 |
| TheJulia | https://tarballs.opendev.org/openstack/ironic-python-agent-builder/dib/files/ | 18:41 |
| TheJulia | enjoy! | 18:42 |
| TheJulia | fwiw | 18:42 |
| TheJulia | by chance are you using LACP at all? | 18:42 |
| trandles | I am not | 18:42 |
| TheJulia | okay | 18:42 |
| trandles | at least not yet | 18:42 |
| trandles | baby steps ;) | 18:42 |
| TheJulia | well, using lacp on ramdisks is super problematic | 18:42 |
| TheJulia | but that ramdisk should have a patch to make it somewhat happier | 18:43 |
| trandles | ah ok | 18:43 |
| TheJulia | you'll still want to set an initial startup delay into ipa, when the time comes we can point you to that config | 18:43 |
| trandles | my testbed does not have enough links for LACP | 18:43 |
| TheJulia | babysteps | 18:43 |
| trandles | production will be a different story | 18:43 |
| TheJulia | ok, if it is passive lacp then you have no need to worry | 18:43 |
| TheJulia | if your active/default on... then network booting is a world of pain | 18:43 |
| * TheJulia suspects that could be a t-shirt, but that she could never wear it out of the house | 18:44 | |
| trandles | I've hit this before in a totally different context | 18:44 |
| trandles | really it boils down to one of the few universal truths: blame the network | 18:44 |
| TheJulia | well, it is not _just_ the network... but yeah | 18:45 |
| TheJulia | Well, the universal truths are more complex than that | 18:46 |
| TheJulia | There may.... or may not be a spoon | 18:46 |
| TheJulia | and we may all be holograms in a simulation | 18:46 |
| trandles | by the by, how does pxelinux.cfg/default get regenerated? restarting ironic-conductor after adding some things to pxe_append_params didn't do anything | 18:47 |
| trandles | I edited by hand | 18:47 |
| TheJulia | oh, that file doesn't get regenerated | 18:47 |
| trandles | it's turtles all the way down | 18:47 |
| TheJulia | you can edit it at will but if you re-install | 18:47 |
| *** ijw has joined #openstack-ironic | 18:48 | |
| openstackgerrit | Merged openstack/ironic bugfix/15.2: Increase memory of tinyipa vms https://review.opendev.org/747455 | 18:51 |
| openstackgerrit | Julia Kreger proposed openstack/ironic master: Guard conductor from consuming all of the ram https://review.opendev.org/726483 | 18:53 |
| *** bnemec has quit IRC | 18:53 | |
| *** AlanMimms has joined #openstack-ironic | 18:56 | |
| AlanMimms | Can someone point me to a list of Redfish functionality/properties/resources requirements for Ironic Stein? I want to be sure our Redfish implementation can work properly and I'm planning the tasks to get it there. | 18:57 |
| TheJulia | AlanMimms: Greetings! | 18:58 |
| AlanMimms | Hi | 18:58 |
| TheJulia | Great question | 18:58 |
| TheJulia | slightly unfortunate we didn't start on an interop profile ages ago | 18:58 |
| AlanMimms | Has someone now created one? <hoping> | 18:59 |
| TheJulia | So the person to ask for an update on that from is rpioso | 18:59 |
| TheJulia | I think he is on staycation this week | 18:59 |
| TheJulia | so stein.. hmmm | 18:59 |
| trandles | Woot woot, got a node enrolled via inspector | 19:00 |
| TheJulia | trandles: congrats | 19:00 |
| trandles | merci | 19:00 |
| TheJulia | largely it is power control under system and the boot mode overrides | 19:00 |
| AlanMimms | Murphy's Law of questions. OK. Would it be best to contact @rpioso next week? | 19:00 |
| TheJulia | AlanMimms: he should be around on Monday, we hold an IRC meeting here at 8 AM US Pacific on Monday morning | 19:01 |
| TheJulia | mornings | 19:01 |
| AlanMimms | Yes, I had guessed as much. I was thinking of looking at the CI tests to see what they do. Maybe that's going to test each Resource/Property? | 19:01 |
| TheJulia | hmmmmm | 19:01 |
| *** bnemec has joined #openstack-ironic | 19:01 | |
| AlanMimms | I will make a note to contact him on Monday then. Thank you VERY much for the pointer. | 19:01 |
| TheJulia | that might work | 19:01 |
| TheJulia | we use an intermediate library called sushy for everything though | 19:01 |
| TheJulia | so it hides a lot of the end resources from us on a daily basis | 19:02 |
| TheJulia | But sushy also has a larger emulator so you can't say "well if I can pretend to be sushy! It must work!" | 19:02 |
| AlanMimms | Yes. Sushy looks very good. But it's general - not specific. So while it can be used to talk to many resources/properties it doesn't say which is actually used. | 19:02 |
| TheJulia | I'm trying to remember the property uesd to change boot mode, and that seems to be the more questionably supported field by vendors. BootModeOverride* is what we use to set boot to disk or boot to network. We don't use the redfish 1.9 uefi specific fields though | 19:03 |
| AlanMimms | Yeah. 1.9 isn't widely avail yet | 19:04 |
| TheJulia | yeah.... Some vendors have also leaked some of it into patches on earlier versions | 19:04 |
| AlanMimms | Yah. And some of these earlier patches may be the genesis from which the DMTF finally adopted the standardized solution. | 19:05 |
| TheJulia | A change in 1.8 bit us too | 19:05 |
| TheJulia | A change in wording that multiple vendors took as "lets break the behavior" | 19:05 |
| TheJulia | :( | 19:05 |
| TheJulia | That was for continious overrides with uefi mode.... | 19:06 |
| *** bnemec has quit IRC | 19:08 | |
| AlanMimms | I'm REALLY hoping for full interoperability when our WIP generation ships. Hence my investigation. Thanks for the pointers. Maybe I should subscribe to the mail list. | 19:08 |
| rpioso | AlanMimms: We have a goal this Victoria release cycle to develop a Redfish Interoperability Profile for the vendor-independent redfish driver/hardware type. It is targeted at master, not the stable branches. However, it could be used as a basis for those. | 19:08 |
| TheJulia | I guess the question also becomes what part of the feature set matters, and if the focus is just on "getting a deployment to work" or "more" | 19:09 |
| AlanMimms | Hi @rpioso! Thanks for that. I'm really looking for high level requirements to start out here. Details and rigorous testing are for 6-9 months from now | 19:09 |
| TheJulia | rpioso: hey, aren't you on staycation?!? | 19:09 |
| rpioso | TheJulia: Yep | 19:09 |
| TheJulia | Or are you downloading MSFT Flight Sim 2020 and bored :) | 19:09 |
| rpioso | TheJulia: Bored, yes :-) | 19:09 |
| AlanMimms | Yes, exactly, TheJulia. It's "build a VW bug" or "build a Tesla". | 19:10 |
| rpioso | AlanMimms: To be sure we're on the same page, are you referring to the profile described by this DMTF specification: https://www.dmtf.org/sites/default/files/standards/documents/DSP0272_1.3.0.pdf? | 19:10 |
| * TheJulia would prefer to just make her FoST and FiST go faster up the mountain instead of "build a tesla" | 19:11 | |
| AlanMimms | Yes that's the one. But I'm looking for high level interop basics at this point. Details are for later. | 19:11 |
| rpioso | AlanMimms: The devil is in the details ;-) | 19:12 |
| AlanMimms | We're going to have to shoot for basic interop and control at first. | 19:12 |
| TheJulia | That should be a really short list then | 19:12 |
| AlanMimms | Yes, but the devil I'm fighting here is estimation of effort and getting a product manager to believe in how much work it will or won't be. | 19:13 |
| AlanMimms | I think it probably is a short list. | 19:13 |
| rpioso | AlanMimms: The DMTF Interoperability Validator, https://github.com/DMTF/Redfish-Interop-Validator, can validate a profile against a Redfish service. | 19:14 |
| AlanMimms | Yep. | 19:14 |
| AlanMimms | That's definitely planned as a testing requirement. | 19:14 |
| rpioso | AlanMimms: Cool. Sounds like you're familiar with the DMTF toolbox: github.com/dmtf/ :-) | 19:15 |
| AlanMimms | I have some familiarity :) - I have been lurking on the Redfish world for a couple of years getting ready for this project. There's another guy we just hired who is the real expert, but I have learned a few things... | 19:17 |
| AlanMimms | At least in theory :) | 19:17 |
| rpioso | AlanMimms: Interesting input re: basics. We're implementing the profile source module-by-module, which translates to ironic hardware interface Redfish implementation-by-implementation -- .../ironic/drivers/modules/redfish/*.py. When that's done we've been planning to combine them into a single profile for the entire driver/hardware type. | 19:19 |
| AlanMimms | Sweet. I'm looking forward to that release. But I think since I'm in (some) control of our Redfish implementation and I need interop with older Ironic releases prominently used in the field I will have to go through, say, Stein or maybe even older and gather requirements to make sure we can function. | 19:20 |
| rpioso | AlanMimms: It sounds like it may be useful to offer both, so folks could choose which interfaces they're interested in and to facilitate development of profiles for the stable branches. | 19:20 |
| TheJulia | So I have this feeling it boils down to: boot BootSourceOverrideEnabled, BootSourceOverrideMode, BootSourceOverrideTarget, system PowerState, the reset action (ResetType) (i think) Naturally we also need the allowed option fields for all of that | 19:20 |
| AlanMimms | OK. | 19:21 |
| TheJulia | that _should_ cover basics | 19:22 |
| rpioso | AlanMimms, TheJulia: Most fundamental would be what management.py and power.py use. | 19:22 |
| AlanMimms | I think because I'm a Redfish VENDOR my perspective is different. I know you work for a vendor too, rpioso. But my guess is you're a consumer of some other internal group's Redfish implementation. | 19:22 |
| AlanMimms | Yes, rpioso, I have done a prelim walk through of those two source files. I need to go back and do that again in detail and with more care. | 19:23 |
| rpioso | AlanMimms: Yes, I am with Dell EMC. May I ask which vendor you're with? | 19:23 |
| TheJulia | AlanMimms: most of that is calling sushy's sushy/resource/system/system.py | 19:23 |
| AlanMimms | Jabil Cloud and Compute and Storage group | 19:23 |
| JayF | We had some folks from DMTF foundation (? I don't remember the exact name) come have a discussion at our mid-cycle. The more we all work together, the easier it is for everyone :) | 19:23 |
| TheJulia | s/foundation// | 19:24 |
| AlanMimms | We're a white box vendor to tier #1 cloud that is looking to expand to tier #2 asap. | 19:24 |
| TheJulia | Distributed Management Task Force | 19:24 |
| TheJulia | AlanMimms: awesome! | 19:24 |
| AlanMimms | Hah. Yes, it is occasionally awesome. Thanks. | 19:24 |
| TheJulia | AlanMimms: well, most of us are here at least on weekdays so if there is anything we can do to help you on your quest, please don't hesitate | 19:25 |
| AlanMimms | wonderbar! I will probably find the occasion to take advantage of that. This community seems to be really strongly interconnected and interactive and that's both useful and refreshing. | 19:25 |
| TheJulia | we try, we're all only human! | 19:26 |
| TheJulia | I think... | 19:26 |
| AlanMimms | Thanks very much. | 19:26 |
| rpioso | JayF: Yep, we've engaged the two DMTF Redfish Forum co-chairs, mraineri, Dell EMC and Jeff Autor, HPE. They are very much involved :-) | 19:26 |
| TheJulia | I joke there may be an AI or two.... | 19:26 |
| AlanMimms | Is there doubt about some? | 19:26 |
| * rpioso doubts himself | 19:26 | |
| JayF | pretty sure openstackgerrit is not a human | 19:26 |
| AlanMimms | Great rpioso. I have talked with Jeff a lot at OCP Summit. He's a good guy to know. | 19:26 |
| TheJulia | oh yes, the bots are definitely NOT human | 19:27 |
| JayF | at least until they are :-O | 19:27 |
| rpioso | AlanMimms: They're both very responsive and helpful. | 19:27 |
| * TheJulia loads up zuul to see if it is time to cry | 19:27 | |
| * TheJulia sighs | 19:29 | |
| TheJulia | I'm going to have to fire up a new vm or something for htese queens patches | 19:29 |
| JayF | TheJulia: hmm. Let me take a real close look at them. I might be able to figure it out. | 19:30 |
| JayF | TheJulia: I have those in ocata working, so it's probably something I already solved... | 19:30 |
| JayF | TheJulia: you wanna link the one that is failing, and how | 19:30 |
| TheJulia | JayF: my conudnrum is right now I can't run the unit tests on any of my mahcines | 19:30 |
| TheJulia | because everything is python 3.7/3.8 | 19:30 |
| TheJulia | and trying to go older just explodes | 19:30 |
| TheJulia | I also need a 2.7 machine | 19:30 |
| * TheJulia may actually have one sitting around | 19:31 | |
| TheJulia | https://review.opendev.org/#/c/747288/ in zuul right now | 19:31 |
| patchbot | patch 747288 - ironic (stable/queens) - Fix for failure in cleaning - 3 patch sets | 19:31 |
| *** arne_wiebalck_ has joined #openstack-ironic | 19:31 | |
| *** alan_ has joined #openstack-ironic | 19:31 | |
| *** AlanMimms has quit IRC | 19:32 | |
| *** arne_wiebalck has quit IRC | 19:32 | |
| *** arne_wiebalck_ is now known as arne_wiebalck | 19:32 | |
| TheJulia | well, 2.7 failed due to the race on mysql | 19:32 |
| alan_ | * was AlanMimms - not sure how to reclaim my nick after reconnect | 19:32 |
| *** gyee has quit IRC | 19:32 | |
| TheJulia | alan_: /nick AlanMimms | 19:32 |
| *** alan_ is now known as AlanMimms | 19:32 | |
| TheJulia | https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_155/747288/3/check/openstack-tox-py27/155ecd7/testr_results.html :( | 19:32 |
| AlanMimms | Off to lunch - thanks TheJulia and rpioso for the encouragement! [cheers] | 19:33 |
| JayF | AlanMimms: if you /msg NickServ, you can register your nickname with a password to save it for yourself. Then you can tell it to kick out imposters (or old leftover connections) to get it back | 19:33 |
| *** rh-jlabarre has joined #openstack-ironic | 19:33 | |
| rpioso | AlanMimms: Enjoy. Great meeting you and hope to "see" you on Monday. | 19:33 |
| TheJulia | oh, i see what the issue is | 19:33 |
| * TheJulia goes to fix the patch... again | 19:34 | |
| JayF | TheJulia: ah, I removed the call downstream because we don't use the drac driver, it's just another instance of deploy_utils.notify_conductor... that needs to instead come from agent_base_vendor.py | 19:34 |
| TheJulia | yup | 19:34 |
| JayF | if it kabooms again, and I see it before you, mind if I try and fix? | 19:34 |
| JayF | like I said, been down this road so I can probably ID the failures by memory, I did this for ocata literally 2 weeks ago | 19:34 |
| TheJulia | sure, fixing how so hopefully ti shoudl be good to go in about 2 minutes | 19:36 |
| JayF | is "q" as far back as we stable maint for now? | 19:36 |
| *** gyee has joined #openstack-ironic | 19:36 | |
| openstackgerrit | Julia Kreger proposed openstack/ironic stable/queens: Fix for failure in cleaning https://review.opendev.org/747288 | 19:37 |
| TheJulia | I'm mainly going to Q for rhosp 13 | 19:37 |
| TheJulia | since upstream we have far more intensive and extensive testing of it before it goes to our downstream folks so basically if it is a patch to be backported, we try to get it in upstream on the same branch for good measure | 19:38 |
| *** jhesketh_ has joined #openstack-ironic | 19:38 | |
| JayF | speaking of, any chance my eventlet patch would qualify for an IPA ussuri backport? | 19:39 |
| JayF | My hunch is no, just because it's probably a touch more invasive than I'd be comfy with for a stable backport | 19:39 |
| TheJulia | JayF: got a link handy? | 19:40 |
| clarkb | TheJulia: https://github.com/pyenv/pyenv is popular for dealing with the python versions problem | 19:40 |
| JayF | https://review.opendev.org/#/c/746774/ | 19:40 |
| patchbot | patch 746774 - ironic-python-agent - Eventlet should be monkey patched as early as poss... - 5 patch sets | 19:40 |
| TheJulia | clarkb: I'm at the point where I just need fully differnet headers to link against to do module reinstalls from source | 19:40 |
| JayF | waiting to merge on the tinyipa fixes :| | 19:40 |
| *** jhesketh has quit IRC | 19:42 | |
| *** rh-jelabarre has quit IRC | 19:42 | |
| *** tzumainn has quit IRC | 19:42 | |
| *** rloo has quit IRC | 19:42 | |
| *** cp- has quit IRC | 19:42 | |
| TheJulia | JayF: I'd be really cautious but if it passes ci on ussuri and the bugfix brancehs I don't see why not | 19:42 |
| JayF | Oh, that's right, we have more than just stable branches now | 19:43 |
| JayF | I'll try to backport it when it merges | 19:43 |
| JayF | same idea -- I'd rather it be tested upstream than carried downstream | 19:43 |
| *** rloo has joined #openstack-ironic | 19:44 | |
| JayF | the follow-on after that, to enable the TLS support from oslo.service, will probably need to be downstream as it adds a feature + config opt, unless you all are much more flexible about stable backports than previously | 19:44 |
| JayF | ( https://review.opendev.org/#/c/747193/ ) | 19:45 |
| patchbot | patch 747193 - ironic-python-agent - If listen_ssl is true, enable TLS on wsgi server - 3 patch sets | 19:45 |
| *** tzumainn has joined #openstack-ironic | 19:46 | |
| TheJulia | JayF: we're flexible when it comes to making life easier/better for the operator, but features like that would be really hard to justify upstream | 19:46 |
| TheJulia | if there is a legitimate case that can be made that it is a bug... that changes the story | 19:46 |
| JayF | yeah; that's my feeling as well with an upstream hat on | 19:46 |
| JayF | but it never hurts to verify | 19:46 |
| TheJulia | yeah | 19:47 |
| JayF | I do think the eventlet patch will have a net-positive effect on removing undefined behavior | 19:47 |
| TheJulia | I'm going to go lay down for a littl ebit. I keep context switching today and feel like that is all I've done for the last two hours | 19:47 |
| TheJulia | ++ | 19:47 |
| JayF | o/ go collect some spoons | 19:47 |
| openstackgerrit | Jay Faulkner proposed openstack/ironic-python-agent master: If listen_ssl is true, enable TLS on wsgi server https://review.opendev.org/747193 | 19:47 |
| *** cp- has joined #openstack-ironic | 19:49 | |
| trandles | TheJulia, I don't see anything upcoming for the upstream institute. Should I email Kendall and ask if one's happening pre-summit? | 20:03 |
| TheJulia | trandles: it is still being organized | 20:14 |
| TheJulia | there was a post seeking volunteers on the mailing list recently | 20:14 |
| trandles | ah, thx | 20:14 |
| trandles | I subscribe to the discuss mailing list...and my mailbox shows 4638 unread :( | 20:14 |
| TheJulia | impressive | 20:15 |
| TheJulia | superhero like even! | 20:15 |
| trandles | when the mailing lists were consolidated to discuss I stopped trying to keep up...and my outlook/exchange filters got totally hosed during an exchange upgrade | 20:15 |
| *** ijw has quit IRC | 20:16 | |
| *** Lucas_Gray has joined #openstack-ironic | 20:24 | |
| *** tzumainn has quit IRC | 20:36 | |
| openstackgerrit | Julia Kreger proposed openstack/ironic master: Detail iPXE + LACP troubleshooting information https://review.opendev.org/743787 | 21:19 |
| *** rloo has quit IRC | 21:33 | |
| openstackgerrit | Merged openstack/ironic-python-agent master: Temporary disable multiple jobs to land CI fixes. https://review.opendev.org/747457 | 21:49 |
| openstackgerrit | Julia Kreger proposed openstack/ironic-python-agent master: Revert "Temporary disable multiple jobs to land CI fixes." https://review.opendev.org/747490 | 21:51 |
| openstackgerrit | Merged openstack/ironic-python-agent master: Increase memory for tinyipa jobs https://review.opendev.org/747355 | 21:59 |
| *** trandles has quit IRC | 22:03 | |
| *** k_mouza has quit IRC | 22:11 | |
| *** ociuhandu has joined #openstack-ironic | 22:16 | |
| *** ociuhandu has quit IRC | 22:21 | |
| *** hjensas has quit IRC | 22:26 | |
| *** jamesdenton has joined #openstack-ironic | 22:31 | |
| *** Lucas_Gray has quit IRC | 22:36 | |
| openstackgerrit | Julia Kreger proposed openstack/ironic-python-agent bugfix/6.3: Increase memory for tinyipa jobs https://review.opendev.org/747492 | 22:39 |
| openstackgerrit | Julia Kreger proposed openstack/ironic-python-agent bugfix/6.2: Increase memory for tinyipa jobs https://review.opendev.org/747493 | 22:39 |
| TheJulia | okay, ipa builder patch rechecked, and revert is in ci now | 22:41 |
| *** AlanMimms has quit IRC | 22:52 | |
| *** sradhakrishna has quit IRC | 23:17 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!
