| arne_wiebalck | Good morning, Ironic! | 06:06 |
|---|---|---|
| janders | hey arne_wiebalck o/ | 07:07 |
| janders | Happy Friday! | 07:07 |
| arne_wiebalck | hey janders! | 07:46 |
| opendevreview | Dmitry Tantsur proposed openstack/bifrost master: Do not install grub2 and shim on the host system https://review.opendev.org/c/openstack/bifrost/+/857935 | 10:00 |
| opendevreview | Dmitry Tantsur proposed openstack/bifrost master: Do not install grub2 and shim on the host system https://review.opendev.org/c/openstack/bifrost/+/857935 | 10:17 |
| opendevreview | Dmitry Tantsur proposed openstack/bifrost master: Rename include_dhcp_server -> enable_dhcp for consistency https://review.opendev.org/c/openstack/bifrost/+/858090 | 10:33 |
| opendevreview | Dmitry Tantsur proposed openstack/bifrost master: Do not install grub2 and shim on the host system https://review.opendev.org/c/openstack/bifrost/+/857935 | 10:56 |
| opendevreview | Dmitry Tantsur proposed openstack/bifrost master: Rename include_dhcp_server -> enable_dhcp for consistency https://review.opendev.org/c/openstack/bifrost/+/858090 | 10:56 |
| opendevreview | Dmitry Tantsur proposed openstack/bifrost master: Do not install grub2 and shim on the host system https://review.opendev.org/c/openstack/bifrost/+/857935 | 11:19 |
| iurygregory | happy friday Ironic o/ | 11:24 |
| opendevreview | Dmitry Tantsur proposed openstack/bifrost master: Do not install grub2 and shim on the host system https://review.opendev.org/c/openstack/bifrost/+/857935 | 11:43 |
| opendevreview | Dmitry Tantsur proposed openstack/bifrost master: Do not install EPEL from the internet if it's already installed https://review.opendev.org/c/openstack/bifrost/+/858099 | 11:46 |
| opendevreview | mitya-eremeev-2 proposed openstack/ironic master: Delete resource provider after node deletion. https://review.opendev.org/c/openstack/ironic/+/827295 | 11:49 |
| opendevreview | Dmitry Tantsur proposed openstack/bifrost master: Do not install EPEL from the internet if it's already installed https://review.opendev.org/c/openstack/bifrost/+/858099 | 12:03 |
| dtantsur | JayF: I think I'm getting somewhere with https://review.opendev.org/c/openstack/bifrost/+/857935/ finally | 12:17 |
| * dtantsur takes a break | 12:17 | |
| dtantsur | JayF: pinged kolla folks about their job failure | 14:06 |
| dtantsur | cannot understand why exactly | 14:06 |
| JayF | dtantsur: What kolla job failure? | 14:07 |
| JayF | oh, I have the wrong change up | 14:07 |
| JayF | aha THAT kolla job failure lol | 14:07 |
| dtantsur | sorry, I'm still in the context of the grub2 issue | 14:07 |
| JayF | No, it's fine, I was too, just staring at a different review thinking it was that change | 14:08 |
| dtantsur | what I think is happening is that they don't allow installing or even downloading packages in the bootstrap phase. hmm. | 14:09 |
| JayF | There is a *lot* of error noise in that kolla run | 14:09 |
| JayF | even if it's not causing the failures | 14:10 |
| JayF | makes it tough to troubleshoot :| | 14:10 |
| dtantsur | https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_fee/857935/6/check/kolla-ansible-ubuntu-source-bifrost/fee5f91/primary/logs/ansible/deploy-bifrost | 14:12 |
| dtantsur | this is the real one | 14:12 |
| JayF | So they don't even have repo mirrors setup at that ... stage in the ansible run? | 14:13 |
| dtantsur | yep. they split the install and boostrap stages | 14:14 |
| dtantsur | so I need to move the actual download to the install stage. sigh. | 14:14 |
| dtantsur | back to the drawing board | 14:14 |
| opendevreview | Jay Faulkner proposed openstack/ironic master: Document existence of non-production "fake" driver https://review.opendev.org/c/openstack/ironic/+/857984 | 14:20 |
| opendevreview | Dmitry Tantsur proposed openstack/bifrost master: Do not install grub2 and shim on the host system https://review.opendev.org/c/openstack/bifrost/+/857935 | 14:37 |
| dtantsur | JayF, I hope this ^^ makes everyone happy | 14:38 |
| JayF | dtantsur: on a re-run of bifrost, if there's a newer efi shim available, will it be used or will the cached copy be used forever? | 14:39 |
| dtantsur | JayF: should be refreshed | 14:39 |
| JayF | ack I'll review once ci is done | 14:42 |
| opendevreview | Dmitry Tantsur proposed openstack/bifrost master: Do not install grub2 and shim on the host system https://review.opendev.org/c/openstack/bifrost/+/857935 | 14:57 |
| opendevreview | Dmitry Tantsur proposed openstack/bifrost master: Do not install grub2 and shim on the host system https://review.opendev.org/c/openstack/bifrost/+/857935 | 15:26 |
| dtantsur | you would assume that a person with my title can write a simple ansible playbooks in less than a dozen of attempts.. no. | 15:26 |
| opendevreview | Dmitry Tantsur proposed openstack/bifrost master: Do not install grub2 and shim on the host system https://review.opendev.org/c/openstack/bifrost/+/857935 | 15:44 |
| arne_wiebalck | JayF: we used the fake driver extensively in production for node adoption ... not sure I added sth to our documentation about this ... but it could be another use case in addition to "testing" (we have used this on 1000s of nodes) | 15:51 |
| opendevreview | Dmitry Tantsur proposed openstack/bifrost master: Do not install grub2 and shim on the host system https://review.opendev.org/c/openstack/bifrost/+/857935 | 15:58 |
| TheJulia | dtantsur: as you get promoted, you forget how to do things except arcane stuff like... sqlalchemy | 16:13 |
| dtantsur | sigh | 16:13 |
| TheJulia | speaking of, I need to make some coffee and try to get test_migrations.py sorted today | 16:13 |
| dtantsur | good luck! | 16:24 |
| dtantsur | and have a nice weekend afterwards | 16:24 |
| JayF | arne_wiebalck: I suspected such a use case existed, but I don't understand it well enough to document it there. Can you help me understand or perhaps propose some language on that PR I can add? | 18:10 |
| JayF | arne_wiebalck: FWIW, I've also used fake driver at places for adoption-like workflows; but bluntly it was not a pattern I'd suggest to most ironic users (and thought it was bad practice at that job, but couldn't fix it) | 18:10 |
| arne_wiebalck | JayF: https://techblog.web.cern.ch/techblog/post/ironic-nova-adoption/ explains what we did. I think we discussed at some point to transform that post into a "recipe" in our docs. If that would happen, you could just reference it :) I can have a look. Unless we think that using the fake driver for this adoption use case is not recommended, you could just mention it for now w/o going into detail. | 18:32 |
| JayF | That's basically the pattern from the other job that I thought was non-ideal; but it's clear I'm the minority on that so I'm happy to document that use case and might even link back to that blogpost | 18:34 |
| arne_wiebalck | oh, that is also a possibility and saves a rewrite | 18:35 |
| JayF | I mean, let me put it this way: I saw a bug about this being documented and it was labelled as low hanging, I didn't think it was low hanging so I just picked it up | 18:35 |
| arne_wiebalck | all I can say is that it worked for us, I have not heard many other opinions yet :) | 18:35 |
| JayF | I think it's reasonable to do a thin layer of comments and link back to the blogpost, and later if we want we can migrate that into a bigger doc for ironic-proper | 18:35 |
| JayF | arne_wiebalck: eh, I think my objections all funnel back into the "snowflakes vs cattle" kind of argument that's been circling openstack for a decade :) | 18:36 |
| arne_wiebalck | JayF: heh | 18:36 |
| TheJulia | Turns out hardware is full of pet cattle | 18:40 |
| arne_wiebalck | TheJulia: yeah! | 18:43 |
| opendevreview | Julia Kreger proposed openstack/ironic master: Phase 3 - SQLAlchemy 2.0 Compatability https://review.opendev.org/c/openstack/ironic/+/857932 | 18:57 |
| TheJulia | I *think* it is good. I think | 19:00 |
| TheJulia | cat py3.out |grep RemovedIn20|wc -l | 19:02 |
| TheJulia | 0 | 19:02 |
| JayF | \o/ | 19:12 |
| TheJulia | that was a bit of a pain to figure out what they actually meant with the errors for test_migrations | 19:12 |
| rloo | JayF: i think the 'snowflake' etc is a bit late, given that we have 'adoption' in ironic. (caveat, i don't know the entire context, just skimming above discussion with arne ;) if i recall, yahoo does a similar thing as cern wrt adoption... | 19:13 |
| JayF | heh, you must have missed the part where I said a previous employer used that flow but I didn't think it was a good pattern :P | 19:13 |
| TheJulia | heh | 19:14 |
| JayF | rloo: TheJulia: we should have a meeting of the minds at some point about https://review.opendev.org/c/openstack/ironic/+/840006 since it seems like rloo and I have nearly-opposite review feedback w/r/t telling the API caller they are limited by abuse prevention | 19:16 |
| TheJulia | yeah :( | 19:16 |
| TheJulia | the whole intent was not just mechanical prevention of process gone awry but also the malicious human attacker | 19:17 |
| rloo | JayF: which particular feedback? | 19:17 |
| JayF | literally the last comment made on that linked PR | 19:17 |
| JayF | e.g. if we should spit out limit value in the API response that was blocked by the ops limits | 19:17 |
| JayF | IMO; we should not, as it's giving information to a malicious attacker about how to improve their attack | 19:18 |
| rloo | oh. i was just wondering. you have a good reason not to. so we shouldn't :) | 19:18 |
| JayF | I'm going to quote that in the PR so it's written down, since I doubt TheJulia wants to go heads-up from the sqla 2.0 work :D | 19:18 |
| TheJulia | I'm semi-brain fried | 19:18 |
| TheJulia | I'm thinking of tossing jumk into the dumpster and going up the home away from home for a week | 19:19 |
| TheJulia | and hopefully jumping into a pool later tonight | 19:19 |
| rloo | sorry, i'll reply. was there anything else there JayF? | 19:19 |
| JayF | that's the only piece of disagreement aiui | 19:20 |
| JayF | I put the comment on it already, you don't have to | 19:20 |
| rloo | thx JayF! | 19:20 |
| TheJulia | that was the only thing I saw when I looked at it last | 19:20 |
| * TheJulia should just eat something and go toss stuff in the dumpster | 19:20 | |
| JayF | Ruby had a large number of comments that are valid other than that one; but I think any contention is gone now | 19:20 |
| JayF | and if you need to go weekend you should go weekend :D | 19:20 |
| JayF | you've done a hell of a job on getting the sqlalchemy 2.0 compatability going | 19:21 |
| TheJulia | I think most of her comments were follow-up worthy, but monday() | 19:25 |
| JayF | yep, no rush | 19:25 |
| JayF | I just wanted to get it unstuck while we were all in IRC :D | 19:26 |
| opendevreview | Jay Faulkner proposed openstack/ironic master: Document existence of non-production "fake" driver https://review.opendev.org/c/openstack/ironic/+/857984 | 19:48 |
| JayF | arne_wiebalck: ^ See what you think about the wording there | 19:48 |
| opendevreview | Jay Faulkner proposed openstack/ironic master: Document existence of non-production "fake" driver https://review.opendev.org/c/openstack/ironic/+/857984 | 19:51 |
| opendevreview | Jay Faulkner proposed openstack/ironic stable/xena: Do not reboot into nowhere after BIOS settings with fast-track https://review.opendev.org/c/openstack/ironic/+/856205 | 20:04 |
| JayF | So it looks like CI is broken on (at least) bugfix/21.0 and bugfix/20.2 | 20:13 |
| JayF | failures that look a lot like the ones here https://review.opendev.org/c/openstack/ironic/+/856191 except I've reproduced them locally without that patch | 20:13 |
| JayF | it's very suspicious because it smells like DB | 20:13 |
| JayF | honestly, makes me wonder if it's an eventlet version thing, perhaps? | 20:14 |
| JayF | given how it's failing | 20:14 |
| JayF | I guess I should check master too, eh | 20:15 |
| JayF | master is working; going to attempt to go down a level (yoga) and see if they work | 20:18 |
| JayF | and yoga works, so it's literally just bugfix/21.0 and 20.2 | 20:20 |
| JayF | ...could it be the oslo.db stuff? | 20:20 |
| JayF | does that fix need backport? | 20:21 |
| JayF | since those are zed-based branches... | 20:21 |
| JayF | https://review.opendev.org/c/openstack/ironic/+/856507 looks like dmitry already backported it, I was +2 | 20:23 |
| JayF | going to ensure unit tests pass here then approve as a single core to fix CI | 20:23 |
| JayF | that looks like the fixer, landing it | 20:27 |
| JayF | rechecking the bugfix/20.2 patch as it had unrelated failures, will land it when 21.0 makes it outta the gate .... if I'm not here then, and you wanna land it, it's here: https://review.opendev.org/c/openstack/ironic/+/856508 | 20:29 |
| TheJulia | JayF: yeah, we need to backport the patch | 20:30 |
| TheJulia | since they can pull in oslodb 12.1 | 20:30 |
| JayF | yeah, I eventually got there lol | 20:30 |
| JayF | the fix for bugfix/21.0 is in the gate now, I just approved it without a second +2 | 20:31 |
| TheJulia | sounds good | 20:31 |
| JayF | I'll watch, when that hits I'll push buttons for 20.2 as well | 20:31 |
| TheJulia | k | 20:31 |
| JayF | weren't you having an early weekend? :) | 20:31 |
| opendevreview | Merged openstack/ironic bugfix/21.0: Fix compatibility with oslo.db 12.1.0 https://review.opendev.org/c/openstack/ironic/+/856507 | 20:43 |
| TheJulia | JayF: eh, yeah, but still getting things together | 21:16 |
| opendevreview | Jay Faulkner proposed openstack/ironic-python-agent stable/xena: Fix software raid output poisoning https://review.opendev.org/c/openstack/ironic-python-agent/+/857772 | 21:32 |
| *** tosky is now known as Guest765 | 21:33 | |
| *** tosky_ is now known as tosky | 21:33 | |
| opendevreview | Merged openstack/ironic bugfix/21.0: Redfish: Consider password part of the session cache https://review.opendev.org/c/openstack/ironic/+/856191 | 21:55 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!