cardoe | This is getting crazy on this RBAC change I keep trying to help get over the line. https://review.opendev.org/c/openstack/ironic/+/928283 timeout on downloading an image now | 04:16 |
---|---|---|
frickler | a) why is ironic using an outdated cirros version? b) why are you downloading the image in CI instead of using a pre-cached copy? | 06:19 |
opendevreview | Takashi Kajinami proposed openstack/ironic master: Drop SQLALCHEMY_WARN_20 https://review.opendev.org/c/openstack/ironic/+/929396 | 06:21 |
opendevreview | Takashi Kajinami proposed openstack/ironic master: Drop SQLALCHEMY_WARN_20 https://review.opendev.org/c/openstack/ironic/+/929396 | 06:25 |
kubajj | good morning ironic! o/ | 06:42 |
dtantsur | frickler: we try to use the cached version, but the code is somewhat fragile since it has to fall back to downloading to support local devstack runs | 09:22 |
dtantsur | this is my general impression of the issue, the details may differ | 09:22 |
dtantsur | JayF: wdyt about landing https://review.opendev.org/c/openstack/ironic-inspector/+/928783? I'm not fond of it (mostly because I don't understand what exactly has changed to cause it) but I also don't want to spend too much time debugging CI for a deprecated project | 12:31 |
JayF | My question for stuff like this is always do we have any idea if it's broken in the real world | 13:15 |
JayF | If this is just devstack shenanigans, +2. If we don't know ... That's more hairy | 13:15 |
JayF | frickler: iirc newer cirros broke us at one point and we pinned it. I'm onboard to revisit | 13:26 |
TheJulia | good morning | 13:29 |
TheJulia | We've hit and pinned like 5-6 times through my entire time in openstack | 13:29 |
TheJulia | I think some of that was added complexity of trying to take it and make a partition image which we can likely revisit | 13:31 |
TheJulia | Sorry, my brain got nerd sniped by product management at 6 AM sharp | 13:31 |
dtantsur | JayF: as far as my limited understanding goes, it's very specific to how the inspector's devstack plugin is organized | 13:53 |
TheJulia | How to know when the day is going to be interesting: When someone puts something so crazy in writing you have to respond with "I'm sorry, what?!" as a reply | 13:57 |
dtantsur | Monday :D | 14:01 |
cardoe | morning all... here's my weekly "ready for workflow" spam... | 14:54 |
cardoe | https://review.opendev.org/c/openstack/ironic/+/927780 https://review.opendev.org/c/openstack/ironic/+/928106 https://review.opendev.org/c/openstack/ironic/+/929364 https://review.opendev.org/c/openstack/ironic/+/929272 https://review.opendev.org/c/openstack/ironic-lib/+/928778 https://review.opendev.org/c/openstack/ironic-lib/+/928776 | 14:57 |
TheJulia | :) | 14:57 |
TheJulia | <3 | 14:57 |
* TheJulia tries to get a slow cooked piece started for dinner in 10 hours | 14:59 | |
cardoe | A couple that don't have a +2 but are likely not that controversial and can be a quick review https://review.opendev.org/c/openstack/sushy/+/929055 https://review.opendev.org/c/openstack/ironic/+/927645 | 15:00 |
JayF | I'll note, Muhammad Ahmad from https://review.opendev.org/c/openstack/ironic/+/929364 reached out to me on linkedin about contributing more to Ironic. I'll respond today but that was nice context around why that patch was posted | 15:01 |
JayF | Who is running meeting this morning? | 15:01 |
opendevreview | Takashi Kajinami proposed openstack/ironic-inspector master: Drop SQLALCHEMY_WARN_20 https://review.opendev.org/c/openstack/ironic-inspector/+/929533 | 15:02 |
JayF | I'm going to run it in lieu of anyone else :) | 15:02 |
JayF | #startmeeting ironic | 15:02 |
opendevmeet | Meeting started Mon Sep 16 15:02:27 2024 UTC and is due to finish in 60 minutes. The chair is JayF. Information about MeetBot at http://wiki.debian.org/MeetBot. | 15:02 |
opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 15:02 |
opendevmeet | The meeting name has been set to 'ironic' | 15:02 |
TheJulia | I ran it last week :) | 15:02 |
JayF | #chair TheJulia | 15:02 |
opendevmeet | Current chairs: JayF TheJulia | 15:02 |
JayF | Hi all, welcome to the Ironic meeting, we hold our meetings under the Openinfra CoC | 15:02 |
JayF | #topic Announcements/Reminders | 15:02 |
masghar | o/ | 15:03 |
JayF | #note Standing reminder to review patches tagged ironic-week-prio and to hashtag any patches ready for review with ironic-week-prio: https://tinyurl.com/ironic-weekly-prio-dash | 15:03 |
dtantsur | o/ | 15:03 |
JayF | #note This is R-2 week. Next week final releases must be cut and the integrated OpenStack release is early october. | 15:03 |
JayF | #link https://releases.openstack.org/dalmatian/schedule.html | 15:03 |
TheJulia | Which means we need to cut *this* week | 15:04 |
cid | o/ | 15:04 |
JayF | TheJulia: the linked thing seems to indicate we could do it next week | 15:04 |
JayF | TheJulia: but I am very +1 to getting it done in advance | 15:04 |
TheJulia | JayF: they will force us for this week most likely, it has happened nearly every time | 15:04 |
TheJulia | anxiety overrules all | 15:04 |
JayF | That's my motto | 15:05 |
JayF | lol | 15:05 |
JayF | So lets cut one this week then, sgtm | 15:05 |
JayF | #note the next OpenInfra PTG which will take place October 21-25, 2024 virtually! Registration is now open! | 15:05 |
JayF | #link https://ptg.openinfra.dev/ | 15:05 |
JayF | #link https://etherpad.opendev.org/p/ironic-ptg-october-2024 | 15:05 |
JayF | Please add your topics to the etherpad, and review them async so we can keep the PTG live sessions to the point and on topic as much as possible. | 15:06 |
JayF | That's all on the agenda for announcements | 15:06 |
JayF | #topic Review Ironic CI Status | 15:06 |
dtantsur | Re releases: rpittau comes back on ?Wednesday? | 15:07 |
dtantsur | Re CI: we put out SOOO many fires last week | 15:07 |
JayF | yeah | 15:07 |
JayF | Our CI has bitrotted significantly, in a handful of different ways | 15:07 |
dtantsur | Broken inspector should be fixed by https://review.opendev.org/c/openstack/ironic-inspector/+/928783 | 15:07 |
dtantsur | then... hopefully.... | 15:07 |
JayF | we wanted to fix some of it with a replacement for ipa; we didn't do that this cycle | 15:07 |
JayF | there's a lot of random failures like Jens alludded to earlier- including downloading cirros from the internet | 15:07 |
JayF | I feel like CI can consume infinite time, but between releases I think it'd pay off to try and tackle as many of these intermittant issues we can | 15:08 |
TheJulia | I'm wondering if we need to instead of looking at CI during hte ptg through the lens of fix/reduction but to radical revisioning, or determination of what we can do to make it easier for ourselves to have more stable/reliable CI | 15:08 |
TheJulia | note, that is a big picture thought, we don't have to discuss now() | 15:09 |
JayF | I agree a lot | 15:09 |
JayF | including documenting our specific CI matrix, what jobs cover what | 15:09 |
* dtantsur has deja vu | 15:09 | |
TheJulia | dtantsur: I said similar in the existing ptg notes | 15:09 |
dtantsur | and in the past.. and in the past.. and in the past | 15:10 |
JayF | We have a few topic on PTG for this: replacing dnsmasq to help remove that periodic CI failure, training up folks on how to troubleshoot CI so it's less of a strain on the longer term folks | 15:10 |
TheJulia | all this has happened before, all this will happen again | 15:10 |
* dtantsur nods wisely | 15:10 | |
JayF | dtantsur: we've been doing this for ten years, I'd be amazed if we don't do this every two years | 15:10 |
JayF | and I'm not sure it's a bad thing :D | 15:10 |
* TheJulia channels https://en.battlestarwikiclone.org/wiki/Pythia | 15:10 | |
JayF | Looks like we'll have a handful of things to chat about CI during PTG | 15:11 |
JayF | Discussion topics in the meeting agenda are old; am not going to cover those | 15:11 |
JayF | #topic Bug Deputy updates | 15:12 |
JayF | cid and masghar had volunteered to be deputies over a couple of weeks, at least when the agenda was updated | 15:12 |
JayF | is there anything to note? | 15:12 |
masghar | I forgot to update the agenda, apologies | 15:12 |
* JayF notes the agenda was last updated 8/26/2024 so that promise is beyond expired | 15:12 | |
masghar | No new bugs for September | 15:12 |
JayF | ++ Thanks | 15:12 |
JayF | Do we have a volunteer to be the next bug deputy? | 15:12 |
cid | No new bugs | 15:13 |
JayF | I can probably take it if nobody else is signing up? | 15:13 |
JayF | I haven't in a month+ | 15:13 |
cid | masghar, we can both keep an eye until after release (?) | 15:13 |
masghar | I wouldnt mind it at all :) | 15:13 |
cid | ++ | 15:13 |
JayF | I'll be around if you have any questions | 15:13 |
masghar | Perfect, thanks | 15:14 |
JayF | #note CID and masghar agree to bug deputy thru release | 15:14 |
cid | JayF: ty | 15:14 |
JayF | #topic RFE Review | 15:14 |
JayF | there are no RFEs posted for review | 15:14 |
JayF | #topic Open Discussion | 15:14 |
TheJulia | braaaaains | 15:14 |
TheJulia | cardoe has some items requiring attention | 15:15 |
TheJulia | I can take a look after the meeting | 15:15 |
JayF | I have a small thing for here, honestly probably a plug more than anything else -- I made a short video about OSSA-2024-003, https://www.youtube.com/watch?v=cz71C4tW3Pw -- feel free to point anyone at it who has questions | 15:15 |
JayF | TheJulia: I landed most of them already I think | 15:15 |
cardoe | JayF: what I linked? That was me scrapping about 20 minutes ago. | 15:15 |
JayF | Yeah, I've been busy | 15:16 |
JayF | Anything else for open discussion? | 15:16 |
TheJulia | uhhh | 15:16 |
TheJulia | Availability in weeks leading up to the PTG> | 15:16 |
TheJulia | ? | 15:16 |
JayF | I'll note it's unlikely I'll be here a week from today | 15:16 |
JayF | as it's my birthday and I usually try not to work | 15:16 |
cid | JayF: Happy birthday in advance | 15:17 |
masghar | ++ ^ | 15:17 |
TheJulia | As an FYI, the first whole week in october I'll be in the EU time zone for meetings. The week after that I'll be in Indiana for OpenInfra days | 15:17 |
TheJulia | The week after that i the PTG | 15:18 |
JayF | I might be out at right around that time, too | 15:18 |
* JayF checks calendar | 15:18 | |
JayF | Yeah like 10/3 - 10/9 or so | 15:18 |
TheJulia | of further note, the week after the ptg, I have jury duty too | 15:18 |
JayF | My brother is getting married :) | 15:18 |
JayF | ** Oct 3 - Oct 9 (I know that was a us-ian way to write that) | 15:19 |
dtantsur | I'll be out for a few days around Oct 1 | 15:19 |
JayF | #note Many cores out first two weeks of October; please set expectations appropriately for getting code landed | 15:19 |
TheJulia | ++ | 15:20 |
JayF | Last call for items for Open Discussion? | 15:20 |
TheJulia | Uhh | 15:20 |
TheJulia | OpenInfra Days NA - Indiana, about a month from now() | 15:20 |
TheJulia | I will have a project update/feedback session, if anyone else will be there | 15:21 |
kubajj | o/ | 15:21 |
TheJulia | o/ kubajj | 15:21 |
JayF | o/ | 15:21 |
JayF | TheJulia: nice; if you want I can try to attend remotely but will not be on site | 15:21 |
TheJulia | I don't know if they are recording forum sessions | 15:22 |
TheJulia | since I'm sort of splitting the time into high level what is going on and then requirements gathering | 15:22 |
JayF | Yeah, that's what I figured, just tossing out the willingness out there | 15:22 |
JayF | I know having you and dtantsur on the line when rpittau and I did BM SIG @ CERN was super helpful | 15:23 |
JayF | (and I think we've incorporated much of that actionable feedback into this release \o/) | 15:23 |
TheJulia | I'll see what I can do, but won't know likely until the day of | 15:24 |
opendevreview | Merged openstack/ironic master: Remove default override for RBAC config options https://review.opendev.org/c/openstack/ironic/+/928283 | 15:24 |
JayF | You know how to get ahold of me if you need :D | 15:24 |
TheJulia | \o/ | 15:24 |
JayF | Last call (part deux) for open discussion items? | 15:25 |
dtantsur | I can haz a review request? | 15:25 |
dtantsur | the nc-si / no-power-off spec is sad and lonely: https://review.opendev.org/c/openstack/ironic-specs/+/926654 | 15:26 |
TheJulia | dtantsur: we can haz cheezeburger? | 15:26 |
dtantsur | mmmm do want | 15:26 |
TheJulia | dtantsur: I can take a look in a day or two, I need to get some other stuff sorted first | 15:26 |
JayF | dtantsur: I have some concerns but I think the only way to resolve them is "don't support this hardware" which isn't reasonable | 15:27 |
JayF | dtantsur: so I'll take a look | 15:27 |
TheJulia | I just independently had a crazy idea without looking at the spec, I wonder how similar it might be?! | 15:27 |
dtantsur | Concerns should at least be listed, mitigated and documented | 15:27 |
TheJulia | ++ | 15:27 |
JayF | Yeah, mainly I think there are cases we can leak a running ramdisk | 15:28 |
dtantsur | The spec is kinda boring, so a crazy idea might turn out something completely different | 15:28 |
JayF | which are ~unavoidable, in the same way the "we flip the power and hope the machine comes up" is unavoidable | 15:28 |
TheJulia | JayF: for network booting? | 15:28 |
JayF | like think abuot | 15:28 |
JayF | what happens if the agent completely goes away | 15:28 |
JayF | the "timeout" bug gets 100000x worse if you can't reboot via oob somehow | 15:28 |
JayF | that goes from "a deploy fails" to "a human must touch this machine" | 15:28 |
dtantsur | You can reboot, you cannot power off | 15:29 |
TheJulia | oh yeah, possible, I was more thinking these cases are likely vmedi abased | 15:29 |
JayF | dtantsur: you can reboot /via oob/? | 15:29 |
dtantsur | Yes, sure. Only power off is a problem. | 15:29 |
JayF | dtantsur: if so that will obviate like 90% of my concerns | 15:29 |
TheJulia | I think we always try inband first | 15:29 |
JayF | okay nice, I was thinking the inband method too | 15:29 |
dtantsur | We do try in-band power off first, which will also need to be changed to a reboot | 15:29 |
TheJulia | oh, yeah | 15:30 |
TheJulia | ugh | 15:30 |
JayF | so it's not like the interface to power off doesn't exist | 15:30 |
TheJulia | or maybe smarter?! | 15:30 |
JayF | it's like, power off for this hardware isn't a usual thing to do? | 15:30 |
dtantsur | Please check the spec, it answers all these questions :) | 15:30 |
TheJulia | changing running default across a fleet is slightly problematic | 15:30 |
JayF | I've read the spec | 15:30 |
TheJulia | ack | 15:30 |
* TheJulia read a much earlier version, I think | 15:30 | |
JayF | probably same | 15:30 |
* TheJulia needs to refresh the brain | 15:30 | |
TheJulia | Oh, have we posted our prelude? | 15:30 |
JayF | Yeah it does not specifically say it's oob, I think there are some unsaid assumptions in there, I'll give it a quality review once meeting is over | 15:31 |
TheJulia | There is a morale imperitive for a silly prelude | 15:31 |
JayF | TheJulia: nope, that's another thing on my list for today if nobody else pushes it | 15:31 |
dtantsur | Happy to elaborate wherever needed | 15:31 |
JayF | I am not a silly writer | 15:31 |
JayF | if you want a boring prelude I will write one | 15:31 |
TheJulia | okay, I'll see what I can hammer out after my next meeting | 15:31 |
* JayF prefers boring for docs that will be read by multiple cultures / esl folks | 15:31 | |
JayF | I don't trust myself to know what is funny for everyone :) | 15:32 |
TheJulia | That might be better | 15:32 |
* dtantsur cannot English well enough for a funny prelude (at least the one that most people will find funny) | 15:32 | |
* TheJulia thinks dtantsur underestimates his english skills | 15:32 | |
JayF | I was going to suggest we highlight the operator-facing enhancements: | 15:32 |
JayF | same | 15:32 |
TheJulia | or German has replaced htem | 15:32 |
JayF | like 2-3 features spceifically asked for at BM SIG have landed this release, ironic-guest-metadata, runbooks | 15:32 |
JayF | we have a lot of operator-happiness changes in this one | 15:32 |
dtantsur | TheJulia: I do have a mess in my head nowadays :D | 15:33 |
kubajj | btw, I discovered a bug (that I caused) in the RAID skip_block_devices and am working on fixing it | 15:33 |
TheJulia | Did docs for runbooks merge? I ask because I had someone send me a feature list and it lacked it | 15:33 |
TheJulia | kubajj: oh, good to know. Is there a bug in launchpad? | 15:33 |
JayF | cid: ^ re: runbook docs | 15:34 |
JayF | I think they did? | 15:34 |
JayF | I think they were alongside | 15:34 |
kubajj | TheJulia: I'll have a look, not sure | 15:34 |
TheJulia | nothing on runbooks in https://docs.openstack.org/ironic/latest/admin/ | 15:34 |
cid | JayF: yes, unless something was left out, it was all part of the same patch. | 15:34 |
TheJulia | and that is where I have folks looking for "features" | 15:35 |
TheJulia | nor https://docs.openstack.org/ironic/latest/user/ | 15:35 |
TheJulia | I bet it is just not linked in | 15:35 |
JayF | Unless objection, going to close the meeting as I think this is rapidly flowing into normal conversation? | 15:35 |
JayF | me too TheJulia | 15:35 |
TheJulia | JayF: sgtm | 15:36 |
JayF | #endmeeting | 15:36 |
opendevmeet | Meeting ended Mon Sep 16 15:36:07 2024 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 15:36 |
opendevmeet | Minutes: https://meetings.opendev.org/meetings/ironic/2024/ironic.2024-09-16-15.02.html | 15:36 |
opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/ironic/2024/ironic.2024-09-16-15.02.txt | 15:36 |
opendevmeet | Log: https://meetings.opendev.org/meetings/ironic/2024/ironic.2024-09-16-15.02.log.html | 15:36 |
JayF | TheJulia: it was added inline to servicing/cleaning docs | 15:36 |
JayF | which tbh makes sense to me versus a dedicated page based on the feedback from the docs review | 15:37 |
TheJulia | JayF: ahh | 15:37 |
JayF | https://docs.openstack.org/ironic/latest/admin/cleaning.html#runbooks-for-manual-cleaning https://docs.openstack.org/ironic/latest/admin/servicing.html#using-runbooks-for-servicing | 15:37 |
TheJulia | kind of yeah, somehow the pm folks looking delineated cleaning/servicing | 15:37 |
JayF | there's no specific guide, outside of client/api docs, on how to create them | 15:37 |
JayF | but that seems mostly OK to me? | 15:37 |
TheJulia | We likely need something user centric on performing actions | 15:39 |
TheJulia | otherwise the bar is a bit high | 15:39 |
TheJulia | and somewhat expectedly so, but an easier on-ramp is always better | 15:39 |
JayF | that's basically what dave said | 15:40 |
JayF | we have a lot of "you can do x" | 15:40 |
JayF | but not a lot of "if you want [thing], do these steps:" | 15:41 |
JayF | so our docs assume basic context going in that you have an idea what people normally use these tools for | 15:41 |
JayF | which may not be true for all folks | 15:41 |
JayF | realistically the docs that were added for runbooks fall into this trap too, but I don't expect folks to overhaul the docs for a feature :) | 15:42 |
TheJulia | Definitely not true for folks who haven't touched ironic in ages and very much not true for people stumbling upon it for the first time or with a "oh, its a thing" context | 15:42 |
JayF | yeah, exactly | 15:42 |
TheJulia | true, yeah | 15:43 |
TheJulia | kubajj: if you can just create something, that would be helpful. Doesn't have to be much, just a skelaton of context | 15:49 |
kubajj | TheJulia: working on it, trying to remember what is the current state before I started fixing stuff :D | 15:51 |
TheJulia | heh | 15:52 |
TheJulia | that happens :) | 15:52 |
TheJulia | I think we had someone notice something else about 5 months ago | 15:52 |
TheJulia | you might just want to double check git blame | 15:52 |
JayF | cardoe: so the other day, you were asking about monitoring/metrics for Ironic iirc, yeah? | 15:54 |
cardoe | yeah | 15:54 |
JayF | You know about the built-in Ironic/IPA metrics support? | 15:55 |
JayF | API/Conductor/IPA will send metrics to statsd; for support w/the prometheus-exporter, you can only send from conductor | 15:55 |
TheJulia | conductor is also two sided, prometheus-exporter stuff was originally for scraping node data up | 15:56 |
TheJulia | but it now captures the metrics stuffs too if so configured | 15:56 |
JayF | so statsd metrics: only application performance metrics | 15:56 |
JayF | prom metrics: conductor perf metrics + node data? | 15:56 |
TheJulia | (in other words, if you want more/other data, we need to understand that for the PTG) :) | 15:57 |
kubajj | TheJulia: will have a look | 15:57 |
JayF | and speak now because statsd support is potentially on the chopping block | 15:58 |
JayF | with ironic-lib getting parted out | 15:58 |
TheJulia | ... and no we're not going to take ironic-lib to Gotham Garage | 15:58 |
kubajj | TheJulia: this? https://review.opendev.org/c/openstack/ironic-python-agent/+/915858 | 15:59 |
TheJulia | (https://www.imdb.com/title/tt8893550/) | 15:59 |
dtantsur | speaking of ironic-lib: https://review.opendev.org/c/openstack/ironic-python-agent/+/928779 | 16:00 |
TheJulia | kubajj: I think there was something else | 16:02 |
dtantsur | and once we get the inspector job fixed, I think the next step should be https://review.opendev.org/c/openstack/ironic-tempest-plugin/+/928284 | 16:03 |
opendevreview | Merged openstack/ironic-lib master: Remove code already migrated to Ironic and IPA https://review.opendev.org/c/openstack/ironic-lib/+/928776 | 16:03 |
dtantsur | I wonder why Ubuntu renamed its OVMF files to OVMF_CODE_4M.fd | 16:08 |
dtantsur | wtf is 4M? should I be worried? | 16:08 |
JayF | what package is that in? | 16:08 |
JayF | https://launchpad.net/ubuntu/+source/qemu/+changelog might have details | 16:08 |
dtantsur | JayF: ovmf | 16:08 |
dtantsur | "I've also read about KVM guys defaulting to the 4M firmware image, to prevent UEFI update failures, as these cert lists become too long to store within the "old" 2M images." - from googling | 16:09 |
opendevreview | Merged openstack/ironic master: Make sure qemu-img command is available in debian/suse https://review.opendev.org/c/openstack/ironic/+/929272 | 16:09 |
dtantsur | this_is_fine.jpg | 16:09 |
opendevreview | Merged openstack/ironic master: CI: Remove scope enforced ci jobs https://review.opendev.org/c/openstack/ironic/+/928106 | 16:11 |
opendevreview | Merged openstack/ironic master: Remove skip check for Python 3.6 https://review.opendev.org/c/openstack/ironic/+/927780 | 16:11 |
JayF | dtantsur: https://review.opendev.org/c/openstack/ironic-specs/+/926654/2#message-fb51f6d8cfaf50a89ab6261b9869aa0d03e98fc3 as promised | 16:11 |
kubajj | TheJulia: I've drafted this: https://bugs.launchpad.net/ironic/+bug/2080871 (managed to fix 1-3 and testing the 4th one) - problem is that deployment is taking ages, but I have access to our qa nodes now | 16:12 |
opendevreview | Dmitry Tantsur proposed openstack/bifrost master: WIP: start working on Ubuntu 24.04 support https://review.opendev.org/c/openstack/bifrost/+/928895 | 16:16 |
dtantsur | JayF: thanks! Typed some quick responses, please check. Will follow-up with everything else tomorrow. | 16:22 |
JayF | dtantsur: tl;dr my reply is "rescue is the model" | 16:23 |
JayF | dtantsur: it's already a place where we swap networking on a running IPA, and we can use the same pattern, just instead of putting it up on the new network, we batten down the hatches in case it stays running | 16:24 |
dtantsur | okay, so basically on receiving $something, IPA stops its API service and heartbeater? | 16:24 |
Pcmalih_ | Hello this is test message | 16:25 |
JayF | hi Pcmalih_ | 16:25 |
JayF | dtantsur: yeah, basically equivalents of `service iptables panic` and `for i in devs; do ip link set $i down` | 16:25 |
JayF | dtantsur: obviously writing that code will be tricky but I think we can do it | 16:26 |
dtantsur | JayF: mmm, I don't quite like running system actions | 16:26 |
JayF | dtantsur: we do it in rescue | 16:26 |
dtantsur | none of these will work on our ramdisks, for instance | 16:26 |
JayF | this is a ramdisk think though | 16:26 |
JayF | **thing | 16:26 |
dtantsur | because IPA is in a container | 16:26 |
JayF | I'm talking about /in the ramdisk/ | 16:26 |
JayF | IPA is not a container in any upstream ramdisk | 16:26 |
JayF | and we pulled it out explicitly to enable this | 16:26 |
Pcmalih_ | Hi @Jay, I have a question about ironic used in kolla-ansible yoga/stable . VIF attach command is not effctive . Once when i try to depliy image it say no ViF found. | 16:27 |
JayF | because I got tired of patching container runtimes breaking our bios updates :) | 16:27 |
dtantsur | well, it is for us and potentially more people | 16:27 |
JayF | Pcmalih_: hey, can I get a little context? I'm wondering if we work together based on versions/ | 16:27 |
JayF | Then allow the container to do those things? | 16:27 |
dtantsur | but okay, I can do some best effort network shutdown after making sure IPA is silent and not reachable | 16:27 |
JayF | the thing I realize now though | 16:27 |
JayF | and what I *thought* you were alluding to | 16:27 |
JayF | is the other direction | 16:27 |
JayF | when we have an installed thing, when would we switch to IPA network | 16:28 |
JayF | we can't do that nearly as safely because we cannot be assured the OS is no longer running | 16:28 |
dtantsur | sigh, yeah | 16:28 |
dtantsur | also: how do we set the reboot command to IPA that has been shut down? | 16:28 |
dtantsur | I need to think this all through once again. but I think there is a reason why I ended up with the spec like this. | 16:28 |
JayF | you don't, you do it oob | 16:28 |
JayF | Well the thing is, for flat networking this is scary, and now I think it's scary for multitenant networking | 16:29 |
dtantsur | we very explicitly start with in-band to make sure we don't lose any information | 16:29 |
JayF | the A++ solution is bidirectional conversation with the switch | 16:29 |
JayF | where we can see switchport go down as a proxy for reboot | 16:29 |
JayF | even though that can be fooled | 16:29 |
dtantsur | OOB is a hard reset. unless we want to get into the business of using soft reboot | 16:29 |
JayF | dtantsur: prepare_for_network_and_reboot() { sync;sync;sync;sync;sync;sync;./kill-the-network.bash } kinda-/s | 16:30 |
JayF | I'll note the original IPA reboot was "echo s > /proc/sysrq-trigger && echo b > /proc/sysrq-trigger" :) | 16:30 |
JayF | (not that I think that's good, just saying that it's a continuum of tradeoffs) | 16:30 |
dtantsur | Are we ready to stop using in-band reboots for *all cases* then? And rely on sync to work? | 16:31 |
dtantsur | The last thing I want to do is to catch data integrity bugs that only happen when this flag is on | 16:31 |
JayF | Why all cases to cater to a single weird hardware behavior? | 16:31 |
dtantsur | You say it's safe to do, so why keep the complexity | 16:31 |
JayF | yeah, that's why I wouldn't wanna for all cases :/ | 16:31 |
JayF | no, I said it's a contiuum of tradeoffs | 16:31 |
JayF | if we don't have the interfaces we need to do everything right, it's a choice of what things we do less-right | 16:31 |
dtantsur | well, so far nobody asked for the neutron interface support | 16:32 |
dtantsur | so it's weird to try make it work at the expense of the people who actually want this feature... | 16:32 |
JayF | the flat support was originally a concern for me before we even went down this path | 16:32 |
opendevreview | Merged openstack/ironic-lib master: Remove the rest of the rootwrap machinery https://review.opendev.org/c/openstack/ironic-lib/+/928778 | 16:32 |
JayF | [DEFAULT]/its_ok_if_ipa_keeps_running=True | 16:32 |
dtantsur | I like your idea of detecting the running IPA though | 16:32 |
JayF | I'm not 100% serious, but right now I'd almost want this behavior hidden behind a flag for safety | 16:32 |
dtantsur | this can very well be done | 16:32 |
JayF | I don't like the idea that anyone who could update driver_info (a project admin or manager in some cases) could open the security crack to try and jam a crowbar into :) | 16:33 |
dtantsur | Flag - maybe. Although, in the RBAC universe, if you can update driver_info, you can do so many things. | 16:33 |
dtantsur | I mean.. wanna know how our folks made it work for now? | 16:33 |
JayF | if you tell me, will I consider it a security bug? | 16:34 |
dtantsur | they literally install a Redfish proxy between Ironic and hardware that intersects power offs and trun it into reboots | 16:34 |
JayF | consider that before making your decision :) | 16:34 |
JayF | that is actually a slick as hell solution | 16:34 |
dtantsur | :D | 16:34 |
JayF | I don't dislike that at all | 16:34 |
dtantsur | also super fragile | 16:34 |
JayF | I might like that better than your spec LOL | 16:34 |
JayF | well yeah, it gets the fragility outta ironic where you wanna put it LOL | 16:34 |
JayF | "we work around it right now by making it not your problem" - sgtm /s ;) | 16:35 |
dtantsur | for some definition of "you"... we broke this hack at some point, and it became my problem | 16:35 |
JayF | Pcmalih_: You disconnected, did you see: hey, can I get a little context? I'm wondering if we work together based on versions/ | 16:35 |
dtantsur | my point is: if you can change driver_info, you can do anything, including fishing for BMC passwords or changing a lot of Ironic logic | 16:35 |
JayF | dtantsur: I mean, yeah, and you know philosophically I try to save all of us time with the hope it gets paid back to upstream, but I do think it's different when we're talking about taking on potential security risk | 16:35 |
JayF | that's where my head is when I say this | 16:36 |
JayF | remember you're talking to someone who just lost a month to testing patches and doing security fixes | 16:36 |
dtantsur | What's the threat model? A person who can change driver_info but is not trusted to make an informed decision on powering off vs rebooting? | 16:36 |
pcmalih_ | @JayF: I am able to enroll Supermicro using Redfish driver. It entoll, manageable and available but at deploy command gailed with no vif errro. I did create vir using vif attach and its visible using nshow node command. Im using kolla-ansible (manually enabled ironic) | 16:37 |
opendevreview | Merged openstack/ironic-inspector master: Try enabling port 5050 on PUBLIC_BRIDGE https://review.opendev.org/c/openstack/ironic-inspector/+/928783 | 16:37 |
opendevreview | Merged openstack/ironic master: Bring back the metal3-integration job https://review.opendev.org/c/openstack/ironic/+/929286 | 16:37 |
JayF | a tenant who has understanding of this can potentially prevent a reboot from working, causing the tenant OS to be alive in IPA | 16:37 |
dtantsur | how? | 16:37 |
JayF | I've 100% experienced, personally, hardware that out of the box you could essentially make the BMC useless if you loaded the right set of commands | 16:37 |
JayF | this was 10 years ago+IPMI and I hope it doesn't exist still, but ever since then I've been very suspect of BMC actions that we can't verify actually happened | 16:38 |
JayF | power off [did we power off?] power on [did we power on?] mitigates this in a way a single reboot we can't confirm does not | 16:38 |
dtantsur | if you can mess with the BMC, you can work around this limitation too (just make the frontend return the right value without actually changing the power state) | 16:39 |
JayF | that's not how the exploit I personally experienced worked | 16:40 |
dtantsur | if what you're looking for is a flag to disable this feature, I'm happy to provide it | 16:40 |
JayF | I'm looking to have the conversation to see if two smart people with an audience can figure out the safe path through the woods :D | 16:40 |
pcmalih_ | I am able to enroll Supermicro using Redfish driver. It entoll, manageable and available but at deploy command failed with no vif error. I did create vif link by using vif attach command and its visible using show node command. Im using kolla-ansible (manually enabled ironic) | 16:41 |
dtantsur | I don't think there is a 100% safe path to use this feature and also uphold the guarantees of the neutron interface - this is why I opted to fail validation rather than putting people at risk | 16:41 |
JayF | pcmalih_: I'm not sure exactly how to get past that; I would assume something about the tenant networking side is weirdly misconfigured | 16:41 |
JayF | I think it's scary for flat case, you just basically make me go from "it's less scary for neutron" -> "it's more scary for neutron" :D | 16:41 |
dtantsur | we already have quite a few knobs for scary features like disabling cleaning or running firmware updates | 16:42 |
JayF | dtantsur: let me ponder it, as it is, I'd say we'd put it behind a flag that details the security risk in multitenant environments; but I don't wanna give up on thinking of another path | 16:42 |
JayF | dtantsur: if you remember back, I was upset when we added those, too ;) | 16:42 |
* TheJulia exits meeting with scrambled eggs for brains | 16:42 | |
JayF | I am consistenly paranoid :D | 16:42 |
TheJulia | so. much.chatting. | 16:43 |
JayF | pcmalih_: I feel like your nick is slightly familiar but I can't place it, do we know each other? | 16:43 |
pcmalih_ | For tenant networking, can you please confirm that the neutron dns range in the kolla-ansible need s to be same at the physical BMC network or the neutron virtual network created by openstack? | 16:43 |
dtantsur | JayF: I'll also ponder this conversation with a fresher brain, i.e. tomorrow | 16:43 |
JayF | That's probably a better question for the k-a folks, but I can have a look. | 16:43 |
pcmalih_ | Its my first time to chat here but i have attempted to chat in the past. I am part of bluebrint paper release recently. | 16:44 |
TheJulia | pcmalih_: welcome! | 16:45 |
pcmalih_ | This vif unavailability issue is acting as blocker | 16:46 |
JayF | yeah, it's just tough trying to answer questions about years-old releases in install automation I don't know about | 16:46 |
JayF | I'll have a look but please set expectations appropriately with that in mind :) | 16:47 |
pcmalih_ | Ah.. but for the sake of clarity. What is the expected value for neutron dhcp subnet? | 16:50 |
JayF | that's what I'm looking for friend, I just don't know k-a code so it's not immediate | 16:50 |
JayF | that doesn't always 1:1 map with an ironic concept | 16:50 |
JayF | dtantsur: I documented this conversation with a link to the logs and a tldr of my take on the conclusion for now in the spec, so it's not lost | 16:51 |
dtantsur | thx | 16:51 |
pcmalih_ | @JayF: do you mean i need to use Biforst etc instead on Kolla-aansible? | 16:51 |
JayF | Think of Ironic as raw material :) kolla-ansible, bifrost, metal3, openstack-ansible, etc all use it in slightly different ways | 16:51 |
JayF | I know a lot about the raw material but not so much about this shape of it :D | 16:52 |
pcmalih_ | Okay so what is the role of VIF? What configurations it expects? | 16:52 |
JayF | pcmalih_: jay in kolla-ansible on unmaintained/yoga via v3.11.9 \n ❯ rg neutron_dhcp_subnet -- is the field named something different? | 16:53 |
JayF | pcmalih_: well that's just the thing: it depends on context; if using neutron network interface you'd have (I believe?) a vif in the cleaning *and* tenant networks | 16:53 |
TheJulia | what is going on with vif availability? | 16:54 |
TheJulia | the quick tl;dr please | 16:54 |
pcmalih_ | Do it need ml2 driver configuration to get vif detected by node? | 16:57 |
TheJulia | vifs are not detected by the node, ironic gets told the user required vif mapping(s) | 16:58 |
TheJulia | ironic creates/deletes some others as needed for cleaning/deployment/rescue | 16:59 |
TheJulia | but it always cleans those up | 16:59 |
TheJulia | There were some issues around that, but I think I got them all fixed by Train | 16:59 |
*** awb_ is now known as awb | 17:00 | |
TheJulia | And being one of the people who spent a lot of time chasing down and trying to remedy those issues, I'm curious to understand *exactly* what is taking place | 17:01 |
pcmalih_ | @TheJulia: would you please send link to those fixes in Train? We also find Train to be working one. But yoga/stable is strucking my neck! | 17:01 |
TheJulia | (how is that as service, when the board chair chimes in between emails) | 17:01 |
* JayF adds another train ironic installation to his sadness board /s | 17:02 | |
TheJulia | https://review.opendev.org/q/I8d683d2d506c97535b5a8f9a5de4c070c7e887df was one of them, but that was in queens days | 17:05 |
TheJulia | which makes me wonder if there is something entirely separate happening outside of our control | 17:05 |
pcmalih_ | Unmaintained/yoga with bifrost works fine but stable/yoga with k-a is failing at vif nit found | 17:07 |
pcmalih_ | Same machine, same driver etc | 17:07 |
TheJulia | so kolla-ansible is failing where reporting vif not found? | 17:08 |
pcmalih_ | #TheJuliayeso | 17:08 |
TheJulia | So yoga could be an RBAC configuration issue, but that where is going to be critical to understand | 17:09 |
TheJulia | since Wallaby is where we and other projects started evolving the RBAC model heavily | 17:10 |
pcmalih_ | @TheJulia yes k-a is failing to detect VIF | 17:12 |
TheJulia | pcmalih_: so I don't understand what you mean when you say that | 17:13 |
TheJulia | if you can supply a complete error or sniplet of log, that would help orient the discussion | 17:13 |
* TheJulia sighs | 17:13 | |
Pcmalih_ | VIF list command didn’t show anything. Even if i create VIF using vif create command. | 17:16 |
TheJulia | so if your using ironic, and your asking for a vif list, you see nothing? | 17:16 |
TheJulia | what user roles are you running your account with? | 17:16 |
Pcmalih_ | Yes openstack baremetal node vif list | 17:17 |
Pcmalih_ | User role is Admin. | 17:18 |
TheJulia | So the tl;dr is you do "openstack baremetal node vif create x y z", and then you do "openstack baremetal node vif list" and there is nothing? | 17:19 |
TheJulia | no intermediate step/action taken? | 17:20 |
TheJulia | Pcmalih_: can you verify you have the reader role? https://github.com/openstack/ironic/blob/unmaintained/yoga/ironic/common/policy.py#L833-L840 | 17:22 |
Pcmalih_ | First I enable ironic in k-a, set ironic_dnsmasq_interface to the physical interface, ironic_dnsmasq_dhcp_range to range of Ip ands sbnet created by neutron where dhcp_agent and subnet is attached. | 17:23 |
Pcmalih_ | Set ironic_cleaning_network: to public (default created network). S set ironic_dnsmasq_boot_file to pxelinux.0 | 17:24 |
Pcmalih_ | Then create node and set its redfish and node paramenters. Then create neutron interface and attached it to node with vif create command. Then do vif list. | 17:25 |
TheJulia | wow | 17:27 |
TheJulia | Pcmalih_: your proxy/router/browser dislikes the web irc client interface | 17:28 |
Pcmalih_ | My webclient sessionkeep on disconnecting after few minute | 17:29 |
TheJulia | yeah | 17:29 |
JayF | Pcmalih_: I suggest a free sub to irccloud.com; that might work better for you | 17:29 |
TheJulia | Every 2.5 minutes most likely, your network/router/proxy is likely slaying the sockets | 17:30 |
TheJulia | since that is the maximum "theoretical" stale keepalive time | 17:30 |
TheJulia | Pcmalib: JayF just recommended using irccloud, it is free and has extra logic to handle things like breaking connections | 17:30 |
Pcmalib | @TheJulia: I may have missed your message after [22:25] <Pcmalih_> Then create node and set its redfish and node paramenters. Then create neutron interface and attached it to node with vif create command. Then do vif list | 17:31 |
TheJulia | Pcmalib: so, we need you to confirm the roles your user has from keystone because honestly, it sort of sounds like your missing the reader role | 17:31 |
JayF | Pcmalib: fwiw, also https://meetings.opendev.org/irclogs/%23openstack-ironic/ exists to find things you might have missed :) although I do recommend a better client | 17:32 |
TheJulia | so if secure rbac is enabled by kolla-ansible on that version and you somehow don't have the reader role, your *should* end up with an empty list or an access denied | 17:32 |
* TheJulia doesn't remember exactly which but thinks it is an empty list in herently if you have a valid user | 17:32 | |
Pcmalib | Do you want me to manually add readers role in adminrc file and use that? | 17:34 |
TheJulia | uhh, it is whatever is in keystone, not what is defined in adminrc | 17:34 |
TheJulia | so you have to look at the rights keystone has on file | 17:34 |
TheJulia | becuase ironic turns around, takes your token, goes and asks keystone for "what is it really" | 17:34 |
TheJulia | and uses *that* | 17:34 |
TheJulia | can't trust an authenticated user as an inherently authorized user | 17:35 |
JayF | openstack user show $user ## might tell you the roles | 17:36 |
JayF | that is a command that exists, unsure if it gives you what you need | 17:36 |
TheJulia | I think it does | 17:36 |
TheJulia | I think | 17:36 |
TheJulia | it has been a while since I've invoked it | 17:36 |
jssfr | `openstack role assignment list --user $user --names` | 17:36 |
jssfr | is what you're looking for. I don't think `user show` shows the roles. | 17:37 |
TheJulia | jssfr: your a life saver! | 17:37 |
* JayF says loudly to nobody in particular that he'd like a bag full of money | 17:37 | |
JayF | I just figured while people were dropping in and granting wishes :D | 17:37 |
JayF | (ty jss) | 17:37 |
Pcmalib | @Julia Appreciate that and will give it a try today asap. | 17:38 |
* TheJulia goes turns on the surround sound and starts Rammstein Radio on Pandora | 17:38 | |
TheJulia | .... I was supposed to touch code today, wasn't I ? | 17:38 |
jssfr | you're welcome | 17:38 |
jssfr | also pro tip: don't mix up `openstack role remove` and `openstack role delete`. | 17:38 |
jssfr | the latter will have you find out the hard way ~~how to restore data from mariadb WALs~~ if your backups work. | 17:38 |
Pcmalib | @JayF how to tag someone in the reply? Like i am trying my luck @ keywork | 17:39 |
TheJulia | jssfr: I'm twitching already | 17:39 |
JayF | most clients just need the person's name said to highlight | 17:39 |
JayF | the @blah vs blah: vs ... are all just aestetic choices | 17:39 |
TheJulia | for example, I type jay<tab> and irccloud turns that into JayF, and I then just add a : | 17:39 |
JayF | irccloud adds the ': ' for me, too | 17:40 |
JayF | is that configurable? | 17:40 |
TheJulia | it might be | 17:40 |
JayF | not that I care, I'm just intrigued if we have idfferent behavior | 17:40 |
TheJulia | I've been using irccloud for like 10 years, so I might be missing some settings | 17:40 |
TheJulia | oh, if it is the beginning of a line, it does add it automatically | 17:40 |
TheJulia | So, Time for Ethyl 2-cyanoacrylate ? | 17:42 |
clarkb | weechat will only append : if it happens at the beginning of a message | 17:42 |
TheJulia | so next irc release would be 26.1 right? | 17:44 |
clarkb | IRC is specified in RFC which are simply numbered and don't have relationship to previous versions? Anyway https://modern.ircdocs.horse/ looks like a collection of things that could be the next rfc (but that probably won't happen) | 17:48 |
JayF | IRCv3 is a thing that exists, and irccloud supports it | 17:52 |
JayF | the irccloud network is an IRCv3 network | 17:53 |
JayF | if more things used v3 we'd have less need for matrix and friends tbh | 17:53 |
TheJulia | err, not IRC, Ironic | 17:54 |
TheJulia | sorry, brain needs migraine meds | 17:54 |
opendevreview | Julia Kreger proposed openstack/ironic master: Add Prelude for end of cycle release https://review.opendev.org/c/openstack/ironic/+/929564 | 17:56 |
TheJulia | JayF: ^ | 17:56 |
TheJulia | edit at will | 17:57 |
JayF | I wonder if it's worth specifically calling out that was feedback given in a BM SIG that we followed up on | 17:58 |
JayF | seems weird for a prelude but maybe an oppo to show people the value of participation | 17:58 |
TheJulia | I focused mainly on features in our reno, I didn't get back to the sig etherpad | 17:59 |
* TheJulia takes migraine meds | 17:59 | |
* TheJulia also takes the meds which should have been taken with breakfast | 18:00 | |
* JayF +2s it | 18:00 | |
JayF | I couldn't find a way to make it sound not-bad to do what I suggested | 18:01 |
JayF | and you covered the things I was specifically thinking should be in there | 18:01 |
TheJulia | I know the block bios stuff was in that etherpad | 18:01 |
* TheJulia expects operators to scream in joy to being able to block people from adding bios nodes | 18:01 | |
JayF | I was surprised to hear people wanted that | 18:02 |
JayF | cid: ^ she's talking about your feature, btw | 18:02 |
TheJulia | When you think about it, it makes a lot of sense | 18:02 |
TheJulia | because users get stuck in their ways and may not really grok the difference there | 18:02 |
JayF | It basically allows operators with clouds where multiple people are enrolling to remove a decoder ring :) | 18:02 |
TheJulia | I can tell you, with my more PM-ey hat on, I'm acutely worried about baremetal hardware users perceptions as we move forward over the next couple years | 18:03 |
TheJulia | since eventually Centos/RHEL won't be bios bootable at all anymore too | 18:03 |
TheJulia | Yup | 18:03 |
* cid starts reading 2+ hours of chat log to get upto speed with the discussion. | 18:03 | |
JayF | no need cid, just pointing out TheJulia said opers were gonna jump for joy over your features | 18:03 |
TheJulia | cid: eh, maybe 20 minutes :) | 18:03 |
JayF | specifically the allowed boot modes stuff | 18:04 |
TheJulia | cid: thank you for putting in the effort on that! | 18:04 |
JayF | TheJulia: looking at the etherpad; we filed 3 RFEs, one complete, one in progress (awaiting feedback from CERN actually), one unmoved | 18:04 |
JayF | We have an email out to CERN to help us figure out behavior of stress-ng with >1 GPU to enable burnin | 18:05 |
TheJulia | stevebaker[m]: speaking of, I bet we might want to start asserting that config in openstack-k8s-operators at some point since I think we're going to hit a major pain point when we get to our next major version if we don't toggle it out of the gate | 18:05 |
TheJulia | what was the unmoved one? | 18:05 |
JayF | https://bugs.launchpad.net/ironic/+bug/2069083 | 18:05 |
JayF | tl;dr enable burnin as an inspection thing | 18:05 |
TheJulia | ahh, yeah | 18:06 |
JayF | so you can get metrics on speed and catch anomolous hardware | 18:06 |
TheJulia | and that makes sense only with the rest of the pipeline there | 18:06 |
TheJulia | ++ | 18:06 |
TheJulia | makes tons of sense | 18:06 |
TheJulia | I... wonder if someone like NobodyCam might be able to make use of similar | 18:06 |
JayF | oh, we should mention image types in there too | 18:07 |
* JayF does it | 18:08 | |
JayF | yeah nevermind it's in upgrade | 18:09 |
JayF | and I don't want our happy prelude to have sad padlocks | 18:09 |
* cid feels a sense of accomplishment, having run through all of the chats now. | 18:35 | |
cid | JayF, TheJulia: Oh, ty! It really was a team effort. | 18:36 |
JayF | yep, you've been here the whole release | 18:36 |
JayF | it's nice to take a minute, take a deep breath, and realize you achieved something | 18:36 |
JayF | then get back to work! :D | 18:36 |
JayF | lol | 18:36 |
cid | Indeed | 18:37 |
opendevreview | cid proposed openstack/ironic master: [WIP] Add inspection rules https://review.opendev.org/c/openstack/ironic/+/918303 | 20:58 |
cid | masghar: ^^ (take another look when you can). | 21:08 |
* cid EOD | 21:08 | |
TheJulia | g'night | 21:12 |
TheJulia | :) | 21:12 |
opendevreview | cid proposed openstack/ironic master: [WIP] Add inspection rules https://review.opendev.org/c/openstack/ironic/+/918303 | 21:23 |
opendevreview | Julia Kreger proposed openstack/ironic master: trivial: fix http result code on ImageInvalid https://review.opendev.org/c/openstack/ironic/+/929574 | 21:32 |
stevebaker[m] | TheJulia: Hey I have one comment on the prelude | 21:43 |
TheJulia | stevebaker[m]: oh? | 21:44 |
JayF | TheJulia: that patch ^^ makes me wonder if there's ever a world where we would hook up image safety check to a validate call | 21:44 |
stevebaker[m] | just clarity | 21:44 |
TheJulia | JayF: that would make me tableflip and drink tons | 21:44 |
TheJulia | since that is a fundimental meaning of the call change | 21:44 |
TheJulia | given it reutnrs a bunch of data today which is cursory | 21:44 |
JayF | fair | 21:44 |
TheJulia | anyway | 21:44 |
opendevreview | Julia Kreger proposed openstack/ironic master: Add Prelude for end of cycle release https://review.opendev.org/c/openstack/ironic/+/929564 | 21:46 |
TheJulia | stevebaker[m]: done | 21:46 |
stevebaker[m] | TheJulia: cool, do you care that you lost the backtick formatting on that edit? | 21:48 |
TheJulia | anywayugh | 21:48 |
opendevreview | Julia Kreger proposed openstack/ironic master: Add Prelude for end of cycle release https://review.opendev.org/c/openstack/ironic/+/929564 | 21:49 |
stevebaker[m] | sweet, approved :) | 21:51 |
JayF | readded my +2 as well, thanks Julia \o/ | 21:55 |
opendevreview | Merged openstack/ironic master: Add Prelude for end of cycle release https://review.opendev.org/c/openstack/ironic/+/929564 | 22:14 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!