| *** JasonF is now known as JayF | 00:57 | |
| rpittau | good morning ironic! o/ | 08:08 |
|---|---|---|
| rpittau | JayF: awesome, thanks for the reno fix, reviewing the patches now | 08:10 |
| rpittau | if another core has a moment please review https://review.opendev.org/c/openstack/ironic-inspector/+/871788 https://review.opendev.org/c/openstack/ironic/+/871787 https://review.opendev.org/c/openstack/ironic-python-agent/+/871786 to fix the reno job and unlock CI, thanks! | 08:15 |
| dtantsur | done | 08:52 |
| opendevreview | Merged openstack/ironic-python-agent master: Make reno ignore bugfix eol tags https://review.opendev.org/c/openstack/ironic-python-agent/+/871786 | 09:03 |
| opendevreview | Merged openstack/ironic-inspector master: Make reno ignore bugfix eol tags https://review.opendev.org/c/openstack/ironic-inspector/+/871788 | 09:08 |
| opendevreview | Merged openstack/ironic-python-agent master: Readd usedevelop true to tox.ini https://review.opendev.org/c/openstack/ironic-python-agent/+/871728 | 09:28 |
| kubajj | dtantsur: is it possible that a failed test locked a node and never released it on my local vm? | 09:40 |
| dtantsur | kubajj: a unit test or something else? | 09:44 |
| kubajj | dtantsur: yes, a unit test | 09:45 |
| kubajj | I am getting: ironic.common.exception.NodeLocked: Node 1be26c0b-03f2-4d2e-ae87-c02d7f33c123 is locked by host fake-mini, please retry after the current operation is completed. | 09:45 |
| dtantsur | unit tests operate on a temporary in-memory database. so it must be something within the test run itself | 09:45 |
| kubajj | dtantsur: ok | 09:45 |
| kubajj | thanks | 09:45 |
| kubajj | dtantsur: I think my problem is that I'm acquiring a task on a node that I'm about to destroy | 09:52 |
| kubajj | That shouldn't work, right? | 09:52 |
| dtantsur | kubajj: we always acquire a task for any changes | 09:53 |
| dtantsur | you're possibly doing it twice :) | 09:53 |
| samuelkunkel[m] | Good Morning, is someone still missing from https://review.opendev.org/c/openstack/sushy/+/870888? rpittau already reviewed it. I am not 100% familiar how the process goes from here :) | 10:04 |
| opendevreview | Merged openstack/ironic master: Make reno ignore bugfix eol tags https://review.opendev.org/c/openstack/ironic/+/871787 | 10:19 |
| kubajj | dtantsur: yes, I think that's the problem. The destroy_node calls it from its body as well. Can I mock the task that's returned? | 10:26 |
| dtantsur | kubajj: I don't think mocking is the answer: you may get the same problem in the real code | 10:27 |
| dtantsur | where is the 2nd task created? | 10:27 |
| kubajj | dtantsur: I wanted to make sure that the cleanup swift entries function is called once with a task, but I don't see how to get the task created in destroy node. | 10:28 |
| dtantsur | kubajj: not sure what you mean. You have a task already in https://review.opendev.org/c/openstack/ironic/+/871394/3/ironic/conductor/manager.py | 10:29 |
| dtantsur | you just need to use it and avoid creating more tasks | 10:29 |
| kubajj | dtantsur: this is the test I'm trying to fix https://paste.opendev.org/show/bTHSTB7phANgn1IkgJhC/ | 10:33 |
| opendevreview | Merged openstack/bifrost master: Support PXE network boot with grub https://review.opendev.org/c/openstack/bifrost/+/807220 | 10:50 |
| dtantsur | samuelkunkel[m]: approved | 10:56 |
| opendevreview | Riccardo Pittau proposed openstack/ironic master: Bump cirros to version 0.6.1 https://review.opendev.org/c/openstack/ironic/+/871812 | 10:56 |
| dtantsur | (you need 2 reviewers in most cases) | 10:56 |
| dtantsur | kubajj: I'm a bit puzzled, could you upload the updated patch? | 10:56 |
| samuelkunkel[m] | ahh, good to know :) | 10:56 |
| samuelkunkel[m] | thanks | 10:56 |
| rpittau | sorry samuelkunkel[m] I gave as assumed that you knew that, my bad :) | 10:58 |
| samuelkunkel[m] | no worries | 10:58 |
| samuelkunkel[m] | maybe its written somewhere within the contributors guide | 10:59 |
| samuelkunkel[m] | and then RTFM is valid | 10:59 |
| opendevreview | Jakub Jelinek proposed openstack/ironic master: Erase swift inventory entry on node deletion https://review.opendev.org/c/openstack/ironic/+/871394 | 11:02 |
| kubajj | dtantsur: this is it ^ | 11:03 |
| *** adam_ is now known as adamrozman | 11:08 | |
| dtantsur | kubajj: I see "NameError: name 'task' is not defined" | 11:36 |
| iurygregory | good morning Ironic | 11:37 |
| dtantsur | o/ | 11:37 |
| kubajj | dtantsur: yeah, that's because I don't know how to mock/get the task in the test | 11:37 |
| dtantsur | it's not critical, you can use mock.ANY | 11:38 |
| opendevreview | Merged openstack/sushy master: workaround: requests verify handling if env is set https://review.opendev.org/c/openstack/sushy/+/870888 | 12:29 |
| kubajj | dtantsur: this might be a very dumb question, but what should in the case you mention in https://review.opendev.org/c/openstack/ironic/+/871394/comments/1590c652_c033cfeb I can add the side effect to raise the exception, but what should I then assert? | 12:39 |
| opendevreview | Kaifeng Wang proposed openstack/ironic master: Fixes console port conflict occurs in certain path https://review.opendev.org/c/openstack/ironic/+/868342 | 12:39 |
| opendevreview | Jakub Jelinek proposed openstack/ironic master: Erase swift inventory entry on node deletion https://review.opendev.org/c/openstack/ironic/+/871394 | 12:56 |
| kubajj | dtantsur: managed to figure out what was the problem. I was adding side effects after calling the function. | 13:10 |
| opendevreview | Merged openstack/ironic master: Docs: Troubleshooting: how to exit clean failed https://review.opendev.org/c/openstack/ironic/+/871129 | 13:25 |
| opendevreview | Kaifeng Wang proposed openstack/ironic master: Relaxing console pid looking https://review.opendev.org/c/openstack/ironic/+/871262 | 13:28 |
| rpittau | JayF, TheJulia, dtantsur, iurygregory: for the bugfix branches I'm still waiting for a couple of patches to merge, please double-check the list I've made: https://etherpad.opendev.org/p/outstanding-patches thanks! | 14:05 |
| iurygregory | rpittau, ack | 14:05 |
| vanou | JayF: Regarding backport of iRMC driver patch. In my understanding, backport of https://review.opendev.org/c/openstack/ironic/+/862616 till Zed and https://review.opendev.org/c/openstack/ironic/+/862552 to Wallaby is first denied but approved in the end. Is my understanding correct? | 14:59 |
| * iurygregory brb going to reboot the laptop (vpn problems) | 15:03 | |
| Sandzwerg[m] | Hi, short question. Looking at the release notes they mention that for retirement to work automatic cleaning needs to be enabled, but the documentation doesn't mention it. But it seems automatic cleaning is needed so the documentation is missing this? | 15:04 |
| dtantsur | arne_wiebalck: ^^^ | 15:06 |
| dtantsur | Without looking into any details, I'd assume cleaning itself is not required.. but let's see what Arne says | 15:06 |
| TheJulia | I think it might be actually, since one of the original requirements I think was to prepare the machine for removal | 15:07 |
| TheJulia | which would involve wiping it | 15:07 |
| dtantsur | Logically yes. But have we encoded this requirement? | 15:08 |
| TheJulia | looking | 15:08 |
| dtantsur | opinions on the failure in https://review.opendev.org/c/openstack/ironic-inspector/+/869719? | 15:09 |
| dtantsur | I don't understand why this change affected coverage.. | 15:09 |
| TheJulia | I've had a few cases recently where I've skirted along the edge of code coverage in inspector recently with minor refactoring, but it was always a bit more than that | 15:12 |
| dtantsur | I wonder if it was a good idea at all | 15:12 |
| TheJulia | the lock? | 15:12 |
| TheJulia | you *could* just nuke it honestly | 15:13 |
| TheJulia | that feels evil, but it feels appropriately evil | 15:13 |
| TheJulia | so yes, to retire a node, you do need to make it through cleaning | 15:13 |
| dtantsur | no, the coverage limit | 15:13 |
| TheJulia | oh... | 15:14 |
| TheJulia | the bulk of what it is complaining about is things like dbsync.py and stuff that is not in the normal testing path | 15:15 |
| TheJulia | because it is a wrapper, or just for the tests to run anyhow | 15:15 |
| dtantsur | right | 15:15 |
| TheJulia | ... (which is confusing to begin with...) | 15:15 |
| TheJulia | I'd +1 a change to lower it to 79% | 15:15 |
| TheJulia | err, +2 | 15:15 |
| TheJulia | with the small size of the code base, 80% is... tough. | 15:16 |
| TheJulia | I was looking for like individual lines I could add testing for last time | 15:16 |
| dtantsur | or just stop failing at all? | 15:16 |
| dtantsur | if we keep lowering the minimum, it's not really better than dropping it | 15:17 |
| TheJulia | well, I think that relates to the timeline on merging | 15:17 |
| TheJulia | true | 15:17 |
| opendevreview | Dmitry Tantsur proposed openstack/ironic-inspector master: Stop gating on unit test coverage https://review.opendev.org/c/openstack/ironic-inspector/+/871827 | 15:18 |
| dtantsur | proposal ^^ | 15:18 |
| Sandzwerg[m] | Regarding retiring and cleaning: https://opendev.org/openstack/ironic/src/branch/master/ironic/conductor/allocations.py#L104 seems to assume there can't be available nodes with retired =true but when I set one active node of ours to retired and afterwards deleted the instance it ended up in available again and currently we have disabled automatic cleaning, that's why I started to look. We are currently running ironic on Xena | 15:19 |
| dtantsur | I have a feeling it's a bug. Even though disabling cleaning is not recommended, we shouldn't just break the retirement feature. | 15:20 |
| TheJulia | so the documentation sets the context forward that the system state will be cleaned | 15:21 |
| Sandzwerg[m] | We want to enable automatic cleaning sooner or later but it feels at least like bug in the documentation if it is a requirement. | 15:21 |
| TheJulia | so it feels more like contractual design requirement with the expectation set | 15:21 |
| TheJulia | I've got a doc update half written at the moment | 15:21 |
| dtantsur | JayF: FYI we're on the list for https://etherpad.opendev.org/p/openstack-pypi-maintainers-cleanup | 15:22 |
| TheJulia | i'm fairlyl sure aeva would not object | 15:24 |
| dtantsur | and Lucas :) | 15:26 |
| dtantsur | well, actually, I won't object either even though I'm still active here | 15:26 |
| dtantsur | unused privileges shall be removed | 15:26 |
| opendevreview | Julia Kreger proposed openstack/ironic master: [DOC[ Set cleaning requirement with retirement https://review.opendev.org/c/openstack/ironic/+/871829 | 15:32 |
| TheJulia | Sandzwerg[m]: Please take a look at ^^^ | 15:32 |
| kubajj | dtantsur, TheJulia: if you had a minute, could you please have a look at https://review.opendev.org/c/openstack/ironic/+/871394 ? | 15:35 |
| TheJulia | so if we have a transient failure, should we orphan data in swift? | 15:36 |
| * TheJulia is going to need to switch over to standing desk mode :( | 15:37 | |
| Sandzwerg[m] | TheJulia: perfect. Also answers the question if I should open a issue for this :) | 15:37 |
| dtantsur | TheJulia: it's not ideal; neither is blocking deletion.. I'm not sure | 15:39 |
| dtantsur | could be a periodic task ;) | 15:39 |
| TheJulia | but to have a periodic, you'd need to record then remove | 15:40 |
| TheJulia | so we would need a deleted field and the nightmare with it, or... a journal of some sort | 15:41 |
| TheJulia | Sandzwerg[m]: You can still file a bug.. The challenge is ultimately kind of trying to find the appropriate balance, and a bug would drive wider discussion. | 15:43 |
| TheJulia | kubajj: what do you think? w/r/t the question of orphaning data in swifth potentially | 15:44 |
| Sandzwerg[m] | Alright I'll take a look and create an account | 15:45 |
| kubajj | TheJulia: I have to read last our of discussion :) | 15:45 |
| TheJulia | no rush, I'm swapping my desk to standing mode so my back stops complaining | 15:46 |
| TheJulia | and done | 15:46 |
| kubajj | TheJulia: so this is orphaning when node deletion fails? | 15:50 |
| TheJulia | The file, yes | 15:50 |
| kubajj | Or what does transient failure mean? | 15:50 |
| kubajj | What happens now? | 15:51 |
| TheJulia | transient failure is like... if the service crashed | 15:51 |
| TheJulia | or... the cable got knocked out in the data center | 15:51 |
| TheJulia | or... fiber cut... or switch rebooted | 15:51 |
| kubajj | I see | 15:52 |
| TheJulia | connectivity is the most common (imho) driver of failures | 15:52 |
| TheJulia | human stood infront of the wifi link for too long, created too much noise, jammed the neighboring link out | 15:52 |
| TheJulia | (humans, being mostly water... and wifi at 2.4GHz... etc. | 15:53 |
| TheJulia | ) | 15:53 |
| kubajj | And the period task would check that the file still exists and if not remove the swift object? | 15:53 |
| kubajj | Sorry for so many questions. ;) | 15:53 |
| TheJulia | how would it knows the file exist though? | 15:53 |
| kubajj | TheJulia: great question. | 15:55 |
| kubajj | So what was the periodic task Dmitry and you suggested? | 15:56 |
| TheJulia | well, I think he is proposing it as a means of cleaning up, the challenge is, how would it know what to clean up | 15:57 |
| TheJulia | I think it might just be better to fail the destroy(), tbh | 15:57 |
| TheJulia | if any exception is not file not found | 15:57 |
| kubajj | TheJulia: how can you recover from failing to destroy though? | 15:59 |
| TheJulia | you don't, the user would have to get an error and retry... | 15:59 |
| kubajj | Then I would say just failing is alright. | 16:03 |
| opendevreview | Merged openstack/ironic-python-agent master: Make logs collection a hardware manager call https://review.opendev.org/c/openstack/ironic-python-agent/+/871708 | 16:26 |
| opendevreview | Merged openstack/ironic-inspector master: Use new get_rpc_client API from oslo.messaging https://review.opendev.org/c/openstack/ironic-inspector/+/871183 | 16:27 |
| opendevreview | Merged openstack/ironic master: Clarify release docs: bugfix releases optional https://review.opendev.org/c/openstack/ironic/+/871537 | 16:28 |
| arne_wiebalck | Sandzwerg[m]: Hi! | 16:39 |
| * arne_wiebalck is reading the backlog | 16:40 | |
| Sandzwerg[m] | 👋 | 16:45 |
| arne_wiebalck | Sandzwerg[m]: IIRC, what the retirement features does are two things: a) not block auto-cleaning (like maintenance would) and b) move the node to manageable (rather than available) at the end of cleaning. | 16:46 |
| opendevreview | Verification of a change to openstack/ironic master failed: Fix grub config path default https://review.opendev.org/c/openstack/ironic/+/869134 | 16:47 |
| arne_wiebalck | Sandzwerg[m]: With manual cleaning b) would not be relevant, I guess, as we started from manageable. | 16:47 |
| arne_wiebalck | Sandzwerg[m]: My question is: what would you expect to happen when you manually clean a node with the retirement flag set? | 16:48 |
| arne_wiebalck | Sandzwerg[m]: I never did that or thought about this :) | 16:49 |
| arne_wiebalck | Sandzwerg[m]: So I cannot say quickly what would happen. | 16:49 |
| Sandzwerg[m] | I saw a case were the node didn't end up in manageable after a active instance was with retired=true was deleted the node ended up in available again | 16:49 |
| arne_wiebalck | aha | 16:49 |
| arne_wiebalck | oh, so this is not for manual cleaning, but for deletion | 16:50 |
| Sandzwerg[m] | That's why I thought that automated cleaning might be requirement even when the current documentation doesn't mention it but the release notes did | 16:50 |
| opendevreview | Merged openstack/ironic master: Use new get_rpc_client API from oslo.messaging https://review.opendev.org/c/openstack/ironic/+/871160 | 16:51 |
| opendevreview | Merged openstack/ironic-inspector master: Stop gating on unit test coverage https://review.opendev.org/c/openstack/ironic-inspector/+/871827 | 16:51 |
| Sandzwerg[m] | In our case we want to implement port groups and have done so for all available nodes, but we thought we could use retired as a feature to make sure that nodes which get returned are not deployed before we configure the port group | 16:52 |
| rpittau | good night! o/ | 16:53 |
| arne_wiebalck | yeah ... I think you're right: this would only work if auto-cleaning is enabled as the new state is picked somewhere at the end of auto cleaning | 16:53 |
| arne_wiebalck | interesting! | 16:54 |
| arne_wiebalck | I see TheJulia has put up a patch to clarify the docs already. | 16:54 |
| arne_wiebalck | but if you need this as a feature, it might be good to file this as a "bug", so we do not forget | 16:55 |
| arne_wiebalck | bye rpittau o/ | 16:55 |
| Sandzwerg[m] | Yeah, I've now opened a bug as well: https://storyboard.openstack.org/#!/story/2010552 | 16:56 |
| Sandzwerg[m] | Maybe it can be closed if the doc patch is merged but I found one place where a assumption is broken by this, not sure if this breaks anything else | 16:57 |
| opendevreview | Verification of a change to openstack/ironic bugfix/21.0 failed: Fix selinux context of published image hardlink https://review.opendev.org/c/openstack/ironic/+/871033 | 16:57 |
| Sandzwerg[m] | We're probably fine by reactivating cleaning, in the worst case we need to disable the steps that are currently broken for us until we get around to make our own cleaning steps | 16:58 |
| arne_wiebalck | Sandzwerg[m]: Yes, that sounds like it would do what you want. | 17:00 |
| arne_wiebalck | Sandzwerg[m]: Thanks for bringing this up! | 17:00 |
| Sandzwerg[m] | Thanks for the quick help, I thought after some time of idling I could use the matrix bridge to see if my assumption was right :) | 17:02 |
| dtantsur | Folks, we discussed with metal3 people from Ericsson the revival of this idea: https://storyboard.openstack.org/#!/story/2010551 | 17:09 |
| dtantsur | This is a fake redfish emulator backend to enable massive scale testing of Ironic and Metal3 | 17:10 |
| JayF | dtantsur: arne_wiebalck: in 15m? | 17:15 |
| JayF | I just got back from taking the cat to the vet | 17:15 |
| dtantsur | JayF: works for me | 17:15 |
| arne_wiebalck | JayF: in 45min, no? | 17:15 |
| * arne_wiebalck hears the Italian food cooking | 17:16 | |
| JayF | arne_wiebalck: it can be at 10a if you want; I'm offering to claw it back to 9:30am again to overlap our time better | 17:16 |
| * arne_wiebalck scheduled dinner before our call and would upset his Italian mother in law when changing the sensitive timing of things | 17:16 | |
| dtantsur | :) | 17:17 |
| JayF | I would never want to get between a guy and Italian home cooking | 17:17 |
| dtantsur | dangerous even! | 17:17 |
| arne_wiebalck | dtantsur: in 43 mins would work for you? | 17:17 |
| dtantsur | I think yes if we don't take too long | 17:17 |
| arne_wiebalck | I think that is the plan anyway :) | 17:17 |
| TheJulia | dtantsur: seems... reasonable I guess. | 17:19 |
| TheJulia | the story, that is | 17:19 |
| JayF | vanou: I'm not sure I see how the patches are different regarding if they can be backported beyond zed | 17:22 |
| JayF | vanou: it's not just my decision though, fwiw. I do think we have to be intentional about making sure we don't backport changes which would change behavior that operators are already relying on or expecting | 17:23 |
| opendevreview | Dan Smith proposed openstack/ironic master: DNM: Test ironic against nova stable uuid change https://review.opendev.org/c/openstack/ironic/+/871838 | 17:34 |
| NobodyCam | Good Morning Ironic Folks | 17:41 |
| opendevreview | Merged openstack/sushy stable/wallaby: Fix volume deletion on newer iDRACs https://review.opendev.org/c/openstack/sushy/+/869934 | 17:43 |
| opendevreview | Verification of a change to openstack/ironic bugfix/19.0 failed: Fix selinux context of published image hardlink https://review.opendev.org/c/openstack/ironic/+/871038 | 17:47 |
| opendevreview | Verification of a change to openstack/ironic stable/yoga failed: Fix selinux context of published image hardlink https://review.opendev.org/c/openstack/ironic/+/871053 | 17:47 |
| TheJulia | blah, evil timer failure | 17:48 |
| JayF | dtantsur: looking at that pypi stuff has been on my list, so I took the time this morning | 17:52 |
| JayF | dtantsur: and holy crap, it's incredibly disturbing. | 17:52 |
| JayF | dtantsur: there's a nonzero chance there's some rackspace shared pypi account (or e.g. Russell) out there with access to IPA; given it's got an extra maintainer of "login.launchpad.net_154" | 17:52 |
| JayF | dtantsur: and I assure you in that case, it'd be a password that hasn't changed in a long, long time | 17:53 |
| opendevreview | Verification of a change to openstack/ironic stable/zed failed: Fix selinux context of published image hardlink https://review.opendev.org/c/openstack/ironic/+/871032 | 17:54 |
| dtantsur | ouch | 17:57 |
| opendevreview | Merged openstack/ironic bugfix/18.1: Fix selinux context of published image hardlink https://review.opendev.org/c/openstack/ironic/+/871040 | 18:01 |
| opendevreview | Verification of a change to openstack/ironic bugfix/20.2 failed: Fix selinux context of published image hardlink https://review.opendev.org/c/openstack/ironic/+/871052 | 18:13 |
| opendevreview | Verification of a change to openstack/ironic stable/xena failed: Fix selinux context of published image hardlink https://review.opendev.org/c/openstack/ironic/+/871039 | 18:26 |
| JayF | dtantsur: https://review.opendev.org/c/openstack/ironic-specs/+/871535 needs to land since you landed the docs update lol | 18:37 |
| JayF | dtantsur: arne_wiebalck: that etherpad has been updated, feel free to muck with it further if you don't like what I did with it :) | 19:05 |
| arne_wiebalck | JayF: thx | 19:09 |
| arne_wiebalck | JayF: LGTM | 19:09 |
| JayF | \o/ | 19:09 |
| arne_wiebalck | bye everyone, see you tmrw o/ | 19:10 |
| JayF | dtantsur: I assume you're onboard to have your user removed from any pypi projects that are OpenStack-governed? Looks like you're the plurality-winner in terms of project creation :P | 19:11 |
| opendevreview | Dan Smith proposed openstack/ironic master: DNM: Test ironic against nova stable uuid change https://review.opendev.org/c/openstack/ironic/+/871838 | 19:16 |
| stevebaker[m] | good morning | 19:32 |
| opendevreview | Jay Faulkner proposed openstack/ironic master: Minor spelling/grammar fixes for release docs https://review.opendev.org/c/openstack/ironic/+/871845 | 19:32 |
| *** tzumainn_ is now known as tzumainn | 19:44 | |
| dansmith | JayF: I left out some words in my reply just now. Ironic shouldn't be affected by that nova change, so I'm running your jobs to make sure it isn't | 19:50 |
| JayF | dansmith: I trust ya, wanted to just ask to make sure you had thought of that :D | 19:51 |
| dansmith | the reason I'm running your jobs against it _because_ I know we're flying close to the ironic third rail and I want to make sure we don't break you | 19:52 |
| TheJulia | dansmith: much appreciated | 19:54 |
| JayF | TBH; I thought we were somewhat co-gated with you all | 19:54 |
| JayF | I guess probably just for ironic driver changes? | 19:54 |
| * JayF can't always keep track of what used to be true vs what the actual state of things is today :| | 19:55 | |
| dansmith | JayF: we don't run most of your jobs, just like you don't run most of ours | 19:55 |
| JayF | ooooh, so you're testing the balance | 19:55 |
| JayF | instead of just the sliver | 19:55 |
| JayF | we really do appreciate that \o/ | 19:55 |
| dansmith | yes, I'm trying to avoid the case where this patch passes our little ironic smoke test, but breaks one of your more complicated jobs after it lands | 19:55 |
| JayF | If there's another core around; https://review.opendev.org/c/openstack/ironic-specs/+/871535 needs another +2A -- the docs update got merged ahead of the policy update (yikes) so I'd like to land this soon please | 20:03 |
| opendevreview | Julia Kreger proposed openstack/ironic master: [DOC] Set cleaning requirement with retirement https://review.opendev.org/c/openstack/ironic/+/871829 | 21:43 |
| opendevreview | Julia Kreger proposed openstack/ironic master: [CI] Swap anaconda urls https://review.opendev.org/c/openstack/ironic/+/871855 | 22:01 |
| TheJulia | That *should* fix the issue why the tempest gate is failing on the anaconda job at the moment | 22:02 |
| JayF | AIUI; the right way to fix bugfix branches release notes is "don't generate release notes from bugfix branches", right? | 23:07 |
| JayF | stable/* should get the yaml config fix | 23:07 |
| JayF | yeah? | 23:07 |
| opendevreview | Jay Faulkner proposed openstack/ironic stable/zed: Make reno ignore bugfix eol tags https://review.opendev.org/c/openstack/ironic/+/871765 | 23:08 |
| JayF | ^^ needed to unbreak stable gate, please land quickly if there are any cores up cc: TheJulia | 23:09 |
| opendevreview | Jay Faulkner proposed openstack/ironic-inspector stable/zed: Make reno ignore bugfix eol tags https://review.opendev.org/c/openstack/ironic-inspector/+/871766 | 23:09 |
| TheJulia | ahh for the cannot concatonate error? | 23:09 |
| opendevreview | Jay Faulkner proposed openstack/ironic-python-agent stable/zed: Make reno ignore bugfix eol tags https://review.opendev.org/c/openstack/ironic-python-agent/+/871767 | 23:09 |
| JayF | TheJulia: the actual thing that happened is reno was seeing bugfix-x.y-eol | 23:10 |
| JayF | assuming that mapped to stable/x.y (due to a regexp) | 23:10 |
| TheJulia | ack, okay | 23:10 |
| TheJulia | k | 23:10 |
| JayF | so we have to exclude bugfix from that regexp | 23:10 |
| JayF | but AFAICT, there's zero reason to run release notes jobs on bugfix branches, yeah? | 23:11 |
| TheJulia | just wanted to make sure because I was starting to look at some failures | 23:11 |
| JayF | master should be good | 23:11 |
| JayF | this is backporting to fix stables | 23:11 |
| TheJulia | only to validate if we need to merge an override | 23:11 |
| JayF | and trying to sus out the best way to fix bugfix | 23:11 |
| TheJulia | uhh.... I'm not quite so sure about master branch | 23:11 |
| TheJulia | but... I'll double check | 23:11 |
| JayF | master branch release notes jobs are good | 23:11 |
| JayF | the known issues are resolved | 23:11 |
| JayF | the unknown issues are seemingly-limitless | 23:11 |
| TheJulia | eh, hmm | 23:12 |
| JayF | I'm just going to backport the relnotes fix thru bugfix branches | 23:12 |
| JayF | since there's no good answer here :) | 23:12 |
| TheJulia | failure at 8:47 am local, but that patch merged on master at 2-something am | 23:12 |
| TheJulia | oh, the answer is if we need a version specific override, it has to be merged on that branch | 23:12 |
| opendevreview | Jay Faulkner proposed openstack/ironic bugfix/21.2: Make reno ignore bugfix eol tags https://review.opendev.org/c/openstack/ironic/+/871768 | 23:13 |
| opendevreview | Jay Faulkner proposed openstack/ironic bugfix/21.0: Make reno ignore bugfix eol tags https://review.opendev.org/c/openstack/ironic/+/871769 | 23:13 |
| opendevreview | Jay Faulkner proposed openstack/ironic bugfix/20.2: Make reno ignore bugfix eol tags https://review.opendev.org/c/openstack/ironic/+/871870 | 23:14 |
| opendevreview | Jay Faulkner proposed openstack/ironic stable/yoga: Make reno ignore bugfix eol tags https://review.opendev.org/c/openstack/ironic/+/871871 | 23:14 |
| opendevreview | Jay Faulkner proposed openstack/ironic bugfix/19.0: Make reno ignore bugfix eol tags https://review.opendev.org/c/openstack/ironic/+/871872 | 23:14 |
| opendevreview | Jay Faulkner proposed openstack/ironic stable/xena: Make reno ignore bugfix eol tags https://review.opendev.org/c/openstack/ironic/+/871873 | 23:15 |
| opendevreview | Jay Faulkner proposed openstack/ironic bugfix/18.1: Make reno ignore bugfix eol tags https://review.opendev.org/c/openstack/ironic/+/871874 | 23:15 |
| opendevreview | Jay Faulkner proposed openstack/ironic stable/wallaby: Make reno ignore bugfix eol tags https://review.opendev.org/c/openstack/ironic/+/871875 | 23:15 |
| opendevreview | Jay Faulkner proposed openstack/ironic stable/victoria: Make reno ignore bugfix eol tags https://review.opendev.org/c/openstack/ironic/+/871876 | 23:15 |
| opendevreview | Jay Faulkner proposed openstack/ironic stable/ussuri: Make reno ignore bugfix eol tags https://review.opendev.org/c/openstack/ironic/+/871877 | 23:15 |
| opendevreview | Jay Faulkner proposed openstack/ironic stable/train: Make reno ignore bugfix eol tags https://review.opendev.org/c/openstack/ironic/+/871878 | 23:16 |
| opendevreview | Jay Faulkner proposed openstack/ironic-inspector bugfix/11.0: Make reno ignore bugfix eol tags https://review.opendev.org/c/openstack/ironic-inspector/+/871879 | 23:17 |
| opendevreview | Jay Faulkner proposed openstack/ironic-inspector bugfix/10.12: Make reno ignore bugfix eol tags https://review.opendev.org/c/openstack/ironic-inspector/+/871880 | 23:17 |
| opendevreview | Jay Faulkner proposed openstack/ironic-inspector stable/yoga: Make reno ignore bugfix eol tags https://review.opendev.org/c/openstack/ironic-inspector/+/871881 | 23:17 |
| opendevreview | Jay Faulkner proposed openstack/ironic-inspector bugfix/10.9: Make reno ignore bugfix eol tags https://review.opendev.org/c/openstack/ironic-inspector/+/871882 | 23:18 |
| opendevreview | Jay Faulkner proposed openstack/ironic-inspector stable/xena: Make reno ignore bugfix eol tags https://review.opendev.org/c/openstack/ironic-inspector/+/871883 | 23:18 |
| opendevreview | Jay Faulkner proposed openstack/ironic-inspector bugfix/10.7: Make reno ignore bugfix eol tags https://review.opendev.org/c/openstack/ironic-inspector/+/871884 | 23:18 |
| opendevreview | Jay Faulkner proposed openstack/ironic-inspector stable/wallaby: Make reno ignore bugfix eol tags https://review.opendev.org/c/openstack/ironic-inspector/+/871885 | 23:19 |
| opendevreview | Jay Faulkner proposed openstack/ironic-inspector stable/victoria: Make reno ignore bugfix eol tags https://review.opendev.org/c/openstack/ironic-inspector/+/871886 | 23:19 |
| opendevreview | Jay Faulkner proposed openstack/ironic-inspector stable/ussuri: Make reno ignore bugfix eol tags https://review.opendev.org/c/openstack/ironic-inspector/+/871887 | 23:20 |
| opendevreview | Jay Faulkner proposed openstack/ironic-inspector stable/train: Make reno ignore bugfix eol tags https://review.opendev.org/c/openstack/ironic-inspector/+/871888 | 23:20 |
| opendevreview | Jay Faulkner proposed openstack/ironic-python-agent bugfix/9.0: Make reno ignore bugfix eol tags https://review.opendev.org/c/openstack/ironic-python-agent/+/871889 | 23:20 |
| opendevreview | Jay Faulkner proposed openstack/ironic-python-agent bugfix/8.6: Make reno ignore bugfix eol tags https://review.opendev.org/c/openstack/ironic-python-agent/+/871890 | 23:20 |
| opendevreview | Jay Faulkner proposed openstack/ironic-python-agent stable/yoga: Make reno ignore bugfix eol tags https://review.opendev.org/c/openstack/ironic-python-agent/+/871891 | 23:21 |
| opendevreview | Jay Faulkner proposed openstack/ironic-python-agent bugfix/8.3: Make reno ignore bugfix eol tags https://review.opendev.org/c/openstack/ironic-python-agent/+/871892 | 23:24 |
| opendevreview | Jay Faulkner proposed openstack/ironic-python-agent stable/xena: Make reno ignore bugfix eol tags https://review.opendev.org/c/openstack/ironic-python-agent/+/871893 | 23:24 |
| opendevreview | Jay Faulkner proposed openstack/ironic-python-agent bugfix/8.1: Make reno ignore bugfix eol tags https://review.opendev.org/c/openstack/ironic-python-agent/+/871894 | 23:24 |
| opendevreview | Jay Faulkner proposed openstack/ironic-python-agent stable/wallaby: Make reno ignore bugfix eol tags https://review.opendev.org/c/openstack/ironic-python-agent/+/871895 | 23:24 |
| opendevreview | Jay Faulkner proposed openstack/ironic-python-agent stable/victoria: Make reno ignore bugfix eol tags https://review.opendev.org/c/openstack/ironic-python-agent/+/871896 | 23:24 |
| opendevreview | Jay Faulkner proposed openstack/ironic-python-agent stable/ussuri: Make reno ignore bugfix eol tags https://review.opendev.org/c/openstack/ironic-python-agent/+/871897 | 23:25 |
| opendevreview | Jay Faulkner proposed openstack/ironic-python-agent stable/train: Make reno ignore bugfix eol tags https://review.opendev.org/c/openstack/ironic-python-agent/+/871898 | 23:25 |
| JayF | can someone please land those without me doing a for patch in patches; set hashtag ironic-week-prio; done :D | 23:26 |
| JayF | I'll tag any tomorrow that aren't approved | 23:26 |
| TheJulia | errr /home/jkreger/ironic/releasenotes/source/liberty.rst:3:Duplicate explicit target name: "irmc driver documentation". | 23:26 |
| TheJulia | JayF: doing so | 23:26 |
| JayF | TheJulia: hmm, is that local-only? | 23:26 |
| TheJulia | yeah, with latest state of master branch | 23:26 |
| JayF | TheJulia: I haven't seen that in any of the jobs, and for mine to have merged overnight it had to generate notes... | 23:26 |
| TheJulia | I am not sure the job runs with -W though.... | 23:27 |
| * JayF going to clean gerrit/master and running the tox job | 23:27 | |
| TheJulia | I'd have to dig at it | 23:27 |
| TheJulia | ++ | 23:27 |
| TheJulia | i've got a patch and some trash in my working folder | 23:27 |
| TheJulia | so ++++++ | 23:27 |
| JayF | my testing was the exact gate command run | 23:27 |
| TheJulia | i was running tox -ereleasenotes | 23:28 |
| JayF | it's breaking for me locally now too | 23:28 |
| JayF | WTF | 23:28 |
| JayF | current status: having a breakdown over release notes live on youtube | 23:29 |
| JayF | it runs with -W in the gate | 23:30 |
| JayF | did a new version get released or something? | 23:30 |
| TheJulia | I would suspect so, but have not looked | 23:31 |
| TheJulia | yet | 23:31 |
| TheJulia | are you seriously going wtf live?! | 23:31 |
| JayF | yes | 23:31 |
| TheJulia | awesome! | 23:31 |
| JayF | https://zuul.opendev.org/t/openstack/builds?job_name=build-openstack-releasenotes&project=openstack/ironic-python-agent IPA is happy, I explicitly tested IPA yesterday | 23:32 |
| JayF | it's a nonzero possibilty that I didn't test ironic | 23:32 |
| TheJulia | it happens, just surprised it merged | 23:33 |
| JayF | I think it never ran the job | 23:33 |
| JayF | I'm digging that moreso than the fix right now | 23:33 |
| JayF | for sanity sake lol | 23:33 |
| TheJulia | dunno | 23:33 |
| JayF | https://review.opendev.org/c/openstack/ironic/+/871787 | 23:34 |
| JayF | releasenotes job did not run | 23:34 |
| TheJulia | whiskey tango | 23:34 |
| TheJulia | it never ran the releasenotes job on ironic | 23:34 |
| JayF | what repo are the openstack job templates in? | 23:34 |
| TheJulia | https://review.opendev.org/c/openstack/ironic/+/871787 | 23:35 |
| opendevreview | Merged openstack/ironic-python-agent stable/zed: Make reno ignore bugfix eol tags https://review.opendev.org/c/openstack/ironic-python-agent/+/871767 | 23:35 |
| JayF | TheJulia: yeah, I'm a little ahead of you ^^^ lol | 23:35 |
| TheJulia | k | 23:35 |
| JayF | OH! | 23:35 |
| JayF | OH! I know why! | 23:35 |
| JayF | it's in root of repo /zuul.yaml | 23:35 |
| JayF | s/zuul/reno/ | 23:35 |
| JayF | in IPA / inspector it was releasenotes/config.yaml | 23:35 |
| JayF | I'm going to move the ironic config so we can get a blown-up PR to work off of | 23:35 |
| TheJulia | k | 23:36 |
| JayF | my hope is the regexp is busted | 23:36 |
| JayF | to not exclude kilo-eol or w/o | 23:36 |
| JayF | w/e | 23:36 |
| opendevreview | Jay Faulkner proposed openstack/ironic master: Move reno config so that releasenotes job will run https://review.opendev.org/c/openstack/ironic/+/871862 | 23:37 |
| JayF | TheJulia: ima go abandon a bunch of backports right now :| | 23:37 |
| TheJulia | :( | 23:37 |
| TheJulia | move quickly | 23:38 |
| TheJulia | don't bother with the comment | 23:38 |
| JayF | W-1 will stop a merge, yeah? | 23:38 |
| TheJulia | it should | 23:38 |
| opendevreview | Verification of a change to openstack/ironic bugfix/21.0 failed: Make reno ignore bugfix eol tags https://review.opendev.org/c/openstack/ironic/+/871769 | 23:38 |
| TheJulia | just the ironic chnage, the other ones are good it looks like it? | 23:38 |
| JayF | yes | 23:39 |
| JayF | I think I caught 'em all | 23:39 |
| JayF | So we know now... 1) Ironic was never fixed, 2) Why the releasenotes job didn't catch it in CI | 23:40 |
| JayF | we don't know why Ironic is not working still though | 23:40 |
| JayF | confirmed #871862 is running the releasenotes job | 23:42 |
| JayF | so at least that piece of the mystery is resolved | 23:42 |
| JayF | so my main working theory is that we're not properly excluding kilo-eol OR that there's some unique case to Ironic (like a change only being on a bugfix branch, maybe?) that's breaking us | 23:42 |
| JayF | liberty is adjacent to one of the retired bugfix branches... | 23:42 |
| TheJulia | crazy question, was liberty one fof the branches that you closed out? | 23:43 |
| JayF | I did a lot of bugfix/x.y -> bugfix-x.y-eol | 23:43 |
| JayF | branch -> tag | 23:43 |
| JayF | there were liberty-adjacent bugfix branches retired, I suspect | 23:43 |
| JayF | given how many I retired | 23:43 |
| TheJulia | eh, we werent doing them back in liberty days | 23:43 |
| * TheJulia checks remote branch state | 23:43 | |
| TheJulia | it *might* be this repo | 23:43 |
| TheJulia | on my lcoal machine | 23:44 |
| JayF | I have reproduced the failure | 23:44 |
| JayF | on a clean branch | 23:44 |
| JayF | it's not you | 23:44 |
| TheJulia | show the error on y our channel | 23:44 |
| TheJulia | okay, that is different from what I get | 23:44 |
| TheJulia | nah, double checking what we did with kilo real quick | 23:45 |
| TheJulia | one minute | 23:45 |
| JayF | TheJulia: it's on my stream | 23:45 |
| opendevreview | Merged openstack/ironic-inspector stable/zed: Make reno ignore bugfix eol tags https://review.opendev.org/c/openstack/ironic-inspector/+/871766 | 23:47 |
| JayF | this has to be b/c we changed the match groups coming out of the regexp unintentionally | 23:47 |
| JayF | and it's relying on match groups | 23:47 |
| * TheJulia needs a new desktop | 23:48 | |
| JayF | because that err is in an `if match:` branch | 23:48 |
| TheJulia | hmm. getting errors on mitaka now | 23:48 |
| JayF | but at no point does reno check to ensure it has the match groups it expects | 23:48 |
| TheJulia | le sigh | 23:48 |
| opendevreview | Merged openstack/ironic-inspector bugfix/10.12: Make reno ignore bugfix eol tags https://review.opendev.org/c/openstack/ironic-inspector/+/871880 | 23:48 |
| TheJulia | it implies a match, but never validates it | 23:48 |
| JayF | yeah | 23:48 |
| JayF | but either way, I think it'd just break differently | 23:48 |
| JayF | like, I will put a pin in this to fix in reno | 23:49 |
| JayF | and give a better error | 23:49 |
| JayF | but I think this is secondary and the real problem is ours | 23:49 |
| TheJulia | I'm going to go wf-1 other stuffs | 23:49 |
| JayF | https://zuul.opendev.org/t/openstack/build/58dab4a3ba0a4ea3a18fccdca529ae4d that failure is a good sign | 23:49 |
| JayF | at least we aren't crazy | 23:49 |
| JayF | yeah, I'm going to poke at this for another 10 | 23:49 |
| JayF | see if I can figure something out | 23:49 |
| opendevreview | Merged openstack/ironic-inspector stable/victoria: Make reno ignore bugfix eol tags https://review.opendev.org/c/openstack/ironic-inspector/+/871886 | 23:50 |
| opendevreview | Verification of a change to openstack/ironic-python-agent bugfix/8.6 failed: Make reno ignore bugfix eol tags https://review.opendev.org/c/openstack/ironic-python-agent/+/871890 | 23:50 |
| opendevreview | Verification of a change to openstack/ironic-python-agent stable/victoria failed: Make reno ignore bugfix eol tags https://review.opendev.org/c/openstack/ironic-python-agent/+/871896 | 23:50 |
| opendevreview | Verification of a change to openstack/ironic-python-agent stable/wallaby failed: Make reno ignore bugfix eol tags https://review.opendev.org/c/openstack/ironic-python-agent/+/871895 | 23:50 |
| opendevreview | Verification of a change to openstack/ironic-python-agent stable/ussuri failed: Make reno ignore bugfix eol tags https://review.opendev.org/c/openstack/ironic-python-agent/+/871897 | 23:50 |
| opendevreview | Verification of a change to openstack/ironic-python-agent stable/xena failed: Make reno ignore bugfix eol tags https://review.opendev.org/c/openstack/ironic-python-agent/+/871893 | 23:51 |
| opendevreview | Verification of a change to openstack/ironic-python-agent stable/train failed: Make reno ignore bugfix eol tags https://review.opendev.org/c/openstack/ironic-python-agent/+/871898 | 23:51 |
| JayF | got it | 23:51 |
| JayF | TheJulia: ^^ | 23:51 |
| opendevreview | Verification of a change to openstack/ironic-inspector stable/wallaby failed: Make reno ignore bugfix eol tags https://review.opendev.org/c/openstack/ironic-inspector/+/871885 | 23:51 |
| opendevreview | Verification of a change to openstack/ironic-inspector stable/ussuri failed: Make reno ignore bugfix eol tags https://review.opendev.org/c/openstack/ironic-inspector/+/871887 | 23:51 |
| opendevreview | Verification of a change to openstack/ironic-inspector stable/xena failed: Make reno ignore bugfix eol tags https://review.opendev.org/c/openstack/ironic-inspector/+/871883 | 23:52 |
| opendevreview | Verification of a change to openstack/ironic-inspector stable/yoga failed: Make reno ignore bugfix eol tags https://review.opendev.org/c/openstack/ironic-inspector/+/871881 | 23:52 |
| opendevreview | Jay Faulkner proposed openstack/ironic master: Move and fix reno config for releasenotes job https://review.opendev.org/c/openstack/ironic/+/871862 | 23:53 |
| JayF | TheJulia: https://review.opendev.org/c/openstack/ironic/+/871862 | 23:53 |
| JayF | TheJulia: I cropped an 'r' off the front of the regexp that Adam sent me | 23:53 |
| TheJulia | g'night :) | 23:54 |
| TheJulia | so looks like inspector was good | 23:56 |
| TheJulia | eh, other failures. tomorrow | 23:59 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!