Monday, 2023-01-30

vanougood morning ironic04:49
*** akahat|rover is now known as akahat07:05
dtantsurkubajj: re Red Hat: the office in Brno is hiring nearly constantly. I know you're not really fond of Brno though ;)09:30
rpittaugood morning ironic! o/09:37
kubajjMorning everyone!11:01
kubajjdtantsur: I've seen some remote jobs on the website, so I think I could work from Prague. But I think I still have time to figure stuff out.11:03
dtantsurtrue11:03
kubajjdtantsur: do you agree that if swift object (either inventory or plugin) exists, but fails to delete, we just log it and let the node delete?11:04
opendevreviewMerged openstack/ironic master: Imported Translations from Zanata  https://review.opendev.org/c/openstack/ironic/+/87206011:11
dtantsurkubajj: I'm largely undecided. That was what I suggested, but I think TheJulia objected? Or do I remember it wrong?11:11
iurygregorygood morning Ironic11:16
kubajjdtantsur: I think TheJulia doesn't like to orphan things, and JayF suggested we orphan and make loud noises about orphaning11:16
dtantsurkubajj: option #3 which we already use in some cases: only allow orphaning in maintenance mode11:22
dtantsurWe do this because logs are not really visible for e.g. a CLI user11:31
kubajjdtantsur: when is maintainance used?11:50
dtantsurkubajj: it's a boolean flag on a node that lets operators mark a node as "undergoing maintenance" or not operating properly12:17
dtantsurwhen this mode is on, we allow actions with such a node that are not allowed normally12:17
dtantsurlike forced deletion12:17
kubajjdtantsur: would anyone destroy a node without maintenance then?12:18
dtantsurkubajj: yeah, it's normal to delete nodes that are not in maintenance. but Ironic is more picky about this.12:19
dtantsure.g. it will let you delete nodes in "available" and "manageable" states, but not "active"12:19
kamleshChauvhanHi Ironic.13:11
*** tkajinam is now known as Guest288213:11
kamleshChauvhancan someone please help to second review for backported patch https://review.opendev.org/c/openstack/sushy/+/86877713:11
opendevreviewRiccardo Pittau proposed openstack/ironic-python-agent bugfix/8.3: Fix CI  https://review.opendev.org/c/openstack/ironic-python-agent/+/87198013:39
opendevreviewRiccardo Pittau proposed openstack/ironic-python-agent bugfix/8.3: Fix CI  https://review.opendev.org/c/openstack/ironic-python-agent/+/87198013:42
TheJuliadtantsur: ahh, good idea w/r/t maintenance14:14
dtantsurgood morning TheJulia 14:22
* TheJulia attempts to wake up14:25
opendevreviewJakub Jelinek proposed openstack/ironic master: Erase swift inventory entry on node deletion  https://review.opendev.org/c/openstack/ironic/+/87139414:50
kubajjdtantsur, or TheJulia: could either of you see why test_inspect_utils.SwiftCleanUp.test_clean_up_swift_entries_with_exception throws the SwiftObjectStillExists exception even though I set http_status to 404 in the change above?14:51
TheJuliaWould anyone be willing to review the sharding patches? We're starting to run short on time and we need them merged to finally begin fixing the issue with nova14:55
TheJuliakubajj: weeeird14:55
TheJuliakubajj: line 96 should be return, not pass14:56
TheJuliasorry!14:56
kubajjTheJulia: I see. Thanks14:56
JayFTheJulia: fwiw I'm going to focus on the port querying part of that today14:58
JayFTheJulia: explicitly at the expense of almost anything else14:58
JayFso someone else is gonna have to guard the gates lol14:58
* TheJulia is sad she never got a guard the gate pin14:59
JayFthis is OSS; you want a pin you gotta build it yourself lol14:59
JayFactually, you did15:00
TheJuliawell, yeah :)15:00
JayF#startmeeting ironic15:00
opendevmeetMeeting started Mon Jan 30 15:00:20 2023 UTC and is due to finish in 60 minutes.  The chair is JayF. Information about MeetBot at http://wiki.debian.org/MeetBot.15:00
opendevmeetUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.15:00
opendevmeetThe meeting name has been set to 'ironic'15:00
JayFGood morning, who all is here?15:00
iurygregoryo/15:00
dtantsuro/15:00
TheJuliao/15:00
rpittauo/15:01
* TheJulia stumbles across the channel saying "brraaaaainns"15:01
JayF#topic Announcements15:01
JayFAs always, please hashtag your patches #ironic-week-prio when they are ready for review, and use the priority dashboard in the whiteboard to do reviews.15:01
JayF#link https://ironicbaremetal.org/blog/baremetal-sig-2023q1/15:01
JayFQ1 2023 Baremetal SIG is happening Weds, f15:02
JayF**Weds, Feb 8 from 1500-1700 UTC.15:02
JayFPlease see the linked blogpost for details.15:02
* TheJulia sighs since she will be in Dallas, TX15:02
JayFAight, next up15:02
JayF#topic review previous action items15:03
JayFI am going to push my action about LP; releasenotes job stole that time15:03
JayF#action JayF to bring a launchpad migration plan to a future meeting15:03
JayFI did fulfill the actions I had to 1) update bugfix branch policies (these have merged) and 2) clean up core list to remove now-inactive members :( 15:04
JayFvanou had an item to update our security docs; I haven't seen that come across15:04
JayFvanou: do you have that or should we push it too?15:04
kubajjo/15:04
JayFI don't think vanou is here; I'll push the action so we can check next week15:05
TheJuliaI suspect vanou is asleep15:05
JayF#action vanou follow up on ironic security docs action from 1/23 meeting15:05
JayFThat's it for action items.15:05
JayF#topic Review Ironic CI status15:05
JayFIronic CI has been bonkers.15:05
JayFI fixed issues caused in the releasenotes job by the presence of the bugfix-x.y-eol tags -- we now configure reno to ignore them.15:06
JayFI know rpittau has continued to work on making some stable branches happy, too. Thanks for that.15:06
JayFI'd say I expect master should work, but some stable/bugfix branches may still be struggling15:06
rpittauJayF: I'm removing the reno jobs from the bugfix branches, doesn't make sense to have that there15:06
JayF++15:06
opendevreviewJakub Jelinek proposed openstack/ironic master: Erase swift inventory entry on node deletion  https://review.opendev.org/c/openstack/ironic/+/87139415:07
JayFif you find something wrong, please put it in the whiteboard or comment in IRC :)15:07
JayFMoving on15:07
JayF#topic review 2023.1 workstream progress15:07
JayFI'll note I haven't found much time for getting the last bit of sharding done; I'm going to try and hide somewhere today and complete it.15:07
JayF#link https://etherpad.opendev.org/p/IronicWorkstreams2023.115:08
JayFlooks like sqlalchemy 2.0 is just waiting on the metal3 CI job15:08
JayFand merging inspector into ironic; kubajj has made great progress ... but I'm not sure it seems we'll get totally migrated this cycle? How is it looking?15:08
dtantsurTotally - absolutely not15:09
dtantsurthe introspection data API is just the first step15:09
TheJuliaDoes a list or rough plan exist someplace?15:09
dtantsuryes, on storyboard. hold on15:09
dtantsur#link https://storyboard.openstack.org/#!/story/2010275 Plan on merging inspector in ironic15:10
JayFOK; that is super useful, thank you15:10
TheJuliaI've added the link to the etherpad15:10
JayFTBH at this point I assume storyboard data is wrong unless someone tells me otherwise :| 15:10
JayFThank you for that work. We need to land much of this as the release will be here before we realise it :D 15:11
JayFmoving on15:11
JayF#topic Project Maintainer Audit @ PyPI15:11
JayF#link https://etherpad.opendev.org/p/openstack-pypi-maintainers-cleanup15:11
JayFThe results found for ironic in ^ can only be best described by an emoji screaming15:12
JayFWe have multiple repos with maintainers whom are no longer associated with the community, and in one case, not identifyable at all15:12
JayFI am asking for unanimous consent to move forward with the TC plan to remove all existing maintainers except openstackci. 15:12
* iurygregory see his name in the list XD15:12
rpittauabsolutely, let's do it15:13
JayFTBH; I'm not sure this is something that we as a community haven't even already consented to, and it's the right thing to do, but I wanted to provide a venue in case there is any disagreement15:13
TheJulia... *blink* *blink*15:14
iurygregoryno objections from me 15:14
* TheJulia wonders how monty ended up a maintainer on bifrost15:14
JayFgenerally speaking; it's whoever created the PyPI project originally15:15
JayFlike I suspect the `login.launchpad.net_154` is probably one of my ex-managers at Rackspace, when we created it15:15
JayF(it being IPA)15:15
JayFI'm going to consider this agreed.15:16
JayF#agreed Ironic is onboard with TC plans to unify PyPA maintainership under openstackci15:17
* dtantsur has no objections15:17
JayF#undo15:17
opendevmeetRemoving item from minutes: #agreed Ironic is onboard with TC plans to unify PyPA maintainership under openstackci15:17
JayF#agreed Ironic is onboard with TC plans to unify PyPI maintainership under openstackci15:17
JayFAight, next up.15:18
JayFNo RFEs to review, skipping that15:18
JayF#topic virtualpdu situation15:18
JayFrpittau: o/15:18
TheJuliadid they respond?15:18
rpittauyes, the status is not good15:18
rpittauthey did not respond15:19
JayFSo can we be clear how Ironic is impacted by the issue with VirtualPDU?15:19
TheJulia++15:19
JayFJust to reset back to the original trigger15:19
TheJuliaI would like to understand that15:19
rpittauironic jobs depend on virtualpdu15:19
dtantsurIt's used in the SNMP job15:19
TheJuliayes15:19
rpittauvirtualpdu is not python3 compatible15:19
TheJuliaoh, heh15:20
rpittauwe can't move to jammy15:20
iurygregoryoh wow15:20
rpittauthis is in short15:20
TheJulia... and without an active maintainer.. ugh15:20
JayFSo we really don't have a lot of time to resolve this either; then15:20
rpittauthis https://review.opendev.org/c/openstack/ironic/+/869052 is failing because of that 15:20
dtantsurwhich options we have? 1) fork virtualpdu, 2) put it in a container with old ubuntu, 3) drop the job?15:20
JayF4) wait longer15:20
TheJulia5) Ask to adopt it15:21
TheJulia?15:21
dtantsurWe cannot ask if they don't respond15:21
JayFTheJulia: #5 has been done; we've not gotten any response15:21
dtantsurIf we decide to wait, the container approach can be a workaround15:21
dtantsurto proceed with jammy15:21
rpittauthe project is abandoned since years now btw15:21
JayFHonestly, I think it makes sense to fork it15:21
TheJuliano response does sort of lend ourselves to lazy consensus ability15:21
JayFWe're the only people in the world with a real need to reliably support fake-hardware emulation lol15:22
dtantsurThink about this: we can always remerge out fork if they wake up15:22
rpittaulast update was 4 years ago https://opendev.org/x/virtualpdu15:22
JayFYep. 15:22
JayFSo it's under opendev already?15:22
rpittauyeah15:22
JayFDoes anyone have any idea what the process is for this?15:22
JayFand/or want to own the communication bits of this? 15:22
rpittauI can handle that15:22
TheJuliathere likely needs to be a dialog with the opendev folks to figure out what they are most comfortable with15:23
rpittaujust need some tips :015:23
JayFI'd go into #opendev and tell them the situation15:23
TheJuliagiven being in the x/ namespace, it was once likely in openstack/15:23
TheJulia++15:23
JayFthat we've agreed in Ironic to adopt it15:23
rpittausounds good15:23
JayFand we're willing to do that as a fork if needed15:23
JayFor not as a fork if hijacking is an option (I probably prefer not? but they might have context we don't)15:23
JayF#action rpittau to communicate with opendev/openstack folks to determine how best for Ironic community to adopt/fork https://opendev.org/x/virtualpdu15:24
JayFrpittau: goes without saying, if you have any trouble or confusion please ask :) I don't have much special knowledge but I can ask everyone :D 15:25
rpittauof course :)15:25
JayFAnything else on virtualpdu?15:25
rpittauthat's all I guess15:25
JayF#topic Open Discussion15:25
JayFanything for open discussion?15:25
TheJulianothing from my point of view at this time15:26
JayFAight, I'm going to call it.15:26
JayF#endmeeting15:26
opendevmeetMeeting ended Mon Jan 30 15:26:45 2023 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)15:26
opendevmeetMinutes:        https://meetings.opendev.org/meetings/ironic/2023/ironic.2023-01-30-15.00.html15:26
opendevmeetMinutes (text): https://meetings.opendev.org/meetings/ironic/2023/ironic.2023-01-30-15.00.txt15:26
opendevmeetLog:            https://meetings.opendev.org/meetings/ironic/2023/ironic.2023-01-30-15.00.log.html15:26
TheJuliaiurygregory: o/15:29
iurygregoryo/15:29
TheJuliagot a little bit to chat about i-p-e?15:30
iurygregorygoing for lunch now, can be when I'm back? 15:30
TheJuliasure15:30
opendevreviewJulia Kreger proposed openstack/ironic-prometheus-exporter master: WIP: Support extraction of ironic internal metrics  https://review.opendev.org/c/openstack/ironic-prometheus-exporter/+/86950915:34
opendevreviewRiccardo Pittau proposed openstack/ironic-python-agent bugfix/8.3: Fix CI  https://review.opendev.org/c/openstack/ironic-python-agent/+/87198016:07
iurygregoryTheJulia, I'm back o/16:18
TheJuliaiurygregory: cool16:22
* TheJulia looks for one of the windows she had open16:22
TheJuliaiurygregory: https://meet.google.com/vbm-vqku-mhe16:24
TheJuliaand https://paste.openstack.org/show/b2AeycAXKHCNoa5rMyUp/16:24
iurygregoryok16:25
opendevreviewJulia Kreger proposed openstack/ironic-prometheus-exporter master: WIP: Support extraction of ironic internal metrics  https://review.opendev.org/c/openstack/ironic-prometheus-exporter/+/86950916:57
TheJuliaiurygregory: https://paste.openstack.org/show/bzclDetApkB5UdslepcX/17:06
TheJuliawith that patch above :)17:06
iurygregoryTheJulia, ack17:07
rpittaugood night! o/17:11
TheJuliagoodnight rpittau 17:12
JayFSo I'm implementing querying ports by shards21:53
JayFand I found what appears to be a bug in our API code21:53
JayFHOWEVER fixing it could throw 400 errors to existing clients21:53
JayF(those clients wouldn't be getting the results they wanted anyway)21:53
JayFit proactively raises here if we specify both a node_ident and portgroup_ident: https://github.com/openstack/ironic/blob/master/ironic/api/controllers/v1/port.py#L25421:54
JayFbut in the actual if statement, just below in L257-285, it's obvious that *address* is mutually exclusive, too21:54
JayFsince I'm implementing shards, and that would be mutually exclusive, it's trivial to make this case error instead of returning incorrect results, but I'm unsure if that's the right path21:55
JayFTheJulia: ^ thoughts?21:55
JayFtl;dr: if you query ports with both a node||portgroup AND an address, the address gets completely ignored21:57
JayFwhereas correct behavior would indicate returning a 400 as it's a bad query21:57
TheJuliaso node_ident is largely going to be populated, for the most part, via subclassed resources of /v1/nodes/node/node-1/portgroups/21:58
TheJuliagetting the collection asking for specifics21:59
TheJuliahmmm21:59
TheJuliayeah, you can't filter at that level21:59
TheJuliaso, I *think* it is valid to add shard to that list as well to just raise 40021:59
JayFso there /is no/ current way for someone to say21:59
JayFfor instance21:59
JayF /v1/ports?node_ident=some_uuid&address=some-mac21:59
TheJuliaso that is a different query path22:00
TheJuliahmmmm22:00
JayFthat doesn't call _get_ports_collection?22:00
TheJulialooking22:00
TheJuliait would be /v1/ports/?node=node-1&address=blah22:01
JayFthat still populates node_ident at that level afaict, yeah22:02
TheJuliaaddress is still valid22:02
TheJuliait is portgroup_ident which is for the subclassed resource view22:03
JayFnot implemented that way, no22:03
JayFso if you look22:03
JayFyou get in PortsCollection.get_all()22:03
JayFwhen node is passed in22:03
JayFit converts that to a node_ident which is passed to _get_ports_collection()22:03
JayFwhich is where we're at22:03
TheJuliaoh22:03
TheJuliai see22:03
TheJuliahttps://github.com/openstack/ironic/blob/4e8705dc9f6219b0af8d6d2769fb5aa554f458c5/ironic/api/controllers/v1/port.py#L257-L28522:03
TheJuliaparameter precedence22:03
JayFexactly22:04
JayFif you set node and address, address is ignored22:04
TheJuliacorrect22:04
JayFwhen an operator might assume, based on our api-ref, that querying by node+address would return a port ONLY if the port's address matched22:04
JayFwhich breaks our API contract22:04
JayFso I lean towards making it a 400 ... but maybe hiding that behavior behind a microversion :| 22:05
TheJuliaI think we would need to double check the precise wording of the docs22:05
TheJuliaconundrum here is node_ident is used by >1 means22:05
JayFhttps://docs.openstack.org/api-ref/baremetal/?expanded=list-ports-detail#id8322:05
JayFTheJulia: is your suggestion we might need to actually hook up support for address+node_ident? 22:06
TheJuliano22:07
TheJuliaI think we need to just explicitly say "this is how it works"22:07
JayFSo you're saying we should retcon the docs and call it expected behavior, more or less?22:07
TheJuliayeah22:07
JayFlol22:07
JayFit's the obviously right thing to do22:07
TheJuliabecause unwinding that is going to be crazy22:08
TheJuliaand breaking in multiple ways :(22:08
JayFif I couldn't fix it with the docs; I'd probably go the route of22:08
TheJuliasince we would need to delineate everything out and if the caller is specific22:08
TheJuliaand that would spaghettize some of the code path too22:08
JayFputting a check for if address+node_ident is set to bail22:08
JayFbut I like fixing the docs a lot better lol22:09
TheJuliaconundrum is that would break way down elsewhere22:09
* TheJulia doublechecks something esle22:09
TheJuliahttps://docs.openstack.org/api-ref/baremetal/?expanded=list-ports-detail,list-ports-by-node-detail#node-management-nodes22:10
TheJuliaaddress is not listed there :)22:10
TheJulianor here: https://docs.openstack.org/api-ref/baremetal/?expanded=list-ports-detail,list-ports-by-node-detail,list-portgroups-by-node-detail#node-management-nodes22:10
opendevreviewJay Faulkner proposed openstack/ironic master: Fix API docs to document port filtering behavior  https://review.opendev.org/c/openstack/ironic/+/87222922:13
JayFTheJulia: look under /v1/ports, not /v1/ports/detail22:13
TheJuliaJayF: I know, but those other paths is that same code path22:14
JayFI'm not sure I understand why this is relevant?22:14
JayFaddress being None is OK is all cases22:14
JayFaddress being not-none and node/node_uuid/portgroup being set is not great22:15
TheJuliait is the code path why it is dual used code22:15
JayFand what I updated the docs to improve22:15
TheJuliabecause of parent node_idents or portgroup_ident22:15
JayFyeah, that's why node_ident can be passed in at api query or class creation time, yeah22:15
JayFeither way, I think my patch ^^ covers it22:16
JayFI'm going to take feedback on that patch and move back to making sharding work22:16
TheJulia++22:16
vanougood morning ironic23:14
vanouJayF: I'm working on writing security doc. I think I can propose draft this week. Sorry for slow progress.23:15
JayFvanou: it's not urgent; we were just following up on the action item in the meeting23:15
JayFvanou: since you're asleep then, if you wanna put an update for it in the meeting agenda (under follow up on action items) you can; or I'll just keep a lookout for this :) 23:16
JayF(do not wake up early / stay up late to go to the meeting though; sleep is important :D)23:16
vanouUnderstand. Thanks a lot :)23:18
JayFWhere are you in the world, vanou 23:19
JayFjust curious23:19
JayFI'm in Washington state in the USA, top left corner of the US. :) 23:19
vanouI'm in Japan23:19
JayFThat's what I assumed but wasn't sure :D23:20
* TheJulia misses Japan23:20
vanouDC is place I want to go. Good place.23:20
vanouJayF TheJUlia Have you gone Japan?23:21
JayFI had a chance to go many years ago when the summit was in Tokyo; I passed23:21
JayFI am a large man even by US standards (6' over 300 lbs), so I didn't think Tokyo would be well suited for me :D23:21
TheJuliaI've been to Japan twice, First time was for the Summit, the second time was for a vacation. I spent lots of time roaming around tokyo by foot23:22
* TheJulia goes and jumps in the car for a few hours of driving :(23:23
JayFI have been around the world a bit; Japan would be a place I'd like to visit but that's probably 100 lbs and many many years away :D 23:23
vanouJayF TheJulia: Nice :) Maybe summit related to OSS?23:24
JayFWe'll see :D I can really only travel freely during the summer; going to conferences and such while school is in session is difficult for us right now23:24
JayF(my wife is a teacher and we have animals that can't be left alone/with a sitter)23:25
opendevreviewJay Faulkner proposed openstack/ironic master: Allow port queries by shard list  https://review.opendev.org/c/openstack/ironic/+/87223523:26
JayFdon't get excited, ^ it's a wip that needs tests :P 23:26
vanouJayF: I see. Animal family gives nice life. I have cat too. He is important part of my life.23:26
JayFbut everything is hooked up end to end 23:26
JayFvanou: yeah, we have a cat who has cancer and gets medicine daily, and a dog who can't be left alone all day :/23:27
JayF(The sick cat is 100% in remission right now, and is doing incredibly well; but he still gets the medicine)23:27
opendevreviewJay Faulkner proposed openstack/ironic master: Allow port queries by shard list  https://review.opendev.org/c/openstack/ironic/+/87223523:28
vanouJayF: Then long/short travel should be avoided, naturally. I wish his health better23:29
vanouTheJulia: Regarding backport patch https://review.opendev.org/c/openstack/ironic/+/870880. This patch consists of 2 part: (1)Verify step to check HTTP(S) connection possible (2)Vendor interface which allows fetching iRMC firmware version. To deal with firmware HTTP(S) incompatibility which commit message tells, only first part of code is required. Second part of code was to23:36
vanoube23:36
vanouused to deal with firmware IPMI incompatibility storyboard tells. I think adding new vendor interface in backport patch is not good, as you said. So I think I remove second part of code from23:36
vanou backport patch. How do you t23:36
vanou^ How do you think about it?23:37
JayFYou'd still keep the logic though, yeah? So existing operators would be seeing new traffic to their nodes (e.g. using IPMI in some cases that it's not currently?) ... or am I confusing multiple patches?23:45

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!