| opendevreview | Goutham Pacha Ravi proposed openstack/governance master: Resolution: Provide an AI Policy Interpretation https://review.opendev.org/c/openstack/governance/+/1012080 | 07:59 |
|---|---|---|
| opendevreview | Goutham Pacha Ravi proposed openstack/governance master: Resolution: Provide an AI Policy Interpretation https://review.opendev.org/c/openstack/governance/+/1012080 | 08:00 |
| opendevreview | Goutham Pacha Ravi proposed openstack/governance master: Resolution: Provide an AI Policy Interpretation https://review.opendev.org/c/openstack/governance/+/1012080 | 08:00 |
| opendevreview | Goutham Pacha Ravi proposed openstack/governance master: Resolution: Provide an AI Policy Interpretation https://review.opendev.org/c/openstack/governance/+/1012080 | 08:03 |
| opendevreview | Goutham Pacha Ravi proposed openstack/governance master: Resolution: Provide an AI Policy Interpretation https://review.opendev.org/c/openstack/governance/+/1012080 | 08:06 |
| *** elodilles_pto is now known as elodilles | 08:52 | |
| *** jlarriba_ is now known as jlarriba | 10:21 | |
| fungi | on the llm use topic, https://lists.debian.org/debian-devel/2026/11/msg00174.html struck me as an interesting way to reframe concerns in a way that resonates with experiences our community also has: "...how we can use new tools responsibly without turning the additional capacity they provide into unsustainable obligations" | 18:14 |
| clarkb | fungi: I think that is in line with the things I've mentioned about reorganizing effort and your comments on using llms to create a lot more of the straightforward work which creates more work for htose in positions that are harder to automate (because judgement or security or whatever) | 18:20 |
| fungi | yes, also short-term savings that cost us more in the long run, a.k.a. "borrowing from the future" | 18:21 |
| clarkb | essentially we made writing new features much easier. But we haven't (yet anyway) made the process of accepting and maintaining new features easier. So put the breaks on the feature side we don't need more effort there necessarily. Instead figure out how to use the tools to improve the actual bottlenecks we have | 18:22 |
| clarkb | s/breaks/brakes/ | 18:22 |
| dansmith | yeah, and that was my point in -nova a couple hours ago when it was said that some contributor(s) were leaving claude running autonomously to fix things and push to gerrit, presumably from further review comments or zuul test results while they sleep | 18:31 |
| dansmith | if I'm having a conversation with your agent over gerrit comments but I think I'm talking to a human, I'm very mad when I find out the truth | 18:32 |
| dansmith | and also, I have my own agent to do things I tell it if that's what I want | 18:32 |
| fungi | as has been rephrased repeatedly elsewhere, if anyone wants to have a conversation with a computer they already know where to get that | 18:33 |
| dansmith | yeah | 18:33 |
| gouthamr | :( yeah, code review conversations are a huge part of the "community" aspect as well.. automated responses to human review seem callous. i've seen a lot of places where maintainers are asking "did you just copy-paste what your LLM said?" - but more recently, there's no copy-pasting.. seems like people are handing out their gerrit creds and automating that part. I hate -2, but this would be one where i would unfortunately | 18:38 |
| gouthamr | like a collective -2 - not to say i'm the only one that feels that way | 18:38 |
| fungi | how i've explained it elsewhere is that a lot of people think llm use is saving work, but it's really just an effective means of shifting their work onto someone else | 18:40 |
| fungi | so it's saving *them* work, while creating additional work for others | 18:41 |
| fungi | there are things llms are efficient at, and things the're worthless at. unfortunately, the former is speeding up and we haven't, collectively, figured out how to shift people onto focusing on the latter instead | 18:42 |
| gouthamr | i still believe the agentic-workflows effort is worth it to keep maintainer sanity.. but, we need more folks to participate in the opinion making there | 18:45 |
| clarkb | fungi: yes that is exactly how it feels with the problems I've run into with ai contributions. It can often feel like I'm the first person spending any effort to understand and apply judgemtn to the proposals | 18:48 |
| clarkb | But this also hasn't been a universal experience. I think there are people doing the work upfront on their end before making it someone elses problem. The issue is that thsi isn't universal and ai tools make it so easy to skip this step | 18:49 |
| fungi | if llm technology makes people 10x better at solving one problem we have but doesn't help with another problem we have, then we only need 1/10 as many people working on the first thing and the other 9/10 of them can come help with the other | 18:49 |
| clarkb | I suspect it gets harder and harder to do over time too as AI tooling erodes at those skills/abilities | 18:49 |
| dansmith | gouthamr: yes, I have said "please stop just proxying to your LLM. If I wanted that I'd just do it locally and cut out the latency" | 18:54 |
| clarkb | it really feels like a situation where people leading through example will be critical, but most of the examples we have are not compelling (because they don't address the actual bottlenecks) or are actually negatives | 18:54 |
| gouthamr | dansmith++ | 19:01 |
| gouthamr | clarkb: any ideas on how we can surface "people leading with examples"? | 19:02 |
| gouthamr | maybe we have some of these people write some blog posts? | 19:03 |
| ChristianBerendt[m] | I used to really like the guide on the wiki with commit messages that included both good and bad examples. Maybe we should pick out a few reviews that were created with AI assistance and were acceptedand the kinds we don't want to see and post them in an official place so we can refer to them. I'm afraid, however, that acceptance itself depends very heavily on the specific projects. With commit messages, it's basically the | 19:06 |
| ChristianBerendt[m] | same thing. | 19:06 |
| fungi | (once we get the wiki back online) | 19:06 |
| dansmith | it's also kinda hard to convey what was going on behind the scenes where you realized (late) that you were talking to a machine | 19:07 |
| fungi | it's the uncanny valley of code review | 19:07 |
| dansmith | commit messages are (kinda) either good or bad, but the bad behavior here is a bit harder to call out and mark the bad part red | 19:07 |
| gouthamr | problem is the bad ones when asked for an explanation or a change of direction were quick to point out that (1) we don't have a written policy (2) we don't have documentation of the expectations - and then we're back in a circle :D i've been backfilling docs where i concede.. but where there is clear policy or documentation and yet the contributor or their tooling ignores these i put my hands up | 19:21 |
| dansmith | gouthamr: I commented on your patch to make it explicit :) | 19:21 |
| gouthamr | ++ ty dansmith | 19:22 |
| clarkb | gouthamr: I don't think blog posts are the right idea. I think pushing code your llm wrote for you that is ready for review, actually solves the problem listed on the tin, etc is what needs to happen | 19:27 |
| clarkb | before you could have an llm do this work for you you had to invest personal time into understanding the problem well enough to get close to a solution. This necessarily craeted an understanding on the part of the person working with the code that solved many of these issues. | 19:29 |
| clarkb | But now people can skip those steps, assume the llm is correct, and make it my problem to figure out | 19:29 |
| * TheJulia blinks and reads the chat | 21:27 | |
| TheJulia | gouthamr: I think at some point, it may be the needful to revise the code of conduct. | 21:28 |
| TheJulia | And, I realize suggesting that it is basically a chilling effect, but the reality is no matter how much some people obsess with making llms do everything, we still require humans in the loop and communities are about human engagement. | 21:29 |
| gouthamr | > But now people can skip those steps, assume the llm is correct, and make it my problem to figure out | 21:52 |
| gouthamr | Yes, and that's what the TC's AI policy is arguing as well.. we're asking contributors and maintainers to apply the same standards to AI assisted contributions as they would regular contributions.. "AI said it was correct" shouldn't be entertained | 21:52 |
| gouthamr | > gouthamr: I think at some point, it may be the needful to revise the code of conduct. | 21:55 |
| gouthamr | TheJulia: yes, i agree. we must cross link the AI policy there perhaps, although i'm trying to emphasize an over-arching "maintainers decide what's acceptable, and contributors must adhere to these" line in the TC's interpretation.. | 21:55 |
| fungi | which i think https://openinfra.org/legal/ai-policy sort of dances around too where it states, "This policy does not supersede any project specific requirements around actions to be taken prior to the submission of changes into review." | 21:56 |
| gouthamr | ++ don't think it can be more prescriptive at that level.. | 21:59 |
| TheJulia | Yeah, the intent is not to handycap but to try and guide at the highest level | 21:59 |
| enricokern | for new features and the use of AI i am also kind of skeptical. But for bugfixing etc. Anyway whatever endlevel of understanding of the llm end-user it, it should everytime be proper tested, and with this i dont mean zuul tests | 22:11 |
| enricokern | i mean it makes no sense if people submit features or fixes without any kind of openstack understanding | 22:12 |
| enricokern | in the end this will NOT go away. If its fine to get out shitload of CVEs it will be fine to fix shitload of BUGs :) | 22:13 |
| enricokern | for reviewers thats the main issue. How do people scope with reviewing all that stuff thats coming in? Thats the hard part | 22:14 |
| TheJulia | Policy requires humans in the loop, and that is fine at the higher level policy. Varying perceptions may exist in regards to CVE handling, but the point is if items are being worked such that they are being reviewed and released in line with existing processes, then that is what is important. | 22:29 |
| TheJulia | Reviews are a huge issue and I'm hearing that from projects inside and outside our community. I think the base issue is AI, and the endless demands for more in the name and because of AI tooling basically is pushing everyone to a breaking point. | 22:30 |
| TheJulia | people only have so much capacity. | 22:30 |
| fungi | yes, somehow this ai revolution hasn't made my job any easier, it's only given me way more work when i already didn't have enough time to do it all | 22:32 |
| enricokern | you both nail it | 22:33 |
| TheJulia | I took last week off, still had to deal with some stuff, and basically 11-12 hour days. I'm literally going to a resturant/bar in about an hour to stop my workday because otherwise I'm going to literally begin flipping tables | 22:33 |
| fungi | if there are other people whose jobs it has made far easier, it would be great for them to take all the spare time it leaves them with and come help with the stuff that hasn't gotten easier | 22:33 |
| enricokern | and it would not work without people like you doing that reviews. and now you get hammered | 22:33 |
| enricokern | thats not a openstack/openinfra issue, its a global issue | 22:34 |
| TheJulia | yup, speed over process, paperwork, and everything else in the name of "as fast as possible" | 22:34 |
| enricokern | i like AI, i did my patches with it, but i babysit my stuff and test it along time internal before i did that. You can put this in a policy, but there is no real way to really control check this | 22:35 |
| enricokern | and with test it i do not mean a devstack i spooled up | 22:35 |
| TheJulia | Funny enough, I got feedback that expectations are shifting to "years of works" in "months" is now the expectation thanks to AI. I quite literally have had to engage a standards body to get things into a suitable place, which means thats an automatic 1.5 year lag now. | 22:35 |
| enricokern | so im on one side pro AI, on the other side its damnit shit | 22:35 |
| enricokern | also speed should not be a factor. It worked 15 years as it is | 22:36 |
| enricokern | and the sometimes slow processes made it especially valueable imho for openstack | 22:36 |
| TheJulia | enricokern: its an honor system, but if folks are responding in the middle of the night with overly verbose responses into gerrit, that sounds like disregard for engaging. | 22:36 |
| enricokern | we do not do software for 20 people but thousands | 22:37 |
| fungi | the ond addage was that it takes 9 months to make a human, no matter how many engineers you throw at the problem. but now we have ai, so we can definitely improve throughput | 22:37 |
| fungi | er, old addage | 22:37 |
| fungi | adage | 22:37 |
| fungi | my typing is gone and it's only midweek | 22:37 |
| enricokern | curious how most will find a solution for that | 22:37 |
| enricokern | it will just get only worse | 22:37 |
| TheJulia | fungi: if I had a teleporter, you'd like the bar I'm going to | 22:37 |
| * TheJulia sighs | 22:38 | |
| fungi | if the bar has a teleporter, count me in | 22:38 |
| enricokern | bambi xy user will find a new feature he wants and just spools up fable on it | 22:38 |
| enricokern | as example | 22:38 |
| enricokern | in the end AI will review too. that wont work i think | 22:38 |
| enricokern | on the other hand looking as it is now. devs are hard. old devs are super safe, new ones and new generations are hard to get by now | 22:39 |
| enricokern | this shifted a whole ecosystem | 22:39 |
| fungi | llm reviewers trained on the same data as llm developers sounds like a recipe for model collapse | 22:39 |
| enricokern | thats also a thing, imho now the models will train on content generated mainly on there own garbage they may find. fingerprinting solves this to some extend, but stil | 22:40 |
| enricokern | human stuff will decrease immense | 22:40 |
| enricokern | sad times | 22:40 |
| enricokern | human factor imho should be a thing in this policy | 22:41 |
| enricokern | you do shit with ai? well be human explain your stuff | 22:41 |
| enricokern | you "let it do" | 22:41 |
| enricokern | weekly meetings, ptgs whatever. join it. take part. if you cant explain what and why its not relevant | 22:42 |
| enricokern | irc too... | 22:42 |
| enricokern | doesnt mean all the time but at least try. i also have a hard time constraint. im in my 40ths too with family and what not. | 22:43 |
| TheJulia | I guess its time to begin to call it a day... the bar opens at 4pm | 22:49 |
| TheJulia | line starts forming around 3:30 | 22:49 |
| enricokern | enjoy your free time | 22:52 |
| enricokern | same for me. almost midnight. time for bed | 22:52 |
| fungi | sounds less like free time, more like self care! | 22:55 |
| enricokern | both is good :) | 22:56 |
| enricokern | where are you based fungi ? | 22:57 |
| fungi | on an island off the atlantic coast of the usa | 22:58 |
| enricokern | I didn't study geography. Could be manhattan or what else :O I was alot in the US and around it but ok. :) | 23:00 |
| fungi | barrier islands off the north carolina coast | 23:00 |
| fungi | we're approximately 10mi/16km offshore | 23:00 |
| enricokern | nice. never was in that part. I was in Texas, California, Florida, New Jersey, Washington and Tenesse mainly | 23:01 |
| fungi | so close to the mainland | 23:01 |
| fungi | in that case we're about halfway between florida and new jersey, about 500mi/800km east of tennessee | 23:02 |
| enricokern | well guess wont coming back so soon as long this orange is there. but nothing for here :P | 23:03 |
| enricokern | going to bed too. enjoy off time. Thanks for your great work! | 23:04 |
| fungi | thanks for yours as well! | 23:04 |
Generated by irclog2html.py 4.1.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!