| ianw | clarkb: i tested the !groupa:!groupb and I think https://paste.opendev.org/show/bj5zn1PPBHZ7m3axkZTZ/ shows that it indeed means "don't run if this host is in groupa or in groupb" | 00:08 |
|---|---|---|
| Clark[m] | Cool I did end up leaving a +1 after finding the extra docs. Off to dinner now but a lot more confident that does the correct thing now | 00:14 |
| opendevreview | Ian Wienand proposed openstack/project-config master: gerrit/acl : check for function/s-r in normalize https://review.opendev.org/c/openstack/project-config/+/875997 | 01:58 |
| opendevreview | Ian Wienand proposed openstack/project-config master: gerrit/acls : fix some missed NoOp functions https://review.opendev.org/c/openstack/project-config/+/877569 | 01:58 |
| opendevreview | Ian Wienand proposed openstack/project-config master: gerrit/acl : check for capital booleans in normalize https://review.opendev.org/c/openstack/project-config/+/877571 | 02:37 |
| opendevreview | Ian Wienand proposed openstack/project-config master: gerrit/acl : fix some missed NoOp functions https://review.opendev.org/c/openstack/project-config/+/877569 | 02:38 |
| opendevreview | Ian Wienand proposed openstack/project-config master: gerrit/acl : check for function/s-r in normalize https://review.opendev.org/c/openstack/project-config/+/875997 | 02:38 |
| opendevreview | Ian Wienand proposed openstack/project-config master: gerrit/acl : check for capital booleans in normalize https://review.opendev.org/c/openstack/project-config/+/877571 | 02:38 |
| opendevreview | Ian Wienand proposed opendev/system-config master: system-config-run-review : add review priority and backport labels https://review.opendev.org/c/opendev/system-config/+/868054 | 02:59 |
| ianw | ok that is everything i had on my todo list for copyConditions/submit-requirements cleared out | 03:00 |
| ianw | (modulo +1 verified :) | 03:01 |
| ianw | clarkb: i think ^^ is kind of good to add the trigger votes section, seems to work -> http://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_e70/868054/4/check/system-config-run-review-3.6/e70a1f6/bridge99.opendev.org/screenshots/gerrit-change-page-1.png | 04:02 |
| *** gibi_pto is now known as gibi | 08:04 | |
| *** jpena|off is now known as jpena | 08:21 | |
| *** jpena is now known as jpena|off | 08:43 | |
| *** jpena|off is now known as jpena | 08:52 | |
| mnasiadka | clarkb: I did sign up without paying for anything - but it was last year, and our plan allows 0 private repositories for openstack.kolla organization - should be for free | 09:18 |
| *** elodilles_pto is now known as elodilles | 09:26 | |
| frickler | clarkb: I'll be off early today and away until tuesday, so you'll have to discuss docker without me. I would certainly prefer not to pay them anything in reward for their behaviour | 09:54 |
| *** dhill is now known as Guest7939 | 12:29 | |
| opendevreview | Merged opendev/system-config master: dns variables : move to canonical locations https://review.opendev.org/c/opendev/system-config/+/876935 | 13:32 |
| opendevreview | Merged opendev/system-config master: bind9 : drop obsolete option for later versions https://review.opendev.org/c/opendev/system-config/+/876937 | 13:39 |
| opendevreview | Merged opendev/system-config master: system-config-run-dns : update nodes to jammy https://review.opendev.org/c/opendev/system-config/+/876930 | 13:39 |
| clarkb | frickler:ack | 15:15 |
| dtantsur | https://quay.io/organization/metal3-io also has a free account | 15:26 |
| clarkb | ya I'm about to try signing up this morning. I think the thing that trips us up is that all of the docs say you need to set up at least a developer level account whihc is not free but then also says you can use quay.io for free. The messaging is confusing and now I just need to give it a go | 15:28 |
| clarkb | "user account for this email already exists" thats news to me /me recovers an account insteadof creating a new one | 15:33 |
| clarkb | interesting when I did that it redirected me to a page where i have to add all the extra info corvus was talking about so couldn't sneak around that with an old account | 15:37 |
| clarkb | which then results in "We're sorry an internal server error occurred" | 15:37 |
| clarkb | and if I try to login to quay I get redirected to the page to add extra details again and it fails again | 15:38 |
| clarkb | so ya thats fun | 15:40 |
| fungi | maybe they're buried under dockerhub evacuees and the system has fallen over | 15:51 |
| clarkb | or I've got a really old account they don't know how to make modern | 15:51 |
| hashar | hi, is there any guide as to which python jobs I could add to a repo? I am trying to polish up the old `jjb/python-jenkins` which uses `openstack-python35-jobs` and I guess I can replace it with `tox-py35`? | 15:53 |
| hashar | I guess I can try :] | 15:53 |
| clarkb | hashar: zuul lists them all for you at https://zuul.opendev.org/t/openstack/jobs | 15:54 |
| clarkb | hashar: tox-py36 37 38 39 310 311 should all work but you may need to select a test nodeset that supports that version of python | 15:55 |
| hashar | Niceeee | 15:55 |
| clarkb | I've tried both chrome and firefox to make sure this wasn't a browser difference breaking red hat. Same behavior with both | 15:55 |
| clarkb | If I try to login via another location it asks me to choose if my account is corporate or developer. I'll set this personal email account to a regular developer account I guess | 15:56 |
| clarkb | and see if that fixes things | 15:56 |
| hashar | clarkb: I will try thanks | 15:56 |
| clarkb | one problem with figuring out how to login to red hat is that red hat sso is both a product they sell and a tool they run | 15:57 |
| clarkb | I'm getting turned around in circles here :/ | 15:57 |
| fungi | could it be that all the people saying "just upload to quay, it's free" got developer accounts automatically by being employed at red hat? | 16:00 |
| clarkb | fungi: well I'm not even trying to do anything quay related yet | 16:01 |
| clarkb | I'm just trying to login at sso.redhat.com | 16:01 |
| clarkb | I filled out the form it wanted there and it redirected me back to the same empty form after hitting submit | 16:01 |
| clarkb | no explicit error this time but I'm not sure it worked iether | 16:01 |
| fungi | right, you're trying to sign up for a developer account. i'm just wondering if anyone outside of rh is actually able to sign up for those successfully | 16:01 |
| fungi | and whether rh employes get signed up for it automatically so don't realize the sign-up is busted | 16:02 |
| clarkb | *I'm trying to update my existing account to a modern rh developer account | 16:02 |
| clarkb | I suspect the issue is in upgrading the account. I can try creating a new account with a different email addr | 16:02 |
| fungi | oh, that's probably an even less well-tested path ;) | 16:02 |
| clarkb | I started this by trying to sign up for a new account and being told an account already existed with that email address | 16:03 |
| clarkb | so then I reset the password successfully but am now stuck in update your details limbo | 16:03 |
| JayF | We have enough RH-adjacent people in the community that surely one of them could at least connect you to someone who could fix it? | 16:03 |
| clarkb | maybe. But games of telephone are never fun. | 16:04 |
| JayF | Oh, I dislike that they hide so much behind login as well. Not really open in the 4-opens sense... but it's probably better to send an email than bang your head against it | 16:04 |
| JayF | make them feel some of the pain from those choices too :/ | 16:05 |
| clarkb | I also wonder if that account got associated with hp or itnel or some corporate account way back when and now its basically dead. But the easiest way to deal with this is a new account I think | 16:06 |
| clarkb | yup that worked | 16:09 |
| clarkb | infra-root: ok I had to provide an email address, phone number, and physical address for this personal account. I did not have to provide any billing info. When I login and view the billing info area of the settings page it says 0 private repos is the maximum allowed by your plan and doesn't seem to force me to sign up with billing info anywhere | 16:11 |
| clarkb | I think this should work if you can amange to create an account :) | 16:11 |
| clarkb | looks like you can also set the account type to be an organization? I'm not sure if it is better to create a new organization via a normal user account or create a new normal user account for our purposes then convert it to an aorganization? | 16:13 |
| clarkb | also it looks like you set a quay docker protocol password independently of your red hat sso which is nice | 16:14 |
| fungi | and that's what would be put in the zuul secret for the publishing job? | 16:14 |
| clarkb | yup | 16:14 |
| fungi | cool | 16:14 |
| clarkb | I need to grab breakfast but I'll push a copy of our base python images to my personal account there just to make sure all the plumbing works | 16:15 |
| clarkb | When you click create organiztion you set an organization name and email (this email addr must differ from your account) and then you choose from a set of prices. One of them is "Open Source" which is 0 private repos and $0 | 16:17 |
| fungi | makes sense | 16:17 |
| clarkb | I think that means we have the option of having an org+regular user for "infra-root" but with different email addrs. Or just make an infra-root regular user for opendevorg/ or whatever and share that account | 16:18 |
| clarkb | I suspect the "correct" thing to do is create an org and a push user | 16:18 |
| clarkb | rather than just a user for everything | 16:18 |
| clarkb | the whole phone number and unique emails requirement is a bit annoying but I thinkwe can make that work assuming infra-root+quayopendevorg and infra-root+quayopendevuser work | 16:19 |
| clarkb | ok breakfast now. Then I'll try pushing an image and see if others can fetch it to ensure the actual workflows work | 16:21 |
| fungi | do they test that the phone number is actually in service? | 16:22 |
| clarkb | fungi: I have not recieved any phone contact from them since creating the account. Admittedly not that long ago | 16:24 |
| clarkb | no sms or phone calls etc | 16:24 |
| clarkb | it is probably useful in a recovery situation and may make sense to set to a value that is likely to function. | 16:25 |
| clarkb | Oh! also when you sign up to quay they ask for slightly less info than signing up via sso.redhat.com. I'd suggest we sign up via quay for this reason | 16:25 |
| clarkb | (it drops the whole corporate vs personal question) | 16:25 |
| fungi | oh, nice | 16:25 |
| opendevreview | Antoine Musso proposed zuul/zuul-jobs master: Remove ignored success-url job attribute https://review.opendev.org/c/zuul/zuul-jobs/+/877700 | 16:52 |
| clarkb | https://quay.io/repository/clarkboylan/python-builder/manifest/sha256:a55d63d1fdddde72a597b1ede31fe79f3e6384936c0cc8ee4cbeba1d938bb19b I think that worked. If someone else wants to pull quay.io/clarkboylan/python-builder:3.11-bullseye you can confirm the public access | 16:59 |
| corvus | clarkb: wfm. pulled manifest and i had all the layers locally :) | 17:03 |
| clarkb | excellent then despite some account management clunkyness and messaging confusion I think this should work | 17:03 |
| corvus | pulled on another machine without layers cached and that worked too | 17:03 |
| corvus | and obvs neither of these had quay logins, so was anonymous | 17:04 |
| clarkb | I find the UI a bit clunky too but I think that isn't a problem 99% of the time | 17:06 |
| clarkb | lol the security checker for quay complains about debian libc due to a cve that debian has marked as not an issue | 17:07 |
| clarkb | I can see how notices like that will cause people to very quickly ignore the security checkers | 17:08 |
| clarkb | our debian libc package is up to date too | 17:08 |
| clarkb | I'm going to guess that centos and fedora images do not trip these messages | 17:09 |
| clarkb | their checker also shows fixes for some things in package versions that don't show up in the debian package search | 17:11 |
| clarkb | If I have any real criticism of quay so far it is their security scanner produces a bunch of noise which will lead people to ignore it | 17:11 |
| clarkb | the last thing I need to test is that the update to the quay docker cli password settings really did leave my sso credentials alone | 17:15 |
| clarkb | heh if you sign out all sessions then immediate click sign in it does so autoamtically... | 17:16 |
| * clarkb opens a new browser serssion | 17:16 | |
| clarkb | yup the passwords are distinct which is what we want | 17:18 |
| *** jpena is now known as jpena|off | 17:23 | |
| clarkb | I think what we do is have admin user accounts tied to us as individuals. We create organizations for opendevorg/ zuul/ etc and under the umbrella of an organization we can create robot accounts. These robot accounts are the ones that should go into our zuul stuff | 17:24 |
| fungi | perfect | 17:26 |
| fungi | hopefully this means our discussion in ~1.5 hours will be a quick one | 17:26 |
| clarkb | it isn't clear to me if a robot account can create new rpos | 17:26 |
| clarkb | "For an organization-owned robot account, a robot account can be granted permission to create repositories if placed under a team with the creator permission. Otherwise, a robot account must be granted individual permissions." | 17:28 |
| clarkb | I think this should all work | 17:28 |
| clarkb | quay docs also say you can just push to a name to create a repository. | 17:42 |
| clarkb | let me test this | 17:43 |
| clarkb | ok direct push works but it seems to mark the image private by default I think this is what mnasiadka was referring to | 17:47 |
| clarkb | looks like there are apis to set these values too but the api docs don't document valid parameter values just that the parameters exist :/ | 17:51 |
| clarkb | https://access.redhat.com/solutions/6966410 is something I don't get the full text for | 17:52 |
| clarkb | this seems solveable if we end up manually toggling that value in the short term | 17:53 |
| clarkb | of course the time we'll want it most is when we create a bunch of new images for the things we use today | 17:54 |
| clarkb | I wonder if you create an open source org if it solves that for you actually. Since the implication there is very much that repos will all be public. I don't want to create orgs for oepndev and/or zuul until we have a bit more consensus on this though so I'll avoid testing that for now | 17:55 |
| NeilHanlon | clarkb: meat of that article is | 18:04 |
| NeilHanlon | Add and set following parameter CREATE_PRIVATE_REPO_ON_PUSH: false in quay config.yaml file. This helps create a public repository when first pushing the image to the quay registry. | 18:04 |
| NeilHanlon | notably: there is also a CREATE_NAMESPACE_ON_PUSH parameter | 18:05 |
| clarkb | NeilHanlon: oh so that is for people running their own local quay deployments. That won't apply here unfrotuntaely | 18:06 |
| NeilHanlon | Trying to figure that, but my suspicion is the same.. that it's a server-side config | 18:07 |
| NeilHanlon | yeah, looks to be server side | 18:08 |
| NeilHanlon | https://github.com/quay/quay/blob/ff66a93eb7c1b466b2dffd5e62187d7824e6ccad/endpoints/v2/v2auth.py#L308 | 18:08 |
| fungi | so no mention of any similar parameter that can be passed in the api call | 18:09 |
| NeilHanlon | not that I can tell | 18:09 |
| NeilHanlon | there is some default permission settings in an organization.. but I cannot see if there's a way to grant read to 'all' | 18:11 |
| NeilHanlon | https://drop1.neilhanlon.me/irc/uploads/1c37b9837d029fb6/image.png | 18:11 |
| clarkb | ya Idon't think it is the end ofthe world considering there appears to be an api endpoint to manage it. Our flow might just be create repo with api endpoint then push | 18:15 |
| NeilHanlon | yeah that's probably smart | 18:16 |
| NeilHanlon | it also might just be me, but I've experienced trouble uploading to quay... interested to hear if you all have similar issues | 18:17 |
| NeilHanlon | sometimes their proxy just hangs up, it seems | 18:17 |
| fungi | so we should implement retries, sounds like | 18:17 |
| NeilHanlon | I would recommend it, yeah | 18:18 |
| fungi | fwiw, we've seen random failures with trying to publish on dockerhub too | 18:18 |
| fungi | though our usual workflow is to push images built from approved changes, and then retag them in the repository with a known label after they merge | 18:18 |
| clarkb | I'm told emilienm may have a blog post about this | 18:19 |
| clarkb | I'm looking now | 18:19 |
| clarkb | ya we allow three attempts to docker hub iirc | 18:19 |
| fungi | er, retag them in the registry i mean | 18:19 |
| NeilHanlon | clarkb https://my1.fr/blog/moving-container-images-from-docker-io-to-quay-io/ ? | 18:19 |
| fungi | i think it's mainly been tag deletes where we end up with failures? | 18:19 |
| clarkb | https://my1.fr/blog/moving-container-images-from-docker-io-to-quay-io/ | 18:19 |
| clarkb | NeilHanlon wins | 18:20 |
| clarkb | so ya this should work for us. We'll just need zuul jobs that create the public repo before we push to it if it doesn't exist already | 18:20 |
| corvus | sounds like we could make a role to do that and include it first thing in the promote jobs | 18:50 |
| clarkb | yup I was looking at what it takes to get a bearer token and I think we can make an application for an organization (instead of a robot) and use the resulting token for the application to talk to the api and do the image push commands | 18:57 |
| clarkb | the distinction between an application and a robot is a bit fuzy to me but it see that only an org can have an application but orgs and users can have robots? | 18:57 |
| clarkb | infra-root it is 19:00 which is when we agreed to spend some time making decisions about the shutdown of free docker orgs | 19:00 |
| ianw | o/ | 19:00 |
| clarkb | https://etherpad.opendev.org/p/MJTzrNTDMFyEUxi1ReSo is the etherpad where I've collected ideas/thoughts/concerns so far to try and ensure we keep the info as centralized as possible. Though I haven't put what I have learned about quay there yet | 19:00 |
| clarkb | Long story short on April 14 we'll lose access to manage our docker hub orgs for opendevorg/ and zuul/ images. I believe the public images hosted there will be accessible for 30 days after april 14 | 19:01 |
| corvus | we meeting here or -meeting? | 19:01 |
| clarkb | corvus: would you like us to keep notes and use -meeting? we can do that | 19:01 |
| * fungi is ambivalent about meeting minutes, good with either channel choice | 19:02 | |
| corvus | oh no opinion, just want to make sure i'm in the right place :) | 19:03 |
| clarkb | after you said it I realized its nice to have the logs separated for ease of discovery later | 19:03 |
| clarkb | things can get lost in the daily scrollback in here | 19:03 |
| corvus | to the meeting room then! | 19:04 |
| artom_ | So out of curiosity, if we have a Zuul tox job that fails consistently 100% of the time, but we can't reproduce locally, is it conceivable to hold a VM so that we can poke around after the job? | 19:32 |
| clarkb | artom_: yes, if you note the job name and change you'll trigger it with we can set a hold. We'll also need a copy of your public ssh key | 19:33 |
| fungi | artom_: it is, but have a link to the build result page? we mightbe able to spot it | 19:33 |
| clarkb | also that | 19:33 |
| fungi | some of us have become rather attuned to the assumptions users make when running tests locally | 19:33 |
| opendevreview | Jeremy Stanley proposed openstack/project-config master: Restore rax-ord quota but lower max-concurrency https://review.opendev.org/c/openstack/project-config/+/877715 | 19:56 |
| fungi | clarkb: corvus: ^ after reviewing the graphs for rax-ord. i think the max-concurrency has helped but may still be too high, so want to chop it in half while we redouble the max-servers back to the original capacity | 20:03 |
| fungi | https://grafana.opendev.org/d/a8667d6647/nodepool-rackspace?orgId=1 | 20:04 |
| fungi | seems like it does a good job of using nodes there as long as it manages to boot them | 20:04 |
| clarkb | fungi: wfm | 20:04 |
| corvus | fungi: should we think about rate too? that might help stretch out delete serialization if that's a problem (since i don't think max-concurrency does anything for deletes) | 20:05 |
| corvus | (but also approved) | 20:05 |
| fungi | we can, though i already attenuated the rate by an order of magnitude earlier | 20:06 |
| fungi | but 100/sec may still be too high, yeah | 20:06 |
| fungi | there's not really a substantial gap between used/deleting in the graphs, that i can see | 20:06 |
| * clarkb finds lunch | 20:07 | |
| fungi | corvus: and it really seems like the underlying problems there aren't the number of api calls, rather their nova scheduler probably takes way too long to pick a host and process the create | 20:09 |
| fungi | and the time that takes seems to get significantly worse the more of those we ask for at once | 20:09 |
| fungi | but the graph is looking much closer to how their other two regions behave now, at least | 20:11 |
| corvus | fungi: yep; someone mentioned the delete instance thundering herd problem to me the other day and that's fresh in my mind | 20:13 |
| corvus | but i don't have any evidence it applies here | 20:13 |
| corvus | just something to keep in mind | 20:13 |
| fungi | will do, thanks! | 20:13 |
| corvus | erm, stupid question: are we still infra-root at openstack.org ? | 20:16 |
| fungi | corvus: yes, that's still the address | 20:16 |
| corvus | kk. i will register with "infra-root+quay-zuul-org" at ... | 20:16 |
| opendevreview | Merged openstack/project-config master: Restore rax-ord quota but lower max-concurrency https://review.opendev.org/c/openstack/project-config/+/877715 | 20:16 |
| corvus | bad news! zuul already exists. i wonder if it's owned by jeblair@redhat.com or mordred@redhat.com ... | 20:18 |
| fungi | d'oh! | 20:19 |
| fungi | zuulci, everyone's favorite italian dessert | 20:20 |
| clarkb | I'm doing opendevorg/ to keep the url portion the same with what we had and infra-root+quay-opendev-org at ... | 20:23 |
| fungi | though it's an opportunity to have opendev there if it's not already squatted | 20:23 |
| clarkb | opendev/ is greeen if we want it | 20:24 |
| corvus | i didn't find out it was unavailable until i hit submit | 20:24 |
| fungi | we ended up using opendevorg on dh only because opendev was taken | 20:24 |
| clarkb | corvus: ah | 20:24 |
| clarkb | do we prefer opendev/ then? I can try that | 20:24 |
| fungi | shorter is probably better, but i'm not all that concerned either way | 20:25 |
| corvus | i have slight preference for opendev/ | 20:25 |
| clarkb | cool trying that then | 20:25 |
| clarkb | it already exists. Now trying opendevorg/ | 20:25 |
| corvus | i wonder if the same helpful person squatted both... | 20:26 |
| clarkb | hahaha | 20:26 |
| clarkb | corvus: do you think it worthwhile to wait on that or just move ahead? | 20:26 |
| fungi | or if the recent dockerhub announcement has resulted in everything getting squatted | 20:26 |
| corvus | clarkb: for opendev i don't think i'd wait, i'd say keep going with opendevorg; since we already use it | 20:27 |
| clarkb | https://quay.io/organization/opendevorg done | 20:28 |
| corvus | for zuul, i sent an email to.... support@coreos.com (since apparently that's what the faq says?) to ask | 20:28 |
| corvus | i also asked in #zuul in case someone helpfully squatted it and we forgot | 20:28 |
| clarkb | I'm going to pause here. Both coruvs and I are owners of the opendevorg org and will happily add other infra-root once you let us know what your usernames are | 20:34 |
| clarkb | Its bright and sunny and almost warm outside today so I want to get out on my bike. When I get back I'll look at setting up an "Application" in the org which can be used to talk to the api and push images | 20:34 |
| fungi | looks like i previously had a red hat account because of working ansiblefest, but now it's busted | 20:37 |
| clarkb | oh! | 20:37 |
| clarkb | that may explain what I ran into | 20:37 |
| clarkb | fwiw I don't see any emails after creating the org | 20:38 |
| clarkb | but maybe when I create an application we'll get email | 20:38 |
| corvus | me too, so i have 2 potentially busted redhat accounts | 20:39 |
| fungi | tried the password reset, but i receive no reset email | 20:40 |
| clarkb | ianw: fwiw I reviewed your new acl stack and they lgtm but I dind't approve since I wasnt' sure I could pay attention to them earlier | 20:40 |
| clarkb | ianw: but I think you can self approve if you like | 20:40 |
| clarkb | fungi: I did get a reset email. It was what ahpepend after setting my new password with trying to set personal info that failed | 20:41 |
| fungi | okay, i am now https://quay.io/user/fungi/ | 20:48 |
| fungi | apparently your quay username isn't tied to your red hat username | 20:48 |
| corvus | fungi: i invited you; you should have a notification in the bell at the top right | 20:53 |
| corvus | (it showed up for me after i set my cli password; i don't know if that's because i needed to do that first, or it just took a few minutes) | 20:53 |
| fungi | accepted. thanks! | 20:55 |
| fungi | and yeah, i had already seen and acted on the password setting notification | 20:56 |
| ianw | clarkb: thanks, will do today to clear that, i started updating the checklist page but will finish that off | 21:02 |
| ianw | clarkb: are you ok with me trying out the linaro thing too? i imagine that will take some fiddling to get right, so i'll watch that closely | 21:05 |
| clarkb | I am but that one is probably good to get a second reviewr on? | 21:07 |
| ianw | corvus: i am now https://quay.io/user/ianw | 21:08 |
| ianw | fungi: maybe if you have a sec the change in question is https://review.opendev.org/c/opendev/system-config/+/877436, which adds linaro as an "unamanged" node so we can deploy users and stuff, but not take over iptables, etc. with a full base install | 21:10 |
| fungi | sure, taking a look | 21:11 |
| ianw | it currently won't do anything, i'd just like to validate the path that it can actually run something. also kevinz has agreed to us using it like that | 21:11 |
| artom_ | clarkb, fungi, belated thanks! Was in a call, so have to drop now, but will follow up in the next few days | 21:13 |
| fungi | yeah, lgtm, and i saw the response from kevinz in favor as well | 21:13 |
| artom_ | jparker, ^^ (srcoll to 15:33:02) | 21:13 |
| artom_ | fungi, so https://zuul.opendev.org/t/openstack/build/ad915e775fb648fb8f3e9ea9c0755beb is the build page, in case you're still around | 21:15 |
| *** artom_ is now known as artom | 21:15 | |
| artom | The error is very specific to the project iself | 21:15 |
| artom | And does not happen then I just run `tox -re py310` locally on Jammy | 21:16 |
| opendevreview | Merged openstack/diskimage-builder master: Fix double-keyed json https://review.opendev.org/c/openstack/diskimage-builder/+/876292 | 21:39 |
| opendevreview | Merged openstack/diskimage-builder master: Repeat to umount filesystem when exception occurs https://review.opendev.org/c/openstack/diskimage-builder/+/872430 | 21:39 |
| fungi | artom: got it, looks like oslo_config isn't finding expected configuration options, and this started between 2022-12-13 and 2023-01-23 (last recorded success and first in the current string of failures). seems like py36 jobs are working but py38, py39 and py310 fail? | 21:57 |
| jparker | fungi: yes its just py38,py39, and py310 | 21:58 |
| fungi | artom: fwiw, i get the exact same error running tox -e py310 on my workstation | 21:58 |
| fungi | (debian sid, not ubuntu jammy, but similar) | 21:59 |
| artom | fungi, oh that's some Twilight Zone stuff | 22:12 |
| fungi | if it's working on py36 it probably makes sense to compare what versions of packages are ending up installed, because a lot of libs have dropped py36 support and it's probably a newer version of one of them, or maybe newer tox | 22:15 |
| fungi | actually, that's right around when tox 4.0.0 happened, yeah? | 22:15 |
| fungi | are you testing with tox 3.x locally? | 22:15 |
| fungi | artom: remove the skipsdist from tox.ini and *bam* it works | 22:16 |
| fungi | there's your problem | 22:17 |
| fungi | the timing was a dead giveaway | 22:17 |
| fungi | if you upgrade your local tox you'll likely see the same errors otherwise | 22:17 |
| fungi | jparker: ^ | 22:19 |
| jparker | fungi: thank you! Let me try that now | 22:20 |
| fungi | basically the reason for the confusing error is that the tempest-whitebox-plugin isn't getting installed into the venv | 22:21 |
| fungi | lots of our projects used to rely on usedevelop making the project always get installed, even if they set skipsdist. tox maintainers decided that was a nonsensical misbehavior, and made it so that skipsdist means never install the project even if usedevelop is specified | 22:22 |
| jparker | fungi: thanks debugging and explaining the issue that's extremely helpful, artom I've got something testing now https://review.opendev.org/c/openstack/whitebox-tempest-plugin/+/877720 | 22:25 |
| fungi | my pleasure. it was faster than setting up an autohold and giving someone access to the node | 22:27 |
| artom | fungi, the annoying thing is, we went through this with Nova, I just wasn't paying attention enough | 22:28 |
| artom | fungi, much appreciated, thank you! | 22:28 |
| fungi | any time | 22:28 |
| opendevreview | Ian Wienand proposed openstack/project-config master: openstack/release : return to non-blocking submit rule https://review.opendev.org/c/openstack/project-config/+/877721 | 22:44 |
| clarkb | ianw: you should see an invite in quay I just got back from a bike ride and added you | 23:03 |
| ianw | great, looks like I'm in | 23:04 |
| clarkb | I'm looking at the application stuff and its weird that it says it will act on behalf of me I think because I was the user creating it | 23:07 |
| clarkb | ok looks like there may be a way to do this with a robot instead so I'll explore that avenue | 23:07 |
| clarkb | from https://groups.google.com/g/quay-sig/c/-d8ay9egB78?pli=1 I think that there may not be a way to perform application actions as a robot. In that case I think we probably want two secrets. One is a bearer token for an admin that is able to create new repos. This will create the initial repo. Then another which is the robot which can do all of he pushing | 23:19 |
| clarkb | what I've done is create a robot and put it in a team with "creator" perms | 23:19 |
| clarkb | and now I'll create an application that will act as me apparently to create a repo and see if I can then push as the robot | 23:20 |
| opendevreview | Merged openstack/project-config master: gerrit/acl : fix some missed NoOp functions https://review.opendev.org/c/openstack/project-config/+/877569 | 23:20 |
| clarkb | ok I am/was a derp and triedto create a repo under opendev instead of opendevorg and that is why I was not authorized. So I got a new token with more perms and tried some more before I realized. There doesn't seem to be a good way to remove tokens that I see so I guess I remove the entire application? I'll test that in a bit | 23:42 |
| clarkb | confirmed that deleting the application properly clears out the tokens. | 23:49 |
| opendevreview | Merged opendev/system-config master: infra-prod: run job against linaro https://review.opendev.org/c/opendev/system-config/+/877436 | 23:51 |
| clarkb | ok here is a rundown of what I did in quay that seems to work for creating and pushing a public image/repo: 1) create robot account 2) create automationtools team 3) set default permissions to give org owners admin rights when anyone creates a new image 4) set default permissions to give write rights to the automationtools team when anyone creates a new image 5) create a new | 23:54 |
| clarkb | application to generate a bearer token that will act as me (I can't find a way around this yet) 6) use emilienm's documented curl stuff to create a public repo 7) use robot account to push image content | 23:54 |
| clarkb | if we need to remove bearer tokens the only option seems to be to delete the application and create a new one but this does seem to work | 23:55 |
| clarkb | this is how opendevorg/clarkbtest:base-3.11-bullseye was created and pushed | 23:56 |
| clarkb | because the application is apparently acting as me I dleted the old one and made a new one that very specifically identifies this as a token related to me | 23:56 |
| clarkb | make management of it easier | 23:56 |
| clarkb | Another option would be to docker image push to have the robot account create the image. Then use the bearer token to modify its visibility. Not sure that is any better and I like the flow of creating it public upfront | 23:57 |
| clarkb | ok good the log shows that the application is acting on behalf of a person rather than just that person doing things | 23:59 |
| clarkb | and the log captures everything I just wrote down. THats a nice feature | 23:59 |
| clarkb | I've run out of time, but I'll pick up the ansiblification of this tomorrow | 23:59 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!
