Wednesday, 2022-09-21

opendevreviewMerged zuul/zuul-jobs master: configure-mirrors: fix typo in 9-stream enablement list
opendevreviewMerged openstack/diskimage-builder master: Start running dib-lint again
*** ysandeep|out is now known as ysandeep00:29
*** dasm is now known as dasm|off00:37
opendevreviewIan Wienand proposed opendev/system-config master: Abstract name of bastion host for testing path
opendevreviewIan Wienand proposed opendev/system-config master: Convert production playbooks to bastion host group
ianwok, now we've got liniting i'll make a dib release00:55
ianwprobably the best thing to merge is clarkbs linter fix to nodepool, then that will pull in a new release00:56
ianwwell, switching *back* to a gpg key from a ssh key is a bit of a PITA01:05
ianwfor reference, it's "git config gpg.format openpgp" to get back to "regular" signing; and set user.signingkey to the gpg key, not the ssh one01:15
ianwi guess it would be nice if gerrit's signed tag push allowed ssh signatures01:15
opendevreviewIan Wienand proposed opendev/system-config master: Convert production playbooks to bastion host group
*** pojadhav|PTO is now known as pojadhav03:08
*** ysandeep is now known as ysandeep|afk03:31
*** Guest931 is now known as prometheanfire03:47
opendevreviewIan Wienand proposed openstack/diskimage-builder master: Add Rocky 9 ARM64 functional test
*** ianw is now known as ianw_pto05:09
*** ysandeep|afk is now known as ysandeep05:58
fricklerI'm still worried by the "infinite" lifetime of ssh keys. also I don't think they're useable without gerrit and gitea support05:58
mnasiadkaianw_pto: I see you mentioned it builds, thanks for adding functest in DIB07:28
zigoHas the IRC channel for murano changed? Where should I go?07:36
zigoMurano has the same defect with oslo.db as Designate and Magnum (ie: autocomit)...07:37
zigoI just fixed it on the Debian side of things.07:37
zigoAnd forwarded my patch ...07:37
*** jpena|off is now known as jpena07:38
zigoLooks like Vitrage has the issue too (I'm on it...).07:38
mnasiadkaMagnum is now merging07:47
ianw_ptofrickler: you can choose the length of time to trust the key as in  and i think that github will show things correctly signed even if your key is expired (but not revoked) because it was too confusing that people's old commits looked invalid08:21
*** pojadhav is now known as pojadhav|afk08:22
fricklerI guess we also need cryptographically verifiable timestamps on sigs. but yes, I'd expect the valid-until to limit the timeframe where valid signatures can be made with that key, not limit the validity of existing signatures08:27
*** tobias-urdin6 is now known as tobias-urdin08:49
opendevreviewRafal Lewandowski proposed openstack/diskimage-builder master: Added cloud-init growpart element
opendevreviewFrikin Evgenii proposed openstack/diskimage-builder master: Add variable for check installing python3 in yum element
*** ysandeep is now known as ysandeep|lunch10:06
*** rlandy|out is now known as rlandy|ruck10:29
*** ysandeep|lunch is now known as ysandeep10:58
*** pojadhav|afk is now known as pojadhav11:14
fungizigo: looks like we did log a #murano channel but it had gone unused for so long we removed it from our configuration before the move off freenode11:20
opendevreviewRafal Lewandowski proposed openstack/diskimage-builder master: Added cloud-init growpart element
fungifrickler: yes, the general idea with key expiration is that you shouldn't trust a signature made later than the key validity (especially in the case of revocations), and you don't want to encrypt messages to expired or revoked keys obviously11:27
fungibut also, how you tell for sure that a signature was made after the key expired (and wasn't just backdated) is not always a solved problem11:28
fungii haven't looked into how signed commits are implemented, but as long as the signature is an integral part of the commit itself and factors into subsequent commit hashes, that should be fairly easy to confirm. if signatures can be attached later and subsequent timestamped data doesn't depend on them, then you don't really know for sure when they were made11:30
*** frenzyfriday is now known as frenzyfriday|lunch11:44
zigofungi: Is murano a dead project ?!?11:54
zigoI wrote this:
zigoI'd like to make sure core reviewers will see it ...11:56
fricklerat least murano won't run any jobs until the queue config is fixed. not sure it can be called dead, but it sure smells funny IMO12:09
fricklermaybe wait a couple of days for reactions and take it up with the TC if nothing happens?12:10
*** ysandeep is now known as ysandeep|afk12:26
*** ysandeep|afk is now known as ysandeep12:39
*** frenzyfriday|lunch is now known as frenzyfriday12:46
opendevreviewRafal Lewandowski proposed openstack/diskimage-builder master: Added cloud-init growpart element
opendevreviewRafal Lewandowski proposed openstack/diskimage-builder master: Added cloud-init growpart element
*** dasm|off is now known as dasm14:05
opendevreviewJames E. Blair proposed zuul/zuul-jobs master: Fix CORS and endpoint in AWS log upload
*** ysandeep is now known as ysandeep|out14:16
opendevreviewJames E. Blair proposed zuul/zuul-jobs master: Fix CORS and endpoint in AWS log upload
slittle1please add me as the first core for starlingx-app-sts-silicom-core.   I'll set up the rest.  Thanks15:23
*** marios is now known as marios|out15:31
clarkbslittle1: give me a couple minutes to load keys and I can do that15:34
clarkbslittle1: you've been added15:40
clarkbinfra-root I've discovered via codesearch (so likely incomplete listing) that we're using ansible-version for ansible 2.8/2.9 in a handful of places15:43
clarkbSince support for that is going away reall soon I'll get changes up to remove that asap15:44
slittle1clarkb: thanks15:44
opendevreviewClark Boylan proposed opendev/base-jobs master: Remove ansible-version: 2.8
opendevreviewClark Boylan proposed opendev/system-config master: Remove ansible-version: 2.9
clarkbinfra-root and rlandy|ruck chandankumar
rlandy|ruckclarkb: thanks - will look into those - we probably needs those values specified elsewhere15:54
clarkbrlandy|ruck: well in this case I'm proposing you don't hardcode them15:54
clarkbthat way you'll be kept up to date automatically when the zuul tenant config updates15:54
clarkbwhcih is what I had expected happend a while back when we moved everyone to ansible 5 ...15:54
clarkbrlandy|ruck: is the zuul change and opendev auto deploys zuul updates each weekend at the latest15:55
rlandy|ruckok - should be fine - if the CI on that does not fail then we aren't relying on some specified version15:58
fungiclarkb: oh! i did codesearch for ansible_version not ansible-version, oops!15:58
fungithanks for spotting those15:59
opendevreviewJames E. Blair proposed zuul/zuul-jobs master: Fix CORS and endpoint in AWS log upload
clarkbA change to nodepool just landed which should've updated dib in the builder image. The next hourly deployment should upgrade the builders and then we can rebuild fedora-36 images16:22
clarkbI'm going to go ahead and single core approve the ansible-version removals from base-jobs and system-config16:23
clarkbthe base-jobs change only affects base-test16:24
clarkband in system-config it is self testing16:24
opendevreviewMerged opendev/base-jobs master: Remove ansible-version: 2.8
clarkbslittle1: thinking a bit more about the linux kernel idea. I think if ya'll wanted to push that along further the best next step is to push a change to opendev/system-config that adds the kernel as a test repo in system-config-run-review-base and system-config-run-gitea jobs. That won't give us a direct comparison to prod performance but will give us data we can start to work with16:33
*** jpena is now known as jpena|off16:40
opendevreviewMerged opendev/system-config master: Remove ansible-version: 2.9
opendevreviewJames E. Blair proposed zuul/zuul-jobs master: Fix CORS and endpoint in AWS log upload
corvusi'm going to launch the tracing server now17:11
corvusour docs don't seem to be up to date:
corvusi get openstack.exceptions.BadRequestException: BadRequestException: 400: Client Error for url:, Bad networks format17:15
corvusis there a different env i should run the launch script out of?17:15
clarkbcorvus: it is possible that openstacksdk no longer supports rackspace server apis :/17:16
clarkbcorvus: I've got a venv in my homedir that is there for rax cinder voluem management17:16
corvusclarkb: hrm.  i guess to verify i should make a venv and pip install openstacksdk?17:16
clarkbcorvus: ya or try if ~clarkb/venv which is an older sdk for cinder stuff also fixes the servers api17:17
corvusi'll try yours17:17
corvusthat did not immediately blow up, so that's progress17:18
corvuson a lighter note, while i'm waiting... this had me chuckling: `opendev/`  it's basically a path that just says "opendev" and "zone" over and over again :)17:18
corvus"i heard you like opendev and zones"17:19
clarkbwe put more dns into your domain and vice versa17:19
clarkbinfra-root the builder image and container have updated on nb01 and nb02. I've manually requested a rebuild of fedora-3617:20
corvusthe playbook failed with: AttributeError: module 'platform' has no attribute 'linux_distribution'17:22
corvusi'm trying to work out how fatal that is...17:22
clarkbgrepping for platform and linux_distribution returns no results in system-config so the failure is happening in ansible itself I guess?17:23
clarkbah google says its a python/pip thing17:24
corvushere's the full output
clarkbcorvus: maybe the ansible in my env is too old and expecting older python3 on the node (I'm assuming it is focal or jammy which is 3.8 or 3.10)17:26
clarkbso ya may need to construct a new env with newer ansible but older opnstacksdk (I hope that doesn't introduce new issues)17:27
clarkboh ya the ansible in that env is pretty ancient17:28
clarkb(I normally only use openstackclient in that env)17:28
clarkbcorvus: checking ansible's git log I think ansible 2.7.0 or newer fixes that issue17:31
corvusi will try ansible 2.7.18 and openstacksdk 0.4117:37
clarkbre linux kernel I'm tailing the fedora-36 image builds and we'd need to handle the large repo there too (it will add significant disk usage to our images if we cache the repo there)17:41
clarkbBut I still think starting with the Ci jobs we've got to collect initial data is the way to go if slittle1 wants to pursue it17:42
corvusshould this be focal or jammy?17:54
clarkbcorvus: I think jammy if it works (it generally does in CI, but this would be our first real one)17:55
corvus(current launch-node default is focal, so that's what's booting now, but it just occurred to me maybe we want to change that?)17:55
fungithe mm3 testing we've been doing has all been on jammy as well, with the expectation that's what the new ml server will run17:55
clarkbput another way, if jammy explodes on you I would proceed with focal and we can sort out jammy separately17:56
fungiupdating the launch script sounds fine to me, tes17:56
fungier, yes17:56
clarkbbut if jammy works (and our testing indicates it should) then go for it17:56
fungiagreed, standing up a new service on focal just means one more server we'll have to work out how to upgrade to jammy later17:57
corvushrm, same issue with ansible17:57
corvusi will try ansible 2.917:59
corvusi accidentally installed ansible outside the venv, i'm repairing that now by reinstalling ansible==4.0.0, but be aware that could have caused issues18:02
funginoted, thanks for the warning!18:02
clarkbThe update to using a venv for this can't come soon enough :)18:03
fungiclarkb: do you think we should follow up to the thread from june/july about the ansible 5 default to say that support for older ansible is going away imminently? in case people have pinned it in stable branches or something18:04
clarkbfungi: ++18:04
clarkbyou can like to my topic of changes too to give people examples of what to do18:04
fungi was our last communication on the subject18:04
clarkbthe tripleo-ci change appears to be happy so far. One job failed but it is non voting so I'm unsure of how important that is18:05
fungiwill do18:05
clarkbthe bootstrap bridge job just reported success in the hourly infra-prod jobs fwiw18:06
corvuszomg we have ~root/launch-node-venv ~root/launch-env and i was just about to create ~root/launch-venv18:09
corvusgood news and bad news18:10
fungii think it's time for me to find a lunch-venv in fact18:10
corvusansible works, but the inital playbook now fails with "userdel: user ubuntu is currently used by process 1559"18:10
corvusthat may be a jammy fault18:10
corvusretrying with focal18:10
fungisome service running as that user, or maybe it's the shell itself?18:11
corvusalso, it might just be that jammy checks that and focal didn't18:11
fungientirely possible18:11
clarkbI think we may try to login as root and the older ubuntu images allowed that. But if that fails fallback to ubuntu? But ya I think we can proceed with focal given that and sort this out later18:12
corvuswe could add -f18:12
corvuswe probably use an ansible task for that though18:12
corvusso "just add -f" may not be simple18:12
fungiannouncement for the ansible 2.x removal in jobs has been sent. i'm taking a long-ish late lunch/early dinner a ways up to the north end of the island, so will probably be afk for a couple of hours18:20
opendevreviewJames E. Blair proposed opendev/ master: Add tracing server to DNS
opendevreviewJames E. Blair proposed opendev/system-config master: Add tracing server to inventory
clarkbrlandy|ruck: did get a +1 from zuul20:37
rlandy|ruckand third party20:43
rlandy|ruckdo you need that merged now?20:44
rlandy|ruckotherwise will get a second core on that tomorrow morning (europe time)20:44
rlandy|ruckclarkb: ^^20:44
clarkbrlandy|ruck: I don't need it to be merged. Just know the zuul deployment may update over the weekend and break your jobs20:44
clarkbtomorrow morning is probably fine20:45
rlandy|ruckok - will get that sorted tomorrow20:45
*** ScottSolkhon[m] is now known as scottsol[m]21:22
clarkbinfra-root I've cleaned up my old fedora-36 node in rax dfw and am booting another with the newly built image21:28
clarkbI successfully logged into a new fedora-36 instance in rax booted off the new image21:34
clarkbI have deleted the host now too. I think that ends the fedora-36 safa21:34
*** rlandy|ruck is now known as rlandy|bbl22:05
corvusi'd like to restart the zuul web servers to pick up the rest api change for semaphores23:02
corvusthere's no internal zk api changes, so running with a version skew shouldn't be an issue23:04
Clark[m]Fine with me.23:04
corvusi can do one at a time to make rollback easier in case something goes terribly wrong23:04
Clark[m]May as well do them rolling since we can. I'm off my computer for a moment but will jump back in a few minutes and can help should that become necessary 23:05
corvus01 is restarting23:07
corvusstopping 02 now23:29
corvussanity check still looks good, starting 02 now23:29
corvus is new info23:30
corvus  is a preview build with a ui for that23:33
fungiawesome--thanks for the update!23:33

Generated by 2.17.3 by Marius Gedminas - find it at!