| *** DSpider has quit IRC | 00:12 | |
| *** roman_g has joined #opendev | 01:19 | |
| *** ysandeep|holiday is now known as ysandeep | 02:12 | |
| *** redrobot has joined #opendev | 02:54 | |
| *** elod has quit IRC | 03:35 | |
| *** elod has joined #opendev | 03:37 | |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: Refactor fetch-sphinx-tarball to be executor safe https://review.opendev.org/753199 | 04:15 |
|---|---|---|
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: update-json-file: add role to combine values into a .json https://review.opendev.org/746834 | 04:19 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: ensure-docker: Linaro MTU workaround https://review.opendev.org/747063 | 04:19 |
| *** roman_g has quit IRC | 04:29 | |
| *** ykarel has joined #opendev | 04:33 | |
| *** marios has joined #opendev | 05:00 | |
| AJaeger | mnaser: vexxhost CI is producing POST_FAILURES on zuul-jobs repo like https://review.opendev.org/#/c/753199/ | 06:08 |
| AJaeger | infra-root, translate.openstack.org looks dead, I cannot reach it in the browswer | 06:09 |
| ianw | AJaeger: looking | 06:11 |
| AJaeger | thanks, ianw | 06:12 |
| *** tkajinam has quit IRC | 06:13 | |
| ianw | i can get a console link, but can't get up a console | 06:13 |
| *** tkajinam has joined #opendev | 06:13 | |
| ianw | error 1006 | 06:14 |
| ianw | OS-EXT-STS:vm_state | error | 06:15 |
| *** ralonsoh has joined #opendev | 06:16 | |
| ianw | #status log rebooted translate01.openstack.org which had gone into error state | 06:17 |
| openstackstatus | ianw: finished logging | 06:17 |
| AJaeger | ianw: it's up again, thanks! | 06:19 |
| *** eolivare has joined #opendev | 06:40 | |
| *** roman_g has joined #opendev | 06:42 | |
| openstackgerrit | Merged zuul/zuul-jobs master: update-json-file: add role to combine values into a .json https://review.opendev.org/746834 | 06:43 |
| openstackgerrit | Carlos Goncalves proposed openstack/diskimage-builder master: Add Fedora 33 support https://review.opendev.org/755750 | 06:55 |
| openstackgerrit | Carlos Goncalves proposed openstack/diskimage-builder master: Auto find greatest Fedora cloud image sub-release https://review.opendev.org/755992 | 06:55 |
| *** hashar has joined #opendev | 06:56 | |
| *** slaweq has joined #opendev | 07:03 | |
| roman_g | Good morning, everyone. | 07:05 |
| roman_g | The repository 'http://mirror.ca-ymq-1.vexxhost.opendev.org/ubuntu focal Release' is not signed. | 07:05 |
| roman_g | This is the message I get from apt on Ubuntu for focal focal-updates and focal-backports. | 07:06 |
| roman_g | Is there anything what I need to do prior to using those mirrors? | 07:06 |
| roman_g | Or is it a bug in mirroring? I'm not very good at apt mirrors structure. | 07:07 |
| *** andrewbonney has joined #opendev | 07:12 | |
| *** ykarel_ has joined #opendev | 07:20 | |
| *** ykarel has quit IRC | 07:23 | |
| *** ykarel_ is now known as ykarel | 07:23 | |
| *** rpittau|afk is now known as rpittau | 07:29 | |
| *** fressi has joined #opendev | 07:37 | |
| AJaeger | roman_g: this is expected, see also https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/configure-mirrors/templates/apt/etc/apt | 07:40 |
| *** tosky has joined #opendev | 07:42 | |
| roman_g | AJaeger, got it. Thanks! "[ trusted=yes ]" would help me, as I understand. | 07:42 |
| *** ykarel_ has joined #opendev | 07:48 | |
| *** ykarel has quit IRC | 07:51 | |
| *** moppy has quit IRC | 08:01 | |
| *** moppy has joined #opendev | 08:01 | |
| *** ykarel_ is now known as ykarel | 08:02 | |
| *** Dmitrii-Sh has joined #opendev | 08:23 | |
| openstackgerrit | Javier Peña proposed openstack/diskimage-builder master: Ensure yum-utils is installed in epel element https://review.opendev.org/756010 | 08:29 |
| *** dtantsur|afk is now known as dtantsur | 08:32 | |
| *** redrobot has quit IRC | 08:41 | |
| *** redrobot has joined #opendev | 08:43 | |
| *** ykarel_ has joined #opendev | 09:09 | |
| *** DSpider has joined #opendev | 09:10 | |
| *** ykarel has quit IRC | 09:12 | |
| *** ykarel_ has quit IRC | 09:18 | |
| *** ykarel_ has joined #opendev | 09:19 | |
| *** ykarel_ is now known as ykarel | 09:21 | |
| *** ykarel has quit IRC | 09:42 | |
| *** sshnaidm|afk is now known as sshnaidm | 09:47 | |
| *** ykarel has joined #opendev | 09:54 | |
| *** marios has quit IRC | 10:39 | |
| *** DSpider has quit IRC | 10:40 | |
| *** priteau has joined #opendev | 10:44 | |
| *** DSpider has joined #opendev | 11:24 | |
| *** marios has joined #opendev | 11:31 | |
| *** priteau has quit IRC | 11:34 | |
| *** priteau has joined #opendev | 11:35 | |
| openstackgerrit | sean mooney proposed openstack/diskimage-builder master: [WIP] add apk element https://review.opendev.org/755411 | 11:54 |
| openstackgerrit | sean mooney proposed openstack/diskimage-builder master: [WIP] add alpine element https://review.opendev.org/755410 | 11:54 |
| openstackgerrit | sean mooney proposed openstack/diskimage-builder master: [WIP] bootloader-alpine-support https://review.opendev.org/755412 | 11:54 |
| openstackgerrit | sean mooney proposed openstack/diskimage-builder master: [WIP] openssh-server-alpine-support https://review.opendev.org/755413 | 11:55 |
| openstackgerrit | sean mooney proposed openstack/diskimage-builder master: [WIP] simple-init-alpine-support https://review.opendev.org/755414 | 11:55 |
| fungi | roman_g: by way of explanation, we generate our own package repositories by copying packages from official mirrors, which allows us to generate package indices exactly matching the available packages and so not risk mismatches between mirrored indices and mirrored packages. as a result of not using the official signed indices, our options are to either sign them with an unofficial key or not sign them at | 12:49 |
| fungi | all. the latter helps us reinforce that these repository copies are unofficial and not to be used outside our ci environment (previously when we maintained actual mirrors with official signatures on the indices, random people would set them as their production package mirrors instead of an official mirror) | 12:49 |
| fungi | ianw: AJaeger: rackspace also flooded us with roughly 50 e-mail messages from their ticketing system letting us know about a host outage impacting translate01, i suppose that was related | 12:52 |
| AJaeger | fungi: I guess so - thanksk | 13:01 |
| *** Topner has joined #opendev | 13:13 | |
| *** fressi has quit IRC | 13:26 | |
| *** redrobot has quit IRC | 13:27 | |
| *** redrobot has joined #opendev | 13:31 | |
| *** slaweq has quit IRC | 13:33 | |
| *** slaweq has joined #opendev | 13:39 | |
| openstackgerrit | Pierre Riteau proposed openstack/project-config master: Add Review-Priority flag for cloudkitty https://review.opendev.org/756072 | 13:40 |
| *** tkajinam has quit IRC | 13:49 | |
| *** tkajinam has joined #opendev | 13:49 | |
| *** priteau has quit IRC | 13:52 | |
| roman_g | fungi, thank you for the explanation. It now makes sense to me, and it appeared to be a right choice of me using those repos only during CI here, and letting users supply their own repos during software use or rebuilds. | 13:57 |
| *** priteau has joined #opendev | 13:58 | |
| *** slaweq has quit IRC | 14:47 | |
| *** slaweq has joined #opendev | 14:49 | |
| openstackgerrit | Pierre Riteau proposed opendev/irc-meetings master: Add Rafael Weingärtner to CloudKitty meeting chairs https://review.opendev.org/756087 | 14:51 |
| *** ysandeep is now known as ysandeep|away | 14:52 | |
| *** mlavalle has joined #opendev | 15:05 | |
| *** priteau has quit IRC | 15:11 | |
| fungi | infra-root: reminder, the following production databases are going to experience brief (~5 minute) outages around 03:00-05:00 utc friday: paste, survey, review, grafana | 15:19 |
| fungi | review is probably the one to watch most closely... we might need to restart gerrit to get it to reconnect (though i expect it will be fine other than throwing errors for any attempted interactions when disconnected from its db) | 15:20 |
| fungi | clarkb: ^ might be worth a mention during the meeting too | 15:20 |
| clarkb | I'll add it to the announcements on the agenda when I put that together today | 15:21 |
| fungi | appreciated | 15:22 |
| clarkb | roman_g: fungi: and the reason for going through all of that trouble is debian and ubuntu mirrors have a tendency to get desynced unless they are top level and receive notifications from the distro directly | 15:23 |
| clarkb | that leads to job failures because packages aren't available due to the lack of synchronization | 15:23 |
| roman_g | clarkb, is it like a design issue? | 15:24 |
| clarkb | roman_g: yes its a fundamental issue with the package repo design | 15:24 |
| fungi | they serve indices listing the versions and checksums of all packages | 15:24 |
| fungi | but depending on the state the mirror is in when you try to copy from it, the packages listed in the index may not be present | 15:24 |
| clarkb | we work around it by serving a not quite mirror | 15:24 |
| fungi | yeah, basically unless you are part of their official mirror networks, you can't easily create guaranteed a coherent mirror using their indices | 15:25 |
| roman_g | "depending on the state the mirror" - interesting. I get it. I have had issues related to partially synced ubuntu mirrors previously. | 15:25 |
| clarkb | we do have https enabled on the mirrors now so any newer distro can use https without adding apt https support (its there by default) and that mitigates much of the risk here | 15:26 |
| clarkb | (aiui reprepro is validating what it gets then ssl helps to ensure you're talkign to reprepro) | 15:26 |
| fungi | yep, though also reprepro is creating indices of the exact packages it copied, rather than copying indices and hoping they match the packages | 15:27 |
| *** hashar has quit IRC | 15:47 | |
| *** rpittau is now known as rpittau|afk | 16:06 | |
| openstackgerrit | Pierre Riteau proposed opendev/irc-meetings master: Fix pep8 failures https://review.opendev.org/756123 | 16:11 |
| *** ykarel is now known as ykarel|away | 16:16 | |
| *** dtantsur is now known as dtantsur|afk | 16:21 | |
| *** tosky has quit IRC | 16:22 | |
| *** hamalq has joined #opendev | 16:23 | |
| *** marios is now known as marios|out | 16:30 | |
| openstackgerrit | Pierre Riteau proposed opendev/irc-meetings master: Fix pep8 failures https://review.opendev.org/756123 | 16:34 |
| *** ykarel|away has quit IRC | 16:34 | |
| *** marios|out has quit IRC | 16:48 | |
| clarkb | fungi: catching up on my change backlog this morning. Any chance you have time for https://review.opendev.org/755123 and https://review.opendev.org/755355 I can keep any eye on them if they get approved | 16:49 |
| fungi | sure, can review while i grab some lucnh | 16:50 |
| fungi | lunch | 16:50 |
| clarkb | https://review.opendev.org/754869 is related to the first one but probably less urgent (we aren't udpating our mirror at all currently :( ) | 16:50 |
| johnsom | Hi opendev folks, Could we remove the fedora 28 test image: https://tarballs.opendev.org/openstack/octavia/test-images/test-only-amphora-x64-haproxy-fedora-28.qcow2 it is by far stale at this point. We can save you some bit space. grin | 16:51 |
| johnsom | A quick code search shows no one is using it. (I wouldn't have expected anyone to have used it for a long time) | 16:53 |
| clarkb | johnsom: is there any reason to keep it up? for example if I have an old cloud will building a current image work with it? | 16:53 |
| clarkb | though I guess it is under test-images/ | 16:53 |
| clarkb | ideally no one is running that in prod? | 16:54 |
| johnsom | Well, we strongly encourage people to not use those in any deployment | 16:54 |
| johnsom | Plus, we don't really recommend fedora images.... | 16:55 |
| *** hashar has joined #opendev | 17:03 | |
| *** andrewbonney has quit IRC | 17:08 | |
| *** eolivare has quit IRC | 17:10 | |
| clarkb | fungi: any reason to not rm https://tarballs.opendev.org/openstack/octavia/test-images/test-only-amphora-x64-haproxy-fedora-28.qcow2 ? I'm aklog'd and ready to do it if you have no objects | 17:20 |
| clarkb | *objections | 17:20 |
| fungi | i usually just move stuff like that into my homedir for a while in case someone regrets the removal later | 17:20 |
| fungi | no objections though | 17:21 |
| fungi | or objects for that matter | 17:21 |
| * fungi is not especially object-oriented | 17:21 | |
| johnsom | We could always recreate the job that builds it, but that hasn't run for over a year | 17:22 |
| clarkb | johnsom: I've moved it off the afs filesystem. I'll make a note to rm it from my homdir in a couple weeks | 17:22 |
| johnsom | +1 | 17:23 |
| clarkb | johnsom: you'll need to wait for a publishing pulse which I think happens every 5 minutes? | 17:23 |
| johnsom | Yeah, no rush. Just housekeeping | 17:23 |
| openstackgerrit | Merged opendev/system-config master: Exclude private dirs when mirroring fedora https://review.opendev.org/755123 | 17:24 |
| clarkb | infra-root (and other channel lurkers) it is my understanding that docker container ulimits are based on the ulimits for the dockerd process if not otherwise set. We currently set ulimits on at least the gerrit container which will limit the open files to 8192 as well as set core file size to 0 | 17:55 |
| clarkb | I think we may have taken the file limit from our old puppetry which was used to increase the file limit | 17:55 |
| clarkb | should we drop the ulimit for file county entirely since the intent was the increase it but now we are decreasing it? | 17:55 |
| clarkb | fungi: ^ this is in prep for testing the offline reindex fwiw since we set ulimits on the daemon container but not the run commands container. I think we're good since we should allow even more files in the run container which will do the offline reindex | 17:57 |
| openstackgerrit | Merged opendev/system-config master: Make gitea description update failures nonfatal https://review.opendev.org/755355 | 17:58 |
| fungi | clarkb: in theory we can use ulimit to tell us what the actual limit in the container is both before and after | 18:09 |
| clarkb | ya I did that locally at least to confirm ehat I think is the drfault behavior | 18:11 |
| clarkb | we can double check on review-test too | 18:11 |
| fungi | oh, good enough for me, but sure we can check it there too | 18:11 |
| clarkb | also doing 11am lunches ti sync with kids schedule is weird and difficult :) | 18:11 |
| fungi | when you get to be old like me, 11am lunch and 4pm dinner will seem normal ;) | 18:12 |
| *** ralonsoh has quit IRC | 18:19 | |
| jrosser | i have a new type of failure when switching from ansible to ansible-base and relying more on ansible collections http://paste.openstack.org/show/798690/ | 18:40 |
| jrosser | seems ansible-galaxy now supports installing from git+file:// which might make it possible to carry a set of collections in the CI images | 18:41 |
| clarkb | jrosser: we cache the git repos we host in our images and zuul can act as an intermediary with github | 18:51 |
| clarkb | zuul acts like the cache in those cases | 18:51 |
| jrosser | in this case it's things that would have previously been integral to ansible | 18:52 |
| clarkb | I thought those are still included in the `ansible` package on pypi just not in the git repo called ansile? | 18:52 |
| jrosser | but ansible-base breaks almost everything out into an external collection which needs to be downloaded/installed | 18:53 |
| clarkb | ya ansible and ansible-base are distinct | 18:54 |
| clarkb | and ansible is supposed to continue to be a kitchen sink if installed via pypi aiui | 18:54 |
| jrosser | yes thats right - though we've had such bad luck with anything we care about being merged (open PR for periods of years for rabbitmq) that we needed the new collections anyway | 18:55 |
| clarkb | for git repos in general we've tried to avoid caching them arbitrarily We'd never fit everything onto our images if we did. We're happy to cache what we host since the vast majority of code interaction on the test nodes happens there. And we can hook zuul to external sources as well. I know zuul has talked about doing things to better support collections as well but that would be for the ansible layer zuul | 19:04 |
| clarkb | uses to execute jobs. Not sure how that would interact with nested ansible | 19:04 |
| jrosser | ok well i just thought it was worth knowing about | 19:07 |
| jrosser | imho increasingly the bugfixes to useful modules will be in the collections | 19:07 |
| *** Topner has quit IRC | 19:14 | |
| *** tosky has joined #opendev | 19:29 | |
| *** Topner has joined #opendev | 20:09 | |
| *** roman_g has quit IRC | 20:09 | |
| openstackgerrit | Merged opendev/irc-meetings master: Fix pep8 failures https://review.opendev.org/756123 | 20:19 |
| openstackgerrit | Merged opendev/irc-meetings master: Update Airship meeting details https://review.opendev.org/755334 | 20:19 |
| *** mnasiadka has quit IRC | 20:25 | |
| *** jroll has quit IRC | 20:25 | |
| *** jroll has joined #opendev | 20:27 | |
| openstackgerrit | Pierre Riteau proposed opendev/irc-meetings master: Add Rafael Weingärtner to CloudKitty meeting chairs https://review.opendev.org/756087 | 20:47 |
| *** slaweq has quit IRC | 20:50 | |
| *** hashar has quit IRC | 21:14 | |
| ianw | jrosser: yeah in the system-config gate we run a ansible devel job | 21:21 |
| ianw | so you can already add a few collections as required projects, see project-config | 21:22 |
| jrosser | osa have 'gone early' on 2.10.1 + collections https://review.opendev.org/#/c/752963/ | 21:22 |
| fungi | in the zuulosphere we're still trying to nail down what collections in job definitions look like | 21:23 |
| ianw | https://opendev.org/openstack/project-config/src/branch/master/zuul/main.yaml#L1444 | 21:24 |
| jrosser | theres a regular burble of the jobs failing somehow inside the ansible-galaxy command when installing the collections | 21:24 |
| jrosser | not much, but enough i've noticed its a thing | 21:24 |
| fungi | there are times when i get the impression we're the only community who gets particular about maintaining a stable ci system and solving tooling problems rather than just expecting folks to keep retrying until it magically works | 21:25 |
| fungi | the biggest spanners in the works seem to be (lack of) reliability of the internet and clouds | 21:26 |
| clarkb | is galaxy still a front end for github? | 21:27 |
| clarkb | looks like the tarballs are hosted in s3 | 21:29 |
| fungi | so s/github/amazon/ | 21:29 |
| fungi | or probably more accurately, amazon's cdn | 21:31 |
| jrosser | if theres interest in avoiding constantly downloading collections i'm happy to test it out | 21:32 |
| jrosser | i can probably do that for community.general already by the look of it | 21:32 |
| *** roman_g has joined #opendev | 21:53 | |
| clarkb | https://mirror.dfw.rax.opendev.org/fedora/releases/ woo that updated finally | 22:16 |
| *** roman_g has quit IRC | 22:26 | |
| *** DSpider has quit IRC | 22:35 | |
| *** qchris has quit IRC | 22:41 | |
| *** qchris has joined #opendev | 22:53 | |
| *** tosky has quit IRC | 23:04 | |
| *** mlavalle has quit IRC | 23:09 | |
| *** Topner has quit IRC | 23:33 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!