| *** spatel_ has joined #openstack-ansible | 00:31 | |
| *** spatel_ is now known as spatel | 00:31 | |
| *** sshnaidm|afk has quit IRC | 02:06 | |
| *** evrardjp has quit IRC | 02:33 | |
| *** evrardjp has joined #openstack-ansible | 02:33 | |
| *** jamesdenton has quit IRC | 02:45 | |
| *** jamesdenton has joined #openstack-ansible | 02:46 | |
| *** fresta has quit IRC | 03:17 | |
| *** fresta has joined #openstack-ansible | 03:18 | |
| *** spatel has quit IRC | 03:44 | |
| *** jamesdenton has quit IRC | 04:03 | |
| *** jamesden_ has joined #openstack-ansible | 04:03 | |
| *** raukadah is now known as chkumar|ruck | 04:06 | |
| *** jamesden_ has quit IRC | 04:37 | |
| *** jamesdenton has joined #openstack-ansible | 04:37 | |
| *** jamesdenton has quit IRC | 05:14 | |
| *** jamesdenton has joined #openstack-ansible | 05:15 | |
| *** yasemind has joined #openstack-ansible | 05:25 | |
| *** kleini has joined #openstack-ansible | 05:40 | |
| *** miloa has joined #openstack-ansible | 06:04 | |
| jrosser | jonher: you'll probably need to squash 782837 into 783219 as i'm not sure either will be able to merge without the other | 06:36 |
|---|---|---|
| noonedeadpunk_ | mornings | 06:37 |
| *** miloa has quit IRC | 06:38 | |
| jrosser | jonher: then we need to look at what other components need to be installed alongside cloudkitty to make the deployment meaningful for those tests, which could be gnocchii and ceilometer | 06:38 |
| jrosser | morning | 06:38 |
| noonedeadpunk_ | yeah, we might need sort of telemetry added | 06:39 |
| jonher | morning, it deploys without 782837 but one of the processes keeps on loop-crashing in the background, but if you think they should be squashed into one i can do that | 06:39 |
| noonedeadpunk_ | somewhere here https://opendev.org/openstack/openstack-ansible/src/branch/master/tests/roles/bootstrap-host/vars/main.yml#L50 | 06:39 |
| noonedeadpunk_ | but maybe just ceilometer and gnocchi would be enough | 06:40 |
| jonher | i managed to get a working config this night for cloudkitty so i will need to push some more updates to that repo | 06:40 |
| jonher | keystone gnocci ceilometer should be OK for testing i think | 06:41 |
| jrosser | i was thinking we should do something more minimal than just add the whole telemetry stack, the runtime will get too long otherwise | 06:41 |
| jonher | and it stores to mysql, but that will be there | 06:41 |
| jrosser | there is a cloudkitty tempest plugin too | 06:41 |
| jrosser | so we'd add something like this https://review.opendev.org/c/openstack/openstack-ansible-os_tempest/+/754044 | 06:42 |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Set buster jobs to NV https://review.opendev.org/c/openstack/openstack-ansible/+/782681 | 06:43 |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Bump SHAs for master https://review.opendev.org/c/openstack/openstack-ansible/+/783577 | 06:43 |
| jrosser | looks like we have an old master SHA bump patch lying around | 06:44 |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Disable ssl for rabbitmq during upgrade https://review.opendev.org/c/openstack/openstack-ansible/+/782996 | 06:45 |
| noonedeadpunk_ | yeah | 06:45 |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Disable ssl for rabbitmq during upgrade https://review.opendev.org/c/openstack/openstack-ansible/+/782996 | 06:45 |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Bump SHAs for master https://review.opendev.org/c/openstack/openstack-ansible/+/783577 | 06:46 |
| noonedeadpunk_ | don't we want to use newer pip? | 06:46 |
| jrosser | sure | 06:47 |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Bump SHAs for master https://review.opendev.org/c/openstack/openstack-ansible/+/783577 | 06:47 |
| noonedeadpunk_ | Oh, https://review.opendev.org/c/opendev/system-config/+/777968 has merged! | 07:01 |
| noonedeadpunk_ | So we can try out Bullseye in https://review.opendev.org/c/openstack/openstack-ansible/+/782681 | 07:01 |
| *** luksky has joined #openstack-ansible | 07:03 | |
| noonedeadpunk_ | probably worth making a follow-up anyway... | 07:04 |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: [DNM] Add Debian Bullseye support https://review.opendev.org/c/openstack/openstack-ansible/+/783606 | 07:10 |
| *** jamesdenton has quit IRC | 07:10 | |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: [DNM] Add Debian Bullseye support https://review.opendev.org/c/openstack/openstack-ansible/+/783606 | 07:10 |
| *** jamesdenton has joined #openstack-ansible | 07:11 | |
| jrosser | noonedeadpunk_: i think we need to look at making bullseye images really first | 07:12 |
| noonedeadpunk_ | yeah. making patch to nodepool | 07:12 |
| jrosser | oh cool | 07:13 |
| noonedeadpunk_ | I just though it was already there for some reason :( | 07:13 |
| jrosser | i did notice on some of our stable branches we use debian-stable == stretch | 07:13 |
| jrosser | we should probably change that before getting a surprise | 07:13 |
| *** rpittau|afk is now known as rpittau | 07:15 | |
| * jrosser wonders why all the roles that start with 'm' are the ones which break | 07:20 | |
| noonedeadpunk_ | lol | 07:41 |
| noonedeadpunk_ | yep, we probably should getting rid of stretch... | 07:42 |
| noonedeadpunk_ | but porbably not the highest prio | 07:42 |
| *** shyamb has joined #openstack-ansible | 08:09 | |
| *** tosky has joined #openstack-ansible | 08:28 | |
| *** noonedeadpunk_ is now known as noonedeadpunk | 09:08 | |
| *** shyamb has quit IRC | 09:31 | |
| *** shyamb has joined #openstack-ansible | 09:33 | |
| *** CeeMac has quit IRC | 09:37 | |
| noonedeadpunk | looks like CI is totally broken... | 09:48 |
| noonedeadpunk | on neutron db expand | 09:49 |
| noonedeadpunk | doh | 09:49 |
| noonedeadpunk | have we merged smth for neutron... | 09:50 |
| noonedeadpunk | see only https://opendev.org/openstack/openstack-ansible-os_neutron/commit/e06c25eaa15c305ec2276fe256fe66719ce0d7d3 | 09:51 |
| noonedeadpunk | https://8b7bc396c22f90223dfa-906a8885044ac3b62edc724702c6de01.ssl.cf1.rackcdn.com/783577/5/check/openstack-ansible-deploy-aio_metal-centos-8/00474e0/logs/ara-report/results/1378.html | 09:52 |
| *** frickler has joined #openstack-ansible | 09:55 | |
| *** shyamb has quit IRC | 10:03 | |
| *** shyamb has joined #openstack-ansible | 10:04 | |
| noonedeadpunk | looking at trove config now, there's a lot more then just single bug with innobackupex | 10:10 |
| *** sshnaidm|afk has joined #openstack-ansible | 10:18 | |
| noonedeadpunk | jrosser: I guess this ends up weird to what I see in aio https://opendev.org/openstack/openstack-ansible-lxc_hosts/src/branch/master/tasks/lxc_cache_create.yml#L89 http://paste.openstack.org/show/803999/ | 10:22 |
| noonedeadpunk | have we accidentaly change behaviour with https://opendev.org/openstack/openstack-ansible-lxc_hosts/commit/7fc5e9950ea9e8b379295837fede70f864502439 ? | 10:22 |
| noonedeadpunk | (non related to recent neutron failure) | 10:23 |
| noonedeadpunk | oh, hm... there's even config for `ubuntu-bionic-amd64` | 10:25 |
| noonedeadpunk | http://paste.openstack.org/show/804000/ | 10:25 |
| jrosser | noonedeadpunk: did i break something? | 10:32 |
| jonher | in regards to cloudkitty+tempest it looks like it's already enabled in the tempest role https://github.com/openstack/openstack-ansible-os_tempest/search?q=tempest_service_available_cloudkitty and should match/test unless i'm overlooking something which could be very likely | 10:35 |
| noonedeadpunk | well, no, but having ubuntu-bionic-amd64 among lxc-ls is weird. And not sure how/when it appeared there | 10:35 |
| noonedeadpunk | Just guessed you might have a clue since you was looking through everything we're doing there pretty recently | 10:36 |
| jrosser | hmm | 10:38 |
| jrosser | jonher: yes i took a look and it's already there, though it doesnt actually seem to run any tests for cludkitty | 10:38 |
| *** CeeMac has joined #openstack-ansible | 10:46 | |
| jonher | hm yeah you are right, they are listed in "os_tempest : List tempest tests" but not executed according to the logs | 10:55 |
| jrosser | jonher: if you have your AIO might be worth having a poke at that, see if there is some allow/block list on the tempest tests, maybe we just don't enable them? | 11:03 |
| jonher | i've just deployed a VM to start testing that, so will give it a shot | 11:03 |
| noonedeadpunk | btw regarding tempest.. | 11:12 |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_tempest master: Log tempestconf inside tempest_log_dir https://review.opendev.org/c/openstack/openstack-ansible-os_tempest/+/783637 | 11:12 |
| *** fanfi has joined #openstack-ansible | 11:22 | |
| *** sshnaidm|afk is now known as sshnaidm|off | 11:45 | |
| *** rh-jelabarre has joined #openstack-ansible | 11:57 | |
| jrosser | noonedeadpunk: i have ubuntu-bionic-amd64 show up in lxc-ls on my victoria environment, so thats not new | 12:06 |
| noonedeadpunk | aha... | 12:06 |
| noonedeadpunk | that's good and bad news at the same time I guess | 12:07 |
| jrosser | i wonder if the original image gets started at some point | 12:07 |
| jrosser | becasue it's State: STOPPED with lxc-info | 12:07 |
| noonedeadpunk | as I don't have this in my upgraded envs | 12:07 |
| jrosser | theres no sign of it in /var/log/lxc/... though which suggests it has never been run | 12:09 |
| noonedeadpunk | `btrfs qgroup limit {{ item }} /var/lib/lxc/{{ lxc_container_base_name }}` makes operation against this exact path.. | 12:09 |
| noonedeadpunk | so we might use it just for cache preparation | 12:10 |
| noonedeadpunk | and the thing that was not intended is having config | 12:11 |
| jrosser | it's odd becasue i thought the result of the cache prep ended up in /var/cache/lxc/download/.... | 12:12 |
| noonedeadpunk | indeed | 12:13 |
| jrosser | looking in a random CI job log i see no reference to ubuntu-focal-amd64 until right at the end when it does log collection | 12:14 |
| noonedeadpunk | well, apt-file not sure helpfull in searching for packages that could provide config | 12:18 |
| noonedeadpunk | but indeed it appeared 8 mins before any container on the aio | 12:19 |
| jrosser | if you have an AIO then you could look at the same timestamps in /openstack/log/ansible-logging/.. | 12:20 |
| noonedeadpunk | and has different permissions... | 12:20 |
| jrosser | that would show which tasks were running | 12:20 |
| *** shyamb has quit IRC | 12:20 | |
| jrosser | here on my real environment the /var/lib/ubuntu-bionic-amd64 directory is much older than the dirs/files for the rest of the containers | 12:21 |
| jrosser | maybe this only happens with the image download/build | 12:22 |
| noonedeadpunk | um, log does not contain this time at all... | 12:23 |
| noonedeadpunk | also it's super weird as consist like from facts? | 12:25 |
| jrosser | is this a new AIO you have? | 12:26 |
| noonedeadpunk | yep | 12:26 |
| noonedeadpunk | just spawned today | 12:26 |
| noonedeadpunk | to see wtf is going on with neutron | 12:27 |
| jrosser | oh well, i wonder if this is a side-effect of using COW backing | 12:27 |
| jrosser | i have the rootfs tgz in /var/cache/lxc/download/ubuntu/bionic/amd64/default/ | 12:28 |
| jrosser | but thats no good for the ZFS backing store i have here, so the base image needs to be put into the ZFS somehow then the actual containers are snapshots off that | 12:29 |
| jrosser | i expect the same it true for btrfs if thats using snapshots too | 12:29 |
| noonedeadpunk | eventually it does not contain anything except gathered facts o_O | 12:29 |
| *** pcaruana has quit IRC | 12:30 | |
| jrosser | hmm? http://paste.openstack.org/show/804007/ | 12:30 |
| jrosser | https://github.com/openstack/openstack-ansible-lxc_container_create/blob/master/tasks/lxc_container_create_cow.yml#L34 | 12:32 |
| *** fanfi has quit IRC | 12:37 | |
| *** pcaruana has joined #openstack-ansible | 12:39 | |
| *** fanfi has joined #openstack-ansible | 12:40 | |
| *** cloudnull has joined #openstack-ansible | 12:41 | |
| jonher | i have 2 aio's that's 30days+ old if you want me to check something | 12:46 |
| noonedeadpunk | jrosser: oh, really looks like that's it | 12:49 |
| jrosser | maybe we don't expect to find that with dir backing i guess | 12:50 |
| jrosser | i would expect each container to be a freshly unpacked copy of whats in /var/cache/lxc in that case perhaps | 12:51 |
| noonedeadpunk | but I guess in aio we should have `lxc_container_backing_store: dir` | 12:52 |
| noonedeadpunk | yeah | 12:53 |
| * jrosser looks at config_template broken jobs | 12:54 | |
| jrosser | wtf is this test stuff doing 8-O | 12:54 |
| noonedeadpunk | oh, there's really complicated testing there... | 12:54 |
| noonedeadpunk | it's more like unit tests with mock on ansible | 12:55 |
| jrosser | somehow with the host and a container too | 12:55 |
| jrosser | but only looks like it tests one set of outcomes | 12:55 |
| jrosser | unless i totally misunderstand how handlers work | 12:55 |
| *** spatel_ has joined #openstack-ansible | 13:11 | |
| *** spatel_ is now known as spatel | 13:11 | |
| noonedeadpunk | hm, they used to work recently | 13:12 |
| spatel | noonedeadpunk jrosser not sure why it failed - https://review.opendev.org/c/openstack/openstack-ansible/+/783318 | 13:13 |
| noonedeadpunk | but agree, that they are weird and not easy readable | 13:16 |
| jrosser | what i see is this https://github.com/openstack/ansible-config_template/blob/master/tests/test-yaml.yml#L26 | 13:22 |
| jrosser | returns "prepared": "{\n \"added\": {},\n \"changed\": {},\n \"removed\": {}\n}" | 13:22 |
| jrosser | so i wonder if config_template is no longer doing the right thein for ansible run with --diff | 13:22 |
| jrosser | which could have broken with an ansible upgrade | 13:22 |
| noonedeadpunk | iirc it didn;'t work properly with diff for a while | 13:24 |
| noonedeadpunk | I can recall dmsimard has pointed to this a while back when tried out having a diff recorded in ara or smth like that | 13:25 |
| jrosser | this whole section of asserts is relying on the diff results being correct https://github.com/openstack/ansible-config_template/blob/master/tests/test.yml#L81-L112 | 13:27 |
| *** d34dh0r53 has joined #openstack-ansible | 13:32 | |
| noonedeadpunk | found what's wrong with our CI overall | 13:48 |
| noonedeadpunk | it's importlib-metadata 3.9.0. 3.9.1 has been released half an hour ago and fixes the issue | 13:48 |
| *** MrClayPole has quit IRC | 13:49 | |
| jrosser | thats pinned in u-c | 13:49 |
| noonedeadpunk | the question is why in the world I did have importlib-metadata-3.9.0 installed | 13:50 |
| noonedeadpunk | yeah | 13:50 |
| * jrosser looks at new resolver | 13:50 | |
| *** d34dh0r53 has quit IRC | 13:52 | |
| *** MrClayPole has joined #openstack-ansible | 13:54 | |
| noonedeadpunk | I think it's neutron only though | 13:55 |
| *** d34dh0r53 has joined #openstack-ansible | 13:55 | |
| noonedeadpunk | nova for example has 3.7.2 installed | 13:55 |
| noonedeadpunk | aha, for neutron we retrieve content and do some replacement | 13:56 |
| *** spatel has quit IRC | 13:57 | |
| noonedeadpunk | jrosser: seems, that content is trimmed to 400 lines | 13:59 |
| jrosser | oh no | 14:00 |
| noonedeadpunk | http://paste.openstack.org/show/804012/ | 14:00 |
| *** spatel_ has joined #openstack-ansible | 14:00 | |
| *** spatel_ is now known as spatel | 14:00 | |
| jrosser | this maybe https://github.com/openstack/openstack-ansible-os_neutron/blob/master/tasks/neutron_install.yml#L54-L65 | 14:01 |
| noonedeadpunk | yeah | 14:01 |
| noonedeadpunk | I was jsut looking at it, and my /var/www/repo/os-releases/22.0.0.0rc2.dev114/neutron-22.0.0.0rc2.dev114-source-constraints.txt does not simply have importlib-metadata | 14:01 |
| jrosser | 400 is just too much a round number | 14:03 |
| noonedeadpunk | well, at least we'vecatched that... | 14:04 |
| jrosser | yeah | 14:05 |
| jrosser | could be in all sorts of places, uri only returns 400 lines, or only 400 in the registered var, ...... | 14:05 |
| noonedeadpunk | not sure about workaround though... | 14:07 |
| noonedeadpunk | maybe we should save it as usual, just drop content with lineinfile... | 14:07 |
| jrosser | it needs to end up as a file on the repo server | 14:08 |
| noonedeadpunk | and pass list of things to drop with another var to python_venv_build? | 14:08 |
| jrosser | yeah i think thats the only way, as we don't know where the file needs to be places | 14:08 |
| noonedeadpunk | yeah | 14:08 |
| jrosser | only python_venv_build knows that | 14:08 |
| noonedeadpunk | and since we can't jsut pass content for some reason... | 14:09 |
| jrosser | seems like a good improvement, new var for patterns to drop, then remove those tasks i added in the roles | 14:09 |
| jrosser | theres a few places that was needed | 14:09 |
| noonedeadpunk | well yes, but we will have to loop with removing line by line | 14:09 |
| noonedeadpunk | while you did that in memory... so not sure about how much improvement that is.. | 14:09 |
| jrosser | maybe some debug statements to see where the data gets lost | 14:10 |
| jrosser | i think that the tripleo people needed something similar | 14:10 |
| noonedeadpunk | it fails on get url I believe | 14:13 |
| noonedeadpunk | `"transfer_encoding": "chunked"` | 14:13 |
| spatel | jamesdenton are you around? I have few question related OVN | 14:15 |
| jrosser | i wonder if we can set some accept_encoding to prevent chunked | 14:17 |
| jrosser | noonedeadpunk: is this valid? https://github.com/openstack/ansible-config_template/blob/master/tests/test.yml#L78-L79 | 14:22 |
| jrosser | it seems to ignore the delegate, running all the tasks again against localhost | 14:23 |
| noonedeadpunk | you mean after ansible update? dunno. but I guess it used to be | 14:25 |
| jrosser | i think that the diff is coming out empty because it targets localhost again rather than container1 | 14:26 |
| jrosser | i see it run again here https://zuul.opendev.org/t/openstack/build/31c89962d3354d33a4e821c84f474bc6/log/job-output.txt#10766-10769 | 14:26 |
| jrosser | with "ok" rather than "changed" | 14:26 |
| fanfi | Hi, could I ask for help? during installing new deployment i got following error. http://paste.openstack.org/show/804004/ | 14:32 |
| jrosser | fanfi: you are using the master branch i think? | 14:34 |
| fanfi | good point...i will check it | 14:34 |
| jrosser | best to pick the most recent tag on a stable branch unless you are doing development work | 14:35 |
| fanfi | git branch | 14:36 |
| fanfi | * master | 14:36 |
| openstackgerrit | Jonathan Rosser proposed openstack/ansible-config_template master: Run tests against localhost and container https://review.opendev.org/c/openstack/ansible-config_template/+/783661 | 14:46 |
| fanfi | jrosser and I am up to date with master repo | 14:47 |
| jrosser | fanfi: the master branch is the one that will become the *next* release of openstack-ansible, i.e Wallaby | 14:48 |
| jrosser | if you want a Victoria deployment, which is the current stable release then you need to use a tag from the stable/victoria branch | 14:48 |
| jrosser | i knew you have master checked out becasue the 'admin' user on the database is something thats not yet in a released stable version | 14:49 |
| noonedeadpunk | btw I didn't thought it should fail like that | 14:53 |
| jrosser | no, though i wonder if thats ever been tested on a multinode though | 14:54 |
| jrosser | on well infra jobs should do that | 14:55 |
| jrosser | *oh | 14:55 |
| jrosser | did you see there is a mariadb jira about the 10.5.8->10.5.9 upgrade troubles? | 14:55 |
| noonedeadpunk | nope, I haven't | 14:55 |
| jrosser | seems they have a mistake with superuser grants being br0k across the upgrade | 14:55 |
| jrosser | https://jira.mariadb.org/browse/MDEV-25030 | 14:56 |
| noonedeadpunk | doh. so it's not our issue with 10.5.9 atm | 14:56 |
| noonedeadpunk | or we should switch to admin earlier? | 14:57 |
| jrosser | well, we did some AIO work here last week and gshippey sees something similar but different with the root user getting broken | 14:57 |
| jrosser | after the upgrade we saw access: "0", i.e root user looses all grants totally | 14:57 |
| noonedeadpunk | I wonder if we can jsut create admin user _before_ galera upgrade... | 14:58 |
| noonedeadpunk | with upgrade scripts | 14:58 |
| jrosser | right, yes | 14:58 |
| jrosser | we still rely on the root user for replication though | 14:59 |
| noonedeadpunk | well, according to bug, replication should left as a permission | 14:59 |
| noonedeadpunk | but yeah, not sure... | 15:00 |
| fanfi | jrosser when I switch branch 'stable/victoria' from 'origin' and pull the repo i got message Already up to date. | 15:03 |
| jrosser | fanfi: ok, i think thats normal | 15:04 |
| jrosser | git usually keeps all the branches locally for you | 15:04 |
| fanfi | perfect thanks, i will try it ...and are you editing any mariadb parameters like max_connections ..atc ? | 15:09 |
| fanfi | or you have it all in default | 15:10 |
| *** jamesdenton has quit IRC | 15:12 | |
| *** jamesdenton has joined #openstack-ansible | 15:13 | |
| *** macz_ has joined #openstack-ansible | 15:15 | |
| jrosser | fanfi: the defaults applied are here https://github.com/openstack/openstack-ansible-galera_server/blob/master/defaults/main.yml#L148-L152 | 15:22 |
| noonedeadpunk | maybe it was just that interminent issue in galera we see sometimes in CI | 15:34 |
| *** rh-jelabarre has quit IRC | 15:43 | |
| *** rh-jelabarre has joined #openstack-ansible | 15:43 | |
| *** rh-jelabarre has quit IRC | 15:44 | |
| *** rh-jlabarre has joined #openstack-ansible | 15:44 | |
| *** Guest76033 is now known as jroll | 15:50 | |
| noonedeadpunk | o_O we catch the same issue with importlib-metadata on V for buster https://zuul.opendev.org/t/openstack/build/931b57810ec2442189d7e3771595ac74/log/logs/host/keystone-wsgi-public.service.journal-10-53-22.log.txt#275 | 15:50 |
| jrosser | https://github.com/redhat-openstack/infrared/blob/master/infrared/common/roles/create_venv/tasks/main.yml#L118-L137 | 15:52 |
| noonedeadpunk | well... | 15:52 |
| noonedeadpunk | anyway, wtf on V | 15:52 |
| jrosser | just looking on my V deployment here and i have 3.0.0 in the keystone container | 15:54 |
| noonedeadpunk | dog | 15:55 |
| noonedeadpunk | *doh | 15:55 |
| noonedeadpunk | `importlib-metadata===1.7.0;python_version=='3.6'` | 15:55 |
| noonedeadpunk | buster has 3.7 | 15:55 |
| noonedeadpunk | *facepalm* | 15:55 |
| noonedeadpunk | https://opendev.org/openstack/requirements/src/branch/stable/victoria/upper-constraints.txt#L269 | 15:56 |
| noonedeadpunk | that is so... frustrating... | 15:58 |
| jrosser | here is where that came in https://github.com/openstack/requirements/commit/2d7a41b64c8f891a0c6c0a76f30ff7fe7557163a | 15:58 |
| * jrosser asks the requirements people | 16:00 | |
| noonedeadpunk | honestly - there're also pretty a number of constraints for 3.6 and 3.8 only | 16:00 |
| noonedeadpunk | that does not take into account 3.7 | 16:00 |
| jrosser | with the new resolver this does kind of point to needing "remove these lines" / "insert these other lines" function for python_venv_build | 16:01 |
| noonedeadpunk | do we want to add some extras as well? | 16:02 |
| jrosser | i guess that depends if we can get suitable additions/changes made to u-c for python versions like 3.7 | 16:03 |
| jrosser | if we can't, then new releases of stuff with "unhelpful" entries in UC will break us on buster, or anything else with a slightly out of the ordinary python version | 16:04 |
| noonedeadpunk | Shouldn't we just push to have u-c set properly? | 16:05 |
| jrosser | ideally yes, agreed | 16:05 |
| jrosser | i guess that needs == changing to >= | 16:05 |
| noonedeadpunk | regarding things where we can do nothing (ie tempest, gnocchi, dunno) yes, we should remove from contraints.. | 16:05 |
| noonedeadpunk | for python? yeah.. | 16:06 |
| noonedeadpunk | doesn't really clear what to do with https://opendev.org/openstack/requirements/src/branch/master/upper-constraints.txt#L257-L258 though | 16:06 |
| noonedeadpunk | or https://opendev.org/openstack/requirements/src/branch/master/upper-constraints.txt#L385-L386 | 16:06 |
| *** rpittau is now known as rpittau|afk | 16:08 | |
| jrosser | i guess we are testing those unconstrained right now on osa/buster | 16:09 |
| jrosser | which means that the 3.8 version is probably OK | 16:09 |
| jrosser | feels really like this should be a lint test on u-c | 16:10 |
| noonedeadpunk | yeah | 16:10 |
| jrosser | prometheanfire: are we getting confused here with the 3.7 issue | 16:22 |
| jrosser | the end result is we have patches which work everywhere, like bionic/focal/centos but blow up on buster | 16:22 |
| jrosser | like here just fails all the buster jobs https://review.opendev.org/c/openstack/openstack-ansible/+/783580/ | 16:23 |
| prometheanfire | ok, we should compare the two requirements batches | 16:25 |
| jrosser | i think we are generally concerned with seeing a lot of ==3.6 and ==3.8 in u-c and what that means for a 3.7 system | 16:26 |
| prometheanfire | well, the officially supported python versions for that release were 3.6 and 3.8 | 16:26 |
| jrosser | well https://governance.openstack.org/tc/reference/runtimes/victoria.html | 16:28 |
| prometheanfire | here are the updates to upper-constraints | 16:28 |
| prometheanfire | https://github.com/openstack/requirements/compare/0e97a6d7a11dce9bf2571f86de2bf3ba48daaa38...628ba9dff2189b5392c8a9b5c232c1f261803a9e | 16:28 |
| prometheanfire | basically, not sure what triggered it here | 16:28 |
| prometheanfire | https://github.com/openstack/requirements/commit/ac8bcb00122ccd4c4c8cc22612c14c3c37624026 was the revert, as part of the bot run | 16:34 |
| prometheanfire | so, maybe something in gate happened, we haven't changed the constraints generation command in a while | 16:36 |
| mgariepy | https://github.com/openstack/requirements/blob/master/upper-constraints.txt#L109-L110 | 16:37 |
| mgariepy | if python is 3.7 maybe some pkg are missing ?? | 16:37 |
| prometheanfire | ya, the constraints generation command doesn't include 3.7 iirc | 16:37 |
| noonedeadpunk | I don't think they're present for 3.6 and for 2.8 but not for 3.7 | 16:37 |
| noonedeadpunk | *3.8 | 16:38 |
| prometheanfire | in project-config | 16:38 |
| prometheanfire | playbooks/proposal/propose_update.sh | 16:38 |
| prometheanfire | the generate-constraints command | 16:38 |
| prometheanfire | generate-constraints -b blacklist.txt -p /usr/bin/python3.6 -p /usr/bin/python3.8 -r global-requirements.txt > upper-constraints.txt | 16:38 |
| jrosser | is that something that just runs for master branch? | 16:39 |
| *** klamath_atx has joined #openstack-ansible | 16:39 | |
| prometheanfire | ya | 16:39 |
| prometheanfire | just pointing out that 3.7 is not included there | 16:39 |
| prometheanfire | that gate runs 3.7 was not expected | 16:39 |
| jrosser | no well that might be OK really, as i'm not sure there is a 3.7 which can run W anyway | 16:40 |
| noonedeadpunk | buster can't so yeah | 16:40 |
| jrosser | our issue is really about V i think | 16:40 |
| noonedeadpunk | +1 | 16:41 |
| prometheanfire | ya, I don't think so, wallaby is explicitly 3.6 and 3.8 | 16:41 |
| prometheanfire | for victoria, 3.7 is only for CI | 16:41 |
| noonedeadpunk | which would be incorrect | 16:41 |
| noonedeadpunk | because of not constrained things | 16:42 |
| prometheanfire | https://governance.openstack.org/tc/reference/runtimes/victoria.html says for CI only | 16:43 |
| noonedeadpunk | and it's broken:) | 16:44 |
| noonedeadpunk | well, at least was, until package has been fixed | 16:44 |
| prometheanfire | what's confusing to me is that it's broken NOW | 16:44 |
| jrosser | up to this point whatever the latest version of importlib was OK, by luck | 16:44 |
| prometheanfire | because the review you provided doesn't include a reqs change that should trigger it | 16:44 |
| prometheanfire | https://github.com/openstack/requirements/compare/0e97a6d7a11dce9bf2571f86de2bf3ba48daaa38...628ba9dff2189b5392c8a9b5c232c1f261803a9e | 16:44 |
| noonedeadpunk | yes, exactly https://pypi.org/project/importlib-metadata/#history | 16:45 |
| prometheanfire | so, submit a patch to victoria for python==3.7 :P | 16:45 |
| noonedeadpunk | they did bunch of releases in 2 days. and it might be broken for 3.8 at some point as well | 16:46 |
| prometheanfire | ya, though for 3.8 we are using builtin, or should be | 16:46 |
| noonedeadpunk | ah, right | 16:47 |
| noonedeadpunk | why do we have it constrained then? | 16:47 |
| prometheanfire | because it's not included for 3.6 (or 3.7) as part of base python | 16:48 |
| noonedeadpunk | still for 3.6? | 16:48 |
| noonedeadpunk | ok, gotcha | 16:48 |
| prometheanfire | basically, copy th 3.6 line, and change 3.6 to 3.7 | 16:48 |
| prometheanfire | for upper-constraints | 16:48 |
| prometheanfire | we don't update stable branches so it'll stick | 16:48 |
| prometheanfire | https://review.opendev.org/c/openstack/requirements/+/783715 | 16:50 |
| noonedeadpunk | also weird about importlib-resources btw | 16:52 |
| noonedeadpunk | it's native only in 3.9? | 16:52 |
| noonedeadpunk | but it's https://github.com/openstack/requirements/blob/stable/victoria/upper-constraints.txt#L501 | 16:52 |
| jrosser | oh - that may be my fault | 16:53 |
| prometheanfire | if the review needs more stuff pinned for 3.7 feel free to update it | 16:54 |
| prometheanfire | given how osa points to a sha though it'll take merging before osa can test in gate I think | 16:55 |
| jrosser | i think we can depends-on it | 16:56 |
| jrosser | oh hrrm actually that may not be true | 16:56 |
| noonedeadpunk | we can depends-on, but eventually things are fixed by themselves in 3.9.1 | 16:56 |
| jrosser | we can depends-on it if we land my u-c cache patches :) | 16:56 |
| prometheanfire | so, since it's unconstrainted you can recheck? | 16:56 |
| *** gyee has joined #openstack-ansible | 16:57 | |
| noonedeadpunk | well, infra pip mirrors will need to get updated as well I guess | 16:57 |
| noonedeadpunk | but generally yes | 16:57 |
| prometheanfire | so, transient failure, but updating constraints would be best | 16:58 |
| noonedeadpunk | yes, as eventually they're here to avoid these things from happening | 16:58 |
| jrosser | noonedeadpunk: whilst we look at u-c things, should re remove setuptools from global-requirement-pins | 17:00 |
| jrosser | that used to be OK with the old pip resolver that we had a specific version in OSA, but the new one breaks every time we are != the version in u-c | 17:01 |
| jrosser | i'm just debugging openstack-ansible-tests broken and thats why | 17:01 |
| noonedeadpunk | I was thinking about that as well. I guess we added it locally since there were periods when they were unconstrained locally | 17:01 |
| noonedeadpunk | s/locally/in u-c/ | 17:01 |
| jrosser | feels like we should take out the pin on our side | 17:02 |
| noonedeadpunk | and eventually it worked nicely with old resolver) | 17:02 |
| noonedeadpunk | yeah, I guess so | 17:02 |
| *** yasemind has quit IRC | 17:03 | |
| jrosser | oh | 17:04 |
| jrosser | we use the same file for the ansible-runtime venv don't we? | 17:04 |
| noonedeadpunk | we do | 17:04 |
| noonedeadpunk | and for all venvs as well | 17:05 |
| noonedeadpunk | https://opendev.org/openstack/openstack-ansible/src/branch/master/inventory/group_vars/all/all.yml#L149-L150 | 17:05 |
| jrosser | the runtime venv is OK https://github.com/openstack/openstack-ansible/blob/master/scripts/scripts-library.sh#L80 | 17:05 |
| jrosser | and i think the others will be covered by u-c, except maybe for things like stackviz where it's not used | 17:07 |
| *** djhankb has joined #openstack-ansible | 17:07 | |
| noonedeadpunk | will push patches to python_venv_build first time in the morning. want to test on fresh head | 17:09 |
| * noonedeadpunk also remembers about patch to nova regarding qemu open files limit | 17:10 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible master: Use the version of setuptools specified in upper-constraints https://review.opendev.org/c/openstack/openstack-ansible/+/783720 | 17:10 |
| *** gyee has quit IRC | 17:14 | |
| *** gyee has joined #openstack-ansible | 17:14 | |
| *** gyee has quit IRC | 17:38 | |
| *** gyee has joined #openstack-ansible | 17:49 | |
| *** cloudnull has quit IRC | 18:43 | |
| *** jamesdenton has quit IRC | 19:02 | |
| *** jamesden_ has joined #openstack-ansible | 19:02 | |
| *** cloudnull has joined #openstack-ansible | 19:21 | |
| *** akahat has quit IRC | 20:56 | |
| *** akahat has joined #openstack-ansible | 21:20 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-tests master: Do not filter setuptools from upper-constraints https://review.opendev.org/c/openstack/openstack-ansible-tests/+/783747 | 21:21 |
| *** jamesdenton has joined #openstack-ansible | 21:56 | |
| *** jamesden_ has quit IRC | 21:59 | |
| *** jamesden_ has joined #openstack-ansible | 22:07 | |
| *** cloudnull has quit IRC | 22:09 | |
| *** jamesdenton has quit IRC | 22:09 | |
| *** spatel has quit IRC | 22:11 | |
| *** dasp has quit IRC | 22:13 | |
| *** dasp has joined #openstack-ansible | 22:19 | |
| *** luksky has quit IRC | 22:23 | |
| *** jamesdenton has joined #openstack-ansible | 22:25 | |
| *** jamesden_ has quit IRC | 22:28 | |
| *** cloudnull has joined #openstack-ansible | 22:52 | |
| *** macz_ has quit IRC | 23:12 | |
| *** macz_ has joined #openstack-ansible | 23:19 | |
| *** macz_ has quit IRC | 23:23 | |
| *** macz_ has joined #openstack-ansible | 23:55 | |
| *** macz_ has quit IRC | 23:59 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!