| snadge | Invalid input for operation: Flat provider networks are disabled during tempest install | 00:00 |
|---|---|---|
| snadge | thats simple i don't have a flat network defined, i'll override it to vlan | 00:02 |
| snadge | ok so i uninstalled gcc on compute1, and TASK [os_nova : Install kvm pip packages] succeeded on compute1.. but if i log into it gcc isn't installed system wide, im assuming this is expected behaviour? | 04:54 |
| *** rpittau|afk is now known as rpittau | 08:17 | |
| opendevreview | Andrew Bonney proposed openstack/openstack-ansible master: Rename black/white list variables https://review.opendev.org/c/openstack/openstack-ansible/+/796636 | 08:36 |
| opendevreview | Merged openstack/ansible-hardening master: Remove references to unsupported operating systems https://review.opendev.org/c/openstack/ansible-hardening/+/781061 | 09:17 |
| *** raukadah is now known as chandankumar | 09:26 | |
| opendevreview | Merged openstack/openstack-ansible-os_nova master: Drop CentOS 7 specific task https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/796830 | 10:05 |
| opendevreview | Merged openstack/openstack-ansible-openstack_hosts master: Don't fail when openstack_pki_install_ca not defined https://review.opendev.org/c/openstack/openstack-ansible-openstack_hosts/+/796163 | 10:11 |
| opendevreview | Merged openstack/openstack-ansible-galera_server master: Replace systemd unit overrides with role https://review.opendev.org/c/openstack/openstack-ansible-galera_server/+/796041 | 10:15 |
| opendevreview | Merged openstack/openstack-ansible-haproxy_server master: Replace whitelist with allowlist naming https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/796629 | 10:19 |
| opendevreview | Merged openstack/openstack-ansible master: [doc] Update wrapper path https://review.opendev.org/c/openstack/openstack-ansible/+/796042 | 10:30 |
| opendevreview | Arx Cruz proposed openstack/openstack-ansible-os_tempest master: Add tempest_test_extra_test variable https://review.opendev.org/c/openstack/openstack-ansible-os_tempest/+/796818 | 10:37 |
| noonedeadpunk | jrosser: andrewbonney mgariepy I think we should merge https://review.opendev.org/c/openstack/openstack-ansible/+/795851 as we can end up in broken CI | 11:30 |
| mgariepy | good morning | 12:33 |
| opendevreview | Jonathan Heathcote proposed openstack/openstack-ansible master: [doc] Fix 'installing with limited connectivity' reference https://review.opendev.org/c/openstack/openstack-ansible/+/797075 | 12:36 |
| opendevreview | Merged openstack/openstack-ansible stable/ussuri: Bump SHAs for stable/ussuri https://review.opendev.org/c/openstack/openstack-ansible/+/794999 | 12:37 |
| mgariepy | how many retry do we need.. | 12:44 |
| mgariepy | noonedeadpunk, can you review this one : https://review.opendev.org/c/openstack/openstack-ansible/+/795572 | 12:47 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-haproxy_server master: WIP Generate self-signed SSL per listen IP https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/796940 | 12:50 |
| noonedeadpunk | done | 12:55 |
| jonher | i noticed some repos are tracked on master in openstack_services.yml even on a stable branch like stable/victoria is there some logic behind why that is? i was about to backport some cloudkitty things to victoria which raises the question if i can track os_cloudkitty on master branch or if i should aim for fixes to be backported | 12:58 |
| jonher | best case scenario there would be some more activity there and branched would be maintained, but that's not much the case so figured i'd raise the question even though i think i know the preferred | 12:59 |
| noonedeadpunk | there should be close to no services that follow master in openstack_services.yml on stable branches unless they're branchless (ie do not branch with openstack releases) | 13:00 |
| noonedeadpunk | and cloudkitty is for sure not one of these | 13:01 |
| noonedeadpunk | but eventually, you can easily override cloudkitty version to any preffered one from any branch actually | 13:01 |
| noonedeadpunk | just by defining cloudkitty_git_install_branch in user_variables | 13:02 |
| opendevreview | Jonathan Herlin proposed openstack/openstack-ansible stable/victoria: Integrate cloudkitty https://review.opendev.org/c/openstack/openstack-ansible/+/797030 | 13:08 |
| opendevreview | Jonathan Herlin proposed openstack/openstack-ansible stable/victoria: Integrate cloudkitty https://review.opendev.org/c/openstack/openstack-ansible/+/797030 | 13:09 |
| opendevreview | Jonathan Heathcote proposed openstack/openstack-ansible-os_tempest stable/victoria: Install py3-dev when not building wheels https://review.opendev.org/c/openstack/openstack-ansible-os_tempest/+/797031 | 13:10 |
| opendevreview | Jonathan Herlin proposed openstack/openstack-ansible-os_cloudkitty stable/victoria: Cloudkitty role cleanup and config updates https://review.opendev.org/c/openstack/openstack-ansible-os_cloudkitty/+/797032 | 13:11 |
| *** raukadah is now known as chandankumar | 13:13 | |
| opendevreview | Jonathan Herlin proposed openstack/openstack-ansible-os_cloudkitty stable/victoria: Cloudkitty role cleanup and config updates https://review.opendev.org/c/openstack/openstack-ansible-os_cloudkitty/+/797032 | 13:15 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-haproxy_server master: WIP Generate self-signed SSL per listen IP https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/796940 | 13:33 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-haproxy_server master: WIP Generate self-signed SSL per listen IP https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/796940 | 14:07 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-haproxy_server master: WIP Generate self-signed SSL per listen IP https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/796940 | 14:09 |
| *** rpittau is now known as rpittau|afk | 14:09 | |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-haproxy_server master: WIP Generate self-signed SSL per listen IP https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/796940 | 14:11 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-haproxy_server master: WIP Generate self-signed SSL per listen IP https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/796940 | 14:20 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Don't set keystone URI as unsecure https://review.opendev.org/c/openstack/openstack-ansible/+/796809 | 14:23 |
| noonedeadpunk | jrosser: no hurry, but do you know why in the world python inside venv doesn't consume installed root ca while default one does? http://paste.openstack.org/show/806768/ | 14:31 |
| noonedeadpunk | doh, found it :( | 14:32 |
| noonedeadpunk | damn certifi | 14:34 |
| noonedeadpunk | `Certifi does not support any addition/removal or other modification of the CA trust store content` | 14:35 |
| jrosser | hah yes | 14:38 |
| noonedeadpunk | so question is - why we did all of that... | 14:38 |
| jrosser | REQUESTS_CA_BUNDLE=<system_ca_store> | 14:39 |
| jrosser | so that needs to be either in /etc/environment or equivalent environment setting in service units | 14:39 |
| noonedeadpunk | so we need to adjust /etc/environment? | 14:40 |
| jrosser | i would try the first one as it's easy | 14:40 |
| jrosser | openstack_hosts already knows the path for this https://github.com/openstack/openstack-ansible-openstack_hosts/blob/master/vars/debian.yml#L97 | 14:40 |
| noonedeadpunk | but we don't use it anywhere?:) https://codesearch.opendev.org/?q=openstack_host_ca_location&i=nope&files=&excludeFiles=&repos= | 14:41 |
| jnamdar | hi | 14:42 |
| noonedeadpunk | moreover REQUESTS_CA_BUNDLE env var doesn't seem to work ( | 14:42 |
| noonedeadpunk | ah | 14:43 |
| jnamdar | I'm trying to launch a magnum cluster with the default template but I'm getting a really weird error in cinder | 14:43 |
| jrosser | oh right - looks like i used the pki role now in openstack_hosts? | 14:43 |
| noonedeadpunk | it works | 14:43 |
| jrosser | cool | 14:43 |
| noonedeadpunk | yeah, now pki role is used. So I think we can clean that up? | 14:44 |
| jrosser | well, maybe we want global setting of REQUESTS_CA_BUNDLE | 14:44 |
| noonedeadpunk | and maybe env setup should be performed by pki role as well during ca installation& | 14:44 |
| jrosser | yeah so thats kind of the question really, if we want the pki role to do any host level setup | 14:44 |
| noonedeadpunk | but other then that have no idea about available root CAs? | 14:45 |
| jnamdar | basically heat is deploying fine until it has to create cinder volumes, then it fails and I find this https://pastebin.com/1FQ7zVRu | 14:45 |
| jrosser | oh yes well i just wonder if its a pki or openstack_hosts kind of problem really | 14:45 |
| jrosser | becasue setting REQUESTS_CA_BUNDLE is perhaps good anyway even without the pki role | 14:46 |
| noonedeadpunk | but I mean openstack_hosts does call pki role to install CA? | 14:46 |
| noonedeadpunk | also I wonder how to provide several values to REQUESTS_CA_BUNDLE... | 14:47 |
| jrosser | oh i don't think thats possible | 14:47 |
| mgariepy | don't you need to run update-ca-cert after installing a certificate ? | 14:47 |
| noonedeadpunk | REQUESTS_CA_BUNDLE should be path to specific file, not folder like `/usr/local/share/ca-certificates/` | 14:47 |
| mgariepy | so it's accepted system-wide? | 14:47 |
| jrosser | yes, but that file contains all the certs | 14:48 |
| noonedeadpunk | mgariepy: the problem that it's accepted sytem-wide except venvs.... | 14:48 |
| mgariepy | ho | 14:48 |
| jrosser | errr - well python though isnt it | 14:48 |
| jrosser | issue is that python uses requests, requests uses certifi, certifi is not the system CA store | 14:49 |
| jrosser | so you have to just "fix" requests to use the proper CA store | 14:49 |
| noonedeadpunk | if you look into http://paste.openstack.org/show/806768/ - system python is fine with cert, because certifi got updated there, but not actually venv one | 14:49 |
| noonedeadpunk | I actually wonder if we can symlink certifi storage from system... | 14:49 |
| mgariepy | isn't it caused by the ip stuff ? | 14:50 |
| mgariepy | i'm pretty sure i used systemwide certs in the past with keystone for ldap. | 14:50 |
| noonedeadpunk | oh, well. we can set `REQUESTS_CA_BUNDLE` to `/etc/ssl/certs/ca-certificates.crt` | 14:50 |
| noonedeadpunk | or indeed replace /openstack/venvs/utility-23.0.0.0b2.dev5/lib/python3.8/site-packages/certifi/cacert.pem with /etc/ssl/certs/ca-certificates.crt symlink | 14:51 |
| jrosser | i think this is what i try to say :) | 14:52 |
| jrosser | make the requests env var point the the CA store managed by the ca-certificates package, not the one from python3-certifi (apt) or certifi (pip) | 14:53 |
| jrosser | then when we add a custom one and run update-ca-certificates (or whatever the command is), it should be good | 14:53 |
| jrosser | we're doing that install/update already here https://opendev.org/openstack/ansible-role-pki/src/branch/master/tasks/standalone/install_ca.yml#L25-L28 | 14:55 |
| noonedeadpunk | yeah, I think the question if we want to set it via env var, or nasty symlinking.... | 14:55 |
| jrosser | yeah indeed | 14:55 |
| jrosser | if it works in our services then /etc/environment is kind of neat as its hard to break | 14:56 |
| jrosser | tbh i'd not looked at the internal SSL stuff at all, knew this needed doing | 14:57 |
| jrosser | but good to fix it all | 14:57 |
| noonedeadpunk | Well, I'm not sure everything uses requests really... In terms that certifi might be leveraged not only with requests | 14:57 |
| noonedeadpunk | (no idea here) | 14:57 |
| noonedeadpunk | urllib3 does not require certifi | 14:59 |
| noonedeadpunk | (and works just out of the box | 14:59 |
| jrosser | i have an internal environment which sets that env var on all the hosts, but it's not https on the internal endpoint | 15:00 |
| jrosser | so it was good enough for accessing all the mirrors etc during deployment | 15:00 |
| opendevreview | Merged openstack/openstack-ansible master: Rename black/white list variables https://review.opendev.org/c/openstack/openstack-ansible/+/796636 | 15:21 |
| opendevreview | Merged openstack/openstack-ansible master: Unfreeze roles after RC1 https://review.opendev.org/c/openstack/openstack-ansible/+/795851 | 15:23 |
| opendevreview | Merged openstack/openstack-ansible master: add openvswitch to the log collect https://review.opendev.org/c/openstack/openstack-ansible/+/795572 | 15:23 |
| noonedeadpunk | jnamdar: sorry never tried to run things on lvm... but feels like issue with image format that has been created and used in lcuster template | 15:24 |
| noonedeadpunk | I mean glance image | 15:25 |
| noonedeadpunk | which I assume is `47548ecd-7bca-4713-953b-16c18ff5eccb` | 15:25 |
| jnamdar | noonedeadpunk: yes that's the defautl fedora image | 15:27 |
| noonedeadpunk | well, default fedora image can be shipped in different formats | 15:31 |
| noonedeadpunk | and might be compressed or not | 15:31 |
| noonedeadpunk | So I'd try creating any random volume from it manually first to verify that image is fine | 15:31 |
| noonedeadpunk | btw, it must be fedora-coreos image, not regular fedora | 15:32 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-openstack_hosts master: Set REQUESTS_CA_BUNDLE env var https://review.opendev.org/c/openstack/openstack-ansible-openstack_hosts/+/797129 | 15:33 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Don't set keystone URI as unsecure https://review.opendev.org/c/openstack/openstack-ansible/+/796809 | 15:34 |
| jnamdar | noonedeadpunk: yep, I'll try that. i'll try with a more recent fedora core os image as well | 15:35 |
| noonedeadpunk | well, iirc for V later then 31 doesn't work properly | 15:36 |
| jnamdar | what's weird is that heat successfully provisioned a volume in cinder (for the master node I think) but failed with another | 15:36 |
| noonedeadpunk | but not 100% sure | 15:36 |
| jnamdar | and I specified the same images for both masters and workers | 15:36 |
| noonedeadpunk | huh | 15:36 |
| noonedeadpunk | weird indeed | 15:36 |
| jnamdar | oh I just uploaded 34 :'( | 15:36 |
| noonedeadpunk | but it's failing somewhere on later networking step | 15:37 |
| noonedeadpunk | when starting up cluster | 15:37 |
| jnamdar | kk I'll get a <31 | 15:39 |
| noonedeadpunk | ==31) | 15:40 |
| jnamdar | ah | 15:41 |
| jnamdar | i'm running ussuri btw | 15:42 |
| jnamdar | if that changes anyhthing | 15:42 |
| noonedeadpunk | um... might be... | 15:42 |
| noonedeadpunk | I can't actually recall exactly, but I think for ussuri you need just old coreos | 15:42 |
| jnamdar | also I can't seem to figure out how to download older releases of fedora coreos? https://getfedora.org/en/coreos/download?tab=metal_virtualized&stream=stable | 15:42 |
| noonedeadpunk | but might be magnum backported fedora-coreos support - no idea | 15:42 |
| noonedeadpunk | https://builds.coreos.fedoraproject.org/prod/streams/stable/builds/31.20200210.3.0/x86_64/fedora-coreos-31.20200210.3.0-openstack.x86_64.qcow2.xz <- should be valid link | 15:44 |
| jnamdar | how did you find that link? :O | 15:44 |
| jnamdar | btw I reproduced the error: I cannot create a cinder volume from my fedora coresos image | 15:44 |
| noonedeadpunk | took from CI) https://opendev.org/openstack/openstack-ansible/src/branch/stable/ussuri/tests/roles/bootstrap-host/templates/user_variables_magnum.yml.j2#L21 | 15:45 |
| jnamdar | thx | 15:45 |
| noonedeadpunk | jnamdar: I think the isuse might be that you uploaded pached image (in .xz) | 15:45 |
| noonedeadpunk | *packed | 15:46 |
| jnamdar | I guess so too | 15:46 |
| jnamdar | but I didn't do it manually | 15:46 |
| jnamdar | it was already there after the role finished | 15:46 |
| noonedeadpunk | ah | 15:46 |
| jnamdar | so I guessed OSA put it there :D | 15:46 |
| noonedeadpunk | well, role should do things properly I guess... but dunno... | 15:46 |
| noonedeadpunk | at least it works in CI as is... | 15:47 |
| jnamdar | same error with your image :/ | 15:49 |
| jnamdar | I uploaded it like this openstack image create fedora-coreos-31 --disk-format qcow2 --container-format bare --file fedora-coreos-31.20200210.3.0-openstack.x86_64.qcow2.xz | 15:49 |
| noonedeadpunk | I think you need to unpack image first | 15:50 |
| jnamdar | ah | 15:50 |
| jnamdar | my bad | 15:50 |
| noonedeadpunk | but actually I think glance should be capble of doing that... | 15:50 |
| opendevreview | Merged openstack/openstack-ansible-os_neutron master: Change task ordering to perform smooth upgrades https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/788501 | 15:51 |
| jnamdar | gotta use unxz right? | 15:51 |
| noonedeadpunk | I used tar iirc | 15:51 |
| noonedeadpunk | but whatever works I guess | 15:51 |
| jnamdar | mmh with the uncompressed version I get `ImageUnacceptable: Image a2cc0f37-b0b3-4e4f-94b1-f221614b9e61 is unacceptable: Image virtual size is 8GB and doesn't fit in a volume of size 2GB.` | 15:56 |
| jnamdar | I did provide 2GB in size when creating the volume | 15:56 |
| jnamdar | might explain why it failed even with the compressed version? not sure, gonna try | 15:57 |
| jnamdar | to up the volume size to 10GB | 15:57 |
| jnamdar | meh I changed the image to the uncompressed one, and upped the docker_volume_size to 10GB instead of 2 | 16:19 |
| jnamdar | the first volume heat creates is ok, as it was before, this time with 10GB | 16:20 |
| jnamdar | but the 2nd is still created with 2GB.. so I get the errorImage virtual size is 8GB and doesn't fit in a volume of size 2GB | 16:21 |
| jrosser | jnamdar: have you read the documentation for the os_magnum ansible role? https://github.com/openstack/openstack-ansible-os_magnum/blob/master/doc/source/index.rst | 16:36 |
| jrosser | there is an example cluster template there | 16:36 |
| jrosser | also the giant warning `Note that openstack-ansible deploys the Magnum API service. It is not in scope for openstack-ansible to maintain a guaranteed working cluster template as this will vary depending on the precise version of Magnum deployed and the required version of k8s and it's dependancies.` | 16:37 |
| opendevreview | Arx Cruz proposed openstack/openstack-ansible-os_tempest master: Add tempest_test_extra_test variable https://review.opendev.org/c/openstack/openstack-ansible-os_tempest/+/796818 | 16:49 |
| jnamdar | jrosser: thanks, I'll try to make the default one work for now, I'm getting further with the uncompressed fedora coreos image | 16:54 |
| opendevreview | Merged openstack/openstack-ansible-os_neutron master: Add ovn clustering support https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/794266 | 17:01 |
| mgariepy | OMG it merged ! | 17:42 |
| mgariepy | after many many rechecks. | 17:43 |
| opendevreview | Merged openstack/openstack-ansible master: Use openstack_repo_url for requirements_git_url https://review.opendev.org/c/openstack/openstack-ansible/+/796820 | 18:45 |
| opendevreview | Merged openstack/openstack-ansible master: [doc] Fix 'installing with limited connectivity' reference https://review.opendev.org/c/openstack/openstack-ansible/+/797075 | 18:45 |
| opendevreview | Merged openstack/ansible-role-python_venv_build stable/ussuri: Do not drop all wheels with venv_rebuild https://review.opendev.org/c/openstack/ansible-role-python_venv_build/+/773971 | 19:03 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!