| *** altlogbot_1 has quit IRC | 00:50 | |
| *** altlogbot_1 has joined #openstack-ansible | 00:53 | |
| *** altlogbot_1 has quit IRC | 00:54 | |
| *** altlogbot_2 has joined #openstack-ansible | 00:55 | |
| openstackgerrit | Logan V proposed openstack/openstack-ansible master: Update ARA installation and loading https://review.opendev.org/658708 | 00:55 |
|---|---|---|
| openstackgerrit | Logan V proposed openstack/openstack-ansible master: Set nodepool mirror PIP_OPTS consistently https://review.opendev.org/658709 | 00:55 |
| openstackgerrit | Logan V proposed openstack/openstack-ansible master: Move data device detection to Ansible https://review.opendev.org/658710 | 00:55 |
| openstackgerrit | Logan V proposed openstack/openstack-ansible-os_nova master: Use integrated repo for nova-lxd job https://review.opendev.org/658523 | 00:59 |
| openstackgerrit | Logan V proposed openstack/openstack-ansible master: Move data device detection to Ansible https://review.opendev.org/658710 | 01:16 |
| openstackgerrit | Logan V proposed openstack/openstack-ansible master: Move data device detection to Ansible https://review.opendev.org/658710 | 01:44 |
| openstackgerrit | Logan V proposed openstack/openstack-ansible master: Move data device detection to Ansible https://review.opendev.org/658710 | 02:02 |
| openstackgerrit | Logan V proposed openstack/openstack-ansible-os_nova master: Use integrated repo for nova-lxd job https://review.opendev.org/658523 | 02:14 |
| openstackgerrit | Logan V proposed openstack/openstack-ansible master: Update ARA installation and loading https://review.opendev.org/658708 | 02:22 |
| openstackgerrit | Logan V proposed openstack/openstack-ansible master: Move data device detection to Ansible https://review.opendev.org/658710 | 02:32 |
| openstackgerrit | Logan V proposed openstack/openstack-ansible master: Update ARA installation and loading https://review.opendev.org/658708 | 02:32 |
| openstackgerrit | Logan V proposed openstack/openstack-ansible master: Move data device detection to Ansible https://review.opendev.org/658710 | 02:42 |
| *** BjoernT has joined #openstack-ansible | 02:49 | |
| *** BjoernT has quit IRC | 02:53 | |
| *** BjoernT has joined #openstack-ansible | 02:54 | |
| openstackgerrit | tonybrad proposed openstack/openstack-ansible-repo_server master: Updated the git source https://review.opendev.org/657350 | 03:14 |
| *** shyamb has joined #openstack-ansible | 03:37 | |
| openstackgerrit | Merged openstack/openstack-ansible master: Move entirely to dynamic job execution https://review.opendev.org/658532 | 03:40 |
| openstackgerrit | Merged openstack/openstack-ansible master: Clean up bootstrap-aio scenario configuration https://review.opendev.org/658555 | 03:42 |
| *** Smeared_Beard has joined #openstack-ansible | 03:47 | |
| *** SmearedB_ has joined #openstack-ansible | 03:50 | |
| *** dxiri has joined #openstack-ansible | 03:50 | |
| *** Smeared_Beard has quit IRC | 03:53 | |
| *** udesale has joined #openstack-ansible | 03:56 | |
| *** shyamb has quit IRC | 04:22 | |
| *** BjoernT has quit IRC | 04:25 | |
| *** dxiri has quit IRC | 04:31 | |
| *** shyamb has joined #openstack-ansible | 04:31 | |
| *** aedc has quit IRC | 04:40 | |
| *** ivve has quit IRC | 05:07 | |
| *** shyamb has quit IRC | 05:14 | |
| *** shyamb has joined #openstack-ansible | 05:14 | |
| *** shyamb has quit IRC | 05:15 | |
| *** shyam89 has joined #openstack-ansible | 05:15 | |
| *** SmearedB_ has quit IRC | 05:26 | |
| *** shyam89 has quit IRC | 05:27 | |
| openstackgerrit | Merged openstack/openstack-ansible-lxc_container_create master: Do not use 'null' when trying to locate vars files https://review.opendev.org/657084 | 05:41 |
| *** udesale has quit IRC | 05:45 | |
| *** udesale has joined #openstack-ansible | 05:45 | |
| *** udesale has quit IRC | 05:49 | |
| *** starborn has joined #openstack-ansible | 05:49 | |
| *** udesale has joined #openstack-ansible | 05:49 | |
| *** radeks has joined #openstack-ansible | 05:49 | |
| *** kopecmartin|off is now known as kopecmartin | 05:55 | |
| *** shyamb has joined #openstack-ansible | 05:58 | |
| *** pcaruana has joined #openstack-ansible | 06:16 | |
| *** ivve has joined #openstack-ansible | 06:17 | |
| *** cshen has joined #openstack-ansible | 06:32 | |
| *** shyam89 has joined #openstack-ansible | 06:34 | |
| *** DanyC has joined #openstack-ansible | 06:35 | |
| *** shyamb has quit IRC | 06:38 | |
| *** DanyC has quit IRC | 06:39 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible master: Imported Translations from Zanata https://review.opendev.org/658731 | 06:40 |
| *** udesale has quit IRC | 06:47 | |
| *** udesale has joined #openstack-ansible | 06:47 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible stable/stein: Fix Fwaas Dashboard URL https://review.opendev.org/658736 | 07:00 |
| *** shyam89 has quit IRC | 07:01 | |
| fnpanic | morning | 07:08 |
| evrardjp | morning | 07:12 |
| *** shyamb has joined #openstack-ansible | 07:14 | |
| jrosser | morning | 07:21 |
| evrardjp | jrosser: if you're not aware, there are a few things that merged this week-end to change the bootstrap | 07:22 |
| evrardjp | just an FYI | 07:23 |
| jrosser | looks quite different in places | 07:28 |
| *** jbadiapa has joined #openstack-ansible | 07:37 | |
| *** shyamb has quit IRC | 07:40 | |
| *** shardy has joined #openstack-ansible | 07:57 | |
| _janno | Is there a way to split the management of the services from br-mgmt? | 07:57 |
| _janno | Or alternatively the communication of the services. | 07:58 |
| *** shyamb has joined #openstack-ansible | 07:59 | |
| jrosser | _janno: what are you trying to achieve? br-mgmt is specifically for the services to communicate | 08:11 |
| CeeMac | morning | 08:12 |
| _janno | jrosser: but OSA also needs to access the containers through it, or am I wrong? | 08:12 |
| jrosser | _janno: no, ansible accesses the containers via the host, using a connection plugin | 08:13 |
| jrosser | you can have a dedicated interface for host provisioning and ansible'ing if you like | 08:13 |
| _janno | so the ansible host needs no access to br-mgmt? | 08:14 |
| jrosser | if set up correctly you can do it like that if needed | 08:14 |
| _janno | alright, thank you | 08:15 |
| jrosser | _janno: there are some ansible tasks which need to communicate with openstack services, and the database and so on, there is this variable https://github.com/openstack/openstack-ansible/blob/master/etc/openstack_deploy/user_variables.yml#L33 which lets you delegate those tasks from your deployment node to somewhere which has the right connectivity | 08:17 |
| *** DanyC has joined #openstack-ansible | 08:18 | |
| openstackgerrit | Merged openstack/openstack-ansible master: Collect openstack_deploy in gate log collection script https://review.opendev.org/658554 | 08:18 |
| *** DanyC has joined #openstack-ansible | 08:18 | |
| *** DanyC has quit IRC | 08:25 | |
| *** mkuf_ has joined #openstack-ansible | 08:27 | |
| *** DanyC has joined #openstack-ansible | 08:28 | |
| *** mkuf has quit IRC | 08:30 | |
| *** slin has joined #openstack-ansible | 08:31 | |
| openstackgerrit | Merged openstack/openstack-ansible master: Store gate scenario configuration on Zuul executor https://review.opendev.org/658635 | 08:33 |
| *** aedc has joined #openstack-ansible | 08:34 | |
| *** vnogin has joined #openstack-ansible | 08:53 | |
| *** mkuf_ has quit IRC | 08:56 | |
| noonedeadpunk | mornings | 09:03 |
| *** jawad_axd has joined #openstack-ansible | 09:04 | |
| *** mkuf_ has joined #openstack-ansible | 09:05 | |
| _janno | jrosser: thanks | 09:13 |
| jawad_axd | Hi all, Is there any other way to change hostname of one operational openstack compute node without performing migration? E.g. If I just rename hostname in compute stanza in openstack_user_config.yaml , on host, and run playbooks again.Will it rename hypervisor name successfully without interrupting vms running on that host. Ofcourse, I will clean up the previous entry for compute service .Any suggestions plz? | 09:16 |
| *** electrofelix has joined #openstack-ansible | 09:17 | |
| noonedeadpunk | Once you change hostname for computing node, it will be added as the new hypervisor. I don't think this will cause any influence for vm's on that host | 09:20 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-lxc_hosts stable/rocky: Fix image fetching process for leap 15 https://review.opendev.org/648854 | 09:23 |
| jawad_axd | @noonedeadpunk Thanks | 09:24 |
| *** jawad_axd has quit IRC | 09:24 | |
| *** miloa has joined #openstack-ansible | 09:34 | |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-lxc_hosts stable/stein: Use the 'real' name for the LXC image https://review.opendev.org/658774 | 09:34 |
| evrardjp | jrosser: noonedeadpunk could you have a look at https://review.opendev.org/#/c/658736/ please ? | 09:42 |
| *** DanyC has quit IRC | 09:46 | |
| *** DanyC has joined #openstack-ansible | 09:46 | |
| *** shyamb has quit IRC | 09:56 | |
| *** shyamb has joined #openstack-ansible | 09:57 | |
| *** hamzaachi has joined #openstack-ansible | 10:20 | |
| *** aedc has quit IRC | 10:25 | |
| CeeMac | jrosser, morning. I saw in the chat earlier you were talking about whether/when the deployment node might need to talk to containers and you referenced https://github.com/openstack/openstack-ansible/blob/master/etc/openstack_deploy/user_variables.yml#L33 | 10:34 |
| CeeMac | if I read that right, the utility container is being delegated to perform these tasks? Does this include when you want to run some of the maintenance tasks like checking rabbitmq queue and galera cluster status? | 10:35 |
| CeeMac | or would the deployment node use the plugin to pass these through? | 10:35 |
| jrosser | if you mean the ad-hoc maintainance commands that are described in the ops guide then the connection plugin will execute those against the db or rabbit containers as needed, going via the host | 10:36 |
| *** cshen has quit IRC | 10:37 | |
| *** vnogin has quit IRC | 10:40 | |
| *** stuartgr has left #openstack-ansible | 10:42 | |
| jrosser | so has anyone done an OS upgrade from xenial to bionic, re-installing the hosts one by one? | 10:45 |
| jrosser | re-joining the first re-installed rabbitmq to the original infra2/3 nodes seems to fail | 10:46 |
| *** cshen has joined #openstack-ansible | 10:53 | |
| noonedeadpunk | o/ | 10:56 |
| *** strattao has joined #openstack-ansible | 10:56 | |
| noonedeadpunk | oh, no, I was just updating hosts with do-release-upgrade | 10:57 |
| *** cshen has quit IRC | 10:58 | |
| *** vnogin has joined #openstack-ansible | 10:59 | |
| *** tuxjohnson has joined #openstack-ansible | 10:59 | |
| noonedeadpunk | but, I had the failing rabbitmq cluster after re-joining of one of the participants after stop/start container. running rabbitmq-install.yml with rabbitmq-upgrade=True helped | 11:00 |
| *** nicolasbock has joined #openstack-ansible | 11:04 | |
| *** udesale has quit IRC | 11:17 | |
| *** nicolasbock has quit IRC | 11:20 | |
| *** nicolasbock has joined #openstack-ansible | 11:20 | |
| jrosser | noonedeadpunk: yes it seems leaving off the rabbitmq-upgrade=True leaves you with a cluster in an inconsistent state that then requires fixing by hand to recover | 11:22 |
| *** cshen has joined #openstack-ansible | 11:27 | |
| *** cshen has quit IRC | 11:32 | |
| *** cshen has joined #openstack-ansible | 11:36 | |
| *** ansmith has quit IRC | 11:45 | |
| CeeMac | thanks jrosser, i did mean the ad-hoc maintenance commands yes. | 11:49 |
| CeeMac | was i reading the user-variable correctly, the line referenced means the utility container is delegated for running ansible scripts that need to talk direct to containers? | 11:50 |
| jrosser | CeeMac: there are certain tasks that require access to the db/internal API endpoint/.... and so on, and the roles assume that this is "localhost", i.e the deploy host like this https://github.com/openstack/openstack-ansible-os_neutron/blob/master/defaults/main.yml#L30 | 11:51 |
| jrosser | however, it may be that the deploy host only has ssh access to the openstack environment, so you may need to nominate a remote host like the utility container for running those setup tasks | 11:52 |
| jrosser | but none of that really stops you using the deploy host directly for the ad-hoc maintainance things | 11:53 |
| *** shyamb has quit IRC | 11:53 | |
| *** aedc has joined #openstack-ansible | 11:57 | |
| jrosser | CeeMac: and no, it's not about the utility container being delegated to for ansible run things against containers, the connection plugin always ssh's to the physical host, then connects to the required container from the parent host. | 11:57 |
| CeeMac | jrosser, so for the tasks that require access to the db/internal api, the deploy host uses the connection plugin on the host running the utility container to run the delegated tasks? | 12:02 |
| CeeMac | in that example? | 12:02 |
| CeeMac | just trying to clarify as I'll be kicking off a new prod install where the deploy host only has SSH to the metal servers is all | 12:03 |
| CeeMac | out of band that is, as opposed to on br-mgmt network | 12:03 |
| noonedeadpunk | jrosser: Are you on stein already? | 12:04 |
| jrosser | in the example that uses the utility container as the service setup host, the connection plugin will ssh to the phyiscal host of the utility container, and use lxc-attach to the utility container to then execute the ansible task, see here https://github.com/openstack/openstack-ansible-plugins/blob/master/connection/ssh.py#L416 | 12:04 |
| jrosser | CeeMac: my depoy is like that with a seperate metal network that is not br-mgmt | 12:05 |
| CeeMac | thanks jrosser, i remember you saying you were set up that way, hadn't come across this particular scenario in previous chats though :) | 12:06 |
| noonedeadpunk | I mean, whether https://review.opendev.org/#/c/653410/ is already present and HA queues doesn't help with that | 12:08 |
| jrosser | noonedeadpunk: this is all rocky - i need to get to bionic before i can upgrade | 12:09 |
| noonedeadpunk | oh, yeah, really... | 12:09 |
| noonedeadpunk | s/really/indeed/ | 12:10 |
| *** shyamb has joined #openstack-ansible | 12:11 | |
| *** kplant has joined #openstack-ansible | 12:16 | |
| *** vnogin has quit IRC | 12:20 | |
| *** aedc has quit IRC | 12:20 | |
| *** aedc has joined #openstack-ansible | 12:27 | |
| *** mgariepy has quit IRC | 12:27 | |
| *** slin has quit IRC | 12:30 | |
| *** mgariepy has joined #openstack-ansible | 12:31 | |
| guilhermesp | evrardjp: huum seems that ppl are interested in searchlight and monasca roles. Just saw the ML | 12:33 |
| guilhermesp | if so I think we'd need to revert some of these merged prs https://review.opendev.org/#/q/topic:retire-osa-unused-roles+(status:open+OR+status:merged) | 12:33 |
| openstackgerrit | Marc Gariépy (mgariepy) proposed openstack/openstack-ansible master: Add /v2 to mistral api endpoint. https://review.opendev.org/656664 | 12:34 |
| evrardjp | guilhermesp: we can leave them there | 12:37 |
| *** kplant has quit IRC | 12:37 | |
| evrardjp | if ppl want to revive the things, they can | 12:37 |
| evrardjp | just revert said patches | 12:37 |
| evrardjp | it's up to them to do it :) | 12:37 |
| *** ansmith has joined #openstack-ansible | 12:38 | |
| *** vnogin has joined #openstack-ansible | 12:38 | |
| *** kplant has joined #openstack-ansible | 12:41 | |
| guilhermesp | i see but, some of the other roles are included in this topic, like we did a batch of roles to retire in single patches so, I'd just remove the mentioned roles from the scope just to finish the procedure of retirement for those roles ppl haven't showed interest until now | 12:45 |
| *** vnogin has quit IRC | 12:51 | |
| *** pcaruana|afk| has joined #openstack-ansible | 12:55 | |
| *** pcaruana has quit IRC | 12:57 | |
| *** dirk has quit IRC | 12:58 | |
| *** gshippey has quit IRC | 12:58 | |
| *** serverascode has quit IRC | 13:00 | |
| *** djhankb has quit IRC | 13:00 | |
| *** persia has quit IRC | 13:00 | |
| *** persia has joined #openstack-ansible | 13:00 | |
| *** dirk has joined #openstack-ansible | 13:00 | |
| *** gshippey has joined #openstack-ansible | 13:01 | |
| *** serverascode has joined #openstack-ansible | 13:02 | |
| *** BjoernT has joined #openstack-ansible | 13:03 | |
| *** vnogin has joined #openstack-ansible | 13:05 | |
| evrardjp | sounds good too | 13:08 |
| evrardjp | as long as the message is clear | 13:08 |
| *** udesale has joined #openstack-ansible | 13:10 | |
| *** shyamb has quit IRC | 13:14 | |
| *** udesale has quit IRC | 13:18 | |
| *** djhankb has joined #openstack-ansible | 13:18 | |
| *** udesale has joined #openstack-ansible | 13:18 | |
| evrardjp | mgariepy: are you mistral-ing nowadays? | 13:23 |
| mgariepy | i will | 13:25 |
| mgariepy | evrardjp, testing it to expire vms and create project and so on. | 13:25 |
| mgariepy | i stumbled upon cerns repo of mistral workbooks and i kinda need the functionality for my cloud. | 13:26 |
| mgariepy | since we do research, and don't charge users, they tend to forget what they use. (it's not happening if you charge them i guess, or at least you don't care as much) :) | 13:27 |
| *** starborn has quit IRC | 13:29 | |
| *** aedc has quit IRC | 13:29 | |
| noonedeadpunk | that's absolutely true - no need to care much about resources you've been paid for :p | 13:34 |
| mgariepy | haha :) | 13:34 |
| *** flaper87 has quit IRC | 13:45 | |
| *** priteau has joined #openstack-ansible | 13:53 | |
| *** BjoernT has quit IRC | 13:53 | |
| fnpanic | mgariepy: we had something like the cern stuff but with a simple cron and python in our synnefo cloud. now we are moving to openstack we use the money whip | 13:53 |
| fnpanic | works great :-) | 13:53 |
| evrardjp | synnefo | 13:54 |
| evrardjp | oh my god I didn't hear that for a while | 13:54 |
| *** dxiri has joined #openstack-ansible | 13:54 | |
| evrardjp | are you working for grnet fnpanic? | 13:54 |
| fnpanic | no | 13:54 |
| fnpanic | we are the second larget synnefo user :-) | 13:54 |
| evrardjp | haha | 13:55 |
| fnpanic | were using ganeti for ages | 13:55 |
| fnpanic | ;-) | 13:55 |
| evrardjp | makes sense then | 13:55 |
| *** flaper87 has joined #openstack-ansible | 13:55 | |
| fnpanic | but the archipelago layer has a few nasty bugs | 13:55 |
| *** flaper87 has quit IRC | 13:55 | |
| fnpanic | and no garbage collection | 13:56 |
| evrardjp | fnpanic: you can't talk about that outside thursdays | 13:56 |
| *** flaper87 has joined #openstack-ansible | 13:56 | |
| evrardjp | Throwback thursdays* | 13:57 |
| evrardjp | this is 2013 for me :) | 13:57 |
| evrardjp | iirc | 13:57 |
| openstackgerrit | Logan V proposed openstack/openstack-ansible master: Update ARA installation and loading https://review.opendev.org/658708 | 13:58 |
| *** BjoernT has joined #openstack-ansible | 13:59 | |
| logan- | o/ | 14:01 |
| logan- | a few more reviews appreciated on these topics if anyone has time this morning: https://review.opendev.org/#/q/status:open+owner:%22Logan+V+%253Clogan2211%2540gmail.com%253E%22+AND+(topic:dynamic-aio-scenario+OR+topic:tests-cleanup+OR+topic:use-integrated-repo-for-all-tests) | 14:02 |
| *** BjoernT has quit IRC | 14:02 | |
| *** BjoernT has joined #openstack-ansible | 14:03 | |
| *** miloa has quit IRC | 14:33 | |
| *** CeeMac_ has joined #openstack-ansible | 14:34 | |
| *** CeeMac has quit IRC | 14:37 | |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-lxc_hosts master: Use block/rescue for image pre-stage https://review.opendev.org/658837 | 14:45 |
| *** pcaruana|afk| has quit IRC | 14:55 | |
| *** macza has joined #openstack-ansible | 14:57 | |
| openstackgerrit | Logan V proposed openstack/openstack-ansible master: Move data device detection to Ansible https://review.opendev.org/658710 | 14:57 |
| openstackgerrit | Logan V proposed openstack/openstack-ansible master: Run the OSA bootstrap in the Zuul pre phase https://review.opendev.org/658521 | 14:57 |
| *** cshen has quit IRC | 14:58 | |
| openstackgerrit | Logan V proposed openstack/openstack-ansible master: Set nodepool mirror PIP_OPTS consistently https://review.opendev.org/658709 | 15:00 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-os_glance master: Add ability to overrides for glance-image-import https://review.opendev.org/658679 | 15:05 |
| openstackgerrit | Logan V proposed openstack/openstack-ansible master: Update ARA installation and loading https://review.opendev.org/658708 | 15:06 |
| evrardjp | when the dust is settled, I will make sure all the roles have the metal jobs, I will make sure suse 15.0 is in meta for each of those. | 15:08 |
| evrardjp | 15 not 15.0 | 15:08 |
| evrardjp | but not all | 15:08 |
| evrardjp | (like it currently is) | 15:08 |
| noonedeadpunk | evrardjp: first of all it needs to pass lxc_container_create... | 15:09 |
| *** aedc has joined #openstack-ansible | 15:09 | |
| evrardjp | for the metal it doesn't matter | 15:11 |
| evrardjp | metal jobs* | 15:11 |
| noonedeadpunk | but if we're dropping containers support, than yes | 15:11 |
| evrardjp | it's not that we are dropping it, it's that I don't want to step on your toes fixing it | 15:11 |
| evrardjp | :) | 15:11 |
| evrardjp | when lxc is fixed too, then we can simply have both metal and lxc aio jobs | 15:11 |
| noonedeadpunk | I'm not really active in it) It's probably more about nicolasbock, actually | 15:12 |
| *** hamzaachi has quit IRC | 15:14 | |
| *** vnogin has quit IRC | 15:21 | |
| *** vnogin has joined #openstack-ansible | 15:28 | |
| *** yolanda_ has quit IRC | 15:34 | |
| *** aedc has quit IRC | 15:36 | |
| *** yolanda_ has joined #openstack-ansible | 15:37 | |
| *** vakuznet has joined #openstack-ansible | 15:40 | |
| *** pcaruana|afk| has joined #openstack-ansible | 15:44 | |
| *** aedc has joined #openstack-ansible | 16:04 | |
| *** BjoernT has quit IRC | 16:04 | |
| *** aedc has quit IRC | 16:11 | |
| *** aedc has joined #openstack-ansible | 16:22 | |
| *** cshen has joined #openstack-ansible | 16:33 | |
| *** cshen has quit IRC | 16:38 | |
| *** priteau has quit IRC | 16:38 | |
| *** ivve has quit IRC | 16:48 | |
| *** cshen has joined #openstack-ansible | 16:49 | |
| *** mgoddard has quit IRC | 16:50 | |
| *** nicolasbock has quit IRC | 16:50 | |
| *** nicolasbock has joined #openstack-ansible | 16:50 | |
| *** vakuznet has quit IRC | 16:51 | |
| *** mgoddard has joined #openstack-ansible | 16:52 | |
| openstackgerrit | Merged openstack/openstack-ansible master: Allow skipping bootstrap steps in gate check https://review.opendev.org/658520 | 16:53 |
| *** cshen has quit IRC | 16:54 | |
| *** mgoddard has quit IRC | 16:57 | |
| *** mgoddard has joined #openstack-ansible | 16:58 | |
| *** udesale has quit IRC | 16:59 | |
| jrosser | todays rolling OS update has been very difficult | 17:00 |
| jrosser | each reinstalled rabbitmq node wouldnt rejoin the cluster without first being manually dropped from the cluster on the other nodes | 17:00 |
| logan- | you're just doing xenial->bionic (so same rmq verison)? i wonder why it is failing to rejoin | 17:01 |
| jrosser | and then galera ended up in a cluster of 2 / cluster of 1, and bad luck said that haproxy had picked the cluster of 1 as the active node | 17:01 |
| jrosser | which hadn't synced anything across from the previously reinstalled nodes | 17:01 |
| logan- | ouch | 17:01 |
| jrosser | man what a nightmare :/ | 17:01 |
| jrosser | logan-: yes this is xenial->bionic on rocky | 17:02 |
| logan- | i was thinking of trying that on the limestone ci cloud next weekend with the do-release-upgrade method | 17:02 |
| jrosser | restarting galera on the out-of-cluster node brought it all good | 17:02 |
| jrosser | i've got the journal from that showing it not joining intiailly | 17:03 |
| jrosser | but it's suspicious that haproxy check was still happy with the out-of-cluster node, but i don't know how that works | 17:04 |
| logan- | it sounds like the single-node cluster got restarted with some force bootstrap options, because it should have realized its state wouldn't allow it to start | 17:04 |
| logan- | did it have a new regenerated wsrep cluster uuid? | 17:05 |
| jrosser | i think so, the uuid was different from the other two clustered nodes | 17:06 |
| *** vnogin has quit IRC | 17:07 | |
| logan- | ive seen crazy stuff happen like that before when running the role while it is in certain states w/ some of these flags set https://github.com/openstack/openstack-ansible-galera_server/blob/3ebf85c9a7dd78e86a3d523d4b866b08a0452bf9/defaults/main.yml#L22-L24 | 17:08 |
| logan- | probably this one https://github.com/openstack/openstack-ansible-galera_server/blob/master/tasks/galera_cluster_state.yml#L16-L31 | 17:09 |
| logan- | the node should have reported that it was not on the primary cluster (ie quorum) and failed the role there | 17:09 |
| jrosser | hmm | 17:09 |
| logan- | we didn't switch to mariabackup in rocky right? | 17:10 |
| *** aedc has quit IRC | 17:10 | |
| logan- | i think it is stein only | 17:10 |
| logan- | but the rocky->stein galera upgrade is going to be tricky I think. one of my non-osa galera clusters that I track galera_server master with required a lot of manual work to get upgraded from xtrabackup to mariabackup SST when we changed that | 17:11 |
| *** stuartgr has joined #openstack-ansible | 17:12 | |
| *** mgoddard has quit IRC | 17:16 | |
| *** med_ has joined #openstack-ansible | 17:17 | |
| *** jbadiapa has quit IRC | 17:17 | |
| *** kopecmartin is now known as kopecmartin|off | 17:18 | |
| *** DanyC has quit IRC | 17:18 | |
| *** DanyC has joined #openstack-ansible | 17:18 | |
| *** DanyC_ has joined #openstack-ansible | 17:20 | |
| *** cmart has joined #openstack-ansible | 17:20 | |
| *** mgoddard has joined #openstack-ansible | 17:21 | |
| *** DanyC has quit IRC | 17:23 | |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-os_glance master: Add ability to overrides for glance-image-import https://review.opendev.org/658679 | 17:24 |
| *** DanyC_ has quit IRC | 17:24 | |
| *** jroll has quit IRC | 17:34 | |
| nicolasbock | noonedeadpunk: I was holding back a bit too to not step on your toes ;) | 17:34 |
| nicolasbock | I'll go over your changes again and check them | 17:34 |
| *** jroll has joined #openstack-ansible | 17:35 | |
| *** Weifan has joined #openstack-ansible | 17:40 | |
| jrosser | logan-: the other thing that broke in this upgrade was keystone being unable to Auth against the replacement rabbitmq cluster until we restarted all keystone api, that makes even less sense to me. The keystone logs were just spammed with mq Auth errors. | 17:43 |
| logan- | odd | 17:46 |
| *** noonedeadpunk has quit IRC | 17:49 | |
| *** noonedeadpunk has joined #openstack-ansible | 17:50 | |
| jrosser | It felt like some bad state held at the keystone end bacause rabbitmqctl authenticate_user keystone <passwd> all checked out ok | 17:50 |
| *** BjoernT has joined #openstack-ansible | 17:50 | |
| logan- | yeah rmq disconnects used to be a big problem with openstack services | 17:51 |
| logan- | its a lot better now but im not super surprised about that part :) | 17:52 |
| jrosser | I may be finding some corner cases here because I’ve upgraded the db/mq/memcached layer before any api stuff | 17:53 |
| jrosser | Normally a re-doing a whole infra node would do those in a vertical slice through all the services, rather than just the bottom layer | 17:54 |
| logan- | true | 17:54 |
| logan- | good point | 17:55 |
| *** Weifan has quit IRC | 17:56 | |
| *** Weifan has joined #openstack-ansible | 18:06 | |
| logan- | cores, https://review.opendev.org/#/q/status:open+owner:logan2211%2540gmail.com+AND+(topic:dynamic-aio-scenario+OR+topic:tests-cleanup+OR+topic:use-integrated-repo-for-all-tests)+NOT+label:Workflow%253D-1 wraps up the rest of the test revamps needed before moving forward with os_nova placement removal. all are now ready for a shove into the gate. | 18:08 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-lxc_hosts master: Use block/rescue for image pre-stage https://review.opendev.org/658837 | 18:09 |
| logan- | noonedeadpunk: same change is needed on the other file in that patch ^ | 18:09 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-lxc_hosts master: Use block/rescue for image pre-stage https://review.opendev.org/658837 | 18:09 |
| noonedeadpunk | Yeah, I saw once send patch( | 18:10 |
| logan- | also the name is removed from both of those tasks so they'll just show up blank in ARA etc. even though there is a name on the block | 18:11 |
| logan- | so could you add the name back to those tasks real quick | 18:11 |
| noonedeadpunk | I thought that the point of block usage was partially to display only one name... | 18:12 |
| logan- | nope the block is to control execution flow | 18:12 |
| noonedeadpunk | But yeah, no problem | 18:12 |
| logan- | (ie. block/rescue, apply the same condition to a group of tasks, etc) | 18:12 |
| logan- | thx | 18:12 |
| logan- | you shouldn't need to name the block though | 18:13 |
| logan- | if that makes sense | 18:13 |
| *** DanyC has joined #openstack-ansible | 18:13 | |
| logan- | doesn't hurt anything to do it anyway | 18:13 |
| *** DanyC has quit IRC | 18:14 | |
| *** DanyC has joined #openstack-ansible | 18:15 | |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-lxc_hosts master: Use block/rescue for image pre-stage https://review.opendev.org/658837 | 18:17 |
| *** ivve has joined #openstack-ansible | 18:19 | |
| *** DanyC has quit IRC | 18:27 | |
| *** DanyC has joined #openstack-ansible | 18:46 | |
| *** DanyC has quit IRC | 18:48 | |
| *** cmart has quit IRC | 18:59 | |
| *** sreejithp has joined #openstack-ansible | 19:03 | |
| *** cmart has joined #openstack-ansible | 19:04 | |
| *** d34dh0r53 has joined #openstack-ansible | 19:04 | |
| *** Weifan has quit IRC | 19:04 | |
| *** cmart has quit IRC | 19:13 | |
| *** Weifan has joined #openstack-ansible | 19:24 | |
| *** DanyC has joined #openstack-ansible | 19:31 | |
| *** persia has quit IRC | 19:35 | |
| *** persia has joined #openstack-ansible | 19:36 | |
| *** macza has quit IRC | 19:45 | |
| *** macza has joined #openstack-ansible | 19:46 | |
| *** BjoernT has quit IRC | 19:48 | |
| *** dxiri has quit IRC | 19:51 | |
| *** radeks_ has joined #openstack-ansible | 19:57 | |
| *** radeks has quit IRC | 19:59 | |
| *** BjoernT has joined #openstack-ansible | 20:03 | |
| *** tuxjohnson has left #openstack-ansible | 20:14 | |
| *** partlycloudy has quit IRC | 20:27 | |
| *** radeks_ has quit IRC | 20:37 | |
| *** ansmith has quit IRC | 20:43 | |
| *** pcaruana|afk| has quit IRC | 20:46 | |
| *** BjoernT_ has joined #openstack-ansible | 21:09 | |
| openstackgerrit | Merged openstack/openstack-ansible-lxc_hosts master: Use block/rescue for image pre-stage https://review.opendev.org/658837 | 21:09 |
| *** openstackgerrit has quit IRC | 21:09 | |
| *** BjoernT has quit IRC | 21:10 | |
| *** openstackgerrit has joined #openstack-ansible | 21:11 | |
| openstackgerrit | Nicolas Bock proposed openstack/openstack-ansible-lxc_container_create master: Extend apparmor profile check to Leap 15.0 https://review.opendev.org/658922 | 21:11 |
| openstackgerrit | Nicolas Bock proposed openstack/openstack-ansible-lxc_container_create master: Extend apparmor profile check to Leap 15.0 https://review.opendev.org/658922 | 21:15 |
| *** kplant has quit IRC | 21:18 | |
| openstackgerrit | Nicolas Bock proposed openstack/openstack-ansible-lxc_container_create master: Extend apparmor profile check to openSUSE Leap https://review.opendev.org/658922 | 21:24 |
| *** d34dh0r53 has quit IRC | 21:25 | |
| *** d34dh0r53 has joined #openstack-ansible | 21:29 | |
| *** DanyC has quit IRC | 21:30 | |
| *** sreejithp has quit IRC | 21:34 | |
| *** strattao has quit IRC | 21:44 | |
| *** zhongjun2_ has quit IRC | 21:45 | |
| *** hamzaachi has joined #openstack-ansible | 22:08 | |
| *** partlycloudy has joined #openstack-ansible | 22:13 | |
| *** BjoernT_ has quit IRC | 22:17 | |
| *** hamzaachi has quit IRC | 22:38 | |
| *** tuxjohnson has joined #openstack-ansible | 22:48 | |
| *** ChosSimbaOne has quit IRC | 23:01 | |
| *** ChosSimbaOne has joined #openstack-ansible | 23:01 | |
| openstackgerrit | Merged openstack/openstack-ansible master: Make scenario construction more generic https://review.opendev.org/658698 | 23:06 |
| openstackgerrit | Merged openstack/openstack-ansible master: Add Calico networking AIO scenario https://review.opendev.org/645831 | 23:06 |
| *** macza has quit IRC | 23:10 | |
| *** macza has joined #openstack-ansible | 23:11 | |
| *** macza has quit IRC | 23:16 | |
| evrardjp | these are good news ^ | 23:19 |
| logan- | :) | 23:19 |
| evrardjp | logan-: it seems there are two topics | 23:19 |
| evrardjp | or 3... tests-cleanup, use-integrated-repo-for-all-tests and dynamic-aio-scenario | 23:19 |
| logan- | Yep | 23:20 |
| evrardjp | I guess my patches will be conflicting with Jesse's | 23:20 |
| logan- | Yea probably a lot of rebasing to do | 23:20 |
| logan- | A bunch more are about to merge in a few mins | 23:21 |
| evrardjp | I guess tomorrow I will have a map of what needs doing to be consistent across all repos | 23:21 |
| evrardjp | 1) have the right jobs (metal and metal_distro) 2) make sure the suse meta is correct. I will split in two series of patches, so it makes it easier to review. I will put that onto a google docs file or something | 23:22 |
| evrardjp | logan-: what's your next step, calico upgrade jobs? :p | 23:24 |
| logan- | Placement removal in master and backporting calico stuff to get it working in queens (then upgrading to queens and beyond at last) | 23:25 |
| snadge | does anybody know how to resolve "NovaException: Upgrade DB using Essex release first." when trying to install stein, using the same scripts as queens? | 23:27 |
| snadge | or how to point me in the right direction.. im guessing nova is supposed to initialise its own DB.. but I can't find the script which is responsible for populating the DB.. im not sure which step is missing | 23:28 |
| openstackgerrit | Merged openstack/openstack-ansible master: Move data device detection to Ansible https://review.opendev.org/658710 | 23:43 |
| openstackgerrit | Merged openstack/openstack-ansible master: Run the OSA bootstrap in the Zuul pre phase https://review.opendev.org/658521 | 23:43 |
| openstackgerrit | Merged openstack/openstack-ansible master: Update ARA installation and loading https://review.opendev.org/658708 | 23:43 |
| openstackgerrit | Merged openstack/openstack-ansible master: Set nodepool mirror PIP_OPTS consistently https://review.opendev.org/658709 | 23:43 |
| openstackgerrit | Merged openstack/openstack-ansible stable/stein: Fix Fwaas Dashboard URL https://review.opendev.org/658736 | 23:43 |
| openstackgerrit | Merged openstack/openstack-ansible master: Import placement auth vars to global scope https://review.opendev.org/658176 | 23:43 |
| *** ansmith has joined #openstack-ansible | 23:50 | |
| evrardjp | snadge: if installing stein, why not using stein branch? | 23:52 |
| snadge | i thought I was.. well.. i installed the rdo package for stein, and am reusing ansible scripts that were created for queens | 23:53 |
| snadge | so that is probably the problem.. i have looked at the ones referred to by this channel, and don't really know where to begin.. they're very different | 23:53 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!