Tuesday, 2022-08-02

opendevreviewOpenStack Proposal Bot proposed openstack/openstack-ansible master: Imported Translations from Zanata  https://review.opendev.org/c/openstack/openstack-ansible/+/85178602:39
*** NeilHanlon_ is now known as NeilHanlon03:57
*** NeilHanlon is now known as neil03:59
*** neil is now known as NeilHanlon03:59
*** ysandeep|out is now known as ysandeep05:18
opendevreviewMerged openstack/openstack-ansible master: Imported Translations from Zanata  https://review.opendev.org/c/openstack/openstack-ansible/+/85178607:27
mrfHi, i continue in troubles with TASK [pki : Generate certificate private key 07:44
mrfhttps://pastebin.com/PL29MGiv   07:46
jrosser_mrf: do you mean this happens before?07:50
mrfi re-running again setup-hosts.yml  07:51
mrfi think this recreate the pki certs 07:51
admin13mrf, how many controllers do you have ? if its 3, then that task might fail 3 times, so you just need to run it again as it moves from  1 to next07:52
jrosser_admin1: it shouldnt fail at all - if it does then thats wrong07:53
admin1that was my observation and workaround .. just rerun and it will recreate and move on 07:54
jrosser_but it's still wrong :(07:55
jrosser_and it then generates fokelore as a result - we were in this position with the long standing bug with venv_rebuild=true as well07:55
jrosser_mrf: if you have the log for the previous tasks there in the pki role it would be useful, particularly this https://github.com/openstack/ansible-role-pki/blob/master/tasks/main_certs.yml#L2007:57
jrosser_noonedeadpunk: agreed about ironic_server looking bogus - as i remember merging the inspector stuff was pretty terrible so i am not surprised there is a mess08:17
noonedeadpunkthis can be done as follow-up indeed08:17
jrosser_looking at the ironic role i want to refactor a lot of it08:21
jrosser_ironic / inspector tasks are interleaved all through it making it very confusing08:22
jrosser_really they should be dedicated playbooks even if there is a bit of duplicate code becasue it's very hard to follow08:22
noonedeadpunkyeah, inspector was jsut copy/pasting08:22
noonedeadpunkWe were going to look into ironic later this year, smth like October...08:23
jrosser_right - we are just running it up in the lab now08:23
jrosser_stuart is taking a look at it08:23
*** ysandeep is now known as ysandeep|lunch09:05
*** anskiy1 is now known as anskiy09:07
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-rabbitmq_server stable/ussuri: Use cloudsmith repo for rabbit and erlang  https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/85035009:08
mrfquestion, in production do you move the network roles to dedicated nodes? or mantain in the controller nodes?09:22
noonedeadpunkDepends on scale and design kind of. 09:23
noonedeadpunkIf you don't plan to pass public network to compute nodes and will access public nets only through floating ips/l3 routers - you should totally consider having dedicated net nodes09:24
noonedeadpunkAlso depends on the network backend, as with ovn you don't have concept of net node09:25
noonedeadpunkit's also a thing with lxb and ovs09:25
noonedeadpunk*it's only a thing09:25
mrfgood point09:25
mrfi continue stuck on  [pki : Generate certificate private key for haproxy looks like its not generating certificaste in previus steps?09:37
jrosser_mrf: can you paste some more of the output from that?09:46
jrosser_we need to see the previous tasks from the pki role09:46
jrosser_*all the previous tasks09:46
mrfok let me copy to somewhere09:47
mrf logs: https://pastebin.com/i9ahj7su10:05
*** ysandeep|lunch is now known as ysandeep10:21
opendevreviewMerged openstack/openstack-ansible master: Increase ControlPersist timeout to 300 seconds  https://review.opendev.org/c/openstack/openstack-ansible/+/85142610:23
*** tosky_ is now known as tosky10:24
opendevreviewMerged openstack/openstack-ansible master: Add networking-baremetal repo overrides  https://review.opendev.org/c/openstack/openstack-ansible/+/85155810:25
mrfdid you see anything @jrosser_ ?10:27
jrosser_mrf: sorry in and out of meetings a bit10:41
jrosser_mrf: i think that the trouble is here `TASK [pki : Generate certificate private key for haproxy_haproxy01-172.29.236.6/32] `10:43
jrosser_see that it has `/32` on the end?10:43
jrosser_i think that this would be a great time to compare what you are doing with what the standard setup in an all-in-one would do, thats the reference deployment10:44
mrfmmm this is my openstack_user_config i think is a copy of an example: https://pastebin.com/P32wTtmZ10:49
mrfnothing strange atleast for me10:49
mrfok you're correct 10:52
*** dviroel|out is now known as dviroel11:25
opendevreviewMerged openstack/openstack-ansible-openstack_hosts master: Allow to add extra records to /etc/hosts  https://review.opendev.org/c/openstack/openstack-ansible-openstack_hosts/+/85142811:59
*** priteau_ is now known as priteau12:38
mrfhaproxy roll can enable monitoring of haproxy ?12:57
anskiymrf: yes: https://opendev.org/openstack/openstack-ansible-haproxy_server/src/branch/master/defaults/main.yml#L29-L4112:59
mrfyes just checked online! thanks!!12:59
anskiythere is an options to expose prometheus metrics too: https://docs.openstack.org/openstack-ansible-haproxy_server/latest/configure-haproxy.html#adding-prometheus-metrics-to-haproxy13:00
*** frenzyfriday|rover is now known as frenzyfriday|rover|lunch13:28
*** frenzyfriday|rover|lunch is now known as frenzyfriday|rover14:05
*** ysandeep is now known as ysandeep|dinner14:47
noonedeadpunk#startmeeting openstack_ansible_meeting15:00
opendevmeetMeeting started Tue Aug  2 15:00:43 2022 UTC and is due to finish in 60 minutes.  The chair is noonedeadpunk. Information about MeetBot at http://wiki.debian.org/MeetBot.15:00
opendevmeetUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.15:00
opendevmeetThe meeting name has been set to 'openstack_ansible_meeting'15:00
noonedeadpunk#topic rollcall15:00
noonedeadpunko/15:00
damiandabrowskihi!15:00
jrosser_o/ hello15:03
mgariepyo/ hi !15:03
noonedeadpunk#topic office hours15:04
noonedeadpunkso seems, that rocky jobs have not returned back15:04
NeilHanlono/15:05
noonedeadpunkor well, it's only for Yoga I guess. So I likely is wrong here :)15:05
NeilHanlonthey should be resolved today. ianw was working on a dib release, I believe15:05
noonedeadpunkaha, would that involve r9?15:05
NeilHanlonno, just fixing the r8 ones. but my change for r9 should be able to come pretty quickly, I hope15:06
noonedeadpunkok, great!15:07
noonedeadpunkanother thing we were reported recently, is that we have duplicated records in hosts file, if deploy_host == controller as an example15:07
noonedeadpunkbasically we likely should be somehow smarter in https://opendev.org/openstack/openstack-ansible-openstack_hosts/src/branch/master/defaults/main.yml#L4015:08
noonedeadpunknot really sure how to check for that. as an inventory this host won't be == to 'localhost'15:09
noonedeadpunk*in inventory15:09
NeilHanlonhrm..15:09
noonedeadpunkcan't say it brings a lot of issues, as records are identical, but that's confusing at very least15:10
NeilHanlonmaybe instead of the join, we could filter the list or unique it15:11
mrfOpenstack installed ! :D tomorrow will learn how to add more nodes to a deployment thank so much for your work and help!15:12
noonedeadpunknah, it's a bit different. So this variable jsut controlls if to run this task or not https://opendev.org/openstack/openstack-ansible-openstack_hosts/src/branch/master/tasks/openstack_update_hosts_file.yml#L48-L5715:12
NeilHanlon_etc_hosts_content | unique | join('\n') or so https://opendev.org/openstack/openstack-ansible-openstack_hosts/src/commit/f26fbe4c6aef11cc17c4a9e7be37a7b89ae74326/tasks/openstack_update_hosts_file.yml#L5115:12
noonedeadpunkSo we jsut place same content twice15:12
noonedeadpunkunder different blocks15:12
NeilHanlonyeah15:12
noonedeadpunkon L40 and L48. and it's not that content is wrong, we should just somehow avoid doing that15:13
*** dviroel is now known as dviroel|lunch15:14
NeilHanlonoh. my bad, I understand the problem now15:14
noonedeadpunkanyway, it doesn't hurt much and there's a control, so maybe we should just document that better15:16
NeilHanlononly thing I can think of doing is looping over the list and doing a lineinfile to ensure the lines are inserted once and only once; but that's computationally annoying :) 15:16
noonedeadpunkwell, the bigger problem with that approach is how to cleanup obsolete lines15:17
noonedeadpunkin case compute node is dropped or container re-created with different IP15:17
noonedeadpunkand you can not really clean-up everything not in your list, as ppl might have some custom records there15:18
noonedeadpunkYestarday jrosser_ also spotted weird behaviour in our dynamic_inventory script. While we should patch it right now, this brings me to think what should be our futher plan for this>15:19
noonedeadpunkas dynamic_inventories are being deprecated. One way would be moving it to inventory plugin. But maybe we should jsut have a static inventory, that's being generated/managed with help of inventory-manage script15:20
NeilHanlonSimple way would be not supporting the controller == deploy host scenario, but that's not ideal :D 15:20
jrosser_i don't think that the odd inventory behaviour will affect many deployments, but it's just wrong and extremely confusing15:21
jrosser_though it does make a really wierd deployment of ironic on LXC15:21
noonedeadpunkwell, it affects all deployments, but unlikely you will notice that unless do deep dive into it15:21
jrosser_i think we now understand what is happening, but not necessarily yet how to fix it15:22
noonedeadpunkso it's more of a corner-case usage I would say15:22
noonedeadpunkyeah15:22
noonedeadpunkregarding publishing config_template to galaxy - my original didn't work, as you can not read secrets that are stored in different project except it is project-config.15:27
noonedeadpunkI tried to push jobs to project-config, and agreed with keys-keeper to share secret with us, but this job got -1 from infra stuff with suggestion to move it partially to zuul-jobs.15:28
noonedeadpunkI didn't have time to follow-up on that, but getting it to zuul-jobs sounds weird to me, mainly because of the code and I bet I will get -2 there...15:29
*** ysandeep|dinner is now known as ysandeep15:29
noonedeadpunkat the same time I do not want to copy-paste jobs code, but well... this is other way around15:29
noonedeadpunkWe've also landed almost everything to stable/xena to do bump and next release. This in turn would unblock Yoga upgrade jobs.15:31
jrosser_we have done another X->Y upgrade today15:31
jrosser_andrewbonney may have a few small things to fix15:31
jrosser_i think we have some docs errors around upgrades15:32
noonedeadpunkyeah, we don't mention octavia at very least15:32
noonedeadpunkin terms of moving certs to be usable by pki role15:32
noonedeadpunkI don't think I have anything extra to add though15:38
*** ysandeep is now known as ysandeep|out15:43
jrosser_no - do we have any new bugs?15:44
noonedeadpunkI don't think we do?15:48
jrosser_i've not looked :)15:48
* jrosser_ might have fixed the inventory15:48
jrosser_^ extremely unlikley, but possible15:48
noonedeadpunkwe have not closed some recent ones though, but we didn't have new since few weeks I guess15:49
noonedeadpunkand last one is assigned to damiandabrowski :)15:49
noonedeadpunkoh15:49
damiandabrowskiyeah sorry, will try to come back to it during this week :/15:51
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible master: Do not create {hostname}-host_containers group as child of other groups  https://review.opendev.org/c/openstack/openstack-ansible/+/85176415:55
noonedeadpunkJust did some extra cleanup to the removal of host_containers - not fixing state unfortunately15:57
noonedeadpunk#endmeeting15:58
opendevmeetMeeting ended Tue Aug  2 15:58:58 2022 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)15:58
opendevmeetMinutes:        https://meetings.opendev.org/meetings/openstack_ansible_meeting/2022/openstack_ansible_meeting.2022-08-02-15.00.html15:58
opendevmeetMinutes (text): https://meetings.opendev.org/meetings/openstack_ansible_meeting/2022/openstack_ansible_meeting.2022-08-02-15.00.txt15:58
opendevmeetLog:            https://meetings.opendev.org/meetings/openstack_ansible_meeting/2022/openstack_ansible_meeting.2022-08-02-15.00.log.html15:58
*** dviroel|lunch is now known as dviroel|16:15
*** dviroel| is now known as dviroel16:15
opendevreviewJonathan Rosser proposed openstack/openstack-ansible master: Do not create {hostname}-host_containers group as child of other groups  https://review.opendev.org/c/openstack/openstack-ansible/+/85176416:52
NeilHanloncoming soon to a nodepool builder near you.. https://review.opendev.org/c/openstack/diskimage-builder/+/84890119:09
*** dviroel is now known as dviroel|biab19:56
*** dviroel|biab is now known as dviroel20:31
*** dviroel is now known as dviroel|afk20:58
opendevreviewJonathan Rosser proposed openstack/openstack-ansible master: Do not create {hostname}-host_containers group as child of other groups  https://review.opendev.org/c/openstack/openstack-ansible/+/85176421:06
*** anskiy1 is now known as anskiy21:33
opendevreviewMerged openstack/openstack-ansible-lxc_hosts master: Prevent lxc.service from being restarted on package update  https://review.opendev.org/c/openstack/openstack-ansible-lxc_hosts/+/85107122:50
opendevreviewMerged openstack/openstack-ansible master: Allow to provide serial for lxc_hosts  https://review.opendev.org/c/openstack/openstack-ansible/+/85104923:40

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!