Wednesday, 2024-08-28

opendevreviewAravindh proposed openstack/kolla-ansible master: HAProxy: Switch to L7 Healthchecks  https://review.opendev.org/c/openstack/kolla-ansible/+/91843705:46
Core8798SvenKieske How do I give rabbit 16cpus and 32GB of ram, I saw the limits in the globals file, but could not figure out how to properly set it. What should I set it for Cpu_seconds05:51
opendevreviewMichal Nasiadka proposed openstack/kolla master: CI: Move image building to a role  https://review.opendev.org/c/openstack/kolla/+/92214805:58
opendevreviewMichal Nasiadka proposed openstack/kolla master: CI: Move image building to a role  https://review.opendev.org/c/openstack/kolla/+/92214805:59
opendevreviewMichal Nasiadka proposed openstack/kolla master: CI: Move image building to a role  https://review.opendev.org/c/openstack/kolla/+/92214806:05
opendevreviewMichal Nasiadka proposed openstack/kolla master: CI: Move image building to a role  https://review.opendev.org/c/openstack/kolla/+/92214806:06
opendevreviewMichal Nasiadka proposed openstack/kolla master: CI: Move image building to a role  https://review.opendev.org/c/openstack/kolla/+/92214806:07
opendevreviewMichal Nasiadka proposed openstack/kolla master: CI: Move image building to a role  https://review.opendev.org/c/openstack/kolla/+/92214806:07
opendevreviewMichal Nasiadka proposed openstack/kolla master: CI: Move image building to a role  https://review.opendev.org/c/openstack/kolla/+/92214806:11
opendevreviewMichal Nasiadka proposed openstack/kolla master: CI: Move image building to a role  https://review.opendev.org/c/openstack/kolla/+/92214806:11
opendevreviewMichal Nasiadka proposed openstack/kolla master: CI: Move image building to a role  https://review.opendev.org/c/openstack/kolla/+/92214806:23
opendevreviewMichal Nasiadka proposed openstack/kolla master: CI: Move image building to a role  https://review.opendev.org/c/openstack/kolla/+/92214806:30
opendevreviewMichal Nasiadka proposed openstack/kolla master: CI: Move image building to a role  https://review.opendev.org/c/openstack/kolla/+/92214806:32
opendevreviewMichal Nasiadka proposed openstack/kolla master: CI: Move image building to a role  https://review.opendev.org/c/openstack/kolla/+/92214806:38
opendevreviewMichal Nasiadka proposed openstack/kolla master: CI: Move image building to a role  https://review.opendev.org/c/openstack/kolla/+/92214806:47
mnasiadka2024-08-28 06:49:30.517460 | primary |   DEPRECATION: kolla is being installed using the legacy 'setup.py install' method, because it does not have a 'pyproject.toml' and the 'wheel' package is not installed. pip 23.1 will enforce this behaviour change. A possible replacement is to enable the '--use-pep517' option. Discussion can be found at https://github.com/pypa/pip/issues/855906:49
mnasiadkainteresting06:49
mnasiadkaSvenKieske, frickler: so we do need the pyproject.toml file...06:50
opendevreviewMichal Nasiadka proposed openstack/kolla master: CI: Move image building to a role  https://review.opendev.org/c/openstack/kolla/+/92214806:51
opendevreviewMichal Nasiadka proposed openstack/kolla master: CI: Move image building to a role  https://review.opendev.org/c/openstack/kolla/+/92214806:52
opendevreviewMichal Nasiadka proposed openstack/kolla master: Add pyproject.toml to support pip 23.1  https://review.opendev.org/c/openstack/kolla/+/92068406:52
opendevreviewMichal Nasiadka proposed openstack/kolla master: CI: Move image building to a role  https://review.opendev.org/c/openstack/kolla/+/92214806:55
SvenKieskemnasiadka: that's what I always thought. I was confused by messages in openstack land that it might not be needed? But it's a complicated mess.07:02
opendevreviewMichal Nasiadka proposed openstack/kolla master: CI: Move image building to a role  https://review.opendev.org/c/openstack/kolla/+/92214807:05
opendevreviewSven Kieske proposed openstack/kolla-ansible master: harden haproxy TLS configuration  https://review.opendev.org/c/openstack/kolla-ansible/+/91540307:10
opendevreviewMichal Nasiadka proposed openstack/kolla master: CI: Move image building to a role  https://review.opendev.org/c/openstack/kolla/+/92214807:16
*** bodgix2 is now known as bodgix07:20
*** mmalchuk_ is now known as mmalchuk07:20
opendevreviewRoman Krček proposed openstack/kolla master: Refactor dev mode  https://review.opendev.org/c/openstack/kolla/+/92571207:21
opendevreviewRoman Krček proposed openstack/kolla-ansible master: Refactor dev mode  https://review.opendev.org/c/openstack/kolla-ansible/+/92571407:23
opendevreviewMichal Nasiadka proposed openstack/kolla master: CI: Move image building to a role  https://review.opendev.org/c/openstack/kolla/+/92214807:25
*** edebeste1 is now known as edebeste07:25
opendevreviewAravindh proposed openstack/kolla-ansible master: HAProxy: Switch to L7 Healthchecks  https://review.opendev.org/c/openstack/kolla-ansible/+/91843707:28
opendevreviewMichal Nasiadka proposed openstack/kolla master: DNM: Reproduce bug 2074075  https://review.opendev.org/c/openstack/kolla/+/92735207:38
opendevreviewMichal Nasiadka proposed openstack/kolla master: CI: Move image building to a role  https://review.opendev.org/c/openstack/kolla/+/92214807:40
opendevreviewMichal Nasiadka proposed openstack/kolla master: DNM: Reproduce bug 2074075  https://review.opendev.org/c/openstack/kolla/+/92735207:49
opendevreviewMichal Nasiadka proposed openstack/kolla master: CI: Move image building to a role  https://review.opendev.org/c/openstack/kolla/+/92214807:54
opendevreviewMichal Nasiadka proposed openstack/kolla master: Fix symlinking of openstack-base source  https://review.opendev.org/c/openstack/kolla/+/92475908:29
opendevreviewMichal Nasiadka proposed openstack/kolla master: DNM: Reproduce bug 2074075  https://review.opendev.org/c/openstack/kolla/+/92735208:32
mnasiadkaSvenKieske: https://review.opendev.org/c/openstack/kolla/+/924759 - I think that's more elegant solution now08:36
opendevreviewMatúš Jenča proposed openstack/kolla-ansible master: CI: Fix upgrade jobs failing on adding new certificates  https://review.opendev.org/c/openstack/kolla-ansible/+/92628408:38
opendevreviewMatúš Jenča proposed openstack/kolla-ansible master: Add certificates for RabbitMQ internode TLS  https://review.opendev.org/c/openstack/kolla-ansible/+/92138008:38
opendevreviewMatúš Jenča proposed openstack/kolla-ansible master: Add support for RabbitMQ internode tls  https://review.opendev.org/c/openstack/kolla-ansible/+/92138108:38
opendevreviewTony Breeds proposed openstack/kolla master: pyinotify is no longer needed  https://review.opendev.org/c/openstack/kolla/+/91787108:42
SvenKieskemnasiadka: I was looking into writing a custom kolla function to check if /plugins/ - or any folder/"package" - is empty and then call that via the jinja2 template08:44
mnasiadkaSvenKieske: we could just easily rework the install_pip macro in macros.j2 to make it better - but I don't think that's part of this patch08:44
SvenKieskemnasiadka: because I think fundamentally the logic of "if a random directory is empty we don't install totally unrelated stuff" is just broken, no?08:45
SvenKieskemnasiadka: okay, fine with me I guess08:45
SvenKieskeAs I need some more time to find where this function should live, probably somewhere in kolla_worker.py or build.py, I'm not 100% sure08:47
opendevreviewAravindh proposed openstack/kolla-ansible master: HAProxy: Switch to L7 Healthchecks  https://review.opendev.org/c/openstack/kolla-ansible/+/91843709:08
Core8688can some one look at this patch and let me know if I missed any services?09:11
opendevreviewBartosz Bezak proposed openstack/kolla-ansible master: WIP: Add ovn-bgp-agent / FRR / neutron-bgpvpn / Horizon BGPVPN dashboard support  https://review.opendev.org/c/openstack/kolla-ansible/+/89162209:44
opendevreviewMichal Nasiadka proposed openstack/kolla master: Fix symlinking of openstack-base source  https://review.opendev.org/c/openstack/kolla/+/92475909:51
opendevreviewBartosz Bezak proposed openstack/kolla-ansible master: WIP: Add ovn-bgp-agent / FRR / neutron-bgpvpn / Horizon BGPVPN dashboard support  https://review.opendev.org/c/openstack/kolla-ansible/+/89162210:16
opendevreviewMichal Nasiadka proposed openstack/kolla master: CI: Move image building to a role  https://review.opendev.org/c/openstack/kolla/+/92214810:29
opendevreviewVerification of a change to openstack/kolla master failed: Remove incorrect reno  https://review.opendev.org/c/openstack/kolla/+/92724110:31
PrzemekKHow to configure proxy for slack notification in opensearch ?10:53
opendevreviewMatt Crees proposed openstack/kolla master: Revert "Add support for checking incoming RabbitMQ version"  https://review.opendev.org/c/openstack/kolla/+/92737811:05
opendevreviewMichal Nasiadka proposed openstack/kolla master: CI: Move image building to a role  https://review.opendev.org/c/openstack/kolla/+/92214811:13
opendevreviewBartosz Bezak proposed openstack/kolla-ansible master: WIP: Add ovn-bgp-agent / FRR / neutron-bgpvpn / Horizon BGPVPN dashboard support  https://review.opendev.org/c/openstack/kolla-ansible/+/89162211:19
opendevreviewMichal Nasiadka proposed openstack/kolla master: Fix symlinking of openstack-base source  https://review.opendev.org/c/openstack/kolla/+/92475911:21
mnasiadkaSvenKieske: I have no bloody clue why fixing that pycadf/openstack-base-source drama now makes keystone installation failing :)11:22
opendevreviewMichal Nasiadka proposed openstack/kolla master: CI: Move image building to a role  https://review.opendev.org/c/openstack/kolla/+/92214811:28
opendevreviewMichal Nasiadka proposed openstack/kolla master: DNM: Reproduce bug 2074075  https://review.opendev.org/c/openstack/kolla/+/92735211:34
opendevreviewBartosz Bezak proposed openstack/kolla master: Add ovn-bgp-agent / FRR / Horizon BGPVPN dashboard  https://review.opendev.org/c/openstack/kolla/+/89161711:51
opendevreviewMichal Nasiadka proposed openstack/kolla master: WIP: Add support for rpm to repos.yaml  https://review.opendev.org/c/openstack/kolla/+/90987911:52
opendevreviewMichal Nasiadka proposed openstack/kolla master: Add support for rpm to repos.yaml  https://review.opendev.org/c/openstack/kolla/+/90987911:52
opendevreviewMatt Crees proposed openstack/kolla-ansible master: Add a precheck to catch RMQ SLURP upgrades  https://review.opendev.org/c/openstack/kolla-ansible/+/92614612:31
opendevreviewMatt Crees proposed openstack/kolla-ansible master: Add a precheck to catch RMQ SLURP upgrades  https://review.opendev.org/c/openstack/kolla-ansible/+/92614612:32
opendevreviewMerged openstack/kolla-ansible stable/2024.1: [2024.1 only] CI: RMQ version upgrade before SLURP  https://review.opendev.org/c/openstack/kolla-ansible/+/92703712:36
opendevreviewMatúš Jenča proposed openstack/kolla-ansible master: Switch mariadb's loadbalancer from HAProxy to ProxySQL  https://review.opendev.org/c/openstack/kolla-ansible/+/91372412:38
opendevreviewMatúš Jenča proposed openstack/kolla-ansible master: Add backend TLS between MariaDB and ProxySQL  https://review.opendev.org/c/openstack/kolla-ansible/+/90991212:38
opendevreviewMichal Nasiadka proposed openstack/kolla master: Fix symlinking of openstack-base source  https://review.opendev.org/c/openstack/kolla/+/92475912:38
opendevreviewMichal Nasiadka proposed openstack/kolla master: DNM: Reproduce bug 2074075  https://review.opendev.org/c/openstack/kolla/+/92735212:39
opendevreviewBartosz Bezak proposed openstack/kolla-ansible master: WIP: Add ovn-bgp-agent / FRR / neutron-bgpvpn / Horizon BGPVPN dashboard support  https://review.opendev.org/c/openstack/kolla-ansible/+/89162212:42
mnasiadkamgoddard mnasiadka bbezak frickler kevko SvenKieske mmalchuk gkoper jangutter jsuazo jovial osmanlicilegi mattcrees dougszu darmach - meeting in 10 minutes12:50
SvenKieske\o/? :)13:01
bbezak:)13:01
mnasiadka#startmeeting kolla13:01
opendevmeetMeeting started Wed Aug 28 13:01:34 2024 UTC and is due to finish in 60 minutes.  The chair is mnasiadka. Information about MeetBot at http://wiki.debian.org/MeetBot.13:01
opendevmeetUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.13:01
opendevmeetThe meeting name has been set to 'kolla'13:01
mnasiadka#topic rollcall13:01
mnasiadkao/13:01
SvenKieskeo/13:01
r-krcek\o/13:01
frickler\o13:01
mattcreeso/13:01
bbezako/13:01
mhinero/13:02
mnasiadka#topic agenda13:03
mnasiadka* Announcements13:03
mnasiadka* CI status13:03
mnasiadka* Release tasks13:03
mnasiadka* Current cycle planning13:03
mnasiadka* Additional agenda (from whiteboard)13:03
mnasiadka* Open discussion13:03
mnasiadka#topic Announcements13:03
mnasiadkaI'll be off next week - attending OpenInfra Asia13:03
mnasiadkabbezak/frickler: do you want to run the meeting next week - or should we cancel13:03
mnasiadka?13:03
bbezakcan do13:03
SvenKieskeoh, have fun there! I'll also be away in Berlin, I guess.13:04
bbezakif everybody is away on summits, then I think we should cancel13:04
SvenKieskeI guess it's only me from our team :)13:04
mnasiadkaok, then bbezak running next week - unless you change your mind and cancel ;-)13:05
mnasiadka#topic CI status13:05
bbezakyeap13:05
mnasiadkaI guess it's green, but looking a bit flakey lately13:05
fricklerexcept for the arm stuff13:06
mnasiadkayes13:06
mnasiadkathat has storage slower than my laptop13:06
mnasiadka#topic Release tasks13:06
SvenKieskeperiodic publish looks red for stable branches for aarch64, should those get disabled, due to infra issues I heard about? https://zuul.openstack.org/builds?project=openstack%2Fkolla&pipeline=periodic&result=FAILURE&result=RETRY_LIMIT&result=POST_FAILURE&result=NODE_FAILURE&result=SKIPPED&skip=013:06
mnasiadkait is successful sometimes, so I guess not yet13:07
mnasiadkahopefully the status will improve13:07
fricklerLance is working on boosting cinder/ceph iiuc13:08
mnasiadkaNo release tasks, apart the switch back to master on delorean patch that is merging13:08
mnasiadka#topic Current cycle planning13:08
bbezakI've started reviving ovn-bgp-agent patches13:09
mnasiadkaSo, I'm working on two things - Noble 24.04 (https://review.opendev.org/q/topic:%22kolla-ubuntu-noble%22) and MariaDB 11.4 (https://review.opendev.org/q/topic:%22kolla_mariadb_11%22)13:09
mnasiadkaMariaDB drives me more nuts than Noble13:09
mnasiadkaif anybody is interested13:09
opendevreviewMerged openstack/kolla-ansible master: CI: Fix upgrade jobs failing on adding new certificates  https://review.opendev.org/c/openstack/kolla-ansible/+/92628413:10
mnasiadkaand revived https://review.opendev.org/c/openstack/kolla/+/922148 to continue with CI improvements13:10
SvenKieskewhile interested, I fear I have no time to work on the mariadb stuff, need to push other stuff over the line first13:10
mnasiadkaok then, let's go to additional agenda13:11
mnasiadka#topic Additional agenda (from whiteboard)13:11
mnasiadkaSvenKieske[28th of Aug.] please review these, CI is passing and there are no open comments:13:11
mnasiadkahardening haproxy TLS config should be ready to merge/review now: https://review.opendev.org/c/openstack/kolla-ansible/+/91540313:11
mnasiadkavalidate prometheus config https://review.opendev.org/c/openstack/kolla-ansible/+/92620913:11
mnasiadkaharden horizon config (don't bindmount tmp) https://review.opendev.org/c/openstack/kolla-ansible/+/92137113:11
SvenKieskethat last one is very smol, they should all be non controversial now, as they already cooked some time, hopefully, but I'm happy to address any comments.13:12
mnasiadkafirst one LGTM13:12
frickleradding these to my list13:12
mnasiadkaand third one has already my +213:13
mnasiadkar-krcek [28th Aug]13:13
mnasiadkaplease review https://review.opendev.org/c/openstack/kolla-ansible/+/92311013:13
mnasiadkaand https://review.opendev.org/c/openstack/kolla-ansible/+/92544413:13
mnasiadkaand https://review.opendev.org/c/openstack/kolla-ansible/+/59973513:13
mnasiadkafirst one has merge conflict13:13
mnasiadkasecond LGTM13:14
SvenKieskeyep13:14
mnasiadkathird one... interesting, but I don't think we have any CI running this13:15
r-krcekYeah, it was a little controversial last time. :)13:15
SvenKieskecan confirm, so I guess r-krcek: can you add CI coverage for the "check" command?13:16
SvenKieskeah we did discuss this already?13:16
r-krcekSvenKieske: I will attempt, never really did CI, but Ill ask around in my team.13:16
r-krcekWell, I brought it up few weeks ago and some welcomed it, but then in the comments Michal Arbet called it not really necessary. 13:17
r-krcekAfter that silence :) 13:17
SvenKiesker-krcek: I guess this could be added to the stage where we currently run "config-validate", but on the other side, afaik we already have other checks there checking if containers run? mhhm13:17
SvenKieskeit's a bit of an architecture question if each role should check that, especially clustered services, are actually running - the clustered ones do this afaik. or if you want to centrally check if all expected containers run.13:19
SvenKieskefrom an ops perspective I guess there is value in a dedicated command which checks "everything" instead of having to call each role individually13:19
SvenKieskealso I bet some roles don't really have these kind of checks now.13:20
SvenKieske¯\_(ツ)_/¯13:21
mnasiadkaI just think we should get that run anywhere, not saying in every CI job - question how long does that run :)13:22
mnasiadka#topic Open discussion13:23
mnasiadkaOk then, anything anybody?13:23
mattcreesWould be good to get any more eyes here: https://review.opendev.org/c/openstack/kolla-ansible/+/926146 It's the key precheck that will point people on how to handle rabbit with slurp, want to make sure everyone finds it understandable 13:25
fricklersounds reasonable, will read later13:26
fricklerseems like that may be all?13:29
* SvenKieske pokes mnasiadka ;)13:31
mnasiadkayup, seems like all13:31
mnasiadkathanks for coming!13:31
mnasiadka#endmeeting13:31
opendevmeetMeeting ended Wed Aug 28 13:31:46 2024 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)13:31
opendevmeetMinutes:        https://meetings.opendev.org/meetings/kolla/2024/kolla.2024-08-28-13.01.html13:31
opendevmeetMinutes (text): https://meetings.opendev.org/meetings/kolla/2024/kolla.2024-08-28-13.01.txt13:31
opendevmeetLog:            https://meetings.opendev.org/meetings/kolla/2024/kolla.2024-08-28-13.01.log.html13:31
r-krcekthank you guys13:31
SvenKieskethanks for running the meeting13:31
opendevreviewVerification of a change to openstack/kolla master failed: Revert "[release] Change RDO's DELOREAN_DEPS repo to Caracal"  https://review.opendev.org/c/openstack/kolla/+/92698213:39
opendevreviewBartosz Bezak proposed openstack/kolla-ansible master: WIP: Add ovn-bgp-agent / FRR / neutron-bgpvpn / Horizon BGPVPN dashboard support  https://review.opendev.org/c/openstack/kolla-ansible/+/89162213:45
mnasiadkafrickler: https://review.opendev.org/c/openstack/kolla/+/924759 - that would be important for my downstream folks - can you have a look? there's a reproducer patch on top that uses git repo instead of tarballs for requirements repo14:18
mnasiadkafrickler: it's going through zuul check queue now, but everything is green ATM14:19
opendevreviewVerification of a change to openstack/kolla-ansible master failed: hardening horizon: don't mount hosts /tmp  https://review.opendev.org/c/openstack/kolla-ansible/+/92137115:10
fricklermnasiadka: commented15:40
opendevreviewMerged openstack/kolla master: Revert "Add support for checking incoming RabbitMQ version"  https://review.opendev.org/c/openstack/kolla/+/92737815:50
opendevreviewMichal Nasiadka proposed openstack/kolla master: Fix symlinking of openstack-base source  https://review.opendev.org/c/openstack/kolla/+/92475915:56
opendevreviewMichal Nasiadka proposed openstack/kolla master: Fix symlinking of openstack-base source  https://review.opendev.org/c/openstack/kolla/+/92475915:58
mnasiadkafrickler: responded - if you prefer I can break out the keystone fix to a separate patch.16:00
mnasiadka(now that I understand why it's happening)16:00
opendevreviewMichal Nasiadka proposed openstack/kolla master: DNM: Reproduce bug 2074075  https://review.opendev.org/c/openstack/kolla/+/92735216:07
opendevreviewMerged openstack/kolla master: Remove incorrect reno  https://review.opendev.org/c/openstack/kolla/+/92724116:33
opendevreviewMerged openstack/kolla master: Revert "[release] Change RDO's DELOREAN_DEPS repo to Caracal"  https://review.opendev.org/c/openstack/kolla/+/92698217:04
opendevreviewMerged openstack/kolla-ansible master: hardening horizon: don't mount hosts /tmp  https://review.opendev.org/c/openstack/kolla-ansible/+/92137118:25
milanpetricHi all. I am trying to install kolla ansible on Ubuntu 22.04.4 LTS. I am doing that by manual and everything until: kolla-ansible -i ./all-in-one deploy was great. There is a big error output on step: RUNNING HANDLER [common : Restart fluentd container]. I think that main part is: line 39, in create_api_error_from_http_exception\\n    raise cls(e, response=response, explanation=explanation) from e\\ndocker.errors.NotFoun19:25
milanpetricline 39, in create_api_error_from_http_exception\\n    raise cls(e, response=response, explanation=explanation) from e\\ndocker.errors.NotFound: 404 Client Error for http+docker://localhost/v1.46/images/create?tag=yoga-ubuntu-jammy-aarch64&fromImage=quay.io%2Fopenstack.kolla%2Ffluentd: Not Found (\"manifest for quay.io/openstack.kolla/fluentd:yoga-ubuntu-jammy-aarch64 not found: manifest unknown: manifest unknown\")\\n'"19:25
milanpetricDoes anyone know what could be the issue and how to resolve it?19:25
milanpetricI've tried with: openstack_release: "yoga" and openstack_release: "master" - also with kolla_base_distro: "ubuntu" and kolla_base_distro: "rocky". Always the similar output on same step.19:27
samcat116I think yoga images are no longer published. also if youre using aarch64, I think they only publish debian based images for that architecture19:39
milanpetricOk, so I would need to try that combo. Currently I am using openstack_tag_suffix: "-aarch64" in my globals.yml19:41
milanpetric@samcat116 you are right, but now I have other error later20:25
milanpetricfatal: [localhost]: FAILED! => {"changed": false, "cmd": ["docker", "exec", "-t", "kolla_toolbox", "ansible", "--version"], "delta": "0:00:00.021481", "end": "2024-08-28 20:24:27.740662", "msg": "non-zero return code", "rc": 1, "start": "2024-08-28 20:24:27.719181", "stderr": "Error response from daemon: container 00b1003c01e09094c7679bbd145b26f7ef478d4bea911b66e45c54b8dcfea669 is not running", "stderr_lines": ["Error re20:25
milanpetricOn RUNNING HANDLER [common : Initializing toolbox container using normal user]20:27

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!