*** rpittau|afk is now known as rpittau | 07:51 | |
opendevreview | Mark Goddard proposed openstack/kolla-ansible master: Support Ansible max_fail_percentage https://review.opendev.org/c/openstack/kolla-ansible/+/805598 | 08:23 |
---|---|---|
opendevreview | Mark Goddard proposed openstack/kolla-ansible master: nova: don't fail all hosts on service registration failures https://review.opendev.org/c/openstack/kolla-ansible/+/805599 | 08:23 |
kevko | morning :) | 08:24 |
opendevreview | Radosław Piliszek proposed openstack/kolla-ansible stable/ussuri: Fix regression in image pull action v2 https://review.opendev.org/c/openstack/kolla-ansible/+/805602 | 08:29 |
opendevreview | Mark Goddard proposed openstack/kayobe-config-dev master: Use kolla_internal_vip_address instead of aio_vip_address https://review.opendev.org/c/openstack/kayobe-config-dev/+/805240 | 08:30 |
opendevreview | Mark Goddard proposed openstack/kolla-ansible stable/wallaby: Fix haproxy precheck when kolla_externally_managed_cert is used https://review.opendev.org/c/openstack/kolla-ansible/+/805560 | 08:32 |
opendevreview | Mark Goddard proposed openstack/kolla-ansible master: Remove chrony role from kolla https://review.opendev.org/c/openstack/kolla-ansible/+/791743 | 08:56 |
opendevreview | Michal Nasiadka proposed openstack/kolla master: haproxy: Use haproxy 2.2.x https://review.opendev.org/c/openstack/kolla/+/804513 | 09:33 |
mnasiadka | morning | 09:35 |
jingvar | morning | 09:39 |
kevko | \o/ | 09:42 |
opendevreview | Verification of a change to openstack/kolla stable/wallaby failed: Add some system users to fix user uid inconsistencies https://review.opendev.org/c/openstack/kolla/+/805134 | 09:43 |
opendevreview | Radosław Piliszek proposed openstack/kolla-ansible stable/ussuri: Fix regression in image pull action v2 https://review.opendev.org/c/openstack/kolla-ansible/+/805602 | 10:24 |
opendevreview | Merged openstack/kayobe master: Add twentyFiveGigE to port descriptions in inspector rules https://review.opendev.org/c/openstack/kayobe/+/802709 | 10:41 |
opendevreview | wu.chunyang proposed openstack/kolla master: Remove chrony image from kolla https://review.opendev.org/c/openstack/kolla/+/791753 | 10:51 |
opendevreview | Michal Arbet proposed openstack/kolla master: Move clustercheck script to mariadb-base https://review.opendev.org/c/openstack/kolla/+/805613 | 11:12 |
Fl1nt | Goooood whatever chan!!!! | 11:27 |
Fl1nt | Radek, are you there? | 11:29 |
Fl1nt | Question about the Erlang Bump from 22 to 23, what's the reasoning behind that? | 11:29 |
Fl1nt | on RabbitMQ | 11:29 |
Fl1nt | :D | 11:29 |
Fl1nt | cause for those using the CentOS Messaging repo, it's not yet available :-( | 11:30 |
opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Use Docker healthchecks for mariadb-server service https://review.opendev.org/c/openstack/kolla-ansible/+/805616 | 11:33 |
kevko | anyone have some time for some reviews :) ? | 11:39 |
opendevreview | Mark Goddard proposed openstack/kolla-ansible stable/victoria: Add Neutron DHCP agent to OVN networking setup https://review.opendev.org/c/openstack/kolla-ansible/+/805561 | 11:39 |
kevko | Fl1nt: don't know ..but maybe have erlang same in all images ? (bullseye has 1:23.2.6+dfsg-1 in repo https://packages.debian.org/bullseye/erlang-base) | 11:45 |
opendevreview | Merged openstack/kolla-ansible stable/victoria: Use more RMQ flags for less busy wait https://review.opendev.org/c/openstack/kolla-ansible/+/805272 | 11:57 |
opendevreview | Merged openstack/kolla-ansible stable/ussuri: Use more RMQ flags for less busy wait https://review.opendev.org/c/openstack/kolla-ansible/+/805273 | 12:00 |
opendevreview | Merged openstack/kolla-ansible stable/wallaby: Use more RMQ flags for less busy wait https://review.opendev.org/c/openstack/kolla-ansible/+/805271 | 12:02 |
opendevreview | Michal Nasiadka proposed openstack/kolla master: haproxy: Use haproxy 2.2.x https://review.opendev.org/c/openstack/kolla/+/804513 | 12:11 |
Fl1nt | kevko, yeah probably but still aarch64 is still on 22 as it doesn't yet get any 23 release, so would have been more generic or standard to use 22 everywhere isn't? | 12:27 |
Fl1nt | Anyway, I'll override on our template-override :D | 12:27 |
Fl1nt | oh, sweet clever RMQ Patch :D | 12:29 |
Fl1nt | ok understand the 23 release now | 12:29 |
opendevreview | Merged openstack/kolla stable/wallaby: Add some system users to fix user uid inconsistencies https://review.opendev.org/c/openstack/kolla/+/805134 | 12:33 |
kevko | is this working ? https://etherpad.opendev.org/p/kolla-mentorship-signup :) ? | 12:39 |
mnasiadka | judging by the list of names, this is really old... | 12:43 |
kevko | yeah, i thought :D | 12:44 |
yoctozepto | the history shows it's from 2017 | 12:44 |
kevko | but, is there something similar ? or kolla is closed ? | 12:44 |
mnasiadka | if there's a need for something similar, we could revive it - please add a topic to the next meeting and we can discuss it | 12:45 |
yoctozepto | how is kolla closed? | 12:45 |
yoctozepto | we are mentoring you all the time, kevko | 12:45 |
kevko | :D | 12:45 |
kevko | well, i was thinking about to join core team... so I was trying to find some document, procedure whatever - what is needed to be done .. but what I found is only that one document | 12:47 |
mgoddard | kevko: I think I have said this before, but... how to join core team? Demonstrate understanding of the project and trustworthiness through doing code reviews. | 12:51 |
mnasiadka | kevko: it's not something you can sign up to, like a certification :) | 12:51 |
mgoddard | kevko: from what I have seen you would be a suitable candidate, but we would need more examples of code reviewing | 12:53 |
mgoddard | kevko: we have these slides, which show various levels of involvement: https://docs.google.com/presentation/d/1YyRbp-UW9EXTtG9_sTaRov13X6V8FjCztj9c0EkT754/edit | 12:54 |
mgoddard | "Demonstrate responsibility, understanding & care" | 12:54 |
yoctozepto | I am with mgoddard on this one; you certainly are a good candidate but we need more reviews from you; cores focus on reviews and project maintenance, not necessarily feature implementation | 12:55 |
yoctozepto | mgoddard: perhaps we should link this up in a few places around? | 12:55 |
mgoddard | kevko: it would be good to see you in the top 10 reviewer list: https://www.stackalytics.io/?module=kolla-group | 12:55 |
yoctozepto | I like our creation | 12:55 |
mgoddard | kevko: although that is not the whole story: quality >> quantity | 12:56 |
yoctozepto | mgoddard, kevko: ++, and not with ratio too close to 100% because those are often too artificial :P | 12:56 |
yoctozepto | yeah, that's about that quality part there | 12:56 |
mgoddard | tbh, a -1 is more useful than a +1 | 12:57 |
Fl1nt | hum... I think this stackalytics thing may well loose few stats :D | 12:57 |
Fl1nt | I'm not on the whole classement ^^ | 12:57 |
yoctozepto | mgoddard: my thinking indeed | 12:57 |
mgoddard | -1 when warranted of course :) | 12:58 |
Fl1nt | the 37th one is having 1 review and is on the list :p | 12:58 |
yoctozepto | Fl1nt: you are missing entirely? | 12:58 |
mgoddard | Fl1nt: you are on https://www.stackalytics.io/report/contribution?module=kolla-group&project_type=openstack&days=30 | 12:58 |
mgoddard | #39 | 12:58 |
Fl1nt | My colleague Martin is on the list and I'm not he's having 1 review and I'm having way more ^^ | 12:58 |
yoctozepto | I can see your 7 in total Fl1nt | 12:59 |
yoctozepto | but none on Xena | 12:59 |
yoctozepto | somehow they were not counted against the current release | 13:00 |
Fl1nt | yeah, exactly, just saw you were using Xena filter ^^ I'm only contributing on master and then backport :D | 13:00 |
yoctozepto | remember these are reviews | 13:00 |
yoctozepto | Xena is master ;-) | 13:00 |
yoctozepto | these are reviews | 13:00 |
yoctozepto | not change proposals | 13:00 |
Fl1nt | ooooh ok :D | 13:00 |
yoctozepto | https://www.stackalytics.io/?module=opendev.org/openstack/kolla-ansible&release=xena&metric=patches | 13:01 |
Fl1nt | Yeah, I'm bad at reviewing stuff, I'm usually having hard time finding something to say ^^ | 13:01 |
yoctozepto | here you are in xena | 13:01 |
yoctozepto | Fl1nt: ack, no problem | 13:02 |
mgoddard | reviewing takes practice | 13:02 |
mgoddard | but it gets easier | 13:02 |
Fl1nt | Doing my best when I'm highlighted tbh :D | 13:02 |
mgoddard | your input is appreciated Fl1nt | 13:02 |
yoctozepto | I'm dropping kolla-cli | 13:02 |
yoctozepto | mgoddard, Fl1nt ++ | 13:02 |
yoctozepto | mgoddard: I meant kolla-cli from the slides | 13:03 |
mgoddard | yoctozepto: fire++ | 13:03 |
mgoddard | oh | 13:03 |
Fl1nt | like with the SAML2/OIDC patch for instance, but sometimes I'm having a hard time writing in a way it doesn't sounds pedentic to the contributor :( | 13:03 |
mgoddard | yoctozepto: I think we still need to drop the project | 13:03 |
Fl1nt | ++ yoctozepto | 13:03 |
yoctozepto | mgoddard; yeah, it's been only deprecated I think | 13:03 |
yoctozepto | from Victoria precisely | 13:03 |
kevko | sorry, was smoking ..going to read :) | 13:03 |
yoctozepto | so it should be gone entirely with ussuri going away | 13:03 |
yoctozepto | something for the new ptl to consider ensuring happen | 13:04 |
mgoddard | kevko: oh, sorry - no smokers! :p | 13:04 |
yoctozepto | mnasiadka: ^ | 13:04 |
mnasiadka | drop kolla-cli... huh, well, whatever - I'll add it to my list. | 13:05 |
mgoddard | mnasiadka: PTL pro-tip: delegate_to | 13:05 |
mnasiadka | mgoddard: I didn't say I'm going to do it, just add to list and find a volunteer ;-) | 13:06 |
mgoddard | that's the spirit | 13:06 |
yoctozepto | delegate but ensure it's done ;-) | 13:08 |
kevko | hmm, so i will do reviews, but I'm glad I broke the silence all he day with my question :D | 13:14 |
yoctozepto | :D | 13:14 |
yoctozepto | if you have any comments regarding the slides, they are welcome too | 13:14 |
yoctozepto | I like what we have done there | 13:15 |
yoctozepto | I was delivering that presentation and its followup | 13:15 |
yoctozepto | those were the times... | 13:15 |
opendevreview | Merged openstack/kolla-ansible master: Add ability to retry image pulling https://review.opendev.org/c/openstack/kolla-ansible/+/803828 | 13:22 |
Fl1nt | Talking about reviews :D | 13:22 |
Fl1nt | Could someone had a look at: https://review.opendev.org/c/openstack/kolla-ansible/+/800080 | 13:22 |
opendevreview | Radosław Piliszek proposed openstack/kolla-ansible stable/wallaby: Add ability to retry image pulling https://review.opendev.org/c/openstack/kolla-ansible/+/805565 | 13:27 |
opendevreview | Radosław Piliszek proposed openstack/kolla-ansible stable/victoria: Add ability to retry image pulling https://review.opendev.org/c/openstack/kolla-ansible/+/805566 | 13:27 |
opendevreview | Radosław Piliszek proposed openstack/kolla-ansible stable/ussuri: Add ability to retry image pulling https://review.opendev.org/c/openstack/kolla-ansible/+/805567 | 13:27 |
opendevreview | Radosław Piliszek proposed openstack/kolla-ansible stable/ussuri: Add ability to retry image pulling https://review.opendev.org/c/openstack/kolla-ansible/+/805567 | 13:27 |
opendevreview | Maksim Malchuk proposed openstack/kolla master: Unify curl use (customisation options) https://review.opendev.org/c/openstack/kolla/+/805640 | 13:28 |
opendevreview | Radosław Piliszek proposed openstack/kolla-ansible stable/ussuri: Fix regression in image pull action v2 https://review.opendev.org/c/openstack/kolla-ansible/+/805602 | 13:37 |
opendevreview | Radosław Piliszek proposed openstack/kolla-ansible stable/ussuri: Add ability to retry image pulling https://review.opendev.org/c/openstack/kolla-ansible/+/805567 | 13:37 |
yoctozepto | mgoddard: ^ ansible quirks episode 78611 | 13:38 |
yoctozepto | had to fix ansible linter | 13:39 |
yoctozepto | checked it's all fine now | 13:39 |
yoctozepto | well, more like work around the linter | 13:39 |
yoctozepto | but you get the point | 13:39 |
opendevreview | Maksim Malchuk proposed openstack/kolla stable/victoria: Unify curl options https://review.opendev.org/c/openstack/kolla/+/805643 | 13:44 |
mnasiadka | So, we're using the linter just to work it around? :) | 13:45 |
opendevreview | Maksim Malchuk proposed openstack/kolla stable/ussuri: Unify curl options https://review.opendev.org/c/openstack/kolla/+/805644 | 13:46 |
yoctozepto | mnasiadka: if it has bugs... | 13:46 |
yoctozepto | I wonder how it worked outside of the block | 13:47 |
yoctozepto | as it should be hitting the very same code | 13:47 |
yoctozepto | MEH | 13:47 |
yoctozepto | :D | 13:47 |
yoctozepto | kill it with fire before it lays exceptions | 13:47 |
yoctozepto | ^ an old adage | 13:47 |
opendevreview | Gaël THEROND proposed openstack/kolla-ansible master: Add missing cloudkitty documentation. https://review.opendev.org/c/openstack/kolla-ansible/+/805650 | 14:07 |
opendevreview | Merged openstack/kolla master: ironic-neutron-agent: enable for Debuntu binary https://review.opendev.org/c/openstack/kolla/+/799611 | 14:22 |
opendevreview | Merged openstack/kolla master: Allow build debian/ubuntu-binary cloudkitty images https://review.opendev.org/c/openstack/kolla/+/797433 | 14:22 |
opendevreview | Merged openstack/kolla master: Config: Set default home to /var/lib/{user} https://review.opendev.org/c/openstack/kolla/+/802685 | 14:25 |
opendevreview | Gaël THEROND proposed openstack/kolla-ansible master: Add missing cloudkitty documentation. https://review.opendev.org/c/openstack/kolla-ansible/+/805650 | 14:51 |
opendevreview | Merged openstack/kolla-ansible stable/victoria: Add Neutron DHCP agent to OVN networking setup https://review.opendev.org/c/openstack/kolla-ansible/+/805561 | 14:55 |
opendevreview | Gaël THEROND proposed openstack/kolla-ansible master: Add missing cloudkitty documentation. https://review.opendev.org/c/openstack/kolla-ansible/+/805650 | 15:11 |
opendevreview | wu.chunyang proposed openstack/kolla-ansible master: Modernize the swift role https://review.opendev.org/c/openstack/kolla-ansible/+/797498 | 15:19 |
ignaziocassano | Hello All, I am new in kolla deplpyment. I read every week new sources images are available. I have just pulled and pushed new images (date is 31 hours ago) on my local registry then I run kolla-deply but on computes and controllers the images are stil 8 days ago old | 15:26 |
ignaziocassano | Must I run before kolla pull for computes and controllers ? | 15:27 |
ignaziocassano | Please, any help ? | 15:28 |
mnasiadka | ignaziocassano: If you're using the same tag for new images - you need to run `pull` | 15:31 |
ignaziocassano | mnasuadka: an then I must renu deploy ? | 15:32 |
ignaziocassano | mnasuadka: an then I must run deploy ? | 15:32 |
ignaziocassano | I tired it but redeploying with updated images, stop all my instances | 15:34 |
ignaziocassano | This is my procedure: pull and push new images on local registry. Pull new images from local registry to compute and controllers nodes. Deploy | 15:36 |
ignaziocassano | On an installation with the same openstack version (wallaby) it stops all runiing virtual machines) :-( | 15:37 |
mnasiadka | check libvirt log, that shouldn't be the case... | 15:39 |
kevko | yoctozepto: btw, have you some time to review proxysql thing ? :) | 15:57 |
kevko | or, do someone have a time to review ? | 15:57 |
*** rpittau is now known as rpittau|afk | 15:58 | |
ignaziocassano | mnasiadka: please, I would like to ask if my procedure is correct: 1 pulled new images on my local registry 2 pushed new images on my local registry 3 pushed new images from my local registry to computes and controllers nodes (kolla-ansuble pull --limit control,compute. 4 kolla-ansible deploy | 16:01 |
mnasiadka | ignaziocassano: that's correct, if your vms are getting stopped - there is some problem somewhere, you should check most probably in libvirt/nova-libvirt log | 16:04 |
ignaziocassano | I will check it. Thanks | 16:04 |
ignaziocassano | mnasiadka: seems some errors when it redeploy libvirt container 2021-08-23 15:32:38.221+0000: 2288093: error : virNetSocketReadWire:1833 : End of file while reading data: Input/output error | 16:09 |
opendevreview | Gaël THEROND proposed openstack/kolla-ansible master: Add missing cloudkitty documentation. https://review.opendev.org/c/openstack/kolla-ansible/+/805650 | 16:11 |
ignaziocassano | mnasiadka: may be masakary ? During the update the following appears in masakari log:Exception caught: crmadmin command output unexpected host status.: Exception: crmadmin command output unexpected host status. | 16:20 |
ignaziocassano | Must I disable masakari before updating ? | 16:20 |
mnasiadka | ignaziocassano: might be, yoctozepto ^^ something for you | 16:20 |
kevko | ignaziocassano: haha, that's the bug which i fixed already | 16:21 |
kevko | ignaziocassano: what images do you use ? debian ? | 16:21 |
ignaziocassano | kevko: ubuntu source | 16:21 |
kevko | ignaziocassano: yeah, it's the same .. | 16:21 |
kevko | i mean ..same issue | 16:22 |
ignaziocassano | kevko: what must I do ? | 16:22 |
kevko | ignaziocassano: well, you posted here bug which is already fixed (but probably image is not available yet) | 16:22 |
ignaziocassano | kevko: sorry as I wrote before I am new in kolla. How can I check when the image withi your fix is available ? | 16:23 |
kevko | ignaziocassano: but it's only masakari fix .. i don't know if your problem is related .. (because from time to time i had some turned off instances during the upgrade - but not all) | 16:24 |
kevko | but probably it will be related | 16:25 |
kevko | you can switch images in configuration ..or build new images | 16:25 |
ignaziocassano | kevko: what exactly I must do for testing it ? | 16:26 |
ignaziocassano | kevko: Or can I try a workaround.....Can I disable masakary before updating ? | 16:27 |
kevko | ignaziocassano: it will be best if you build images locally and upload to your docker repository | 16:27 |
kevko | ignaziocassano: stable/wallaby kolla repo should have that patches included | 16:27 |
kevko | ignaziocassano: btw, you can check this bug report https://bugs.launchpad.net/kolla/+bug/1934753 | 16:28 |
kevko | ignaziocassano: you can also stop masakari,hacluster containers ..and upgrade with --tag or use --skip-tags | 16:29 |
kevko | you can try and verify if it is masakari problem | 16:29 |
ignaziocassano | kevko: I will read the bug. Meanwhile next week I will try a workaround. I must disable the masakari segment befor updading o stopping the masakari containers ? | 16:30 |
ignaziocassano | kevko: many thanks. You wrote the answer while I was writing :-) | 16:30 |
kevko | well, as I said ..i don't know if it is masakari issue ..but yes I can imagine that masakari is reacting to lost libvirt process via processmonitor ..but it's buggy .. so it will die somewhere in the middle | 16:32 |
kevko | so, if you think it should be your issue .. 1. stopping masakari,hacluster containers on your nodes 2. Be sure that you are upgrading without hacluster,masakari tags/roles | 16:33 |
kevko | you can also build your images from latest stable/wallaby branch | 16:34 |
ignaziocassano | kevko: thanks. I will try it, but when I try to upgrate only masakari it should works, in a second step | 16:41 |
kevko | yeah, but if you will not replace images ..you will have still buggy/not-functional masakari | 16:43 |
yoctozepto | kevko, ignaziocassano: that issue was not affecting ubuntu, only debian, and only binary for some reason... | 16:45 |
yoctozepto | during update it's normal that services report other services going down | 16:46 |
kevko | yoctozepto: nope, ubuntu also | 16:46 |
kevko | yoctozepto: i think | 16:46 |
yoctozepto | kevko: nope, it worked | 16:46 |
yoctozepto | ignaziocassano: have you checked the logs of masakari-engine? has it actually done any action? | 16:46 |
kevko | yoctozepto: ah, yes, i was little bit blind ..output is different | 16:47 |
ignaziocassano | Exception caught: crmadmin command output unexpected host status.: Exception: crmadmin command output unexpected host status. 2021-08-23 17:35:46.582 7 WARNING masakarimonitors.hostmonitor.host_handler.handle_host [-] 'tst2-osctrl01' is unstable state on cluster.: Exception: crmadmin command output unexpected host status. | 16:47 |
ignaziocassano | this is during update | 16:47 |
yoctozepto | yeah, that's fine | 16:47 |
ignaziocassano | yoctozepto: that's fine ? Do you mean I met the bug ? | 16:48 |
yoctozepto | ignaziocassano: no, I mean the problem must be somewhere else | 16:49 |
kevko | btw, yoctozepto what is your upgrade procedure, are u upgrading in one batch or live-migrating instances somewhere else ..upgrade compute and live-migrate to upgraded node ...upgrade another ..etc | 16:50 |
kevko | because I also saw turned-off instances, but not all at once (during upgrade) | 16:50 |
ignaziocassano | No, I am updating without moving instances | 16:51 |
ignaziocassano | one step only | 16:51 |
ignaziocassano | I ha only two instances on the same node. | 16:52 |
ignaziocassano | But the issue happened also two months ago | 16:52 |
ignaziocassano | And if you remember another user wrote about this issue 2 months ago, if I remember well | 16:53 |
yoctozepto | kevko: if I don't need to update the kernel (so I don't need to reboot), then I simply roll out the containers updates starting with one node to check if it does not break the world | 16:53 |
yoctozepto | per flavour and then just roll out all the others | 16:53 |
yoctozepto | ignaziocassano: I've already asked you to check masakari-engine logs as that is the service that is taking actual actions | 16:54 |
yoctozepto | if it acted upon something, then we need to figure why | 16:54 |
yoctozepto | if it didn't, then we need to find another cause | 16:55 |
kevko | yoctozepto: so you never saw stopped instance after upgrade ? | 17:00 |
kevko | yoctozepto: btw, will you have time for proxysql ? so I can fix things at evening :D | 17:01 |
yoctozepto | kevko: no, never | 17:02 |
yoctozepto | this should not be happening | 17:02 |
yoctozepto | kevko: no, I am as sorry as you are even if not more :-( | 17:02 |
yoctozepto | I like the change but too many higher priorities now | 17:03 |
yoctozepto | I need my own team ;D | 17:03 |
opendevreview | Piotr Parczewski proposed openstack/kayobe master: Add OpenStack release name suffix to base_path https://review.opendev.org/c/openstack/kayobe/+/805696 | 19:57 |
opendevreview | Piotr Parczewski proposed openstack/kayobe master: Add OpenStack release name suffix to base_path https://review.opendev.org/c/openstack/kayobe/+/805696 | 20:07 |
-opendevstatus- NOTICE: The Gerrit service on review.opendev.org has been restarted for a patch version upgrade, resulting in a brief outage | 21:42 | |
*** gilou_ is now known as Gilou | 21:43 | |
*** yoctozepto4 is now known as yoctozepto | 23:37 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!