openstackgerrit | Michal Arbet proposed openstack/kolla-ansible master: Refactor mariadb to support shards https://review.opendev.org/c/openstack/kolla-ansible/+/766952 | 00:34 |
---|---|---|
openstackgerrit | Michal Arbet proposed openstack/kolla-ansible master: Rename role haproxy to loadbalancer https://review.opendev.org/c/openstack/kolla-ansible/+/770618 | 00:34 |
openstackgerrit | Michal Arbet proposed openstack/kolla-ansible master: Add proxysql support for database https://review.opendev.org/c/openstack/kolla-ansible/+/770215 | 00:34 |
openstackgerrit | Michal Arbet proposed openstack/kolla-ansible master: Edit services roles to support database sharding https://review.opendev.org/c/openstack/kolla-ansible/+/770216 | 00:34 |
openstackgerrit | Michal Arbet proposed openstack/kolla-ansible master: [CI] Enable proxysql test https://review.opendev.org/c/openstack/kolla-ansible/+/770621 | 00:34 |
*** LinPeiWen84 has joined #openstack-kolla | 00:42 | |
*** kevko has quit IRC | 00:53 | |
*** LinPeiWen84 has quit IRC | 01:02 | |
*** LinPeiWen20 has joined #openstack-kolla | 01:04 | |
*** kevko has joined #openstack-kolla | 01:05 | |
*** kevko has quit IRC | 01:10 | |
*** rouk has joined #openstack-kolla | 02:35 | |
*** ricolin has joined #openstack-kolla | 02:45 | |
*** zzzeek has quit IRC | 03:04 | |
*** zzzeek has joined #openstack-kolla | 03:05 | |
*** bsanjeewa has joined #openstack-kolla | 03:29 | |
*** bsanjeewa_ has joined #openstack-kolla | 03:32 | |
*** bsanjeewa has quit IRC | 03:32 | |
*** kevko has joined #openstack-kolla | 03:43 | |
*** kevko has quit IRC | 03:47 | |
*** bsanjeewa_ has quit IRC | 04:15 | |
*** bsanjeewa has joined #openstack-kolla | 04:16 | |
*** bsanjeewa_ has joined #openstack-kolla | 05:03 | |
*** LinPeiWen20 has quit IRC | 05:06 | |
*** bsanjeewa has quit IRC | 05:07 | |
*** LinPeiWen15 has joined #openstack-kolla | 05:13 | |
*** ricolin_ has joined #openstack-kolla | 05:47 | |
*** ricolin has quit IRC | 05:47 | |
*** vishalmanchanda has joined #openstack-kolla | 05:52 | |
*** cah_link has joined #openstack-kolla | 06:20 | |
*** skramaja has joined #openstack-kolla | 06:45 | |
*** luksky has joined #openstack-kolla | 07:23 | |
*** jonaspaulo has joined #openstack-kolla | 07:32 | |
*** bsanjeewa_ has quit IRC | 07:40 | |
*** kevko has joined #openstack-kolla | 07:46 | |
*** andrewbonney has joined #openstack-kolla | 08:09 | |
*** kevko has quit IRC | 08:11 | |
*** maharg101 has joined #openstack-kolla | 08:13 | |
*** kevko has joined #openstack-kolla | 08:15 | |
*** rpittau|afk is now known as rpittau | 08:17 | |
*** bengates has joined #openstack-kolla | 08:22 | |
*** bengates has quit IRC | 08:42 | |
*** bengates has joined #openstack-kolla | 08:42 | |
*** dougsz has joined #openstack-kolla | 08:52 | |
kevko | morning :) | 08:57 |
mgoddard | morning | 09:08 |
openstackgerrit | Michal Arbet proposed openstack/kolla-ansible master: Add proxysql support for database https://review.opendev.org/c/openstack/kolla-ansible/+/770215 | 09:10 |
openstackgerrit | Michal Arbet proposed openstack/kolla-ansible master: Edit services roles to support database sharding https://review.opendev.org/c/openstack/kolla-ansible/+/770216 | 09:10 |
openstackgerrit | Michal Arbet proposed openstack/kolla-ansible master: [CI] Enable proxysql test https://review.opendev.org/c/openstack/kolla-ansible/+/770621 | 09:10 |
kevko | mgoddard: wondering if you can also do some review for me :) | 09:10 |
kevko | mgoddard: if I am correct, you was writing support DB for nova-cells .. | 09:10 |
kevko | mgoddard: and my patches will solve also this thing :) | 09:11 |
*** e0ne has joined #openstack-kolla | 09:11 | |
kevko | i have all tested and passing zuul | 09:11 |
kevko | also upgrade from previous version | 09:11 |
mgoddard | kevko: dougsz and I worked on cells previously. There is a patch for separate DBs somewhere | 09:12 |
*** luksky has quit IRC | 09:12 | |
*** luksky has joined #openstack-kolla | 09:13 | |
kevko | mgoddard: well, if you mean this -> https://review.opendev.org/c/openstack/kolla-ansible/+/619756 | 09:13 |
mgoddard | kevko: I will get around to looking at your patches | 09:13 |
mgoddard | that's the one | 09:13 |
mgoddard | and this https://review.opendev.org/c/openstack/kolla-ansible/+/619757/ | 09:14 |
mgoddard | kevko: it stalled, but we may pick it up again this year | 09:15 |
kevko | mgoddard: well, I proposed proxysql support and database sharding ...so defining ONE parameter in all.yml (shard id) will configure ONE VIP:port for ALL databases | 09:15 |
kevko | so it is usable for every service ...and it is also usefull for cells ... | 09:15 |
kevko | and not listening on different ports ... | 09:16 |
kevko | check it and leave some questions, comments ... I will explain everything what you want | 09:16 |
kevko | (i've not added it for cells for now , because i wasn't sure ... but basically you just add nova_cell_database_shard_id to cells vars ..and that's it ) | 09:17 |
mgoddard | kevko: there was a proposal to support multiple cell conductors on a single host. I suppose they could share a DB shard | 09:25 |
kevko | yes, of course ... | 09:25 |
kevko | i just wanted to say that with my patches you can direct database where you want ..and you still can have single point of connection VIP:3306 | 09:26 |
kevko | and no need to define additional group for it, install mariadb cluster manually ...etc etc..everything is automatic | 09:27 |
*** bsanjeewa has joined #openstack-kolla | 09:28 | |
mgoddard | kevko: some may say a single point of connection is a bottleneck for cells :) | 09:30 |
mgoddard | that's a different question though. I don't remember if we looked at multiple load balancers or not | 09:30 |
kevko | hh :D well, now is everything via haproxy :D | 09:31 |
*** brinzhang_ has quit IRC | 09:34 | |
kevko | just check it please .. i've spent big amount of time with implement and test ... and I think it is in very good shape now | 09:40 |
hrw | morning | 09:40 |
hrw | today I started developing kolla in VM instance running on kolla build/deployed cloud | 09:41 |
kevko | morning | 09:43 |
kevko | and ? :) | 09:43 |
hrw | s/^/NOTE(hrw): / | 09:43 |
hrw | better? | 09:43 |
*** bengates has quit IRC | 09:45 | |
*** bengates has joined #openstack-kolla | 09:46 | |
mgoddard | kevko: are all patches under the proxysql topic/ | 09:49 |
mgoddard | https://review.opendev.org/q/topic:%22proxysql%22+(status:open%20OR%20status:merged) | 09:49 |
*** brinzhang has joined #openstack-kolla | 09:52 | |
kevko | click better hash tag | 09:56 |
kevko | https://review.opendev.org/q/hashtag:%22proxysql%22+(status:open%20OR%20status:merged) | 09:56 |
kevko | it is one more patch :) https://review.opendev.org/c/openstack/kolla/+/765781 | 09:56 |
kevko | better use hashtag | 09:56 |
kevko | mgoddard: thanks | 09:56 |
mgoddard | #ok | 09:57 |
*** gfidente|afk is now known as gfidente | 10:03 | |
openstackgerrit | Merged openstack/kolla master: [docs] Add link for project's release notes https://review.opendev.org/c/openstack/kolla/+/771483 | 10:11 |
openstackgerrit | Merged openstack/kolla-ansible master: [docs] Add link for project's release notes https://review.opendev.org/c/openstack/kolla-ansible/+/771272 | 10:12 |
*** rpittau is now known as rpittau|bbl | 10:20 | |
*** e0ne has quit IRC | 10:21 | |
openstackgerrit | Radosław Piliszek proposed openstack/kolla stable/train: Remove footer block from intermediate images https://review.opendev.org/c/openstack/kolla/+/770087 | 10:22 |
openstackgerrit | Radosław Piliszek proposed openstack/kolla stable/train: CI: revert to public package mirrors after build https://review.opendev.org/c/openstack/kolla/+/770090 | 10:22 |
openstackgerrit | Radosław Piliszek proposed openstack/kolla stable/train: CI: revert to public package mirrors after build https://review.opendev.org/c/openstack/kolla/+/770090 | 10:24 |
openstackgerrit | Radosław Piliszek proposed openstack/kolla stable/train: Add a storm-base image https://review.opendev.org/c/openstack/kolla/+/770580 | 10:26 |
*** LinPeiWen15 has quit IRC | 10:31 | |
*** priteau has joined #openstack-kolla | 10:32 | |
*** e0ne has joined #openstack-kolla | 10:34 | |
*** shyamb has joined #openstack-kolla | 10:34 | |
*** e0ne has quit IRC | 10:34 | |
*** shyamb has quit IRC | 10:42 | |
*** shyamb has joined #openstack-kolla | 10:51 | |
*** shyamb has quit IRC | 11:23 | |
*** rpittau|bbl is now known as rpittau | 11:24 | |
hrw | ok. victoria/ussuri/train builds of debian/ubuntu source on aarch64 started. will check logs later and send fixes | 11:25 |
*** LinPeiWen52 has joined #openstack-kolla | 11:42 | |
openstackgerrit | ZhouHeng proposed openstack/kolla-ansible master: Replace db-sock with db-nb-sock in template ovn-nb-db https://review.opendev.org/c/openstack/kolla-ansible/+/771594 | 11:44 |
openstackgerrit | ZhouHeng proposed openstack/kolla-ansible master: Replace db-sock with db-sb-sock in template ovn-sb-db https://review.opendev.org/c/openstack/kolla-ansible/+/771617 | 11:52 |
openstackgerrit | Radosław Piliszek proposed openstack/kolla stable/train: CI: Use Docker proxy for pulls https://review.opendev.org/c/openstack/kolla/+/770975 | 11:53 |
openstackgerrit | Radosław Piliszek proposed openstack/kolla stable/train: CI: Use Docker proxy for pulls + Ruby fixes https://review.opendev.org/c/openstack/kolla/+/770975 | 11:55 |
openstackgerrit | Radosław Piliszek proposed openstack/kolla stable/train: CI: Use Docker proxy for pulls + Ruby fixes https://review.opendev.org/c/openstack/kolla/+/770975 | 11:56 |
*** keuko_android has joined #openstack-kolla | 11:57 | |
openstackgerrit | Radosław Piliszek proposed openstack/kolla stable/train: Remove footer block from intermediate images https://review.opendev.org/c/openstack/kolla/+/770087 | 11:58 |
*** keuko_android has quit IRC | 11:58 | |
openstackgerrit | Pierre Riteau proposed openstack/kayobe master: Fix unit tests on macOS https://review.opendev.org/c/openstack/kayobe/+/771620 | 11:58 |
openstackgerrit | Radosław Piliszek proposed openstack/kolla stable/train: CI: revert to public package mirrors after build https://review.opendev.org/c/openstack/kolla/+/770090 | 11:58 |
openstackgerrit | Radosław Piliszek proposed openstack/kolla stable/train: Add a storm-base image https://review.opendev.org/c/openstack/kolla/+/770580 | 11:58 |
*** nikparasyr has joined #openstack-kolla | 11:59 | |
*** keuko_android has joined #openstack-kolla | 12:04 | |
*** bsanjeewa has quit IRC | 12:08 | |
*** keuko_android has quit IRC | 12:10 | |
*** zzzeek has quit IRC | 12:18 | |
*** kevko has quit IRC | 12:20 | |
*** zzzeek has joined #openstack-kolla | 12:21 | |
*** kevko has joined #openstack-kolla | 12:25 | |
openstackgerrit | Merged openstack/kolla stable/victoria: Remove footer block from intermediate images https://review.opendev.org/c/openstack/kolla/+/770085 | 12:31 |
*** keuko_android has joined #openstack-kolla | 12:32 | |
kevko | guys, why kolla using haproxy for monitor mysql *without_password* if there is mysql_clustercheck where password can be injected ? | 12:34 |
kevko | it was reason only when it was checking via haproxy's mysql-check | 12:35 |
yoctozepto | kevko: where is it without password? | 12:35 |
yoctozepto | possibly a bug then | 12:36 |
*** keuko_android has quit IRC | 12:37 | |
kevko | for example here -> ansible/roles/mariadb/handlers/main.yml | 12:37 |
kevko | or here -> ansible/roles/mariadb/defaults/main.yml | 12:38 |
kevko | I can fix it .. i would like to use same user in proxysql for monitor backends ...and i would like to use it also with password :) | 12:39 |
*** e0ne has joined #openstack-kolla | 12:40 | |
kevko | yoctozepto: what do you mean ? | 12:40 |
kevko | *think | 12:40 |
kevko | and also, i think there is no reason to have user for monitor called haproxy ..i think better is monitor or mysql_monitor or similar | 12:42 |
*** keuko_android has joined #openstack-kolla | 12:52 | |
*** keuko_android has quit IRC | 12:54 | |
mnasiadka | kevko: you need to remember that one thing is thinking something is better, second thing is changing an existing deployment that goes through upgrades and thinking what needs to be deprecated (especially thinking about deprecation & removal times as according to policies) :) | 12:55 |
kevko | mnasiadka: I always make changes so that they go through all the tests, even the upgrade tests, to make those changes make sense, and I always consult and change the code according to the comments | 12:57 |
mnasiadka | kevko: I'm not saying you don't, sometimes it's not worth going through all that to just change a username :) | 12:58 |
kevko | mnasiadka: agree, that's the reason why I am using haproxy as monitor user for my proxysql changes ... | 12:59 |
kevko | mnasiadka: one thing is not change name of user (i can live with it ) .. another thing is use monitor user *without password* .. | 13:00 |
mnasiadka | kevko: raise a bug and fix it :) | 13:00 |
kevko | mnasiadka: no problem :) | 13:01 |
*** dasp_ has joined #openstack-kolla | 13:31 | |
*** dasp has quit IRC | 13:34 | |
*** brinzhang has quit IRC | 13:37 | |
*** keuko_android has joined #openstack-kolla | 13:42 | |
openstackgerrit | Merged openstack/kolla stable/victoria: CI: revert to public package mirrors after build https://review.opendev.org/c/openstack/kolla/+/770088 | 13:43 |
*** brinzhang has joined #openstack-kolla | 13:49 | |
*** brinzhang has quit IRC | 13:51 | |
*** brinzhang has joined #openstack-kolla | 13:51 | |
*** keuko_android has quit IRC | 14:01 | |
kevko | is it meeting today ( i mean here in irc ? ) | 14:07 |
hrw | in 50 minutes | 14:10 |
hrw | heh. | 14:14 |
hrw | kolla is mostly I/O and VM instance can lack good I/O ;( | 14:14 |
*** vishalmanchanda has quit IRC | 14:15 | |
*** irclogbot_0 has quit IRC | 14:27 | |
*** irclogbot_2 has joined #openstack-kolla | 14:31 | |
*** gfidente has quit IRC | 14:31 | |
*** bengates has quit IRC | 14:40 | |
*** bengates has joined #openstack-kolla | 14:41 | |
*** rafaelweingartne has joined #openstack-kolla | 14:50 | |
*** vishalmanchanda has joined #openstack-kolla | 14:52 | |
hrw | mgoddard mnasiadka hrw egonzalez yoctozepto rafaelweingartne cosmicsound osmanlicilegi bbezak: meeting soon | 14:58 |
mnasiadka | have another meeting at this time, will try to keep up | 14:59 |
hrw | kevko: https://wiki.openstack.org/wiki/Meetings/Kolla has a list of nicks to ping before meeting ;D | 15:00 |
*** ricolin_ has quit IRC | 15:00 | |
hrw | #startmeeting kolla | 15:00 |
openstack | Meeting started Wed Jan 20 15:00:39 2021 UTC and is due to finish in 60 minutes. The chair is hrw. Information about MeetBot at http://wiki.debian.org/MeetBot. | 15:00 |
openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 15:00 |
hrw | #chair mgoddard | 15:00 |
*** openstack changes topic to " (Meeting topic: kolla)" | 15:00 | |
openstack | The meeting name has been set to 'kolla' | 15:00 |
openstack | Current chairs: hrw mgoddard | 15:00 |
hrw | long time since I chaired, hope that mgoddard wake up ;D | 15:01 |
hrw | #rollcall | 15:01 |
rafaelweingartne | \o | 15:01 |
mgoddard | o/ | 15:01 |
headphoneJames | o/ | 15:01 |
hrw | mgoddard: lead, I need coffee | 15:01 |
mgoddard | thanks for starting hrw, was in another | 15:01 |
hrw | no problem | 15:01 |
hrw | #agenda | 15:03 |
hrw | * Roll-call | 15:03 |
hrw | * Announcements | 15:03 |
hrw | * Review action items from the last meeting | 15:03 |
hrw | * CI status | 15:03 |
hrw | * Hot issues | 15:03 |
hrw | ** https://anonymoushash.vmbrasseur.com/2021/01/14/elasticsearch-and-kibana-are-now-business-risks | 15:03 |
hrw | ** Dockerhub pull rate limits https://etherpad.opendev.org/p/docker-pull-limits | 15:03 |
hrw | ** Cinder active/active https://bugs.launchpad.net/kolla-ansible/+bug/1904062 | 15:03 |
openstack | Launchpad bug 1904062 in kolla-ansible wallaby "external ceph cinder volume config breaks volumes on ussuri upgrade" [High,In progress] - Assigned to Michal Nasiadka (mnasiadka) | 15:03 |
hrw | * Wallaby release planning | 15:03 |
yoctozepto | \o/ | 15:03 |
*** jovial[m] has joined #openstack-kolla | 15:03 | |
hrw | forgot about #topic ;D | 15:03 |
yoctozepto | doh | 15:04 |
hrw | #topic announcements | 15:04 |
*** openstack changes topic to "announcements (Meeting topic: kolla)" | 15:04 | |
hrw | anyone? | 15:04 |
yoctozepto | want me to chair? | 15:04 |
hrw | #chair yoctozepto | 15:05 |
openstack | Current chairs: hrw mgoddard yoctozepto | 15:05 |
yoctozepto | leadership of USA is changing | 15:05 |
yoctozepto | right today | 15:05 |
hrw | not related to openstack | 15:05 |
yoctozepto | that's pretty important in terms of global politics | 15:05 |
yoctozepto | and openstack is tied to US | 15:05 |
yoctozepto | so close enough! :D | 15:05 |
mgoddard | politics-- | 15:05 |
yoctozepto | nobody likes it but it affects us | 15:06 |
yoctozepto | anyhow | 15:06 |
yoctozepto | 3 chairs | 15:06 |
yoctozepto | let's progress :-) | 15:06 |
yoctozepto | #topic Review action items from the last meeting | 15:06 |
*** openstack changes topic to "Review action items from the last meeting (Meeting topic: kolla)" | 15:06 | |
yoctozepto | mnasiadka configure registry mirror for kolla CI | 15:06 |
mgoddard | mnasiadka configure registry mirror for kolla CI | 15:06 |
yoctozepto | he did | 15:06 |
yoctozepto | and I backported | 15:07 |
mgoddard | you running this thing yoctozepto? | 15:07 |
yoctozepto | train pending last time I checked | 15:07 |
mgoddard | I'll go grab a beer then :D | 15:07 |
yoctozepto | mgoddard: a'ight, whatever pleases you | 15:07 |
yoctozepto | so all actions completed | 15:07 |
yoctozepto | and watching for results in Train | 15:08 |
yoctozepto | #topic CI status | 15:08 |
*** openstack changes topic to "CI status (Meeting topic: kolla)" | 15:08 | |
*** TrevorV has joined #openstack-kolla | 15:08 | |
yoctozepto | kolla GREEN | 15:08 |
yoctozepto | train pending but should pass soon | 15:08 |
yoctozepto | in fact let's check now | 15:08 |
yoctozepto | ERROR! Unexpected Exception, this is probably a bug: SQLAlchemy 1.3.0 or greater is required. | 15:09 |
yoctozepto | DUH | 15:09 |
hrw | I am running train/ussuri/victoria/master builds on aarch64 to check how we are there as I saw some issues. Expect some patches | 15:10 |
*** nikparasyr has left #openstack-kolla | 15:10 | |
yoctozepto | noted the above error in the whiteboard | 15:11 |
yoctozepto | unless you know right away what the culprit is, let's move onto k-a | 15:11 |
yoctozepto | oh yes, we stopped all the periodics it seems :-) | 15:12 |
yoctozepto | pretty hard to tell if stable branches are affected | 15:12 |
yoctozepto | anyhow, train surely is as that is k-a job that fails | 15:12 |
yoctozepto | (and it's k-a's part of it) | 15:12 |
yoctozepto | master GREEN for sure | 15:13 |
yoctozepto | other stable unknown | 15:13 |
yoctozepto | well, k victoria passed today | 15:14 |
yoctozepto | so that one should be k-a green | 15:14 |
yoctozepto | ussuri yesterday so let's assume too | 15:14 |
yoctozepto | now unto kayobe | 15:14 |
yoctozepto | master looks green | 15:15 |
yoctozepto | added link for victoria | 15:15 |
yoctozepto | and victoria looks green | 15:15 |
yoctozepto | ussuri too | 15:15 |
yoctozepto | train perhaps affected or soon to be affected | 15:15 |
yoctozepto | but so far looking green | 15:16 |
yoctozepto | stein green | 15:16 |
yoctozepto | others unknown; perhaps broken as indicated | 15:17 |
yoctozepto | please remove from the whiteboard if not cared about | 15:17 |
yoctozepto | a'ight that's it for the CI status | 15:17 |
yoctozepto | #topic https://anonymoushash.vmbrasseur.com/2021/01/14/elasticsearch-and-kibana-are-now-business-risks | 15:17 |
*** openstack changes topic to "https://anonymoushash.vmbrasseur.com/2021/01/14/elasticsearch-and-kibana-are-now-business-risks (Meeting topic: kolla)" | 15:17 | |
yoctozepto | yeah, that's a burning issue | 15:18 |
yoctozepto | we should perhaps pin the versions and issue a communication about it | 15:18 |
mgoddard | I think we're still on 6.x? | 15:18 |
* yoctozepto 's interest in Monasca and Venus intensifies | 15:18 | |
mgoddard | yeah, ELK 6.x on master | 15:19 |
yoctozepto | mgoddard: on master (Wallaby) too? I thought I have seen a move to es7 but yeah, I remember es6 on stable branches | 15:19 |
yoctozepto | mgoddard: ack, great | 15:19 |
mgoddard | possibly proposed, not merged | 15:19 |
yoctozepto | then communication that it's going to stay at this | 15:19 |
yoctozepto | could be | 15:19 |
hrw | there was es7 proposal | 15:19 |
mgoddard | I guess it should be an opendistro proposal | 15:20 |
mgoddard | whether it's a clean migration, I don't know | 15:20 |
yoctozepto | I can't find it in gerrit | 15:20 |
hrw | https://review.opendev.org/c/openstack/kolla-ansible/+/740986 | 15:22 |
yoctozepto | #link https://opendistro.github.io/for-elasticsearch/ | 15:22 |
hrw | "Upgrade service configuration for ELK 7" | 15:22 |
yoctozepto | yup, so that I have seen; thanks hrw | 15:22 |
yoctozepto | opendistro looks interesting | 15:23 |
mgoddard | it was pushed by AWS | 15:23 |
mgoddard | the kings of open source | 15:23 |
hrw | 'the kings' | 15:23 |
yoctozepto | fwiw, they include es and kibana, but not logstash; just what we want | 15:24 |
yoctozepto | and they provide the docker images | 15:24 |
mgoddard | commented on the kolla patch | 15:26 |
mgoddard | move on? | 15:26 |
yoctozepto | https://github.com/opendistro/for-elasticsearch-docs/issues/7 | 15:27 |
yoctozepto | #link https://github.com/opendistro/for-elasticsearch-docs/issues/7 | 15:27 |
yoctozepto | it needs a dump and a restore | 15:27 |
yoctozepto | or so it seems | 15:27 |
*** skramaja has quit IRC | 15:28 | |
yoctozepto | commented too | 15:29 |
yoctozepto | a'ight | 15:29 |
yoctozepto | let's move on | 15:29 |
yoctozepto | #topic Dockerhub pull rate limits https://etherpad.opendev.org/p/docker-pull-limits | 15:29 |
*** openstack changes topic to "Dockerhub pull rate limits https://etherpad.opendev.org/p/docker-pull-limits (Meeting topic: kolla)" | 15:29 | |
yoctozepto | how bad is it? | 15:29 |
yoctozepto | it seems it's not that bad after our actions? | 15:30 |
mgoddard | definitely better | 15:30 |
mgoddard | I think we can live with it for a while | 15:31 |
mgoddard | we have two options open, if/when we decide to use them | 15:32 |
yoctozepto | agreed | 15:33 |
yoctozepto | I'll remove this point from the agenda then | 15:33 |
yoctozepto | should we remove the es one too? | 15:34 |
yoctozepto | probably; because it's already handled in the relevant change | 15:34 |
yoctozepto | not much to say about it | 15:34 |
yoctozepto | let's move on | 15:35 |
yoctozepto | #topic Cinder active/active https://bugs.launchpad.net/kolla-ansible/+bug/1904062 | 15:35 |
*** openstack changes topic to "Cinder active/active https://bugs.launchpad.net/kolla-ansible/+bug/1904062 (Meeting topic: kolla)" | 15:35 | |
openstack | Launchpad bug 1904062 in kolla-ansible wallaby "external ceph cinder volume config breaks volumes on ussuri upgrade" [High,In progress] - Assigned to Michal Nasiadka (mnasiadka) | 15:35 |
yoctozepto | i wrote a summary in the whiteboard | 15:35 |
yoctozepto | but did not do much | 15:35 |
yoctozepto | if anything | 15:35 |
yoctozepto | any comments on it? | 15:36 |
*** rafaelweingartne has quit IRC | 15:36 | |
* mgoddard reading | 15:37 | |
* yoctozepto waiting for mgoddard to finish reading | 15:37 | |
yoctozepto | others are welcome to read as well | 15:37 |
*** bengates has quit IRC | 15:38 | |
mgoddard | seems like we need to understand those -ing states | 15:39 |
yoctozepto | yup, it was raised by mnasiadka | 15:39 |
*** bengates has joined #openstack-kolla | 15:39 | |
yoctozepto | who is not entirely available for a conversation atm | 15:39 |
mnasiadka | well, as long as we don't stop anything while a _working_ agent is doing something with a volume in -ing state, we should be fine. | 15:40 |
mgoddard | is that generally true? | 15:40 |
mgoddard | regardless of this active/passive discussion | 15:40 |
mnasiadka | well, let it do what it needs to do with the volume, and then we can change anything - if we stop it - we might stop it from doing something in the middle of it's job - and it might bring us to some incosistency | 15:41 |
mnasiadka | so my opinion is we need to check for those states during cinder role run, and if it's there - just fail | 15:42 |
yoctozepto | so it's regarding any upgrade whatsoever | 15:42 |
mnasiadka | seems so :) | 15:43 |
mnasiadka | of course we need to remap the volumes to a new agent after we change to cluster | 15:43 |
yoctozepto | they remap themselves | 15:43 |
mnasiadka | oh, cool | 15:43 |
mnasiadka | so just remove the old agent? | 15:43 |
yoctozepto | change config, restart, be happy | 15:43 |
yoctozepto | that's what worked in the CI | 15:43 |
mnasiadka | great | 15:44 |
mnasiadka | so just to be on the safe side - we need a check if any volumes are in -ing state | 15:44 |
yoctozepto | with only -ed states | 15:44 |
yoctozepto | well, we never did | 15:44 |
yoctozepto | it would be a nice addition but we can't help it | 15:44 |
yoctozepto | there could be -ing at any point in time | 15:44 |
yoctozepto | perhaps we should be rolling with disabling agents, waiting and restarting? | 15:45 |
mnasiadka | other option is to not add it, and wait until someone raises a bug (or not) | 15:45 |
yoctozepto | but still, it's orthogonal to active/passive | 15:45 |
mgoddard | is the concern that a volume in an -ing state won't get remapped? | 15:45 |
mnasiadka | rather a concern is that it would be (created|snapshotted|whatever) but with no update to the state in cinder database | 15:46 |
mnasiadka | so when you start a new agent, it does the same thing, and probably fails | 15:46 |
mgoddard | ok | 15:47 |
mgoddard | sounds like a wider issue | 15:47 |
yoctozepto | yup | 15:47 |
yoctozepto | ok, so for active/passive | 15:47 |
yoctozepto | we should perhaps recommend to keep one cinder-volume active? | 15:47 |
yoctozepto | or let users known the tradeoffs involved? | 15:47 |
mgoddard | prechecks would be a good way to tell people their config is broken/risky | 15:48 |
*** gfidente has joined #openstack-kolla | 15:49 | |
yoctozepto | I_KNOW_WHAT_I_AM_DOING_PLEASE_LET_ME-kind of thing? | 15:49 |
mgoddard | yes | 15:49 |
yoctozepto | and the only active-active that we really support is ceph, no? | 15:50 |
mgoddard | as far as I'm aware | 15:52 |
yoctozepto | well, that's awkward | 15:52 |
yoctozepto | but life is life | 15:52 |
yoctozepto | and then | 15:53 |
yoctozepto | should we impose the coordination now? | 15:53 |
yoctozepto | or only use prechecks here as well | 15:53 |
*** kevko_android has joined #openstack-kolla | 15:54 | |
mgoddard | prechecks | 15:54 |
yoctozepto | and now unto the backend_host | 15:56 |
yoctozepto | I had no time to check that scenario | 15:56 |
yoctozepto | it breaks out of the box | 15:56 |
yoctozepto | if you did not investigate it, then I guess there is not much to talk about | 15:56 |
yoctozepto | :-) | 15:56 |
yoctozepto | #topic Wallaby release planning | 15:57 |
*** openstack changes topic to "Wallaby release planning (Meeting topic: kolla)" | 15:57 | |
yoctozepto | gogo Wallaby | 15:57 |
*** e0ne has quit IRC | 15:58 | |
headphoneJames | the letsencrypt feature is ready for initial reviews | 15:58 |
headphoneJames | https://review.opendev.org/c/openstack/kolla-ansible/+/741340 | 15:58 |
headphoneJames | the test case is still absent, that will be my next bit | 15:59 |
yoctozepto | yay, lovely | 15:59 |
mgoddard | nice, thanks headphoneJames | 15:59 |
headphoneJames | not sure if there is precedence for test cases that mock the service | 16:00 |
hrw | allowed to fail will get requested cleanup and done | 16:00 |
yoctozepto | thanks for your summaries | 16:01 |
*** imcsk8 has quit IRC | 16:01 | |
yoctozepto | we have to finish | 16:01 |
yoctozepto | thanks all | 16:01 |
yoctozepto | #endmeeting | 16:01 |
*** openstack changes topic to "IRC meetings on Wednesdays @ 15:00 UTC - agenda @ https://goo.gl/OXB0DL | Whiteboard: https://bit.ly/2MM7mWF | IRC channel is *LOGGED* @ http://goo.gl/3mzZ7b" | 16:01 | |
openstack | Meeting ended Wed Jan 20 16:01:15 2021 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 16:01 |
openstack | Minutes: http://eavesdrop.openstack.org/meetings/kolla/2021/kolla.2021-01-20-15.00.html | 16:01 |
openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/kolla/2021/kolla.2021-01-20-15.00.txt | 16:01 |
openstack | Log: http://eavesdrop.openstack.org/meetings/kolla/2021/kolla.2021-01-20-15.00.log.html | 16:01 |
hrw | thanks | 16:01 |
openstackgerrit | Marcin Juszkiewicz proposed openstack/kolla master: implement "allowed to fail" images https://review.opendev.org/c/openstack/kolla/+/765807 | 16:01 |
mgoddard | thanks yoctozepto | 16:02 |
*** imcsk8 has joined #openstack-kolla | 16:04 | |
*** keuko_android has joined #openstack-kolla | 16:07 | |
*** keuko_android has quit IRC | 16:08 | |
*** keuko_android has joined #openstack-kolla | 16:10 | |
*** kevko_android has quit IRC | 16:11 | |
*** keuko_android has quit IRC | 16:15 | |
*** keuko_android has joined #openstack-kolla | 16:18 | |
*** keuko_android has quit IRC | 16:22 | |
*** ricolin_ has joined #openstack-kolla | 16:22 | |
*** Trevor_V has joined #openstack-kolla | 16:23 | |
*** TrevorV has quit IRC | 16:26 | |
*** bengates has quit IRC | 16:38 | |
*** bengates has joined #openstack-kolla | 16:39 | |
*** bengates has quit IRC | 16:44 | |
*** bengates has joined #openstack-kolla | 16:45 | |
*** keuko_android has joined #openstack-kolla | 16:52 | |
*** keuko_android has quit IRC | 16:58 | |
*** bengates has quit IRC | 17:22 | |
*** rpittau is now known as rpittau|afk | 17:26 | |
*** dougsz has quit IRC | 17:32 | |
*** e0ne has joined #openstack-kolla | 17:38 | |
*** ivan_lin has joined #openstack-kolla | 18:00 | |
*** LinPeiWen has quit IRC | 18:03 | |
*** keuko_android has joined #openstack-kolla | 18:11 | |
*** keuko_android has quit IRC | 18:15 | |
*** keuko_android has joined #openstack-kolla | 18:15 | |
*** ricolin_ has quit IRC | 18:15 | |
*** keuko_android has quit IRC | 18:19 | |
*** mchlumsky6 has quit IRC | 18:25 | |
*** mchlumsky has joined #openstack-kolla | 18:25 | |
openstackgerrit | Merged openstack/kayobe stable/victoria: Fix overcloud host image resolv workaround on CentOS 8.3 https://review.opendev.org/c/openstack/kayobe/+/771244 | 18:31 |
*** gfidente is now known as gfidente|afk | 18:36 | |
*** vishalmanchanda has quit IRC | 18:40 | |
*** e0ne has quit IRC | 18:45 | |
*** bengates has joined #openstack-kolla | 18:54 | |
*** bengates has quit IRC | 18:58 | |
*** keuko_android has joined #openstack-kolla | 19:05 | |
*** keuko_android has quit IRC | 19:07 | |
*** andrewbonney has quit IRC | 19:09 | |
*** bsanjeewa has joined #openstack-kolla | 19:11 | |
*** dcapone2004 has joined #openstack-kolla | 19:13 | |
*** cah_link has quit IRC | 19:22 | |
*** cah_link has joined #openstack-kolla | 19:22 | |
*** cah_link has quit IRC | 19:30 | |
*** e0ne has joined #openstack-kolla | 19:51 | |
*** maharg101 has quit IRC | 20:06 | |
*** bsanjeewa has quit IRC | 20:17 | |
*** dave-mccowan has joined #openstack-kolla | 20:40 | |
*** mchlumsky7 has joined #openstack-kolla | 20:40 | |
*** mchlumsky has quit IRC | 20:42 | |
*** mchlumsky7 is now known as mchlumsky | 20:42 | |
*** dcapone2004 has quit IRC | 21:11 | |
*** Trevor_V has quit IRC | 21:16 | |
*** sean-k-mooney has quit IRC | 21:16 | |
*** e0ne has quit IRC | 21:33 | |
*** jonaspaulo has quit IRC | 21:43 | |
*** maharg101 has joined #openstack-kolla | 22:03 | |
*** maharg101 has quit IRC | 22:07 | |
*** kevko has quit IRC | 22:55 | |
*** bodgix has joined #openstack-kolla | 23:14 | |
*** bodgix_ has quit IRC | 23:14 | |
*** brinzhang has quit IRC | 23:20 | |
*** brinzhang has joined #openstack-kolla | 23:29 | |
*** luksky has quit IRC | 23:33 | |
*** cah_link has joined #openstack-kolla | 23:56 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!