Tuesday, 2021-01-12

« Monday, 2021-01-11 Index Wednesday, 2021-01-13 »
*** brinzhang has joined #openstack-kolla00:13
openstackgerritMichal Arbet proposed openstack/kolla master: Add proxysql image  https://review.opendev.org/c/openstack/kolla/+/76938500:41
*** LinPeiWen has quit IRC01:51
*** LinPeiWen has joined #openstack-kolla02:03
*** kevko has quit IRC02:07
*** wuchunyang has joined #openstack-kolla02:45
*** skramaja has joined #openstack-kolla03:13
*** ricolin has quit IRC03:19
*** wuchunyang has quit IRC04:00
*** zzzeek has quit IRC04:06
*** zzzeek has joined #openstack-kolla04:07
*** ricolin_ has joined #openstack-kolla04:17
*** brinzhang has quit IRC05:29
*** brinzhang has joined #openstack-kolla05:29
*** vishalmanchanda has joined #openstack-kolla05:35
*** wuchunyang has joined #openstack-kolla05:35
*** cah_link has joined #openstack-kolla06:01
*** zzzeek has quit IRC07:02
*** ivan_lin has joined #openstack-kolla07:03
*** devfaz has joined #openstack-kolla07:04
*** zzzeek has joined #openstack-kolla07:07
*** bsanjeewa has joined #openstack-kolla07:20
*** nikparasyr has joined #openstack-kolla07:27
*** sorin-mihai__ has joined #openstack-kolla07:30
*** dave-mccowan has quit IRC07:30
*** hjensas has quit IRC07:59
*** rpittau|afk is now known as rpittau08:08
*** cah_link has quit IRC08:09
*** kevko has joined #openstack-kolla08:09
*** gfidente|afk is now known as gfidente08:13
*** andrewbonney has joined #openstack-kolla08:17
*** bsanjeewa has quit IRC08:20
*** bengates has joined #openstack-kolla08:20
openstackgerritMichal Arbet proposed openstack/kolla-ansible master: Edit services roles to support database sharding  https://review.opendev.org/c/openstack/kolla-ansible/+/77021608:26
*** hjensas has joined #openstack-kolla08:35
openstackgerritMichal Arbet proposed openstack/kolla-ansible master: Add proxysql support for database  https://review.opendev.org/c/openstack/kolla-ansible/+/77021508:45
openstackgerritMichal Arbet proposed openstack/kolla-ansible master: Edit services roles to support database sharding  https://review.opendev.org/c/openstack/kolla-ansible/+/77021608:45
*** bengates has quit IRC08:47
*** bengates has joined #openstack-kolla08:47
*** dougsz has joined #openstack-kolla08:49
*** cah_link has joined #openstack-kolla08:50
*** e0ne has joined #openstack-kolla08:53
*** cah_link has quit IRC08:57
*** cah_link has joined #openstack-kolla09:01
*** cah_link has quit IRC09:05
*** maharg101 has joined #openstack-kolla09:05
*** cah_link has joined #openstack-kolla09:07
kevkomorning ! :)09:20
kevkoi'm probably done with proxysql/mariadb services/sharding thing :) -> https://review.opendev.org/q/topic:%22proxysql%22+(status:open%20OR%20status:merged)09:21
hrwhttps://review.opendev.org/c/openstack/kolla/+/755222 anyone? yoctozepto?09:26
*** mgoddard has joined #openstack-kolla09:32
openstackgerritMark Goddard proposed openstack/kolla stable/victoria: CI: revert to public package mirrors after build  https://review.opendev.org/c/openstack/kolla/+/77008809:33
openstackgerritMark Goddard proposed openstack/kolla stable/victoria: DNM: testing mirror revert  https://review.opendev.org/c/openstack/kolla/+/77009109:33
hrwkevko: https://review.opendev.org/c/openstack/kolla/+/770217 https://review.opendev.org/c/openstack/kolla/+/770219 https://review.opendev.org/c/openstack/kolla/+/769385  https://review.opendev.org/c/openstack/kolla/+/769385 - decide on one please09:34
kevkothat was pushed by mistake ... but i abandoned it09:36
hrwkevko: I am going to -1 it with info why09:38
*** ricolin_ has quit IRC09:39
kevkoto which review ?09:39
hrwactive one09:39
kevkoonly one proxysql image is active and +1 from zuul09:39
*** cah_link1 has joined #openstack-kolla09:39
*** cah_link has quit IRC09:40
*** cah_link1 is now known as cah_link09:40
hrwkevko: aarch64 ci jobs have retry_limit again ;(09:40
kevkohrw: I usually draw from others docker images in tree :) , so, for me as a template was keepalived ..proxysql has aarch64 packages09:44
hrwkevko: not added in repos.yaml09:45
kevkoah, I see09:45
kevkow8, adding09:45
hrw;)09:47
kevkobtw, should I reorder entries in repos.yaml if I see that they are not alphabeticall correct  ?09:49
hrwI would not complain09:50
kevkoor leave it as it is and just add my change ?09:50
hrwsort them09:50
kevkook09:50
openstackgerritMichal Arbet proposed openstack/kolla master: Add proxysql image  https://review.opendev.org/c/openstack/kolla/+/76938509:52
kevkobtw, I am getting this in one of my review -> "This change depends on a change that failed to merge." ... how can i know what happened ? which dependent change failed to merge ?09:54
kevkocan't see any log09:57
hrwrebase?09:58
kevkohmm, going to try10:07
kevkois Depends-on used only when cross dependency ? or i should add depends-on on previous commit in chain ?10:08
kevkoand could you help me with test job to zuul ? i would like to test switch to proxysql10:09
*** wuchunyang has quit IRC10:17
*** Fl1nt has joined #openstack-kolla10:20
Fl1ntHi everyone!10:20
Fl1ntI hope you all had wonderful holydays!10:20
openstackgerritMichal Arbet proposed openstack/kolla-ansible master: Add proxysql support for database  https://review.opendev.org/c/openstack/kolla-ansible/+/77021510:21
openstackgerritMichal Arbet proposed openstack/kolla-ansible master: Edit services roles to support database sharding  https://review.opendev.org/c/openstack/kolla-ansible/+/77021610:21
*** iniazi_ has joined #openstack-kolla10:29
*** iniazi has quit IRC10:31
hrwkevko: only on cross.10:39
hrwkevko: if you need depend in same project then chain10:39
kevkohmm, so, i don't really understand why it fails to merge :(10:39
kevkoif I rebase to upstream master it is up to date :/10:40
*** hjensas is now known as hjensas|afk10:51
openstackgerritMichal Arbet proposed openstack/kolla master: Add support for checking more daemons  https://review.opendev.org/c/openstack/kolla/+/76578110:55
openstackgerritMichal Arbet proposed openstack/kolla master: Add proxysql image  https://review.opendev.org/c/openstack/kolla/+/76938510:56
*** emccormick has quit IRC11:06
*** emccormick has joined #openstack-kolla11:06
*** wuchunyang has joined #openstack-kolla11:06
*** rpittau has quit IRC11:06
*** rpittau has joined #openstack-kolla11:07
openstackgerritMark Goddard proposed openstack/kolla stable/ussuri: Remove footer block from intermediate images  https://review.opendev.org/c/openstack/kolla/+/77008611:10
openstackgerritMark Goddard proposed openstack/kolla stable/ussuri: CI: revert to public package mirrors after build  https://review.opendev.org/c/openstack/kolla/+/77008911:11
openstackgerritMark Goddard proposed openstack/kolla stable/ussuri: CI: revert to public package mirrors after build  https://review.opendev.org/c/openstack/kolla/+/77008911:12
openstackgerritMark Goddard proposed openstack/kolla stable/ussuri: DNM: testing mirror revert  https://review.opendev.org/c/openstack/kolla/+/77009211:13
openstackgerritMark Goddard proposed openstack/kolla stable/train: CI: revert to public package mirrors after build  https://review.opendev.org/c/openstack/kolla/+/77009011:15
openstackgerritMark Goddard proposed openstack/kolla stable/train: DNM: testing mirror revert  https://review.opendev.org/c/openstack/kolla/+/77009311:15
*** e0ne has quit IRC11:16
*** structz has joined #openstack-kolla11:18
openstackgerritMichal Arbet proposed openstack/kolla-ansible master: Add proxysql support for database  https://review.opendev.org/c/openstack/kolla-ansible/+/77021511:21
openstackgerritMichal Arbet proposed openstack/kolla-ansible master: Edit services roles to support database sharding  https://review.opendev.org/c/openstack/kolla-ansible/+/77021611:21
*** LinPeiWen has quit IRC11:29
*** ivan_lin has quit IRC11:30
*** e0ne has joined #openstack-kolla11:39
*** e0ne has quit IRC11:44
*** Luzi has joined #openstack-kolla11:47
*** e0ne has joined #openstack-kolla11:47
*** e0ne has quit IRC11:54
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible stable/train: Remove the cpuset_cpus parameter for nova_libvirt container  https://review.opendev.org/c/openstack/kolla-ansible/+/77027512:00
*** ricolin_ has joined #openstack-kolla12:03
openstackgerritwu.chunyang proposed openstack/kolla-ansible master: octavia: support tenant management network  https://review.opendev.org/c/openstack/kolla-ansible/+/75558912:03
*** wuchunyang has quit IRC12:04
*** structz has quit IRC12:07
openstackgerritMichal Arbet proposed openstack/kolla-ansible master: Add proxysql support for database  https://review.opendev.org/c/openstack/kolla-ansible/+/77021512:08
openstackgerritMichal Arbet proposed openstack/kolla-ansible master: Edit services roles to support database sharding  https://review.opendev.org/c/openstack/kolla-ansible/+/77021612:08
*** cah_link1 has joined #openstack-kolla12:32
*** cah_link has quit IRC12:36
*** cah_link1 is now known as cah_link12:36
*** ricolin_ has quit IRC12:47
kevkohrw: succeded ..is it now OK ?12:48
openstackgerritPedro Henrique Pereira Martins proposed openstack/kolla-ansible master: Add support to OpenID Connect Authentication flow  https://review.opendev.org/c/openstack/kolla-ansible/+/69543212:51
openstackgerritPiotr Parczewski proposed openstack/kolla-ansible master: Add Prometheus 2.x deployment  https://review.opendev.org/c/openstack/kolla-ansible/+/76597313:19
Fl1ntteam, what's the reasoning behind adding proxysql to the roles? I mean, to my knowledge none of the openstack services support sharding. It seems like we won't really benefit any big improvement over using haproxy.13:24
Fl1ntMariaDB being natively Active/Active multi master and because you can set kolla-ansible to deploy your HA to be Active/Active (Keepalived + HAProxy) what kind of feature improvement do we gain from it?13:25
Fl1ntadditionally, that is going against CELL layout as you're supposed to get one DB per cell which is kind of a sharding already.13:28
openstackgerritMerged openstack/kolla-ansible master: Fix remove libvirt apparmor disabled profile  https://review.opendev.org/c/openstack/kolla-ansible/+/76888513:35
Fl1ntat least at nova level.13:36
hrwI have nothing against proxysql image in kolla (have to check license etc first).13:43
hrwwill it replace haproxy in k-a is a thing someone else has to decide13:43
*** wuchunyang has joined #openstack-kolla13:45
openstackgerritMerged openstack/kolla-ansible master: Register resources info for octavia upgrading  https://review.opendev.org/c/openstack/kolla-ansible/+/74600513:46
openstackgerritMerged openstack/kolla-ansible master: Use Docker healthchecks for cloudkitty services  https://review.opendev.org/c/openstack/kolla-ansible/+/76803913:46
openstackgerritMerged openstack/kolla-ansible master: Fix dpdk deploy failed  https://review.opendev.org/c/openstack/kolla-ansible/+/76851013:46
kevkoguys, let's say i introduced change (proxysql) which is changing the way how kolla-ansible deployment works by default depends on some values (enable_proxysql) .. how I can test in zuul that if original and new solution works both ?13:48
kevkosimply said , i need to test scenario with enable_proxysql: false (haproxy enabled) working , and reconfigure with enable_proxysql: true - working13:49
openstackgerritwu.chunyang proposed openstack/kolla-ansible master: octavia: support tenant management network  https://review.opendev.org/c/openstack/kolla-ansible/+/75558913:55
*** ricolin_ has joined #openstack-kolla13:58
*** ricolin_ has quit IRC13:58
*** ricolin has joined #openstack-kolla13:59
yoctozeptokevko: well, the easiest would be to base on one (or more) upgrade jobs13:59
hrwthe more I look at oslo_config the less I understand it14:00
kevkoyoctozepto: could you please refer me to some example upgrade job which i can retrace ?14:01
yoctozeptohrw: in what sense?14:03
hrwyoctozepto: it is easier to add cli option than getting it read entry from config file14:03
Fl1nthrw, I'm not against proxysql, tbh, it's just a question about does it worth it to be integrated if it doesn't bring any value?14:04
hrwoslo_config.cfg.NoSuchOptError: no such option allowed_to_fail2 in group [DEFAULT]14:04
hrw[DEFAULT]14:04
hrwallowed_to_fail2 = test14:04
yoctozeptokevko: wdym? logs? or path to scripts?14:04
yoctozeptohrw: odd, it usually just works with both sources :S14:05
yoctozeptoFl1nt: proxysql is much smarter about down sql backends14:05
yoctozeptowe can get rid of clumsy haproxy checks14:06
yoctozeptothat is one benefit14:06
hrwyoctozepto: "--allowed-to-fail 'test test2'" works ;P14:07
*** kevko has quit IRC14:08
yoctozeptohrw: well, there is the "2" in one of them14:09
hrwok. got it working14:10
yoctozepto:-)14:14
Fl1ntyoctozepto, define much... we have few of them, they're just natively support sharding and L7 ruling, which isn't supported on HAProxy but isn't honestly required anywhere else than on Application traffic.14:17
Fl1ntEven with a multi region, multi continent, multi cell structure our study on proxysql vs haproxy didn't clearly set a performance or operational advantage ^^ hence why I talked about value added14:17
yoctozeptoFl1nt: that's interesting; performance should not really be the main concern in here but I thought our haproxy check for sql backend being up leaves much to desire compared to it being native in proxysql14:18
Fl1nton this specific topic, of course we could improve the MariaDB Cluster check a bit, but all in all, replacing a simple working and efficient process by adding yet another layer is just freaking me out as I know from experience what staking layers is producing ^^14:20
Fl1ntask Ubisoft about that for instance :p14:20
Fl1ntall in all, adding a role that is optional isn't bad in it ^^14:21
Fl1ntand can even be something useful for one specific kolla ops use case, but it need to be maintained, not to be the defacto default option and doesn't really (to my opinion, but I'll be glad to be proved wrong) a lot of value added.14:22
yoctozeptoFl1nt: it is not a layer; just a smarter proxy to sql14:23
yoctozeptohaproxy is well-suited for proxying http14:23
yoctozeptoand just fine for general tcp14:24
Fl1ntIt is adding a layer as you put it behind the HAProxy whatever happens :D14:25
Fl1ntExemple of additional complexity for proxySQL: TLS management.14:25
yoctozeptoFl1nt: it was supposed to be *instead* of haproxy there (kevko please correct me)14:25
Fl1ntit can't14:25
yoctozeptoFl1nt: haproxy can't tls for sql ;-)14:25
yoctozeptoFl1nt: what can't what? :D14:26
Fl1ntGood point for HAProxy TLS on MariaDB as it's TCP level.14:27
Fl1ntif you get ride of HAProxy, how will you balance HTTP APIs services ?14:28
Fl1nthow will you balance SQL requests on ProxySQL node 01 and 02/0x on a HA/HA install?14:30
Fl1ntyou could use Keepalived and DNS Round robin but... meh, it's not really efficient.14:30
hrw 5 files changed, 53 insertions(+), 21 deletions(-)14:30
hrwallowed to fail images are coming ;D14:30
Fl1nt??14:30
Fl1ntwoot?14:31
hrwFl1nt: one of features for Wallaby release14:31
*** Luzi has quit IRC14:31
Fl1ntoh ok ^^ cool :D14:31
hrwFl1nt: we do not care if images x, y, z fail on CI14:32
yoctozeptohrw: gooood14:32
Fl1ntaaah nice ^^14:32
yoctozeptoFl1nt: not get rid of haproxy, just not use it for sql14:32
yoctozeptoFl1nt: it is still there, just not as a layer for proxysql14:32
yoctozeptothat was my point :-)14:33
Fl1ntthen you're trading a fully redundant and HA solution to a HA using only active/passive.14:33
Fl1ntexcept if someone managed to introduce active/active architecture on ProxySQL tbh I didn't checked lately ^^14:34
Fl1ntHere is another argument, most of time, what's overloaded isn't your API database but rather your messaging queue cluster or your CELL database.14:34
yoctozeptoFl1nt: how so? proxysql is active/active as long as you have multiple galera replicas (as we do now)14:35
Fl1ntkevko, I'm curious about your use case with proxySQL.14:35
yoctozeptoyeah, the performance argument is not that strong14:35
yoctozeptoI am buying it for saner backend failure discovery14:35
mnasiadkaFl1nt: you don't have to use it, but it's nice that K-A users will have a choice after this functionality is merged :)14:35
yoctozeptorather than anything else14:36
mnasiadkayoctozepto: anything better than current xinetd + clustercheck script - I'm in :)14:36
Fl1ntyoctozepto, nooo I talk about frontend side of things, yes your proxySQL service is able to balance/shard agains SQL instances, but if you put multiple proxySQL instance in front of your cluster then you get either a active/passive using keepalived or a active/active using keepalived with multiple IPs round robin with DNS and the last option is to get only one ProxySQL instance so SPOF.14:37
yoctozeptomnasiadka: indeed!14:37
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: WIP: implement "allowed to fail" images  https://review.opendev.org/c/openstack/kolla/+/76580714:38
Fl1ntmnasiadka, I'm absolutely 100% with you on having options, however, I already see how hard the team is working to maintain kolla/k-a/kayobe, adding roles like that without discussing it could just lead to more and more dead code on the repo.14:38
yoctozeptoFl1nt: we already have haproxy and a single VIP; forget that sharding and focus on replicas; maybe I am missing something here but I feel like you went down wrong rabbit hole14:38
Fl1ntWe install our cluster using DUAL HA VIP, so I may be biased yes.14:39
mnasiadkaFl1nt: if it will be tested in CI properly, then it's not really dead14:39
Fl1ntwhat about replicas @yoctozepto ?14:39
hrwplease take a look at allowed-to-fail images as it may need some improvements14:40
Fl1ntyou already get replicas as we deploy a multinode multi master mariadb cluster. Or I don't get what you're pointing out?14:41
yoctozeptoFl1nt: I think I know what you wanted to say - that we don't use all proxysql at once14:41
Fl1ntyes14:41
yoctozeptobut that's not getting worse from what we have14:41
Fl1ntIn many enterprise installation case it is.14:41
Fl1ntas we deployed everything using keepalived multi VIP and so address that passive nodes making nothing useful except waiting for someone else to die ^^14:42
Fl1ntonce again, I'm not against using proxySQL, but I'd like to hear from kevko about it's use case as for now, proxySQL is only really usefull when you're able to use it at application level to spread your requests against sharded DBs.14:44
Fl1ntwhich isn't going to happen soon on Openstack services (to my limited knowledge once again).14:44
*** TrevorV has joined #openstack-kolla14:52
*** sorin-mihai__ has quit IRC14:59
*** wuchunyang has quit IRC15:05
*** kevko has joined #openstack-kolla15:10
*** skramaja has quit IRC15:49
*** sorin-mihai__ has joined #openstack-kolla15:50
openstackgerritMark Goddard proposed openstack/kolla stable/train: Remove footer block from intermediate images  https://review.opendev.org/c/openstack/kolla/+/77008715:53
*** Fl1nt has quit IRC15:57
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: WIP: implement "allowed to fail" images  https://review.opendev.org/c/openstack/kolla/+/76580716:11
*** e0ne has joined #openstack-kolla16:18
openstackgerritMark Goddard proposed openstack/kolla master: Add a storm-base image  https://review.opendev.org/c/openstack/kolla/+/77044516:21
*** openstackgerrit has quit IRC16:55
*** e0ne has quit IRC17:12
*** sorin-mihai_ has joined #openstack-kolla17:14
*** sorin-mihai__ has quit IRC17:16
*** maharg101 has quit IRC17:27
*** dougsz has quit IRC17:31
*** rpittau is now known as rpittau|afk17:32
*** e0ne has joined #openstack-kolla17:33
*** e0ne has quit IRC17:35
hrwmeh.17:42
hrwmonasca-thresh failed to fail when I wanted it to fail17:42
*** sorin-mihai has joined #openstack-kolla17:52
*** sorin-mihai_ has quit IRC17:53
*** sorin-mihai_ has joined #openstack-kolla17:57
*** sorin-mihai has quit IRC18:01
yoctozeptohrw: sounds like an unfulfilled masochistic desire the way you put it18:10
*** nikparasyr has left #openstack-kolla18:13
*** cah_link has quit IRC18:19
*** gfidente is now known as gfidente|afk18:19
*** bengates has quit IRC18:48
*** ianw_pto is now known as ianw18:58
*** andrewbonney has quit IRC19:07
*** maharg101 has joined #openstack-kolla19:23
*** maharg101 has quit IRC19:28
*** e0ne has joined #openstack-kolla19:35
*** e0ne has quit IRC19:39
kevkosomeone still here ? :)19:43
kevkoi have weird behaviour on zuul :/19:44
*** sorin-mihai_ has quit IRC19:50
*** sorin-mihai_ has joined #openstack-kolla19:50
*** e0ne has joined #openstack-kolla20:05
*** e0ne has quit IRC20:18
*** e0ne has joined #openstack-kolla20:19
*** e0ne has quit IRC20:34
*** e0ne_ has joined #openstack-kolla20:34
*** e0ne_ has quit IRC20:40
*** vishalmanchanda has quit IRC20:49
*** sorin-mihai_ has quit IRC20:50
*** sorin-mihai has joined #openstack-kolla20:50
*** TrevorV has quit IRC20:56
*** openstackgerrit has joined #openstack-kolla21:00
openstackgerritMichal Arbet proposed openstack/kolla-ansible master: Refactor mariadb to support multiple separate shards  https://review.opendev.org/c/openstack/kolla-ansible/+/76695221:00
openstackgerritMichal Arbet proposed openstack/kolla-ansible master: Add proxysql support for database  https://review.opendev.org/c/openstack/kolla-ansible/+/77021521:00
openstackgerritMichal Arbet proposed openstack/kolla-ansible master: Edit services roles to support database sharding  https://review.opendev.org/c/openstack/kolla-ansible/+/77021621:00
*** sorin-mihai_ has joined #openstack-kolla21:14
*** hjensas|afk is now known as hjensas21:15
*** sorin-mihai has quit IRC21:16
*** sorin-mihai has joined #openstack-kolla21:17
*** sorin-mihai_ has quit IRC21:19
adebergI don't see how I could re-use that proxysql container for a custom service. A bit like the custom services patch that is available for HAproxy.... looks like it's pretty much thought with sharding only in mind. And yes Fl1nt, that's for my wan keystone-dedicated cluster before you ask21:22
*** maharg101 has joined #openstack-kolla21:24
*** maharg101 has quit IRC21:29
*** sorin-mihai_ has joined #openstack-kolla21:51
*** sorin-mihai has quit IRC21:54
*** sorin-mihai_ has quit IRC22:48
*** sorin-mihai_ has joined #openstack-kolla22:49
*** kevko has quit IRC23:22
*** kevko has joined #openstack-kolla23:25
*** sorin-mihai_ has quit IRC23:37
« Monday, 2021-01-11 Index Wednesday, 2021-01-13 »

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!