Thursday, 2019-08-15

*** iniazi_ has joined #openstack-kolla00:01
*** iniazi has quit IRC00:02
*** absubram has joined #openstack-kolla00:14
*** dave-mccowan has quit IRC00:22
*** iniazi has joined #openstack-kolla00:29
*** iniazi_ has quit IRC00:32
*** hongbin has joined #openstack-kolla00:58
*** spsurya has joined #openstack-kolla01:28
*** gregoryo has joined #openstack-kolla02:12
*** absubram has quit IRC02:12
*** BjoernT has joined #openstack-kolla02:36
*** BjoernT_ has joined #openstack-kolla02:43
*** BjoernT has quit IRC02:44
*** altlogbot_3 has quit IRC03:16
*** altlogbot_3 has joined #openstack-kolla03:17
*** hongbin has quit IRC03:27
openstackgerritMerged openstack/kolla-ansible master: Enable iscsid on cinder-backup hosts  https://review.opendev.org/67505203:34
*** gkadam has joined #openstack-kolla04:07
*** gkadam has quit IRC04:08
*** BjoernT_ has quit IRC04:12
*** dave-mccowan has joined #openstack-kolla04:26
*** dave-mccowan has quit IRC04:31
*** cah_link has joined #openstack-kolla05:35
*** mgoddard has quit IRC05:54
*** mgoddard has joined #openstack-kolla06:01
*** hongbin has joined #openstack-kolla06:28
*** hongbin has quit IRC06:32
yoctozeptomorning07:02
*** cloudnull has quit IRC07:08
*** cloudnull has joined #openstack-kolla07:08
*** zhangfei has joined #openstack-kolla07:18
*** gregoryo has quit IRC07:32
*** jaicaa has quit IRC07:36
openstackgerritMark Goddard proposed openstack/kolla stable/stein: There is no Java 8 in Debian 'buster'  https://review.opendev.org/67660707:39
openstackgerritMark Goddard proposed openstack/kolla stable/stein: There is no Java 8 in Debian 'buster' (part 2)  https://review.opendev.org/67661407:44
openstackgerritMark Goddard proposed openstack/kolla stable/stein: Delete default gnocchi & aodh wsgi conf in Ubuntu binary  https://review.opendev.org/67661607:47
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: Allow cinder coordination backend to be configured  https://review.opendev.org/67626107:48
*** ivve has joined #openstack-kolla07:53
openstackgerritScott Solkhon proposed openstack/kolla-ansible master: Decrypt Ceph keyring files using ansible-vault  https://review.opendev.org/67644107:56
*** jaicaa has joined #openstack-kolla07:58
openstackgerritMark Goddard proposed openstack/kolla stable/stein: Ensures monasca-thresh data and worker dirs exist and are empty on start up  https://review.opendev.org/67662108:04
*** jaicaa has quit IRC08:05
openstackgerritMark Goddard proposed openstack/kolla stable/rocky: Ensures monasca-thresh data and worker dirs exist and are empty on start up  https://review.opendev.org/67662208:05
*** jaicaa has joined #openstack-kolla08:06
openstackgerritMark Goddard proposed openstack/kolla-ansible stable/stein: Fix checking mongodb replication status  https://review.opendev.org/67662508:09
openstackgerritMark Goddard proposed openstack/kolla-ansible stable/rocky: Fix checking mongodb replication status  https://review.opendev.org/67662608:09
openstackgerritMark Goddard proposed openstack/kolla-ansible stable/stein: Fix the redis_connection_string for osprofiler and make it generic  https://review.opendev.org/67662708:10
openstackgerritMark Goddard proposed openstack/kolla-ansible stable/stein: Handle more return codes from nova-status upgrade check  https://review.opendev.org/67662908:15
openstackgerritMark Goddard proposed openstack/kolla-ansible stable/stein: [gnocchi] Don't recursively modify file perms on start  https://review.opendev.org/67663308:20
openstackgerritMark Goddard proposed openstack/kolla-ansible stable/rocky: [gnocchi] Don't recursively modify file perms on start  https://review.opendev.org/67663408:20
openstackgerritMark Goddard proposed openstack/kolla-ansible stable/stein: Enable iscsid on cinder-backup hosts  https://review.opendev.org/67663508:22
yoctozeptoMark, the backporter08:22
openstackgerritMark Goddard proposed openstack/kolla-ansible stable/stein: Fix swift log level configuration  https://review.opendev.org/67663608:23
openstackgerritMark Goddard proposed openstack/kolla-ansible stable/rocky: Fix swift log level configuration  https://review.opendev.org/67663708:23
openstackgerritMark Goddard proposed openstack/kolla-ansible stable/stein: Add missing when condition for swift config files  https://review.opendev.org/67663808:24
openstackgerritMark Goddard proposed openstack/kolla-ansible stable/rocky: Add missing when condition for swift config files  https://review.opendev.org/67663908:24
openstackgerritMark Goddard proposed openstack/kolla-ansible stable/rocky: Add missing Octavia policy file to Horizon  https://review.opendev.org/67664108:28
openstackgerritMark Goddard proposed openstack/kolla-ansible stable/stein: Add missing Octavia policy file to Horizon  https://review.opendev.org/67664208:30
openstackgerritMark Goddard proposed x/kayobe stable/rocky: Disable chrony container by default  https://review.opendev.org/67664308:39
openstackgerritMark Goddard proposed x/kayobe stable/rocky: Stop allocating network and broadcast addresses  https://review.opendev.org/67664408:40
openstackgerritMerged openstack/kolla-ansible master: Fix idempotency of fluentd customisations  https://review.opendev.org/67621608:40
openstackgerritMerged openstack/kolla-ansible stable/queens: Add 'allow *' to getting ceph mds keyring  https://review.opendev.org/67504508:40
openstackgerritMark Goddard proposed x/kayobe stable/queens: Stop allocating network and broadcast addresses  https://review.opendev.org/67664508:40
openstackgerritMark Goddard proposed x/kayobe stable/rocky: Restrict ncclient to 0.6.2 to avoid unknown host key issue  https://review.opendev.org/67664608:41
openstackgerritMark Goddard proposed x/kayobe stable/queens: Restrict ncclient to 0.6.2 to avoid unknown host key issue  https://review.opendev.org/67664708:41
openstackgerritMark Goddard proposed openstack/kolla-ansible master: Remove unused template wsgi_python.conf.j2  https://review.opendev.org/67665509:12
openstackgerritwangwei proposed openstack/kolla stable/stein: Add partition detection to fix osd initialization failure  https://review.opendev.org/67665709:13
openstackgerritwangwei proposed openstack/kolla stable/rocky: Add partition detection to fix osd initialization failure  https://review.opendev.org/67665809:14
*** tonythomas has joined #openstack-kolla09:16
*** iniazi has quit IRC09:19
*** zbr|flu is now known as zbr09:33
openstackgerritMark Goddard proposed openstack/kolla-ansible stable/rocky: Add missing Octavia policy file to Horizon  https://review.opendev.org/67664109:47
*** dpawlik has joined #openstack-kolla09:48
openstackgerritMerged x/kayobe stable/rocky: Ignore --limit for localhost plays  https://review.opendev.org/67426209:49
*** hamzy_ has joined #openstack-kolla09:51
openstackgerritMark Goddard proposed openstack/kolla-ansible master: CI: Test accessing dashboard  https://review.opendev.org/67641209:53
*** hamzy has quit IRC09:54
openstackgerritMark Goddard proposed openstack/kolla-ansible master: Fix import of horizon custom_local_settings on python3  https://review.opendev.org/66841009:56
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: WIP: CI: Zun jobs  https://review.opendev.org/67639009:58
openstackgerritScott Solkhon proposed openstack/kolla-ansible master: Wait for MariaDB to be accessible via HAProxy  https://review.opendev.org/67621910:00
*** mgoddard_ has joined #openstack-kolla10:14
*** hongbin has joined #openstack-kolla10:22
mgoddard_morning all. IRC bouncer rejecting me this morning for some reason :/10:51
openstackgerritMark Goddard proposed x/kayobe master: Update packages in virtualenvs  https://review.opendev.org/66663511:01
*** cah_link has quit IRC11:04
*** cah_link has joined #openstack-kolla11:05
*** kplant has joined #openstack-kolla11:14
*** hongbin has quit IRC11:16
openstackgerritMark Goddard proposed x/kayobe stable/rocky: Disable chrony container by default  https://review.opendev.org/67664311:19
*** zhangfei has quit IRC11:23
goldyfruito/11:55
goldyfruitmgoddard_, I did the changes in Masakari and Masakari CI test11:55
openstackgerritMerged x/kayobe stable/queens: Stop allocating network and broadcast addresses  https://review.opendev.org/67664512:02
*** cah_link has quit IRC12:05
*** cah_link has joined #openstack-kolla12:07
*** luksky has joined #openstack-kolla12:22
mgoddard_goldyfruit: thanks. Will take a look12:32
goldyfruitmgoddard_, how things are going with the baby ?12:34
*** dpawlik has quit IRC12:38
*** Blinkiz has joined #openstack-kolla12:39
*** hongbin has joined #openstack-kolla12:43
*** hongbin has quit IRC12:43
BlinkizI have set up ceph external and have a working configuration. My question is, do the control nodes need access to the interface where public ceph (mon) is running? My guess is that only compute nodes need access.12:43
mgoddard_goldyfruit: going well thanks. Starting to get the idea of what to do. I'm sure it will all change soon12:43
mgoddard_Blinkiz: which services are using ceph? Glance would need access to ceph12:44
BlinkizA broader question is on which nodes all these interfaces should be configured on. https://docs.openstack.org/kolla-ansible/stein/admin/production-architecture-guide.html#interface-configuration12:44
yoctozeptoBlinkiz: in default config yes because control has glance-api and it should have access to ceph12:46
yoctozeptoboth mons and osds12:46
yoctozepto;-)12:46
goldyfruitmgoddard_, the first three months are the hardest (you don't know what to do, you don't sleep a lot, the baby doesn't really do anything except eating, sleeping, pooping \o/) then it will continue to not be easy but it will be much funnier (IMO) !!12:46
yoctozeptomgoddard_: wait till teenage :-)12:47
goldyfruitAhah12:47
mgoddard_eat sleep poop repeat12:47
goldyfruitmgoddard_, exactly :D12:48
mgoddard_hard to imagine teenage year right now12:48
yoctozeptosimple algorithms are the best ;-)12:48
goldyfruityoctozepto, except when the algorithms trigger at random time12:48
yoctozeptogoldyfruit: then event-based programming12:49
goldyfruitSometime you don't want to receive the event :D12:49
goldyfruitespecially at 3am12:49
yoctozeptoraise DeepSleepException12:50
goldyfruitmgoddard_, all of that to say, enjoy it's going very fast and you will forget how tiny she was, how quiet her voice was, etc... :)12:50
yoctozepto^ unless you record that12:51
goldyfruityoctozepto, what about the weight ? :p12:51
mgoddard_trying to take some videos, but he doesn't act interesting on demand12:51
yoctozeptogoldyfruit: record, record12:51
goldyfruityoctozepto, the feeling on the arms :p12:52
yoctozeptoyeah, little children are like pets in this regard12:52
yoctozeptogoldyfruit: review weight, buy flour, fill bags, hold it12:52
* goldyfruit is hoping yoctozepto will not have a baby soon (or a pet) xD12:53
yoctozeptolol12:53
*** BjoernT has joined #openstack-kolla13:03
*** luksky has quit IRC13:33
*** priteau has joined #openstack-kolla13:38
yoctozepto.join #openstack-magnum13:41
yoctozeptooops :D13:41
goldyfruityoctozepto, #openstack-containers13:42
yoctozeptothanks goldyfruit13:42
goldyfruitI got +2 on Masakari role \o/13:45
yoctozeptogoldyfruit: oh, I forgot it, will review immediately :D13:46
*** cah_link has quit IRC13:51
openstackgerritGaëtan Trellu proposed openstack/kolla-ansible master: Add Masakari Ansible role  https://review.opendev.org/61571513:58
yoctozeptogoldyfruit: that was meant as a joke that this stuff is so cool that I could only find and extra line in there13:59
yoctozeptoanyways, restoring w+1 ;-)14:00
goldyfruityoctozepto, maybe a joke but true!14:00
goldyfruityoctozepto, thanks!14:00
yoctozeptogoldyfruit: ok, ok, it will a little longer now due to recheck14:00
yoctozeptotake*14:00
yoctozeptoinstead of just gating14:01
yoctozeptobut will pass finally14:01
goldyfruityoctozepto, I waited months for it I could wait few hours more14:01
yoctozeptotrue14:01
openstackgerritPierre Riteau proposed openstack/kolla stable/stein: Add prometheus_client to monasca-agent image  https://review.opendev.org/67670314:06
openstackgerritPierre Riteau proposed openstack/kolla stable/rocky: Add prometheus_client to monasca-agent image  https://review.opendev.org/67670414:08
openstackgerritScott Solkhon proposed openstack/kolla-ansible master: Support configuration of trusted certificate file  https://review.opendev.org/67670914:32
openstackgerritScott Solkhon proposed openstack/kolla-ansible master: Support configuration of trusted certificate file  https://review.opendev.org/67670914:42
BlinkizHmm. Am failing all the time on the task "keystone : Creating the Keystone service and endpoint". Am running deploy against a working lab openstack cluster. When it tries to connect to http://vip.example.com:35357 it receives the error "DiscoveryFailure: Could not find versioned identity endpoints when attempting to authenticate. Please check that your auth_url is correct. Internal Server Error (HTTP 500)"15:00
BlinkizI also get errors like "GatewayTimeout: Gateway Timeout (HTTP 504)"15:01
Blinkizit has happened before but suddenly disappeared. Now it is make again. Failing at this task. I do not understand :)15:01
Blinkizmake => failing15:02
Blinkiz'ansible -a "nmap vip.example.com -p 35357" control1,control2,control3' say port is open from all three machines.15:05
goldyfruitBlinkiz, is your VIP up on one of the controllers ?15:05
Blinkizgoldyfruit, yes, it is up on control3 at the moment. The task that is failing is running on control115:06
goldyfruitif you curl  http://vip.example.com:35357 ?15:08
goldyfruitoh you don't even have keystone ?15:09
Blinkizcurl works. Gives me a 302 redirect against auth/login/?next=/ path15:09
goldyfruitshould not15:10
Blinkizoh, curled wrong port. sorry15:10
Blinkizlet me test again15:10
Blinkizgoldyfruit, curl gives me "300 MULTIPLE CHOICES" code. location against /v3/15:11
Blinkizgoldyfruit, here is the output: https://paste.ubuntu.com/p/jTGSg4fJNz/15:13
openstackgerritMark Goddard proposed openstack/kolla-ansible master: Use internal API for heat -> heat communication  https://review.opendev.org/67671615:13
*** dave-mccowan has joined #openstack-kolla15:22
goldyfruitBlinkiz, so your Keystone is responding15:27
goldyfruitBlinkiz, are you able to run the curl from all the controllers15:27
Blinkizgoldyfruit, yes, it works from all controllers15:28
goldyfruitare you able to run the curl from kolla_toolbox container ?15:28
Blinkizgoldyfruit, I can try. Please wait :)15:28
Blinkizgoldyfruit, yes, it works from kolla_toolbox. It is the same result15:29
goldyfruitBlinkiz, rocky or stein ?15:30
BlinkizAm going to run the deploy again and check the curl command when the task is failing. See if there is something different at that time.15:30
*** dave-mccowan has quit IRC15:30
Blinkizgoldyfruit, am running stein15:30
goldyfruitBlinkiz, your curl the domain or the ip ?15:30
Blinkizgoldyfruit, I curl the domain.15:31
goldyfruitfrom the 3 kolla_toolbox you it worked ?15:31
Blinkizgoldyfruit, The output is here: https://paste.ubuntu.com/p/jTGSg4fJNz/15:31
Blinkizgoldyfruit, yes, same with kolla_toolbox15:31
goldyfruithum15:32
Blinkizgoldyfruit, And now the error is gone.. It is not failing on that task anymore. Like before. And suddenly it is there again.15:33
goldyfruitBlinkiz, does everytime you got the issue the VIP changed of controller ?15:34
goldyfruitwhat the logs from keepalived/haproxy return ?15:34
Blinkizgoldyfruit, Hmm. I do not know. I can force the vip to change and run the deploy again :)15:34
*** ivve has quit IRC15:35
goldyfruitcheck if the 3 controllers see VRRP traffic15:35
openstackgerritMerged x/kayobe master: Add alternative tenks deploy and teardown entrypoints.  https://review.opendev.org/66329215:35
Blinkizokay. vip is now on control2. Let me run deploy again..15:36
goldyfruitBlinkiz, try the curl it could be faster but deploy works too :p15:37
Blinkizgoldyfruit, here is the result of curl. https://paste.ubuntu.com/p/dgRGCmmxdd/15:37
BlinkizLets see if deploy will fail or not this time15:38
BlinkizIt worked this time also. Hmm. Okay. Next time the problem appears I will directly test with the curl command from all nodes and look for signs of failure.15:41
Blinkizgoldyfruit, Thank you for taking your time and helping me :)15:41
goldyfruitBlinkiz, you are welcome15:42
goldyfruitWhen the problem appears, check keepalived/haproxy logs15:42
goldyfruitcheck if the VIP is there or if it is not at 2 place at the same time15:43
goldyfruitCheck MariaDB too, if MariaDB is out Keystone will fail15:43
Blinkizgoldyfruit, okay. will do!15:43
*** luksky has joined #openstack-kolla15:55
openstackgerritRafael Weingärtner proposed openstack/kolla-ansible master: Standardize the configuration of "oslo_messaging" section  https://review.opendev.org/67062616:25
*** absubram has joined #openstack-kolla16:27
*** absubram_ has joined #openstack-kolla16:28
*** absubram has quit IRC16:31
*** absubram_ is now known as absubram16:31
openstackgerritMerged openstack/kolla-ansible master: Add Masakari Ansible role  https://review.opendev.org/61571516:36
*** absubram has quit IRC16:37
goldyfruitYeah \o/16:40
* goldyfruit is crying16:40
*** mgoddard_ has quit IRC17:04
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: Allow to configure docker for Zun  https://review.opendev.org/67673617:07
yoctozeptogoldyfruit: no need to cry17:07
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: WIP: CI: Zun jobs  https://review.opendev.org/67639017:13
openstackgerritMerged openstack/kolla-ansible master: Testing Masakari role in gate  https://review.opendev.org/61605017:26
goldyfruit\o/17:27
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: Allow to configure docker for Zun  https://review.opendev.org/67673617:37
*** priteau has quit IRC17:39
yoctozeptoyeah, goldyfruit, your hard work has paid off17:42
yoctozeptoand today we also merge Rafael's notifications patch17:43
yoctozeptolots of improvements in k-a :-)17:43
yoctozeptoand zun is nearing some proper support17:43
yoctozeptohopefully, the zun-dependent end result is worth it17:46
yoctozeptozun looks a bit abandoned nowadays17:47
goldyfruityoctozepto, https://github.com/openstack/zun/commits/master I can see commits17:49
yoctozeptogoldyfruit: look deeper, they don't seem anything major17:50
goldyfruitYeah no new features, mostly fixes17:50
yoctozeptoand the channel is not too lively either, though it looks working in principles17:51
yoctozeptothe zun, I mean17:51
*** ivve has joined #openstack-kolla18:10
*** absubram has joined #openstack-kolla18:13
*** lile has joined #openstack-kolla18:19
lileAny pointers on how to run [control] and [haproxy] on nodes on one L2 network separately from the [compute] nodes on a different L2 network?  I've tried changing [haproxy] and [network] both but I seem to have keystone issues?18:33
*** spsurya has quit IRC18:53
lile'19:15
*** vesper11 has quit IRC19:30
*** kplant has quit IRC19:47
*** gouthamr is now known as gouthamr|brb20:02
openstackgerritMerged openstack/kolla-ansible master: Standardize the configuration of "oslo_messaging" section  https://review.opendev.org/67062620:04
*** dougsz has joined #openstack-kolla20:10
*** dougsz has quit IRC20:18
*** gouthamr|brb is now known as gouthamr20:30
*** dougsz has joined #openstack-kolla20:33
lileAny pointers on how to run [control] and [haproxy] on nodes on one L2 network separately from the [compute] nodes on a different L2 network?  I've tried  changing [haproxy] and [network] both but I seem to have keystone issues?20:45
*** dougsz has quit IRC20:48
*** lile has left #openstack-kolla20:48
*** lile has joined #openstack-kolla20:49
*** vesper11 has joined #openstack-kolla20:52
*** lile has left #openstack-kolla21:00
*** lile has joined #openstack-kolla21:03
*** lile has left #openstack-kolla21:08
*** BjoernT has quit IRC21:28
*** lemko has joined #openstack-kolla21:34
openstackgerritJames Kirsch proposed openstack/kolla-ansible master: Implement TLS encryption for internal endpoints  https://review.opendev.org/66355521:56
openstackgerritMerged x/kayobe stable/rocky: Stop allocating network and broadcast addresses  https://review.opendev.org/67664421:57
*** luksky has quit IRC22:01
openstackgerritMerged x/kayobe stable/rocky: Restrict ncclient to 0.6.2 to avoid unknown host key issue  https://review.opendev.org/67664622:04
openstackgerritMerged x/kayobe stable/queens: Restrict ncclient to 0.6.2 to avoid unknown host key issue  https://review.opendev.org/67664722:05
openstackgerritMerged x/kayobe master: Generate openrc files during control host bootstrap  https://review.opendev.org/66745122:13
*** goldyfruit has quit IRC22:35
*** goldyfruit has joined #openstack-kolla22:38
openstackgerritJames Kirsch proposed openstack/kolla-ansible master: Allow disabling insecure API endpoints  https://review.opendev.org/54840722:39
*** ivve has quit IRC22:46
openstackgerritJames Kirsch proposed openstack/kolla-ansible master: Implement TLS encryption for internal endpoints  https://review.opendev.org/66355522:46
*** goldyfruit has quit IRC22:56
*** goldyfruit has joined #openstack-kolla22:56
*** absubram has quit IRC23:33
*** igordc has joined #openstack-kolla23:46

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!