Sunday, 2017-09-17

*** manheim has joined #openstack-kolla00:22
*** zhubingbing has joined #openstack-kolla00:25
*** magicboiz has quit IRC00:36
*** magicboiz has joined #openstack-kolla00:36
*** dave-mccowan has joined #openstack-kolla00:41
*** zhubingbing has quit IRC00:45
*** dave-mccowan has quit IRC00:56
*** magicboiz has quit IRC00:56
*** magicboiz has joined #openstack-kolla00:56
-openstackstatus- NOTICE: Zuul has been restarted to pick up a bug fix in prep for Gerrit upgrade. Changes have been reenqueued for you.01:05
*** yangyapeng has quit IRC01:05
*** yangyapeng has joined #openstack-kolla01:05
*** manheim has quit IRC01:06
*** yangyapeng has quit IRC01:10
*** yangyapeng has joined #openstack-kolla01:16
*** huzhengchuan has quit IRC01:21
*** yangyapeng has quit IRC01:21
*** zhubingbing has joined #openstack-kolla01:28
*** yangyapeng has joined #openstack-kolla01:38
*** yangyapeng has quit IRC01:38
*** yangyapeng has joined #openstack-kolla01:40
*** yangyape_ has joined #openstack-kolla01:43
*** yangyapeng has quit IRC01:43
*** yangyapeng has joined #openstack-kolla01:43
*** yangyape_ has quit IRC01:43
*** yangyape_ has joined #openstack-kolla01:45
*** yangyapeng has quit IRC01:45
*** itlinux has joined #openstack-kolla01:47
*** itlinux has quit IRC01:48
*** yangyapeng has joined #openstack-kolla01:49
*** yangyape_ has quit IRC01:49
*** yangyape_ has joined #openstack-kolla01:51
*** yangyapeng has quit IRC01:51
*** yangyape_ has quit IRC01:52
*** yangyapeng has joined #openstack-kolla01:53
*** yangyape_ has joined #openstack-kolla01:55
*** yangyapeng has quit IRC01:55
*** dave-mccowan has joined #openstack-kolla02:01
*** reddydodda has quit IRC02:13
*** jascott1 has joined #openstack-kolla02:14
*** reddydodda has joined #openstack-kolla02:14
*** lxdong has quit IRC02:18
*** jascott1 has quit IRC02:18
*** yangyapeng has joined #openstack-kolla02:19
*** yangyape_ has quit IRC02:19
*** reddydodda has quit IRC02:20
*** itlinux has joined #openstack-kolla02:35
*** yangyape_ has joined #openstack-kolla02:45
*** yangyapeng has quit IRC02:45
*** zhubingbing has quit IRC02:50
*** schwicht has quit IRC03:03
*** lrensing has joined #openstack-kolla03:28
*** dave-mccowan has quit IRC03:57
*** lrensing has quit IRC03:58
*** manheim has joined #openstack-kolla04:07
*** lrensing has joined #openstack-kolla04:07
*** manheim has quit IRC04:11
*** yangyapeng has joined #openstack-kolla04:13
*** yangyape_ has quit IRC04:13
*** kolla-slack has quit IRC04:21
*** kolla-slack has joined #openstack-kolla04:21
*** kolla-slack has quit IRC04:24
*** kolla-slack has joined #openstack-kolla04:24
*** kolla-slack has quit IRC04:25
*** kolla-slack has joined #openstack-kolla04:25
*** itlinux has quit IRC04:27
*** kolla-slack has quit IRC04:27
*** kolla-slack has joined #openstack-kolla04:27
*** magicboiz has quit IRC04:30
*** magicboiz has joined #openstack-kolla04:30
*** yangyapeng has quit IRC04:48
*** lrensing has quit IRC04:55
*** reddydodda has joined #openstack-kolla05:15
*** magicboiz has quit IRC05:27
*** magicboiz has joined #openstack-kolla05:28
-openstackstatus- NOTICE: Zuul is currently not moving any changes into the gate queue. Wait with approving changes until this is fixed.05:49
*** ChanServ changes topic to "Zuul is currently not moving any changes into the gate queue. Wait with approving changes until this is fixed."05:49
*** unicell has joined #openstack-kolla05:54
*** yangyapeng has joined #openstack-kolla06:38
*** yangyapeng has quit IRC06:56
*** yangyapeng has joined #openstack-kolla06:57
*** xinliang has quit IRC07:07
*** reddydodda has quit IRC07:09
*** yangyapeng has quit IRC07:12
*** reddydodda has joined #openstack-kolla07:15
*** xinliang has joined #openstack-kolla07:20
*** jascott1 has joined #openstack-kolla07:34
*** jascott1 has quit IRC07:39
*** jiriprox has quit IRC07:44
*** reddydodda has quit IRC07:50
*** magicboiz has quit IRC07:51
*** magicboiz has joined #openstack-kolla07:51
*** gaoyan has joined #openstack-kolla08:03
*** manheim has joined #openstack-kolla08:07
*** yangyapeng has joined #openstack-kolla08:11
*** manheim has quit IRC08:12
*** magicboiz has quit IRC08:12
*** magicboiz has joined #openstack-kolla08:13
*** yangyapeng has quit IRC08:17
*** yangyapeng has joined #openstack-kolla08:22
*** yangyapeng has quit IRC08:27
*** yangyapeng has joined #openstack-kolla08:29
*** dp4n has joined #openstack-kolla08:31
*** yangyapeng has quit IRC08:36
*** manheim has joined #openstack-kolla08:52
*** yangyapeng has joined #openstack-kolla08:57
*** livelace2 has quit IRC09:00
*** manheim has quit IRC09:01
*** livelace2 has joined #openstack-kolla09:01
*** yangyapeng has quit IRC09:03
*** dp4n has quit IRC09:30
*** dp4n has joined #openstack-kolla09:32
*** leifmadsen_ has quit IRC09:39
*** mgoddard has joined #openstack-kolla09:40
*** bswartz has quit IRC09:41
*** dp4n has quit IRC09:41
*** Jeffrey4l has quit IRC09:41
*** flaper87 has quit IRC09:41
*** dp4n has joined #openstack-kolla09:42
*** leifmadsen has joined #openstack-kolla09:44
*** Jeffrey4l has joined #openstack-kolla09:46
*** flaper87 has joined #openstack-kolla09:53
*** mgoddard has quit IRC09:54
*** mgoddard has joined #openstack-kolla10:13
*** magicboiz has quit IRC10:18
*** magicboiz has joined #openstack-kolla10:19
*** dp4n has quit IRC10:20
*** dp4n has joined #openstack-kolla10:41
*** yangyapeng has joined #openstack-kolla11:06
*** mgoddard has quit IRC11:26
*** mgoddard has joined #openstack-kolla11:53
*** manheim has joined #openstack-kolla12:01
*** manheim has quit IRC12:06
*** magicboiz has quit IRC12:20
*** magicboiz has joined #openstack-kolla12:20
*** mgoddard has quit IRC12:24
*** yangyapeng has quit IRC12:37
*** yangyapeng has joined #openstack-kolla12:38
*** yangyapeng has quit IRC12:38
*** yangyapeng has joined #openstack-kolla12:38
*** manheim has joined #openstack-kolla12:39
*** yangyapeng has quit IRC12:43
*** zhubingbing has joined #openstack-kolla12:46
*** manheim has quit IRC12:47
*** dp4n has quit IRC12:50
*** manheim has joined #openstack-kolla12:50
*** yangyapeng has joined #openstack-kolla12:52
*** yangyapeng has quit IRC12:56
*** livelace has quit IRC13:07
*** kristian__ has joined #openstack-kolla13:31
*** yingjun has joined #openstack-kolla13:31
kristian__hey inc0, are you here?13:33
*** magicboiz has quit IRC13:39
*** magicboiz has joined #openstack-kolla13:40
*** manheim has quit IRC13:47
*** yingjun has quit IRC13:48
*** in2rd2 has quit IRC14:11
*** schwicht has joined #openstack-kolla14:21
*** yangyapeng has joined #openstack-kolla14:26
*** trozet_ has joined #openstack-kolla14:32
*** trozet_ has quit IRC14:32
*** zhubingbing has quit IRC14:46
*** hrw has quit IRC14:48
*** hrw has joined #openstack-kolla14:50
kolla-slack<inc0> Now I am14:51
kolla-slack<inc0> What's up?14:51
kristian__inc0: did you successfully upgrade kolla-ansible from 4.0.2 to 5.0.0?15:20
*** sdake_ is now known as sdake15:20
kolla-slack<inc0> Yes15:20
kristian__because I'm getting an error when upgrading rabbitmq on some gospel node15:20
kolla-slack<inc0> Hmm, rabbitmq became notorious lately15:21
kristian__also how can I access kolla slack?15:21
kristian__and I have f-ed my openstack deployment btw :/15:22
kolla-slack<inc0> It's openstack-kolla on k8s slack15:22
kolla-slack<inc0> Damn15:22
kristian__ok ok15:22
kolla-slack<inc0> Try stopping all rabbitmq and running upgrade15:23
kolla-slack<oxkipo> @inc0 its me kristian__15:24
*** caoyuan has joined #openstack-kolla15:24
kolla-slack<oxkipo> ok, gonna try it15:24
*** kristian__ has quit IRC15:24
kolla-slack<inc0> Cool15:24
*** ChanServ changes topic to "support: ask.openstack.org | New to Kolla: docs @ https://docs.openstack.org/kolla/latest/ | | Kolla IRC meetngs on Wednesdays @ 16:00 UTC - see agenda @ https://goo.gl/OXB0DL - IRC channel is *LOGGED* @ http://goo.gl/3mzZ7b"15:27
-openstackstatus- NOTICE: Zuul has been fixed, you can approve changes again.15:27
*** caoyuan has quit IRC15:27
kolla-slack<oxkipo> @inc0 now this `fatal: [localhost]: FAILED! => {"failed": true, "msg": "The conditional check '(gospel_node.stdout | from_json).changed' failed. The error was: No JSON object could be decoded"}`15:29
kolla-slack<inc0> Hmm that looks bad. Let me get to code15:30
inc0oxkipo try running docker exec -it rabbitmq /usr/local/bin/rabbitmq_get_gospel_node15:33
inc0ahh nvm I know what happened15:34
inc0since I asked you to stop rabbitmq;)15:34
kolla-slack<oxkipo> I stopped and ran upgrade again15:35
kolla-slack<oxkipo> this happened15:35
inc0right15:35
inc0you have multinode?15:35
inc0anyway, start rabbitmq one at the time in any order, run upgrade adn tell me what happend15:36
kolla-slack<oxkipo> aio15:38
*** kristian__ has joined #openstack-kolla15:39
*** itlinux has joined #openstack-kolla15:54
*** kristia__ has joined #openstack-kolla15:56
*** kristian__ has quit IRC15:56
*** itlinux has quit IRC16:00
*** gaoyan has quit IRC16:00
*** sbezverk has quit IRC16:02
kristia__inc0: destroying it and I will redeploy it16:06
inc0that's a bit harsh16:07
inc0what happened in aio upgradE?16:07
kristia__I know, rabbitmq failed me and I tried to run deploy on top of the old deployment (keystone db is messed up,....)16:10
kristia__I will do all upgrades in a vm for testing16:13
kristia__in the future16:13
inc0well what happened to keystone db tho16:13
inc0it should be safe no matter what16:13
kristia__too late, destroyed it16:15
inc0damn16:15
inc0well16:15
inc0too bad16:16
kristia__I couldnt log into the openstack and when I force rebooted the vm (cinder vda) it couldnt find or attach the iscsi device16:16
kristia__so I thought it was f-ed16:16
inc0hmm not necessarily16:16
inc0it might've been galera rescue thing16:17
kristia__whatever, reconstructing the sites in a docker container (one had letsencrypt ssl and I have assigned other ip to it so :( ) at least I have hourly backups16:18
inc0well vms won't get reconstructed16:31
*** manheim has joined #openstack-kolla16:48
*** itlinux has joined #openstack-kolla16:50
*** manheim has quit IRC16:52
*** kristia__ has quit IRC16:57
*** kristian__ has joined #openstack-kolla16:58
*** itlinux has quit IRC16:59
*** kristian__ has quit IRC17:02
*** kristian__ has joined #openstack-kolla17:02
*** robbbe has joined #openstack-kolla17:10
*** kristian__ has quit IRC17:23
*** kristian__ has joined #openstack-kolla17:24
*** kristian__ has quit IRC17:27
*** kristian__ has joined #openstack-kolla17:28
*** robbbe has quit IRC17:30
*** robbbe has joined #openstack-kolla17:31
*** robbbe has quit IRC17:31
*** magicboiz has quit IRC17:31
*** itlinux has joined #openstack-kolla17:31
*** magicboiz has joined #openstack-kolla17:31
*** robbbe has joined #openstack-kolla17:31
*** robbbe has quit IRC17:32
*** robbbe has joined #openstack-kolla17:32
*** robbbe has quit IRC17:32
*** itlinux has quit IRC17:33
*** robbbe has joined #openstack-kolla17:33
*** robbbe has quit IRC17:33
*** robbbe has joined #openstack-kolla17:34
*** robbbe has quit IRC17:34
*** kristian__ has quit IRC17:39
*** kristian__ has joined #openstack-kolla17:40
*** mattmceuen has joined #openstack-kolla17:42
*** dp4n has joined #openstack-kolla17:44
*** kristian__ has quit IRC17:44
*** dave-mccowan has joined #openstack-kolla17:45
*** dave-mcc_ has joined #openstack-kolla17:47
*** reddydodda has joined #openstack-kolla17:47
*** dave-mccowan has quit IRC17:50
*** dp4n has quit IRC17:53
*** openstackstatus has quit IRC17:57
*** openstack has joined #openstack-kolla18:00
*** magicboiz has quit IRC18:14
*** magicboiz has joined #openstack-kolla18:14
*** egonzalez has joined #openstack-kolla18:26
*** Suicky has joined #openstack-kolla18:27
*** kristian__ has joined #openstack-kolla18:34
*** kristia__ has joined #openstack-kolla18:35
*** kristian__ has quit IRC18:35
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible master: WIP: Destroy per service  https://review.openstack.org/50459218:35
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible master: [DNM] test pike branch  https://review.openstack.org/50199618:43
*** dp4n has joined #openstack-kolla18:45
*** magicboiz has quit IRC18:47
*** magicboiz has joined #openstack-kolla18:48
*** kristia__ has quit IRC18:54
*** magicboiz has quit IRC18:56
openstackgerritEduardo Gonzalez proposed openstack/kolla master: Fix dind image build and oraclelinux opstools repo  https://review.openstack.org/50371218:57
*** magicboiz has joined #openstack-kolla18:57
egonzalez^^ this change fix all current build  errors  in kolla, also seems fix libvirt issues in ansible deploy19:02
*** Suicky has quit IRC19:05
*** Suicky has joined #openstack-kolla19:05
openstackgerritEduardo Gonzalez proposed openstack/kolla stable/pike: [DNM] test pike branch  https://review.openstack.org/50479919:06
*** sbezverk has joined #openstack-kolla19:09
*** magicboiz has quit IRC19:10
*** magicboiz has joined #openstack-kolla19:10
*** dp4n has quit IRC19:20
*** Walex has joined #openstack-kolla19:23
WalexHi, I am following the somewhat confused "quickstart" and found workaroundarounds for most of the confusions, but I am a bit stuck on deployment... Context to follow19:24
WalexI am doing a single host install on "metal", with no local registry.19:25
*** kristian__ has joined #openstack-kolla19:25
Walex'kolla-build' work and 'docker images' lists the expected image like:19:25
Walexkolla/centos-binary-keystone-ssh                  5.0.0               7a7a725c53c3        About an hour ago   787.3 MB19:27
Walexbut when I try to do a deploy it looks for "docker.io/kolla/centos-binary-..." not for "kolla/centos-binary-..."19:28
WalexI have made sure that in 'globals.yml' I have:19:29
Walexdocker_registry: ~19:29
Walexdocker_namespace: kolla19:29
WalexUhmmm, now that I am thinking again as soon as I wrote, the error happens on a "memcached" image which I haven't built....19:30
WalexI am trying again with '--profile base' instead of an explicit list.19:31
Walexoh oh just had a look at 'profiless 'kolla/common/config.py' and the 'infra' and 'base' lists are pretty huge.19:37
*** kristian__ has quit IRC19:38
*** reddydodda has quit IRC19:43
*** itlinux has joined #openstack-kolla19:45
*** itlinux has quit IRC19:48
*** schwicht has quit IRC19:50
WalexIndeed the error message is misleading: if 'docker' cannot find the relevant image locally it tries to find it in a registry, and it is not possible to tell 'docker' to use no registry with '--block-registry docker.io'19:59
egonzalezWalex you may try building and pushing to your local registry, ansible is pulling from dockerhub20:00
openstackgerritKurt Taylor proposed openstack/kolla master: At PTG in Denver, we decided to follow the doc team restructuring format guidelines for the Kolla documentation full revamp. This patchset is the initial restructuring of the existing doc files that we will use to build on.  https://review.openstack.org/50480120:02
openstackgerritEduardo Gonzalez proposed openstack/kolla master: Fix dind image build and oraclelinux build  https://review.openstack.org/50371220:02
openstackgerritKurt Taylor proposed openstack/kolla master: Kolla Documentation Restructuring  https://review.openstack.org/50480120:04
Walexegonzalez: I want to avoid having a local registry _server_, so as mentioned above I set "docker_registry: ~"20:04
egonzalezWalex if using multinode will need  registry or copy  all images manually to hosts,or will try to download from DH20:04
Walexegonzalez: the issue seems indeed to be a misleading error message from 'docker', that it looks up the hub registry if an image is not in the local repository (not local registry)20:05
Walexegonzalez: as I wrote above, I am doing a "single host install", "all-in-one" so I don't really need a local registry. This is for a demo, so I am trying to do a minimal install20:06
WalexThe main problem is the colossal number of images that need building even for a minimal install.20:08
Walexbut that's more of an OpenStack prpblem than a Kolla one20:09
Walexfor example it is hard to believe that I have to build *all* the images in the 'infra' profile.20:16
*** magicboiz has quit IRC20:17
Walexso far for a *base* install I have 70 images, and counting...20:17
*** magicboiz has joined #openstack-kolla20:17
Walexof which 16 just for Nova.20:18
*** Suicky has quit IRC20:24
*** Suicky has joined #openstack-kolla20:24
*** magicboiz has quit IRC20:25
*** magicboiz has joined #openstack-kolla20:25
*** itlinux has joined #openstack-kolla20:40
*** magicboiz has quit IRC20:42
*** magicboiz has joined #openstack-kolla20:43
*** manheim has joined #openstack-kolla20:48
*** manheim has quit IRC20:53
*** Suicky has quit IRC20:59
*** jascott1 has joined #openstack-kolla21:02
*** mattmceuen has quit IRC21:04
*** magicboiz has quit IRC21:08
*** magicboiz has joined #openstack-kolla21:09
*** jascott1 has quit IRC21:14
*** magicboiz has quit IRC21:19
*** magicboiz has joined #openstack-kolla21:20
*** dave-mcc_ has quit IRC21:24
*** robbbe has joined #openstack-kolla21:36
*** reddydodda has joined #openstack-kolla21:40
*** mattmceuen has joined #openstack-kolla21:42
*** itlinux has quit IRC21:49
*** egonzalez has quit IRC22:01
*** itlinux has joined #openstack-kolla22:01
*** magicboiz has quit IRC22:05
*** magicboiz has joined #openstack-kolla22:05
*** manheim has joined #openstack-kolla22:11
*** manheim has quit IRC22:12
*** manheim has joined #openstack-kolla22:12
*** Long has joined #openstack-kolla22:22
*** magicboiz has quit IRC22:25
*** magicboiz has joined #openstack-kolla22:25
*** mattmceuen has quit IRC22:27
*** Long is now known as long22:29
*** mattmceuen has joined #openstack-kolla22:38
*** long has quit IRC22:52
*** manheim has quit IRC22:57
*** bmace has quit IRC23:01
*** bmace has joined #openstack-kolla23:01
*** bswartz has joined #openstack-kolla23:05
*** itlinux has quit IRC23:07
Walexnow all prechecks succeed, and some containers start, but I get an error when creating the 'mariadb' one that kolla-toolbox is not running when creating 'haproxy':23:23
Walexbut the 'kolla-toolbox' image is there, just the container has not been started yet.23:24
WalexTASK [mariadb : Creating haproxy mysql user] ****************************************************************************************23:25
Walexfatal: [localhost]: FAILED! => {"changed": false, "failed": true, "msg": "kolla_toolbox container is not running."}23:25
*** reddydodda has quit IRC23:26
WalexI guess it is an ordering issue, and not clear to me how to start the 'kolla-toolbox' container before the 'mariadb' one with 'deploy'23:26
*** magicboiz has quit IRC23:37
*** magicboiz has joined #openstack-kolla23:38
*** huzhengchuan has joined #openstack-kolla23:43
Walexthe reason for all that is that docker cannot mount to /var/lib/docker/devicemapper/mnt/50df4ee555a3f7799f72a4db4539af1869d69b353ab1cd5234945f72dbbc96e7/rootfs/run/secrets becauyse 'rootfs' not does exist under it. Docker issue?23:43
*** reddydodda has joined #openstack-kolla23:46
Walexknown issue with a race condition: https://bugs.launchpad.net/kolla/+bug/166805923:46
openstackLaunchpad bug 1668059 in kolla "Failed to Starting kolla-toolbox container" [Critical,Triaged]23:46
*** Long has joined #openstack-kolla23:47
*** virus_ has joined #openstack-kolla23:47
Walexit is a MountFlags=shared issue but was pretty sure I had specifidd it23:48
WalexIndeed I have:23:48
Walexsoft#  find * -type f | xargs egrep MountFlags23:48
Walexsystemd/system/docker.service.d/kolla.conf:MountFlags=shared23:48
*** virus_ has quit IRC23:50
*** reddydodda has quit IRC23:50
*** virus_ has joined #openstack-kolla23:50
Walexfound that --enable-secrets=false works around the bug23:56
*** manheim has joined #openstack-kolla23:58
*** schwicht has joined #openstack-kolla23:59

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!