| spsurya | morning | 00:04 |
|---|---|---|
| *** faridda has quit IRC | 00:17 | |
| jascott1 | good morning! | 00:17 |
| *** faridda has joined #openstack-kolla | 00:18 | |
| masuberu | good morning | 00:20 |
| Seyong | good morning | 00:33 |
| *** trozet_ has joined #openstack-kolla | 00:41 | |
| *** dixiaoli has joined #openstack-kolla | 00:43 | |
| spsurya | duonghq: hi.. | 00:56 |
| spsurya | when is your session ? | 00:56 |
| *** kolla-slack has quit IRC | 00:59 | |
| *** kolla-slack has joined #openstack-kolla | 00:59 | |
| *** trozet_ has quit IRC | 01:02 | |
| *** unicell1 has quit IRC | 01:04 | |
| openstackgerrit | jimmygc proposed openstack/kolla-ansible master: Add kolla-ansible bash completion script https://review.openstack.org/463298 | 01:07 |
| *** cuongnv has joined #openstack-kolla | 01:14 | |
| *** gomarivera has quit IRC | 01:16 | |
| openstackgerrit | jimmygc proposed openstack/kolla-ansible master: Fix fwaas options https://review.openstack.org/463882 | 01:17 |
| *** zhubingbing_ has joined #openstack-kolla | 01:17 | |
| *** tovin07_ has joined #openstack-kolla | 01:20 | |
| jascott1 | yum k8s install broken? | 01:24 |
| kolla-slack | <inc0> and summit is done | 01:26 |
| *** hrw has quit IRC | 01:27 | |
| *** hrw has joined #openstack-kolla | 01:34 | |
| masuberu | good morning all, wanted to ask, in kolla-ansible it is a requirement for all hosts to have the neutron_external_interface so kolla-ansible can create a br-ex during openstack deployment. My question is: Do the compute nodes actually used that interface? apologize for my little knowledge about neutron but reading the openstack network docs I am under the impression that the external traffic from the vms will go vn running on compute node | 01:39 |
| masuberu | --(br-tun)-->neutron_server running on control plane --(br-ex)-->internet so I don't quite understand why we need br-ex/neutron_external_interface on each compute node... | 01:39 |
| masuberu | vm running on compute node--(br-tun)-->neutron_server running on control plane--(br-ex)-->internet | 01:40 |
| *** GonZo2000 has quit IRC | 01:42 | |
| *** eaguilar has joined #openstack-kolla | 02:15 | |
| *** faridda has quit IRC | 02:17 | |
| *** faridda has joined #openstack-kolla | 02:18 | |
| *** manheim has joined #openstack-kolla | 02:31 | |
| *** goldyfruit has quit IRC | 02:33 | |
| *** manheim has quit IRC | 02:35 | |
| *** faridda has quit IRC | 02:36 | |
| masuberu | is there any plan to integrate cloudkitty on kolla? | 03:09 |
| *** Teck72 has joined #openstack-kolla | 03:18 | |
| *** Teck7 has quit IRC | 03:19 | |
| *** eaguilar has quit IRC | 03:34 | |
| *** david-lyle has joined #openstack-kolla | 03:37 | |
| *** eanylin has quit IRC | 03:47 | |
| *** david-lyle has quit IRC | 03:54 | |
| *** Teck7 has joined #openstack-kolla | 03:57 | |
| *** Teck72 has quit IRC | 03:58 | |
| *** japestinho has joined #openstack-kolla | 04:06 | |
| *** oiahoighoi has joined #openstack-kolla | 04:07 | |
| oiahoighoi | hi | 04:09 |
| *** oiahoighoi has quit IRC | 04:22 | |
| *** cuongnv has quit IRC | 04:47 | |
| *** klindgren has joined #openstack-kolla | 04:50 | |
| *** jaosorior_away has quit IRC | 04:58 | |
| spsurya | kfox1111 hi... | 05:11 |
| spsurya | sdake: Is there any specific reason of not implementing the microservice for swift component of OpenStack ? | 05:13 |
| *** jascott1 has quit IRC | 05:38 | |
| *** jascott1 has joined #openstack-kolla | 05:40 | |
| *** jaosorior has joined #openstack-kolla | 05:41 | |
| masuberu | hi, I am getting this error when I try to run kolla-ansible reconfigure -i multinode https://pastebin.com/raw/3KTi5PGM | 05:41 |
| *** unicell has joined #openstack-kolla | 05:45 | |
| *** unicell1 has joined #openstack-kolla | 05:49 | |
| *** unicell has quit IRC | 05:51 | |
| *** dciabrin has quit IRC | 05:54 | |
| *** tovin07_ has quit IRC | 06:01 | |
| *** manheim has joined #openstack-kolla | 06:07 | |
| *** jtriley has quit IRC | 06:09 | |
| *** manheim has quit IRC | 06:11 | |
| *** pcaruana has joined #openstack-kolla | 06:14 | |
| openstackgerrit | hosup choi proposed openstack/kolla-kubernetes master: Update deployment-guide for ubuntu user https://review.openstack.org/459163 | 06:17 |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla-ansible master: Remove verbose log in memcached container https://review.openstack.org/464134 | 06:17 |
| *** darth_costanza has joined #openstack-kolla | 06:32 | |
| *** darth_costanza has quit IRC | 06:33 | |
| *** unicell has joined #openstack-kolla | 06:38 | |
| *** unicell1 has quit IRC | 06:39 | |
| *** japestinho has quit IRC | 06:46 | |
| *** matrohon_ has joined #openstack-kolla | 06:50 | |
| openstackgerrit | caoyuan proposed openstack/kolla master: Fix the "tox -e docs" faild https://review.openstack.org/464135 | 06:58 |
| openstackgerrit | caoyuan proposed openstack/kolla-kubernetes master: Update the docs index https://review.openstack.org/464137 | 07:11 |
| *** manheim has joined #openstack-kolla | 07:20 | |
| *** manheim has quit IRC | 07:21 | |
| *** manheim has joined #openstack-kolla | 07:21 | |
| manheim | hello | 07:37 |
| *** egonzalez has joined #openstack-kolla | 07:42 | |
| *** dixiaoli has quit IRC | 07:45 | |
| *** gfidente has joined #openstack-kolla | 07:45 | |
| *** gfidente has quit IRC | 07:45 | |
| *** gfidente has joined #openstack-kolla | 07:45 | |
| *** dixiaoli has joined #openstack-kolla | 07:47 | |
| *** unicell has quit IRC | 07:52 | |
| *** dixiaoli has quit IRC | 08:04 | |
| openstackgerrit | Eduardo Gonzalez proposed openstack/kolla stable/newton: Add ssh-keyscan to unblock kolla gate https://review.openstack.org/462025 | 08:10 |
| *** dixiaoli has joined #openstack-kolla | 08:10 | |
| openstackgerrit | Eduardo Gonzalez proposed openstack/kolla-ansible master: Fix cinder-api ubuntu binary deploy https://review.openstack.org/457993 | 08:11 |
| *** mgoddard__ has joined #openstack-kolla | 08:19 | |
| *** jascott1 has quit IRC | 08:26 | |
| *** mgoddard__ has quit IRC | 08:29 | |
| *** salv-orlando has joined #openstack-kolla | 08:40 | |
| *** mgoddard__ has joined #openstack-kolla | 08:41 | |
| *** mgoddard__ has quit IRC | 08:46 | |
| *** mannamne has joined #openstack-kolla | 08:55 | |
| *** janki has joined #openstack-kolla | 08:56 | |
| *** mgoddard__ has joined #openstack-kolla | 09:02 | |
| *** mgoddard__ has quit IRC | 09:07 | |
| *** mgoddard__ has joined #openstack-kolla | 09:08 | |
| *** yee379 has quit IRC | 09:15 | |
| *** yee37920 has joined #openstack-kolla | 09:15 | |
| *** mgoddard__ has quit IRC | 09:15 | |
| *** iniazi has quit IRC | 09:16 | |
| *** iniazi has joined #openstack-kolla | 09:21 | |
| *** lpetrut has joined #openstack-kolla | 09:22 | |
| *** mgoddard__ has joined #openstack-kolla | 09:27 | |
| *** pbourke has quit IRC | 09:29 | |
| *** pbourke has joined #openstack-kolla | 09:31 | |
| *** janki has quit IRC | 09:36 | |
| openstackgerrit | Merged openstack/kolla master: Fix Incorrect timezone in ubuntu horizon images https://review.openstack.org/460770 | 09:42 |
| openstackgerrit | Merged openstack/kolla master: Add net-tools for collection of base networking utilities https://review.openstack.org/463633 | 09:43 |
| *** GonZo2000 has joined #openstack-kolla | 09:47 | |
| openstackgerrit | Eduardo Gonzalez proposed openstack/kolla-ansible master: Fix cinder-api ubuntu binary deploy https://review.openstack.org/457993 | 09:52 |
| *** caowei has quit IRC | 09:55 | |
| *** caowei has joined #openstack-kolla | 09:56 | |
| *** caowei has quit IRC | 09:59 | |
| *** magicboiz has joined #openstack-kolla | 10:10 | |
| openstackgerrit | Pavel Gluschak (scsnow) proposed openstack/kolla-ansible master: Fixed libvirt package removal https://review.openstack.org/464174 | 10:27 |
| *** jascott1 has joined #openstack-kolla | 10:28 | |
| *** dixiaoli has quit IRC | 10:30 | |
| *** dixiaoli has joined #openstack-kolla | 10:32 | |
| *** jascott1 has quit IRC | 10:32 | |
| *** yee37920 has quit IRC | 10:34 | |
| *** yee379 has joined #openstack-kolla | 10:34 | |
| *** dixiaoli has quit IRC | 10:37 | |
| *** mgoddard__ has quit IRC | 10:37 | |
| *** mgoddard__ has joined #openstack-kolla | 10:39 | |
| *** chas has joined #openstack-kolla | 10:45 | |
| *** chas has quit IRC | 10:49 | |
| manheim | hi | 10:54 |
| manheim | is there an issue with the iscsid container for ocata? | 10:54 |
| manheim | http://paste.ubuntu.com/24560062/ | 10:56 |
| hrw | morning | 10:56 |
| openstackgerrit | Marcin Juszkiewicz proposed openstack/kolla master: kubernetes: get it working on AArch64 as well https://review.openstack.org/464045 | 10:56 |
| manheim | i use ubuntu binary 4.0.0 | 10:56 |
| *** mgoddard__ has quit IRC | 11:00 | |
| openstackgerrit | Eduardo Gonzalez proposed openstack/kolla-ansible master: Fix cinder-api ubuntu binary deploy https://review.openstack.org/457993 | 11:00 |
| *** salv-orlando has quit IRC | 11:04 | |
| egonzalez | manheim, have you followed this steps https://github.com/openstack/kolla-ansible/blob/master/doc/cinder-guide.rst#for-ubuntu-and-lvm2iscsi ? | 11:07 |
| *** salv-orlando has joined #openstack-kolla | 11:15 | |
| openstackgerrit | Pavel Gluschak (scsnow) proposed openstack/kolla-ansible master: Fixed libvirt package removal https://review.openstack.org/464174 | 11:24 |
| openstackgerrit | Paul Bourke (pbourke) proposed openstack/kolla-ansible master: Kibana post_config checks only need to run locally https://review.openstack.org/463798 | 11:35 |
| *** rhallisey has joined #openstack-kolla | 11:41 | |
| manheim | ezonzalez uh, didn't know about them, thanks | 11:45 |
| *** blallau has joined #openstack-kolla | 11:49 | |
| *** yingjun has quit IRC | 11:54 | |
| manheim | i had to do this | 11:55 |
| manheim | https://bugs.launchpad.net/kolla/+bug/1631072/comments/4 | 11:55 |
| openstack | Launchpad bug 1631072 in kolla "iscsid container: mkdir /sys/kernel/config: operation not permitted" [High,Confirmed] | 11:55 |
| openstackgerrit | Eduardo Gonzalez proposed openstack/kolla master: Fix if $user is root at extend_start https://review.openstack.org/464181 | 11:57 |
| *** athomas has joined #openstack-kolla | 11:58 | |
| egonzalez | mandre, can you take a look at https://review.openstack.org/464181 | 11:58 |
| hrw | egonzalez: what is policy of adding new macros? | 11:59 |
| hrw | egonzalez: I would like to add install_kubectl one | 11:59 |
| hrw | just two images use kubectl but I need to insert archs there and would be nice to use one version | 11:59 |
| egonzalez | as far i know there is no policy for that, but often macros are used when the feature is reused by more services, like create users, install packages, etc | 12:00 |
| mandre | egonzalez: looking | 12:00 |
| hrw | also kolla uses 3 different kubernetes releases | 12:00 |
| mandre | egonzalez: interesting, what does $USER evaluates to on ubuntu? | 12:02 |
| egonzalez | also root, and whoami is root, weird | 12:02 |
| egonzalez | mandre, in the bug report added an example https://bugs.launchpad.net/kolla/+bug/1690360 | 12:03 |
| openstack | Launchpad bug 1690360 in kolla "wrong if root check at extend_start" [Undecided,In progress] - Assigned to Eduardo Gonzalez (egonzalez90) | 12:03 |
| mandre | egonzalez: yeah, but isn't $USER set to something on ubuntu? what is it? | 12:04 |
| egonzalez | docker exec cinder_api echo $USER | 12:05 |
| egonzalez | root | 12:05 |
| *** magicboiz has quit IRC | 12:05 | |
| mandre | egonzalez: my point is, $USER is posix, while $EUID is not | 12:07 |
| mandre | egonzalez: let me boot an ubuntu container and test by myself | 12:08 |
| *** eaguilar has joined #openstack-kolla | 12:09 | |
| mandre | egonzalez: hmm, $USER is empty on ubuntu | 12:12 |
| *** goldyfruit has joined #openstack-kolla | 12:13 | |
| mandre | egonzalez: there seem to be no usable environment variable in an ubuntu container, http://paste.openstack.org/show/609425/ | 12:17 |
| manheim | cp: cannot stat '/usr/lib/python2.7/site-packages/magnum_ui/enabled/_*[^__].py': No such file or directory | 12:18 |
| manheim | i get this in the horizon container, I enabled magnum and magnum-ui from globals | 12:18 |
| *** athomas has quit IRC | 12:18 | |
| mandre | egonzalez: or centos for that matter :-/ | 12:18 |
| egonzalez | mandre, EUID exist, at least in cinder containers where i checked | 12:19 |
| egonzalez | the last check have a typo s/EID/EUID/ | 12:19 |
| *** rwellum has joined #openstack-kolla | 12:21 | |
| mandre | egonzalez: replied to the review | 12:21 |
| egonzalez | thanks, let me build the images and try | 12:22 |
| *** trozet_ has joined #openstack-kolla | 12:24 | |
| openstackgerrit | Marcin Juszkiewicz proposed openstack/kolla master: introduce 'install_kubectl' macro https://review.openstack.org/464184 | 12:25 |
| *** Pavo has joined #openstack-kolla | 12:30 | |
| pomac | Ehm... | 12:37 |
| pomac | anyone runnning kolla with centos as base? | 12:38 |
| pomac | libvirtError: internal error: unable to execute QEMU command '__com.redhat_drive_add': Device 'drive-virtio-disk1' could not be initialized <- is a interesting problem | 12:38 |
| pomac | it's been known since 2013 apparently - kvm that ships with centos is outdated and doesn't support rbd volumes being added | 12:38 |
| pomac | (A buildtime configuration) | 12:38 |
| pomac | And i know that centos is outdated in someways but WTF ;) | 12:39 |
| pomac | should be upgraded | 12:39 |
| pomac | (Or be clearly documented in the kolla documentation as "do-not-use") | 12:40 |
| openstackgerrit | Eduardo Gonzalez proposed openstack/kolla master: Fix if $user is root at extend_start https://review.openstack.org/464181 | 12:41 |
| bjolo | latest qemu is 2.9 and the one in centos is 1.5.3. released Aug 22, 2013. pretty outdated | 12:41 |
| pomac | exact date Date: Thu Aug 22 11:21:34 2013 -0500 ;) | 12:42 |
| *** chas has joined #openstack-kolla | 12:46 | |
| Pavo | pomac do you have selinux disabled? | 12:47 |
| pomac | Pavo: lookig, this is kolla built containers | 12:50 |
| *** chas has quit IRC | 12:50 | |
| Pavo | I was having the same issue long ago when I had selinux enabled on the host of the containers | 12:50 |
| pomac | Pavo: not enabled in the container atleast | 12:50 |
| pomac | Ah checking | 12:51 |
| Pavo | no in the container, on the host | 12:51 |
| pomac | Pavo: disabled | 12:51 |
| Pavo | type sudo getenforce | 12:51 |
| Pavo | hmmm | 12:51 |
| Pavo | then I'm not sure | 12:51 |
| pomac | i used selinuxenabled ; echo $? | 12:52 |
| pomac | ;) | 12:52 |
| pomac | but i can't attach the device according to the logs anyway | 12:52 |
| *** trozet_ has quit IRC | 12:55 | |
| rwellum | Hi pomac - might be a little quiet here as people get back from the openstack summit that ended yesterday. | 12:58 |
| rwellum | But many people do kolla on Centos 7 | 12:58 |
| *** Pavo has quit IRC | 12:58 | |
| *** athomas has joined #openstack-kolla | 13:02 | |
| *** goldyfruit has quit IRC | 13:12 | |
| *** schwicht has joined #openstack-kolla | 13:13 | |
| *** iniazi has quit IRC | 13:27 | |
| *** gomarivera has joined #openstack-kolla | 13:32 | |
| -openstackstatus- NOTICE: The Gerrit service on http://review.openstack.org is being restarted to address hung remote replication tasks. | 13:37 | |
| *** krtaylor has joined #openstack-kolla | 13:38 | |
| *** paramite has quit IRC | 13:46 | |
| *** jtriley has joined #openstack-kolla | 13:48 | |
| *** blallau has quit IRC | 13:48 | |
| *** iniazi has joined #openstack-kolla | 13:50 | |
| *** salv-orlando has quit IRC | 13:52 | |
| *** gomarivera has quit IRC | 13:54 | |
| *** thm99 has joined #openstack-kolla | 14:03 | |
| *** thm99 has quit IRC | 14:05 | |
| mnaser | nice to meet everyone over the past week | 14:06 |
| mnaser | :) | 14:06 |
| manheim | Running command: 'cinder-api --config-file /etc/cinder/cinder.conf' | 14:16 |
| manheim | I get this for cinder-api contaner for ubuntu binary 4.0.0 | 14:16 |
| egonzalez | manheim, cinder-api not work in ubuntu binary, work is under review atm | 14:17 |
| manheim | ah, ok | 14:17 |
| manheim | so I have a question on best practices | 14:17 |
| egonzalez | source or any other OS works | 14:17 |
| *** GonZo2000 has quit IRC | 14:18 | |
| egonzalez | is just that ubuntu remove cinder-api command in their ocata packages | 14:18 |
| manheim | can-t i just pip/apt install it in the cinder container/ | 14:18 |
| egonzalez | manheim, not recomended because you loose knowledge of what is in each image, next time you deploy again the change is gone | 14:19 |
| egonzalez | better use customizations during package building | 14:19 |
| manheim | or that ofc | 14:20 |
| manheim | but I cannot use image customization if I only pull, right? | 14:20 |
| manheim | btw, I saw that the same issue is for magnum-api | 14:21 |
| egonzalez | nope, customizations are only supported for image building | 14:21 |
| manheim | deployed both cinder and magnum on ubuntu binary | 14:21 |
| egonzalez | isn't magnum-api command present too? | 14:21 |
| manheim | from what i saw it had the same issue | 14:21 |
| openstackgerrit | Vladislav Belogrudov proposed openstack/kolla-ansible master: Fix MySQL DB create failure detection mechanism https://review.openstack.org/460141 | 14:22 |
| manheim | i can clean-containers and repeploy to make sure | 14:22 |
| manheim | this is the best practice, right? | 14:23 |
| manheim | for the clean container tool | 14:23 |
| egonzalez | cannot see any release note for magnum into apache http://lists.openstack.org/pipermail/openstack-dev/2017-January/111069.html | 14:23 |
| manheim | i am re-deploying | 14:24 |
| manheim | i saw on the summit kolla prez that work is done to improve the docs | 14:25 |
| manheim | this would be awesome | 14:25 |
| egonzalez | magnum-api command is present in 4.0.0, may be other issue | 14:26 |
| egonzalez | still need al lot of work imo | 14:26 |
| manheim | or it hay have been for magnum-ui | 14:26 |
| manheim | well, let me know if I can help with the docs | 14:26 |
| egonzalez | hrm, disable magnum-ui for ubuntu binary | 14:26 |
| manheim | i mean i would be a nice way for me to learn kolla as I document it | 14:27 |
| egonzalez | there is no magum dashboard package | 14:27 |
| manheim | i did, only enabled magnum simple | 14:27 |
| egonzalez | only supported for rpm based distros or source installs | 14:27 |
| manheim | what is magnum-ui exactly? | 14:27 |
| egonzalez | add enable_horizon_magnum: no into globals.yml | 14:28 |
| egonzalez | is the horizon plugin | 14:28 |
| egonzalez | manheim, btw, for ubuntu i'd recommend you to use source installs, often are more stable than binary | 14:29 |
| openstackgerrit | Vladislav Belogrudov proposed openstack/kolla-ansible master: Fix MySQL DB create failure detection mechanism https://review.openstack.org/460141 | 14:29 |
| manheim | ok, good to know | 14:30 |
| *** jascott1 has joined #openstack-kolla | 14:31 | |
| manheim | enable_horizon_magnum triggers magnum_ui right? | 14:31 |
| egonzalez | yep, but in ubuntu binary there is not any magnum_ui, so better disable it or horizon will fail copying required files | 14:32 |
| manheim | hmm, i had only magnum, no ui, and horizon gets erorrs about magnum_ui stil | 14:32 |
| manheim | cp: cannot stat '/usr/lib/python2.7/site-packages/magnum_ui/enabled/_*[^__].py': No such file or directory | 14:33 |
| manheim | this is horizon | 14:33 |
| manheim | i had enable_horizon_magnum disabled | 14:33 |
| egonzalez | by default, if you enable_magnum, magnum horizon is enabled too | 14:33 |
| manheim | ok | 14:33 |
| manheim | ah fuck | 14:33 |
| manheim | i get now what you said with enable_horizon_magnum: no | 14:34 |
| manheim | i thought if I comment it out it won't do it | 14:34 |
| manheim | i have to explicitly tell no | 14:34 |
| egonzalez | and do kolla-ansible (deploy/reconfigure), restart container is not enough since need to add the new env variable | 14:34 |
| *** jascott1 has quit IRC | 14:35 | |
| manheim | i used to do deploy -> tools/cleanup-containers -> redeploy | 14:35 |
| manheim | so reconfigure is better? | 14:35 |
| *** gomarivera has joined #openstack-kolla | 14:36 | |
| egonzalez | depends of what you going to do, is change a config file and restart containers, can use deploy/reconfigure | 14:38 |
| egonzalez | if need something in the db schema, upgrade | 14:39 |
| manheim | Running command: 'magnum-api --config-file /etc/magnum/magnum.conf' | 14:39 |
| egonzalez | if is something that is broken and touched many in places, cleanup and start fresh again | 14:39 |
| manheim | magnum containers are stuck here in restarting | 14:41 |
| egonzalez | any error? | 14:42 |
| manheim | nope | 14:42 |
| *** eaguilar has quit IRC | 14:42 | |
| *** gomarivera has quit IRC | 14:43 | |
| *** jistr is now known as jistr|afk | 14:44 | |
| manheim | also I have an old issue with horizon | 14:44 |
| manheim | the html/css are missing | 14:44 |
| manheim | i get 404 on them | 14:46 |
| *** chas has joined #openstack-kolla | 14:47 | |
| manheim | this happens on centos aswell, but there if i put debug on true in local_settings the probles dissapers | 14:47 |
| *** schwicht has quit IRC | 14:49 | |
| openstackgerrit | Eduardo Gonzalez proposed openstack/kolla stable/ocata: Fix Incorrect timezone in ubuntu horizon images https://review.openstack.org/464214 | 14:50 |
| openstackgerrit | Eduardo Gonzalez proposed openstack/kolla stable/ocata: Fix Incorrect timezone in ubuntu horizon images https://review.openstack.org/464214 | 14:51 |
| *** chas has quit IRC | 14:52 | |
| *** schwicht has joined #openstack-kolla | 14:56 | |
| *** manheim has quit IRC | 14:58 | |
| *** gomarivera has joined #openstack-kolla | 14:58 | |
| *** manheim has joined #openstack-kolla | 14:58 | |
| manheim | lol /usr/share/openstack-dashboard/ is inexistent | 14:59 |
| manheim | but in docker logs horizon i can see them copied | 15:00 |
| *** egonzalez has quit IRC | 15:02 | |
| *** egonzalez has joined #openstack-kolla | 15:03 | |
| manheim | http://paste.ubuntu.com/24561038/ this look ok to you? | 15:03 |
| *** gomarivera has quit IRC | 15:07 | |
| *** lpetrut has quit IRC | 15:09 | |
| manheim | nvm my last lines | 15:12 |
| manheim | but i have the same issue https://bugs.launchpad.net/kolla/+bug/1672741 | 15:12 |
| openstack | Launchpad bug 1672741 in kolla "Missing CSS and JS in Horizon" [Undecided,New] | 15:12 |
| manheim | i have debug = true but no result | 15:12 |
| *** eaguilar has joined #openstack-kolla | 15:13 | |
| *** JamesPR has joined #openstack-kolla | 15:13 | |
| JamesPR | hi everyone :) | 15:13 |
| JamesPR | trying kolla need some help | 15:13 |
| JamesPR | anyone available? | 15:13 |
| *** schwicht has quit IRC | 15:14 | |
| manheim | sure, shoot | 15:15 |
| JamesPR | trying to upload a image to glance give timeout error | 15:15 |
| JamesPR | 504 Gateway Time-out: The server didn't respond in time. (HTTP N/A) | 15:16 |
| JamesPR | maybe some setting on haproxy? | 15:16 |
| JamesPR | or apache timeout? | 15:16 |
| manheim | have you done some special configs from the vanilla ones? | 15:16 |
| JamesPR | no | 15:16 |
| manheim | anything logged in glance services? | 15:17 |
| openstackgerrit | Merged openstack/kolla-ansible stable/ocata: Implement Cinder, Nova Placement and mistral logs for kibana https://review.openstack.org/461937 | 15:18 |
| openstackgerrit | Merged openstack/kolla-ansible stable/ocata: Add local directive for chrony https://review.openstack.org/463661 | 15:18 |
| JamesPR | manheim, how do i look? | 15:19 |
| manheim | so there is a fluentd container which groups all services logs | 15:20 |
| manheim | docker exec -it fluentd bash | 15:20 |
| manheim | and cd /var/log/kolla/glance | 15:20 |
| manheim | tail -f * | 15:20 |
| manheim | and try to upload your image again as you watch logs | 15:20 |
| egonzalez | JamesPR the timeout is from the CLI or Horizon? is the image big >7GB ? | 15:21 |
| openstackgerrit | Eduardo Gonzalez proposed openstack/kolla-ansible stable/ocata: Fix glance Upload large image haproxy timeout https://review.openstack.org/464224 | 15:21 |
| egonzalez | JamesPR ^^ there is the backport for what i guess is your issue | 15:22 |
| JamesPR | egonzalez, https://pastebin.com/raw/6wPXhCN3 | 15:24 |
| egonzalez | JamesPR ceph as backend? check ceph status (docker exec ceph_mon ceph -s) | 15:25 |
| JamesPR | egonzalez, I think the issue is on haproxy because way back I configure a cluster manually with haproxy got the same issue I think I resolved with some parameter on haproxy.cfg | 15:25 |
| JamesPR | health HEALTH_ERR | 15:26 |
| JamesPR | lol | 15:26 |
| JamesPR | WTF ? :( | 15:26 |
| JamesPR | maybe clock screwed? | 15:28 |
| *** rhallisey has quit IRC | 15:29 | |
| *** Pavo has joined #openstack-kolla | 15:33 | |
| *** gomarivera has joined #openstack-kolla | 15:35 | |
| rwellum | mnaser: same :) | 15:36 |
| JamesPR | osdmap e6: 0 osds: 0 up, 0 in | 15:38 |
| JamesPR | no osd | 15:38 |
| JamesPR | in ceph | 15:38 |
| egonzalez | JamesPR have you labeled OSD disks before deployment? https://docs.openstack.org/developer/kolla-ansible/ceph-guide.html#preparation | 15:43 |
| JamesPR | egonzalez, I don't think so :( | 15:45 |
| *** schwicht has joined #openstack-kolla | 15:47 | |
| *** gomarivera has quit IRC | 15:48 | |
| openstackgerrit | Eduardo Gonzalez proposed openstack/kolla stable/ocata: Fix Incorrect timezone in ubuntu horizon images https://review.openstack.org/464214 | 15:55 |
| *** JamesPR has quit IRC | 15:57 | |
| *** manheim has quit IRC | 16:04 | |
| openstackgerrit | Merged openstack/kolla master: Fix if $user is root at extend_start https://review.openstack.org/464181 | 16:04 |
| *** Pavo has quit IRC | 16:05 | |
| *** jistr|afk is now known as jistr | 16:06 | |
| *** matrohon_ has quit IRC | 16:06 | |
| *** pcaruana has quit IRC | 16:12 | |
| openstackgerrit | Eduardo Gonzalez proposed openstack/kolla stable/ocata: Fix if $user is root at extend_start https://review.openstack.org/464230 | 16:12 |
| *** gomarivera has joined #openstack-kolla | 16:12 | |
| *** k_mouza has joined #openstack-kolla | 16:12 | |
| *** rhallisey has joined #openstack-kolla | 16:25 | |
| *** imcsk8 has quit IRC | 16:26 | |
| *** gomarivera has quit IRC | 16:26 | |
| *** salv-orlando has joined #openstack-kolla | 16:30 | |
| *** k_mouza has quit IRC | 16:31 | |
| *** jascott1 has joined #openstack-kolla | 16:32 | |
| *** jascott1 has quit IRC | 16:32 | |
| *** imcsk8 has joined #openstack-kolla | 16:35 | |
| *** manheim has joined #openstack-kolla | 16:37 | |
| *** jascott1 has joined #openstack-kolla | 16:41 | |
| *** manheim has quit IRC | 16:42 | |
| *** Manheim has joined #openstack-kolla | 16:43 | |
| *** zhenguo has quit IRC | 16:46 | |
| *** chas has joined #openstack-kolla | 16:48 | |
| *** unicell has joined #openstack-kolla | 16:51 | |
| *** jrist is now known as jrist-afk | 16:51 | |
| *** chas has quit IRC | 16:52 | |
| *** Pavo has joined #openstack-kolla | 16:53 | |
| *** unicell has quit IRC | 16:59 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/kolla master: Updated from global requirements https://review.openstack.org/462547 | 17:09 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/kolla-ansible master: Updated from global requirements https://review.openstack.org/462548 | 17:09 |
| *** jaosorior has quit IRC | 17:09 | |
| *** jrist-afk is now known as jrist | 17:12 | |
| *** salv-orl_ has joined #openstack-kolla | 17:13 | |
| *** eaguilar has quit IRC | 17:16 | |
| *** salv-orlando has quit IRC | 17:16 | |
| openstackgerrit | Justin Scott proposed openstack/kolla-kubernetes master: WIP Playbook implements UserGuide https://review.openstack.org/457317 | 17:22 |
| *** Pavo has quit IRC | 17:28 | |
| openstackgerrit | Justin Scott proposed openstack/kolla-kubernetes master: Playbook implements UserGuide https://review.openstack.org/457317 | 17:30 |
| openstackgerrit | Justin Scott proposed openstack/kolla-kubernetes master: Playbook implements Deployment Guide https://review.openstack.org/457317 | 17:30 |
| *** unicell has joined #openstack-kolla | 17:32 | |
| *** gomarivera has joined #openstack-kolla | 17:48 | |
| rwellum | rhallisey: ping | 17:49 |
| rhallisey | rwellum, hey | 17:49 |
| rwellum | Hey rhallisey - guess not such a long trip home for you as others? :) | 17:50 |
| rhallisey | i know :) | 17:50 |
| rwellum | Don't know if you can help with this or not - but can you check out this pastebin? http://paste.openstack.org/show/609392/ | 17:51 |
| rwellum | This happens to me since the last kolla-ansible cleanup. | 17:51 |
| rhallisey | I'll have a look | 17:52 |
| rwellum | The first one I do not know if it's related or not. But the jinja2 error stops me from proceeding. You had the same issue in the past. | 17:52 |
| rwellum | Cheers - no rush. | 17:52 |
| rhallisey | so hostvars is an ansible specific var | 17:54 |
| rhallisey | http://docs.ansible.com/ansible/faq.html#how-do-i-see-all-the-inventory-vars-defined-for-my-host | 17:54 |
| *** iniazi has quit IRC | 17:56 | |
| *** david-lyle has joined #openstack-kolla | 17:56 | |
| rwellum | Yeah so how am I seeing this - running kolla-kubernenes? From the ansible playbook in the gen config stage? | 17:56 |
| *** Pavo has joined #openstack-kolla | 17:57 | |
| sdake | pomac we use libvirt from the virtualization sig, not 1.5.3 | 17:58 |
| pomac | sdake: we noticed, but __com.redhat_disk_add is not in the qemu sourcecode - dunno how that part works | 17:59 |
| rhallisey | oh so that playbook is only for config | 17:59 |
| rhallisey | that's new | 17:59 |
| rhallisey | rwellum, do you know what file secret-generating is reading? | 18:00 |
| * rhallisey looks | 18:00 | |
| rhallisey | my guess is that file has hostvars in it that ansible never rendered | 18:01 |
| * rwellum looked but it wasn't clear to me | 18:01 | |
| *** eaguilar has joined #openstack-kolla | 18:02 | |
| rhallisey | rwellum, maybe /etc/kolla/passwords.yml | 18:04 |
| rwellum | I don't see 'hostvars' there | 18:05 |
| sdake | sup peeps | 18:09 |
| sdake | I did notice ansible 2.3.0.0 is out and about | 18:09 |
| sdake | (in the centos epel repo) | 18:09 |
| sdake | it is possible this version of ansible is incompatible with kolla | 18:09 |
| rwellum | sdake: specific to my issue above or just in general? | 18:11 |
| sdake | rwellum run pip freeze | grep inja2 | 18:11 |
| *** iniazi has joined #openstack-kolla | 18:11 | |
| rwellum | Jinja2==2.9.6 - but that was an issue before and I swapped to using 2.8.1, which worked but then the issue came back | 18:12 |
| sdake | rwellum also run pip freeze | grep ansible | 18:12 |
| sdake | and rpm -qa | grep ansible | 18:12 |
| *** salv-orl_ has quit IRC | 18:13 | |
| *** Pavo has quit IRC | 18:13 | |
| rwellum | sdake: https://www.irccloud.com/pastebin/9L8y3Z80/ | 18:14 |
| rhallisey | rwellum, I wonder if you have a var that is undefined | 18:14 |
| rhallisey | like something missing from globals or something | 18:14 |
| sdake | rhallisey unlikely | 18:15 |
| *** mannamne has quit IRC | 18:15 | |
| *** yuanying_ has joined #openstack-kolla | 18:17 | |
| rhallisey | https://bugs.launchpad.net/kolla-kubernetes/+bug/1649247 | 18:17 |
| openstack | Launchpad bug 1649247 in kolla-kubernetes "Secret generate fail with hostvars is undefined for kubernetes orchestration engine" [Critical,Confirmed] | 18:17 |
| egonzalez | sdake i have ansible 2.3 and Jinja2==2.8.1, just saw a couple of warnings with reserved ansible variables | 18:17 |
| rhallisey | rwellum, similar error ^ | 18:17 |
| sdake | rwellum my plane internet is slow - ican't seem to open your link | 18:18 |
| sdake | rwellum can you tell me which version of jinja2 you have installed | 18:18 |
| sdake | rwellum and which version of ansible (and whether its rpm or pip based) | 18:18 |
| *** yuanying has quit IRC | 18:19 | |
| rwellum | jinja2 - 2.8.1 and 2.9.6, ansible 2.3.0.0 and kolla-ansible 4.0.0.0rc2.dev542 | 18:21 |
| egonzalez | btw, is kolla-k8s consumeing templates from master or stable/ocata? | 18:21 |
| *** gomarivera has quit IRC | 18:22 | |
| rwellum | rhallisey: I thought that was an old bug but now I see there's been a reply in the last 14 hours | 18:22 |
| sdake | egonzalez kolla-k8s consumes templates from the kolla-kubernetes repository | 18:22 |
| sdake | rwellum why installing kolla-ansible? | 18:22 |
| rwellum | sdake: part of the deployment guide? | 18:22 |
| sdake | rwellum let me change that - moment | 18:23 |
| rwellum | But not running any commands from it. | 18:23 |
| rwellum | sdake: sudo ansible-playbook -e ansible_python_interpreter=/usr/bin/python -e @/etc/kolla/globals.yml -e @/etc/kolla/passwords.yml -e CONFIG_DIR=/etc/kolla ./ansible/site.yml | 18:23 |
| rwellum | [WARNING]: provided hosts list is empty, only localhost is available | 18:23 |
| sdake | rwellum i see | 18:24 |
| sdake | rwellum ok - then thats good | 18:24 |
| sdake | rwellum ok - so you have jinaj 2.81 and 2.9.6 installed at same time - how is that ? | 18:24 |
| sdake | rwellum here is what I'd recommend | 18:24 |
| sdake | rwellum first figure out how they are installed (via pip or via rpm) | 18:24 |
| rwellum | sdake: no I mean I tried both | 18:24 |
| sdake | rwellum i see | 18:24 |
| rwellum | Same issue | 18:24 |
| sdake | rwellum that warning may be ignored | 18:24 |
| *** schwicht has quit IRC | 18:25 | |
| sdake | rwellum how did you install ansible 2.3? | 18:25 |
| sdake | pip or rpm? | 18:25 |
| rwellum | yum | 18:25 |
| *** schwicht has joined #openstack-kolla | 18:25 | |
| rwellum | as in deployment guide | 18:25 |
| sdake | rwellum you ahve to forgive me, i can't really access much of the internet atm so not sure what is in the deployment guide at present :) | 18:26 |
| sdake | i'm on a clone atm, 4% of repo downloaded :( | 18:26 |
| rwellum | That's fine - just pointing out I am not deviating or doing anything weird I think. | 18:26 |
| sdake | rwellum roger | 18:27 |
| sdake | rwellum would you try uninstalling ansible and pip install ansible==2.2.0.0 | 18:28 |
| rwellum | Doing that now sdake and re-running | 18:30 |
| jascott1 | rwellum what os? | 18:37 |
| rwellum | sdake: same issue with ansible 2.2.0.0 | 18:37 |
| rwellum | jascott1: centos7 vm | 18:37 |
| sdake | rwellum can you tell me which version of jinja2 you hae installed now plz | 18:38 |
| rwellum | 2.9.6 | 18:38 |
| sdake | rwellum pip install jinja2==2.8.2 | 18:39 |
| sdake | pip install -U jinja2==2.8.2 | 18:39 |
| sdake | jinja2.9 is a development series, and it is always broken | 18:39 |
| sdake | 2.8.z is the last stable release of jiinja2 | 18:39 |
| sdake | i alwasy found it odd that devs used odd y semver to identify a development version | 18:40 |
| rwellum | sdake: ack - but I've had this issue with 2.8.1 - trying again now | 18:41 |
| sdake | with ansible 2.3.0 :) | 18:41 |
| sdake | it is possible your experience will be the same with ansible 2.2, although we need to verify that | 18:42 |
| sdake | rwellum have you got through step 2 (validating kubernetes) | 18:42 |
| sdake | bunch of versions changing of dependencies = chaos :) | 18:43 |
| rwellum | sdake 2.8.2 does not exist | 18:43 |
| rwellum | :) | 18:44 |
| rwellum | 2.8.1 it is then | 18:44 |
| jascott1 | i have running centos 7 atm if you want package list or something | 18:44 |
| sdake | jascott1 pip freeze | grep ansible | 18:44 |
| sdake | jascott1 pip freeze | grep inja2 | 18:44 |
| jascott1 | oops was in middle or tinkering, will have it in 5m | 18:45 |
| rwellum | thanks jascott1 | 18:45 |
| *** chas has joined #openstack-kolla | 18:48 | |
| rwellum | Same issue sdake with ansible 2.2.0.0 and jinja2 2.8.1 | 18:50 |
| sdake | rwellum can you paste your globals.yml as well as kolla_kubernetes.yml plz | 18:50 |
| rwellum | I can but you won't be able to read it right? | 18:51 |
| sdake | rwellum give it a try | 18:52 |
| sdake | rwellum internet seems to have imporved | 18:52 |
| *** chas has quit IRC | 18:53 | |
| jascott1 | sdake does the gate job output that info? it might be nice for a goto reference | 18:53 |
| rwellum | http://paste.openstack.org/show/609462/ | 18:53 |
| jascott1 | that info = pip versions etc | 18:53 |
| rwellum | What is the second file sdake? kolla_kubernetes.yml? | 18:53 |
| sdake | rwellum ypu - in /etc/kolla-kubernetes directory | 18:54 |
| rwellum | http://paste.openstack.org/show/609464/ | 18:55 |
| rwellum | jascott1: good question on the gates | 19:01 |
| *** zhubingbing_ has quit IRC | 19:01 | |
| jascott1 | ansible==2.3.0.0, kolla-ansible==4.0.0.0rc2.dev542 | 19:02 |
| *** zhubingbing__ has joined #openstack-kolla | 19:02 | |
| rwellum | jascott1: and jinja2 ? | 19:02 |
| *** harlowja has quit IRC | 19:02 | |
| jascott1 | Jinja2==2.9.6 | 19:03 |
| sdake | rwellum cloud.yaml? | 19:04 |
| jascott1 | rwellum heres my pip freeze for ref http://paste.openstack.org/show/609465/ | 19:05 |
| sdake | rwellum these need to be set: | 19:05 |
| sdake | external_vip: 10.23.230.12 | 19:05 |
| sdake | kolla_kubernetes_external_vip: "10.23.230.12" | 19:05 |
| sdake | kolla_kubernetes_external_subnet: "28" | 19:05 |
| sdake | (if your using keepalived and a VIP address) | 19:05 |
| sdake | i recognize there is a duplicate, bug in the code imo | 19:05 |
| sdake | that external vip should be an unused ip on your network | 19:05 |
| sdake | and dns_name should also be set to the same | 19:05 |
| sdake | then you should run keepalived - and victory :) | 19:06 |
| sdake | rwellum although i am buildilng now and preparing to test master | 19:06 |
| *** eaguilar has quit IRC | 19:14 | |
| rhallisey | rwellum, did you get it working? Sorry had to deal with something | 19:16 |
| rwellum | sdake: cloud.yaml: http://paste.openstack.org/show/609468/ | 19:22 |
| rwellum | rhallisey: nah - still working it with sdake help - jascott1 too | 19:22 |
| rwellum | Pretty sure hostvars is coming from: https://github.com/openstack/kolla-kubernetes/blob/master/ansible/group_vars/all.yml | 19:23 |
| rwellum | That seems wrong - if this is ansible specific. | 19:24 |
| rwellum | rhallisey: do you agree this is the hostvars variable it's complaining about? Although sdake thinks this may just be a harmless warning. | 19:27 |
| rhallisey | rwellum, what do you have orchestration_engine set to? | 19:27 |
| sdake | rwellum the harmless warning was the warning about localhost | 19:28 |
| sdake | not about hostvars | 19:28 |
| sdake | there should be no hostvars anywhere in the codebase in kolla-kubernetes | 19:28 |
| rwellum | sdake: duh - sorry | 19:28 |
| sdake | rwellum those hostvars should all be overridden by the globals.yml | 19:29 |
| rwellum | sdake: right here though: https://github.com/openstack/kolla-kubernetes/blob/master/ansible/group_vars/all.yml | 19:29 |
| sdake | rwellum unless globals.yml is missing something | 19:29 |
| sdake | e.g. https://github.com/openstack/kolla-kubernetes/blob/master/ansible/group_vars/all.yml#L57 | 19:29 |
| sdake | this is overridden by api_interface in globals.yml | 19:29 |
| rhallisey | rwellum, can you paste your all.yaml file | 19:29 |
| sdake | not the correct way to do it - globals.yml overrides should jut be merged in that lal.yaml file | 19:29 |
| rhallisey | or you globals | 19:30 |
| rwellum | rhallisey: pasted that above | 19:30 |
| rwellum | rhallisey: http://paste.openstack.org/show/609462/ | 19:31 |
| rhallisey | looks like everything get overwritten correctly | 19:32 |
| rwellum | rhallisey: yeah at this point I am scripting it - hoping to swap to jascott1 ansible playbook before too long. So when it fails it usually isn't something I've done. | 19:33 |
| rhallisey | gotcha | 19:33 |
| *** fooliouno has joined #openstack-kolla | 19:34 | |
| jascott1 | lol | 19:35 |
| jascott1 | werd | 19:35 |
| sdake | rwellum my guess is your globals.yml is not configured properly | 19:38 |
| sdake | rwellum a fresh deploy with my existing config files works | 19:39 |
| sdake | rwellum (with master and ansible 2.3) | 19:39 |
| jascott1 | i can say the playbook gets all the containers running happily but I suspect the config files need some tweaking | 19:42 |
| jascott1 | whats a good smoke test to run against system? | 19:42 |
| *** eaguilar has joined #openstack-kolla | 19:43 | |
| rwellum | Ok sdake will keep looking | 19:43 |
| sdake | rwellum globals: https://da.gd/nDlX -> https://paste.fedoraproject.org/paste/4ILKtEsmMFdhgXfXmEXsJ15M1UNdIGYhyRLivL9gydE=/ | 19:43 |
| sdake | rwellum kolla-kubernetes.yml: https://da.gd/RvMEy -> https://paste.fedoraproject.org/paste/vn1ynfh9MJdeJsTL4R2qJ15M1UNdIGYhyRLivL9gydE=/ | 19:44 |
| sdake | rwellum cloud.yaml: https://da.gd/rdSxR -> https://paste.fedoraproject.org/paste/MgghWdoOu-~-iyDBTvo6uV5M1UNdIGYhyRLivL9gydE=/ | 19:44 |
| sdake | i did notice an error during the scrolling of deployment - it may be the problem you have | 19:45 |
| sdake | I am going to rerun and capture in paste | 19:45 |
| sdake | rwellum what was your original error please? | 19:49 |
| sdake | note it takes 4minutes20seconds to clean up with the destroy playbook :) | 19:49 |
| sdake | jascott1 thats for you bro :) | 19:49 |
| rwellum | sdake: https://www.irccloud.com/pastebin/DsekkVRe/ | 19:50 |
| *** david-lyle has quit IRC | 19:51 | |
| sdake | rwellum can you paste from the beginning of the command to the end | 19:52 |
| sdake | not just the scrollbackp lz | 19:52 |
| sdake | backtrce rather | 19:52 |
| rwellum | http://paste.openstack.org/show/609392/ | 19:52 |
| jascott1 | sdake haha i ran the commands manually earlier.. time approximation is legit | ) | 19:53 |
| *** erobell has joined #openstack-kolla | 19:54 | |
| erobell | I have been following the instructions here: https://docs.openstack.org/developer/kolla-kubernetes/deployment-guide.html with centos 7.3. Everything went fine until I got the nslookup. I get nslookup: can't resolve 'kubernetes' | 19:56 |
| *** athomas has quit IRC | 19:56 | |
| erobell | I haven't found any error or warning logs that indicate why that could happen. | 19:56 |
| erobell | Any suggestions for how to debug this issue? | 19:57 |
| kolla-slack | <vhosakot> @rwellum can you check https://bugs.launchpad.net/kolla-kubernetes/+bug/1649247 and the fix in https://review.openstack.org/#/c/417331/. Looks like a jinja is using a var missing in the inventory. | 19:58 |
| openstack | Launchpad bug 1649247 in kolla-kubernetes "Secret generate fail with hostvars is undefined for kubernetes orchestration engine" [Critical,Confirmed] | 19:58 |
| *** rwallner has joined #openstack-kolla | 19:59 | |
| sdake | erobell getting to stage 2 (a working kubernetes deploymnent) is indeed the hardest step :) | 20:00 |
| sdake | erobell it is unclear to me if the docs have been udpated to dea lwith kubernetes 1.6.3 (as of yet) | 20:00 |
| erobell | Are there other docs to follow, or should I use a previous version? | 20:01 |
| sdake | erobell sadly the upstream kubernetes docs are not particularly good | 20:01 |
| sdake | erobell which is why we document kubernetes | 20:01 |
| jascott1 | PSA: ansible has a --start-at-task comand useful when developing playbooks - http://docs.ansible.com/ansible/playbooks_startnstep.html | 20:02 |
| sdake | erobell whenever a version chagne happens, there is typically chaos for 1 week | 20:02 |
| sdake | erobell version change being kubernetes | 20:02 |
| sdake | jascott1 which version of kubernetes are you using? | 20:02 |
| jascott1 | 1.6.2 | 20:02 |
| sdake | erobell run rpm -qa | grep kubelet | 20:03 |
| erobell | sdake kubelet-1.6.3-0.x86_64 | 20:04 |
| sdake | erobell try downgrading the dependencies (not sure if that will work) | 20:04 |
| sdake | erobell lets start small with yum downgrade kubelet | 20:04 |
| *** rwallner has quit IRC | 20:04 | |
| sdake | erobell and run the rpm -qa command again plz | 20:04 |
| jascott1 | 1.6.3 was busted for me, havent looked into it | 20:04 |
| rwellum | vhosakat - that issue is fixed though? | 20:04 |
| sdake | rwellum you ahve a config problem dude ;) | 20:05 |
| sdake | rwellum which is why i pasted my config files | 20:05 |
| sdake | rwellum granted kolla shouldn't implode on config value problems | 20:05 |
| erobell | sdrake Ok, did that kubelet-1.6.2-0.x86_64 | 20:05 |
| sdake | erobell ok there are other deps there | 20:06 |
| sdake | erobell in the yum install oepration | 20:06 |
| sdake | anything related to kubernetes needs a downgrade | 20:06 |
| sdake | YMMV if that works :) | 20:06 |
| kolla-slack | <vhosakot> @rwellum right, https://review.openstack.org/#/c/417331/ is merged, can you check if you have that variable set in your env. The bug description in https://bugs.launchpad.net/kolla-kubernetes/+bug/1649247 says the var must be defined in kolla-ansible’s group vars too which I do see in https://github.com/openstack/kolla-ansible/blob/master/ansible/group_vars/all.yml#L122. | 20:08 |
| openstack | Launchpad bug 1649247 in kolla-kubernetes "Secret generate fail with hostvars is undefined for kubernetes orchestration engine" [Critical,Confirmed] | 20:08 |
| kolla-slack | <vhosakot> @sdake are you in flight? :slightly_smiling_face: | 20:08 |
| kolla-slack | <vhosakot> I should stop using slack with IRC :slightly_smiling_face: let me login into IRC>.. lol | 20:09 |
| *** vhosakot has joined #openstack-kolla | 20:10 | |
| vhosakot | in IRC now! | 20:10 |
| sdake | vhosakot yup | 20:11 |
| vhosakot | sdake: cool, safe travels. | 20:11 |
| sdake | rwellum if your not in the midst of something would you try 2.7.2 of jinja2? | 20:12 |
| erobell | sdake: Thanks for the help! Downgraded everything, and trying to get things going again. | 20:13 |
| *** Teck72 has joined #openstack-kolla | 20:15 | |
| rwellum | sdake: comparing cloud.yml files - you have a lot that I do not - and is also not in the current documentation? | 20:16 |
| *** manheim_ has joined #openstack-kolla | 20:17 | |
| rwellum | http://paste.openstack.org/show/609469/ | 20:18 |
| sdake | rwellum indeed | 20:18 |
| sdake | erobell np | 20:18 |
| *** Teck7 has quit IRC | 20:19 | |
| *** Pavo has joined #openstack-kolla | 20:19 | |
| sdake | rwellum note i think your config problem is with globals.yml or kolla-kubernetes.yml | 20:19 |
| sdake | not cloud.yaml | 20:19 |
| *** salv-orlando has joined #openstack-kolla | 20:20 | |
| rwellum | https://www.irccloud.com/pastebin/XUOXCuYU/ | 20:20 |
| rwellum | And the keep-alived? | 20:21 |
| sdake | rwellum if your using keepalived - yes | 20:21 |
| sdake | rwellum you can run without keepalived | 20:21 |
| sdake | rwellum i am running with 3 nodes, so I need keepalived | 20:21 |
| rwellum | What about the first two lines? kolla_k8s_external etc? | 20:21 |
| *** manheim_ has quit IRC | 20:22 | |
| pomac | i have two odd things with kolla deploying ocata, one is that nova seems to have changed and get's it's user and rbd_secret_uuid from cinder-volume... The second is that nova sends __com.redhat_disk_add which doesn't seem to be in standard qemu when attaching volumes - we have this problem on newton as well... The third one is that after fiddling with this nothing seems to work - even when back to | 20:22 |
| pomac | normal | 20:22 |
| *** fooliouno has quit IRC | 20:23 | |
| *** Pavo has quit IRC | 20:24 | |
| *** salv-orlando has quit IRC | 20:25 | |
| rwellum | sdake - 2.7.2 - gives me: jinja2.exceptions.TemplateRuntimeError: no test named 'equalto' | 20:25 |
| pomac | feels like nova and cinder shouldn't be separated anymore | 20:28 |
| *** eaguilar has quit IRC | 20:29 | |
| jascott1 | \\]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]] | 20:34 |
| jascott1 | ]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]] | 20:34 |
| jascott1 | ]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]] | 20:34 |
| erobell | sdake I downgraded everything, restarted, but my node status doesn't go to Ready state. Also I notice that my kubectl server is still 1.6.3 even though kubectl was downgraded to 1.6.2 | 20:36 |
| sdake | erobell right - you need to install, then downgrade, then configure and start | 20:36 |
| sdake | erobell the eror you went in was instead install, downgrade, let er rip :) | 20:37 |
| sdake | erobell this second path is not going to work (as you can see) | 20:37 |
| erobell | sdake I attempted to do that by doing a kubeadm reset, then a kubeadm init again. | 20:38 |
| sdake | pathfinder is going to be the death of me | 20:39 |
| sdake | erobell you need to stop the services (or reboot) | 20:39 |
| sdake | jascott1 ure keyboard is busted :) | 20:39 |
| *** eaguilar has joined #openstack-kolla | 20:40 | |
| erobell | sdake Ok, so kubeadm reset, reboot the server, and then try again? When would it be expected that kubectl server would downgrade? | 20:40 |
| *** david-lyle has joined #openstack-kolla | 20:43 | |
| *** Pavo has joined #openstack-kolla | 20:44 | |
| *** david-lyle has quit IRC | 20:48 | |
| *** eaguilar has quit IRC | 20:51 | |
| *** mannamne has joined #openstack-kolla | 20:56 | |
| *** sbezverk has joined #openstack-kolla | 20:56 | |
| *** jtriley has quit IRC | 20:58 | |
| openstackgerrit | Steven Dake proposed openstack/kolla-kubernetes master: DNM: Test of removal of a bunch of crap in pathfinder https://review.openstack.org/464271 | 20:58 |
| *** Pavo has quit IRC | 21:03 | |
| sdake | erobell id really just remove it all and start fres hwith a downgrade after an install | 21:04 |
| sdake | erobell other then that, I'm 36k feet in the air and unable to help you | 21:04 |
| erobell | sdake Thanks again, yes I decided to reformat the drive and start over with only installing 1.6.2 | 21:06 |
| sdake | erobell most folks use vms to avoid a reformat :) | 21:07 |
| *** gomarivera has joined #openstack-kolla | 21:08 | |
| *** gomarivera has quit IRC | 21:12 | |
| rwellum | sdake: think I found one of my issues. I was piping into my cloud.yaml - but I was one (hidden) character off. Only saw it when I diffed with your version.. | 21:13 |
| openstackgerrit | Steven Dake proposed openstack/kolla-kubernetes master: DNM: Test of removal of a bunch of crap in pathfinder https://review.openstack.org/464271 | 21:13 |
| sdake | rwellum victory :) | 21:13 |
| sdake | which hidden character is missing? | 21:14 |
| sdake | globals.yml and kolla_kubernetes.yml need to GTFO | 21:14 |
| sdake | (for kolla-kubernetes) | 21:14 |
| sdake | they are confusing and unnecessary | 21:14 |
| rwellum | I had a control character - and it was causing the format to be off by 1 | 21:14 |
| sdake | that makes sense | 21:14 |
| rwellum | Yeah - lets have 1 file please? cloud.yaml | 21:15 |
| sdake | rwellum make it happen I'll +2 it :) | 21:15 |
| sdake | that is sort of what 464271 is | 21:15 |
| rwellum | Will take a look | 21:15 |
| sdake | there are two tools we use | 21:15 |
| sdake | one is a secret generator | 21:15 |
| sdake | one is a config map generator | 21:15 |
| sdake | if we can refactor those, then none of those config files are necessary | 21:15 |
| sdake | the secret generator is easy to untangle from the config files | 21:16 |
| sdake | kfox1111 has some initital work to do secret generation and configmap generation in a different way | 21:17 |
| sdake | any of these solutions wil lwork - basically we need to delete kollakube as a command :) | 21:18 |
| sdake | (and secret-generator to a lesser degree or untangle it from all but passwords.yml) | 21:18 |
| *** gomarivera has joined #openstack-kolla | 21:22 | |
| jascott1 | oh sh17 sorry guys i think my dog got on the keyboard | 21:24 |
| bmace__ | jascott1: lol :) | 21:25 |
| *** schwicht has quit IRC | 21:28 | |
| rwellum | Maybe he was looking for meeow - jascott1 ? | 21:33 |
| jascott1 | shes sweet dog but just doesnt grok functional programming | 21:33 |
| *** gomarivera has quit IRC | 21:34 | |
| jascott1 | its all in the dew claw | 21:34 |
| *** david-lyle has joined #openstack-kolla | 21:42 | |
| *** jascott1 has quit IRC | 21:43 | |
| *** mannamne has quit IRC | 21:44 | |
| *** jascott1 has joined #openstack-kolla | 21:45 | |
| *** david-lyle has quit IRC | 21:49 | |
| openstackgerrit | Steven Dake proposed openstack/kolla-kubernetes master: Remove pathfinder from call path of secret-generator https://review.openstack.org/464273 | 21:59 |
| *** rwallner has joined #openstack-kolla | 22:01 | |
| *** erobell has quit IRC | 22:02 | |
| *** Pavo has joined #openstack-kolla | 22:26 | |
| *** rwallner has quit IRC | 22:27 | |
| *** harlowja has joined #openstack-kolla | 22:40 | |
| *** Pavo has quit IRC | 22:51 | |
| *** Pavo has joined #openstack-kolla | 22:53 | |
| *** gfidente has quit IRC | 23:15 | |
| *** bmace__ has quit IRC | 23:16 | |
| *** bmace__ has joined #openstack-kolla | 23:16 | |
| kolla-slack | <sdake> @pomac are u using arc or binary builds | 23:17 |
| jascott1 | sdake whats your opinion on including things like terraform files with playbook? I already have templates for aws and openstack in progress | 23:39 |
| *** rwellum has quit IRC | 23:44 | |
| *** goldyfruit has joined #openstack-kolla | 23:46 | |
| sdake | jascott1 not sure what terraform is in detail | 23:48 |
| sdake | jascott1 although I think it would benefit us to reduce any hard dependencies we may ahve as much as feasible | 23:48 |
| jascott1 | cloud automation.. from hasicorp (vagrant, vault, consul etc) | 23:49 |
| sdake | yup i know that much ;) | 23:49 |
| sdake | what I mean is I haven't actually written any terraform work | 23:49 |
| sdake | what license? | 23:49 |
| jascott1 | i have it creating instances and other things at AWS and then creating an ansible inventory file with the IPs | 23:50 |
| jascott1 | its neat | 23:50 |
| *** jtriley has joined #openstack-kolla | 23:50 | |
| jascott1 | not sure one sec | 23:50 |
| kolla-slack | <sdake> ultimately our ansible code doesn't relaly need an inventory (atleast for the workflow bits) | 23:50 |
| jascott1 | mozilla public | 23:51 |
| jascott1 | https://github.com/hashicorp/terraform/blob/master/LICENSE | 23:51 |
| sdake | jascott1 have you seen the playbook i wrote for destroy? | 23:51 |
| jascott1 | i think as long as it just holds the group/ip info and no other meta data its fine | 23:51 |
| sdake | no ip is needed really | 23:51 |
| jascott1 | yes i have, posted a comment | 23:51 |
| sdake | unless deploying kubernetes itself | 23:51 |
| jascott1 | right | 23:51 |
| sdake | which a lot of people are already working on :) | 23:51 |
| *** egonzalez has quit IRC | 23:53 | |
| jascott1 | but all of that together is nice, I can source aws creds, run terraform, then run playbook with inventory from terraform and ive got everything | 23:53 |
| jascott1 | everything = k8s with kolla-kubernetes installed | 23:54 |
| *** Pavo has joined #openstack-kolla | 23:54 | |
| jascott1 | the drinking/guitar hour approacheth! | 23:56 |
| sdake | i htink we can expect people to bring their own kubernetes | 23:59 |
| sdake | (I hope:) | 23:59 |
| sdake | there are teams as large as kolla's entire team working on 1 of 7 different deployment tools for kubernetes | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!