| *** schwicht has joined #openstack-kolla | 00:00 | |
| *** ducttape_ has quit IRC | 00:04 | |
| *** yingjun has joined #openstack-kolla | 00:05 | |
| *** itlinux has joined #openstack-kolla | 00:09 | |
| masber | good morning, is there any documentation to enable SR-IOV in kolla-ansible? thank you very much | 00:14 |
|---|---|---|
| *** manheim has joined #openstack-kolla | 00:16 | |
| *** ducttape_ has joined #openstack-kolla | 00:16 | |
| *** manheim has quit IRC | 00:20 | |
| *** ducttape_ has quit IRC | 00:21 | |
| *** schwicht has quit IRC | 00:25 | |
| *** harlowja has quit IRC | 00:25 | |
| *** lucasxu has joined #openstack-kolla | 00:27 | |
| openstackgerrit | Marcus Williams proposed openstack/kolla master: Update OpenDaylight Container to Carbon SR1 https://review.openstack.org/491638 | 00:30 |
| *** ducttape_ has joined #openstack-kolla | 00:34 | |
| *** mnasiadka has joined #openstack-kolla | 00:43 | |
| *** mnasiadka has quit IRC | 00:47 | |
| *** mdnadeem has joined #openstack-kolla | 00:49 | |
| *** ducttape_ has quit IRC | 00:53 | |
| *** schwicht has joined #openstack-kolla | 00:55 | |
| *** mdnadeem has quit IRC | 00:57 | |
| *** lucasxu has quit IRC | 00:57 | |
| duonghq | morning guys | 01:01 |
| *** ducttape_ has joined #openstack-kolla | 01:03 | |
| *** ipsecguy_ has joined #openstack-kolla | 01:04 | |
| *** ducttape_ has quit IRC | 01:04 | |
| *** ducttape_ has joined #openstack-kolla | 01:05 | |
| *** lucasxu has joined #openstack-kolla | 01:06 | |
| *** ducttape_ has quit IRC | 01:07 | |
| *** ducttape_ has joined #openstack-kolla | 01:07 | |
| *** ipsecguy has quit IRC | 01:07 | |
| *** ducttape_ has quit IRC | 01:08 | |
| *** zhurong has joined #openstack-kolla | 01:11 | |
| *** schwicht has quit IRC | 01:14 | |
| *** yangyapeng has joined #openstack-kolla | 01:18 | |
| *** hieulq has quit IRC | 01:23 | |
| *** zhubingbing has joined #openstack-kolla | 01:27 | |
| *** caowei has joined #openstack-kolla | 01:27 | |
| openstackgerrit | Merged openstack/kolla-ansible stable/ocata: Add default volume_backend_name for rbd driver in cinder volume https://review.openstack.org/491208 | 01:30 |
| *** rwsu has quit IRC | 01:32 | |
| zhubingbing | hi guys | 01:34 |
| *** jtriley has quit IRC | 01:34 | |
| *** rwsu has joined #openstack-kolla | 01:44 | |
| openstackgerrit | Lei Xu proposed openstack/kolla master: Add chrony link in README.rst https://review.openstack.org/491501 | 01:44 |
| openstackgerrit | Lei Xu proposed openstack/kolla master: Add chrony link in README.rst https://review.openstack.org/491501 | 01:46 |
| openstackgerrit | Merged openstack/kolla master: Add chrony link in README.rst https://review.openstack.org/491501 | 01:51 |
| openstackgerrit | zhubingbing proposed openstack/kolla-ansible master: Move placement-api-access log formate to apache_access https://review.openstack.org/491423 | 01:58 |
| *** jamesbenson has joined #openstack-kolla | 02:02 | |
| *** jtriley has joined #openstack-kolla | 02:06 | |
| *** jamesbenson has quit IRC | 02:06 | |
| *** ducttape_ has joined #openstack-kolla | 02:13 | |
| *** jtriley has quit IRC | 02:15 | |
| *** afranc has quit IRC | 02:17 | |
| *** ducttape_ has quit IRC | 02:18 | |
| *** aagate has quit IRC | 02:18 | |
| *** jowisz has quit IRC | 02:18 | |
| *** ducttape_ has joined #openstack-kolla | 02:18 | |
| *** rbergeron has quit IRC | 02:18 | |
| *** jowisz has joined #openstack-kolla | 02:19 | |
| *** rbergeron has joined #openstack-kolla | 02:19 | |
| duonghq | hi zhubingbing | 02:20 |
| *** tovin07_ has joined #openstack-kolla | 02:20 | |
| zhubingbing | hi | 02:20 |
| zhubingbing | hi <duonghq | 02:20 |
| openstackgerrit | Pete Birley proposed openstack/kolla-kubernetes master: DNM: Ironic gate test https://review.openstack.org/491648 | 02:22 |
| *** afranc has joined #openstack-kolla | 02:22 | |
| *** ducttape_ has quit IRC | 02:23 | |
| *** ddyer has quit IRC | 02:25 | |
| *** aagate has joined #openstack-kolla | 02:26 | |
| *** ducttape_ has joined #openstack-kolla | 02:39 | |
| *** schwicht has joined #openstack-kolla | 02:40 | |
| *** mnasiadka has joined #openstack-kolla | 02:43 | |
| *** mnasiadka has quit IRC | 02:47 | |
| *** unicell has quit IRC | 02:51 | |
| *** caowei has quit IRC | 02:52 | |
| *** caowei has joined #openstack-kolla | 02:52 | |
| *** awiddersheim has joined #openstack-kolla | 02:55 | |
| *** ducttap__ has joined #openstack-kolla | 03:03 | |
| *** jtriley has joined #openstack-kolla | 03:03 | |
| *** awiddersheim has quit IRC | 03:03 | |
| *** ducttape_ has quit IRC | 03:06 | |
| *** stefan_bo has joined #openstack-kolla | 03:08 | |
| *** jtriley has quit IRC | 03:08 | |
| *** jtriley has joined #openstack-kolla | 03:09 | |
| *** Teck7 has joined #openstack-kolla | 03:11 | |
| *** stefan_bo has quit IRC | 03:13 | |
| *** jtriley has quit IRC | 03:14 | |
| *** rfxn has quit IRC | 03:15 | |
| *** david-lyle has quit IRC | 03:16 | |
| *** rfxn has joined #openstack-kolla | 03:17 | |
| *** Teck7 has quit IRC | 03:20 | |
| *** david-lyle has joined #openstack-kolla | 03:23 | |
| *** ducttap__ has quit IRC | 03:24 | |
| *** jascott1_ has quit IRC | 03:24 | |
| *** jascott1 has joined #openstack-kolla | 03:25 | |
| *** jascott1 has quit IRC | 03:26 | |
| *** mdnadeem has joined #openstack-kolla | 03:27 | |
| *** jascott1 has joined #openstack-kolla | 03:27 | |
| *** jtriley has joined #openstack-kolla | 03:30 | |
| *** ducttape_ has joined #openstack-kolla | 03:31 | |
| *** yangyapeng has quit IRC | 03:33 | |
| *** jtriley has quit IRC | 03:35 | |
| *** yangyapeng has joined #openstack-kolla | 03:39 | |
| *** dixiaoli has joined #openstack-kolla | 03:39 | |
| *** lucasxu has quit IRC | 03:43 | |
| *** yangyapeng has quit IRC | 03:47 | |
| *** dave-mccowan has quit IRC | 03:47 | |
| *** yangyapeng has joined #openstack-kolla | 03:47 | |
| *** yangyapeng has quit IRC | 03:52 | |
| *** yangyapeng has joined #openstack-kolla | 03:52 | |
| *** yangyapeng has quit IRC | 03:57 | |
| *** yangyapeng has joined #openstack-kolla | 03:57 | |
| *** zhangfei has joined #openstack-kolla | 04:01 | |
| *** yangyapeng has quit IRC | 04:02 | |
| *** yangyapeng has joined #openstack-kolla | 04:07 | |
| *** yangyapeng has quit IRC | 04:14 | |
| *** yangyapeng has joined #openstack-kolla | 04:15 | |
| *** caowei has quit IRC | 04:26 | |
| *** caowei has joined #openstack-kolla | 04:26 | |
| *** Teck7 has joined #openstack-kolla | 04:27 | |
| *** rfxn has quit IRC | 04:30 | |
| *** rfxn has joined #openstack-kolla | 04:32 | |
| *** harlowja has joined #openstack-kolla | 04:35 | |
| *** Teck7 has quit IRC | 04:36 | |
| *** mnasiadka has joined #openstack-kolla | 04:43 | |
| *** duonghq has quit IRC | 04:46 | |
| *** duonghq has joined #openstack-kolla | 04:46 | |
| *** mnasiadka has quit IRC | 04:48 | |
| *** Teck7 has joined #openstack-kolla | 04:51 | |
| *** eswar has joined #openstack-kolla | 04:54 | |
| *** rfxn has quit IRC | 04:54 | |
| eswar | Hi All........Please let me know if any version 0.6/0.5.0.4 is supported for ubuntu 14.04 as its not mentioned in release notes. | 04:56 |
| eswar | kolla-K8S | 04:56 |
| *** rfxn has joined #openstack-kolla | 04:58 | |
| *** Teck7 has quit IRC | 04:59 | |
| *** caowei has quit IRC | 05:01 | |
| *** janki has joined #openstack-kolla | 05:03 | |
| *** skramaja has joined #openstack-kolla | 05:03 | |
| *** eswar has quit IRC | 05:14 | |
| *** harlowja has quit IRC | 05:14 | |
| *** eswar has joined #openstack-kolla | 05:15 | |
| *** eswar has quit IRC | 05:16 | |
| *** eswar has joined #openstack-kolla | 05:23 | |
| eswar | Hi All ..... In https://docs.openstack.org/kolla-kubernetes/latest/deployment-guide.html# its mentioned kolla-k8s is validated on Ubuntu 16.04 | 05:24 |
| eswar | just wanted to know if its supported on ubuntu 14.04 | 05:24 |
| kolla-slack | <jascott1> eswar probably best effort but i wouldnt expect official support | 05:29 |
| jascott1 | thats just a guess | 05:30 |
| eswar | Thanks Justin .... it means its only supported in 16.04 right ..... please correct my understanding ..... any version of kolla-k8s that supports 14.04 ?? | 05:31 |
| *** jamesbenson has joined #openstack-kolla | 05:38 | |
| *** jamesbenson has quit IRC | 05:42 | |
| *** caowei has joined #openstack-kolla | 05:54 | |
| openstackgerrit | zhubingbing proposed openstack/kolla-ansible master: Move placement-api-access log format to apache_access https://review.openstack.org/491423 | 06:03 |
| eswar | Hi All ..... i am new to kolla-k8s .... trying to understand it ..... just wanted to know latest version of kolla-k8s 0.6.0 supports which openstack version docker containers ... is it by default Mitaka ?? | 06:07 |
| *** unicell has joined #openstack-kolla | 06:12 | |
| *** manheim has joined #openstack-kolla | 06:17 | |
| *** coolsvap has joined #openstack-kolla | 06:27 | |
| *** mnasiadka has joined #openstack-kolla | 06:33 | |
| *** dixiaoli has quit IRC | 06:39 | |
| openstackgerrit | zhangfei gao proposed openstack/kolla-ansible master: mariadb: fix permission issue of /var/run/mysqld https://review.openstack.org/491683 | 06:42 |
| openstackgerrit | zhangfei gao proposed openstack/kolla-ansible master: common: fix ansible location https://review.openstack.org/491684 | 06:42 |
| openstackgerrit | Taeha Kim proposed openstack/kolla master: Fix ceilometer-compute fails to build in RHEL https://review.openstack.org/491685 | 06:52 |
| zhangfei | SamYaple: the patch "Remove priviledged mode for nova-api" cause kolla deploy error | 06:53 |
| zhangfei | https://bugs.launchpad.net/kolla/+bug/1709244 | 06:53 |
| openstack | Launchpad bug 1709244 in kolla "nova : Discovering nova hosts fails" [Undecided,New] | 06:53 |
| *** gfidente has joined #openstack-kolla | 06:55 | |
| *** serlex has joined #openstack-kolla | 06:57 | |
| *** manheim has quit IRC | 06:57 | |
| *** zhurong has quit IRC | 06:59 | |
| *** pcaruana has joined #openstack-kolla | 07:00 | |
| kolla-slack | <egonzalez> zhangfei, the error is fixed in master and work for rpm distros, ubuntu has not packaged the change in nova code base yet | 07:00 |
| zhangfei | cool, which patch? would like to try | 07:01 |
| *** manheim has joined #openstack-kolla | 07:01 | |
| *** jascott1 has quit IRC | 07:05 | |
| openstackgerrit | Taeha Kim proposed openstack/kolla stable/ocata: Fix ceilometer-compute fails to build in RHEL https://review.openstack.org/491691 | 07:05 |
| *** jascott1 has joined #openstack-kolla | 07:05 | |
| kolla-slack | <egonzalez> zhangfei https://review.openstack.org/#/c/480765/ | 07:06 |
| *** manheim has quit IRC | 07:06 | |
| zhangfei | kolla-slack: thanks | 07:08 |
| *** manheim has joined #openstack-kolla | 07:09 | |
| *** jascott1 has quit IRC | 07:10 | |
| *** manheim_ has joined #openstack-kolla | 07:13 | |
| *** manheim has quit IRC | 07:13 | |
| *** manheim_ has quit IRC | 07:17 | |
| *** manheim has joined #openstack-kolla | 07:22 | |
| *** manheim has quit IRC | 07:27 | |
| *** dciabrin has quit IRC | 07:33 | |
| *** dciabrin has joined #openstack-kolla | 07:34 | |
| *** duritong has quit IRC | 07:38 | |
| *** duritong has joined #openstack-kolla | 07:45 | |
| *** manheim has joined #openstack-kolla | 07:49 | |
| *** duritong has quit IRC | 07:50 | |
| *** duritong has joined #openstack-kolla | 07:56 | |
| openstackgerrit | Taeha Kim proposed openstack/kolla master: Fix nova-compute fails to build in RHEL https://review.openstack.org/491710 | 08:04 |
| *** shardy has joined #openstack-kolla | 08:06 | |
| *** spamferkel has joined #openstack-kolla | 08:09 | |
| *** mgoddard has joined #openstack-kolla | 08:11 | |
| *** itlinux has quit IRC | 08:11 | |
| *** serlex has quit IRC | 08:14 | |
| *** jamesbenson has joined #openstack-kolla | 08:16 | |
| *** yingjun has quit IRC | 08:20 | |
| *** jamesbenson has quit IRC | 08:20 | |
| *** lpetrut has joined #openstack-kolla | 08:23 | |
| openstackgerrit | Taeha Kim proposed openstack/kolla stable/ocata: Fix nova-compute fails to build in RHEL https://review.openstack.org/491715 | 08:26 |
| *** spamferkel has quit IRC | 08:38 | |
| *** _tomk_ has joined #openstack-kolla | 08:40 | |
| *** _tomk_ has quit IRC | 08:42 | |
| *** thomask has joined #openstack-kolla | 08:43 | |
| *** thomask has quit IRC | 08:44 | |
| *** _tomka_ has joined #openstack-kolla | 08:45 | |
| *** _tomka_ has quit IRC | 08:45 | |
| *** tosika has joined #openstack-kolla | 08:46 | |
| *** matrohon has joined #openstack-kolla | 08:51 | |
| *** zhangfei has quit IRC | 08:51 | |
| *** zhangfei has joined #openstack-kolla | 09:04 | |
| *** stefan_bo has joined #openstack-kolla | 09:10 | |
| *** manheim has quit IRC | 09:12 | |
| *** manheim has joined #openstack-kolla | 09:13 | |
| *** manheim has quit IRC | 09:13 | |
| *** manheim has joined #openstack-kolla | 09:15 | |
| *** stefan_bo has quit IRC | 09:15 | |
| *** manheim has quit IRC | 09:16 | |
| *** manheim has joined #openstack-kolla | 09:16 | |
| *** manheim_ has joined #openstack-kolla | 09:17 | |
| *** manheim has quit IRC | 09:20 | |
| *** sambetts|afk is now known as sambetts | 09:20 | |
| *** manheim_ has quit IRC | 09:21 | |
| *** kevinz has quit IRC | 09:36 | |
| *** serlex has joined #openstack-kolla | 09:37 | |
| *** shardy has quit IRC | 09:43 | |
| *** kornicameister has quit IRC | 09:47 | |
| openstackgerrit | Vladislav Belogrudov proposed openstack/kolla-ansible master: Add possibility to configure tenant VLANs https://review.openstack.org/464675 | 09:50 |
| *** shardy has joined #openstack-kolla | 09:56 | |
| *** kornicameister has joined #openstack-kolla | 10:00 | |
| *** tovin07_ has quit IRC | 10:02 | |
| *** yangyapeng has quit IRC | 10:12 | |
| *** yangyapeng has joined #openstack-kolla | 10:12 | |
| *** ruhe has quit IRC | 10:16 | |
| *** ruhe has joined #openstack-kolla | 10:18 | |
| *** yangyapeng has quit IRC | 10:18 | |
| *** zhurong has joined #openstack-kolla | 10:24 | |
| *** ducttape_ has joined #openstack-kolla | 10:32 | |
| *** ducttape_ has quit IRC | 10:36 | |
| *** livelace has joined #openstack-kolla | 10:43 | |
| *** mdnadeem has quit IRC | 10:53 | |
| *** eswar has quit IRC | 10:58 | |
| *** caowei has quit IRC | 11:00 | |
| *** jascott1 has joined #openstack-kolla | 11:07 | |
| *** jascott1 has quit IRC | 11:12 | |
| *** JoseMello has joined #openstack-kolla | 11:16 | |
| *** manheim has joined #openstack-kolla | 11:22 | |
| *** schwicht has quit IRC | 11:26 | |
| *** ismc has quit IRC | 11:32 | |
| *** zhangfei has quit IRC | 11:35 | |
| *** dave-mccowan has joined #openstack-kolla | 11:40 | |
| *** bmace has quit IRC | 11:47 | |
| *** bmace has joined #openstack-kolla | 11:47 | |
| openstackgerrit | weichuancheng proposed openstack/kolla-ansible master: Add external-ceph-scripts to create configration files https://review.openstack.org/489984 | 11:57 |
| *** jrist has joined #openstack-kolla | 11:58 | |
| *** mdnadeem has joined #openstack-kolla | 11:59 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/kolla-kubernetes master: Updated from global requirements https://review.openstack.org/485905 | 12:13 |
| *** eaguilar has joined #openstack-kolla | 12:17 | |
| *** eaguilar has quit IRC | 12:22 | |
| *** eaguilar has joined #openstack-kolla | 12:22 | |
| *** skramaja has quit IRC | 12:24 | |
| *** schwicht has joined #openstack-kolla | 12:24 | |
| *** schwicht has quit IRC | 12:26 | |
| *** rwsu has quit IRC | 12:28 | |
| *** rwsu has joined #openstack-kolla | 12:29 | |
| *** zhurong has quit IRC | 12:30 | |
| matrohon | gate-kolla-dsvm-build-ubuntu-source-ubuntu-xenial | 12:33 |
| matrohon | hi kolla guys | 12:33 |
| matrohon | is there something wong with the job : gate-kolla-dsvm-build-ubuntu-source-ubuntu-xenial | 12:34 |
| matrohon | it is constantly failing on https://review.openstack.org/#/c/490515/2 | 12:34 |
| *** schwicht has joined #openstack-kolla | 12:34 | |
| *** sbezverk has joined #openstack-kolla | 12:35 | |
| *** ismc has joined #openstack-kolla | 12:39 | |
| *** rhallisey has joined #openstack-kolla | 12:42 | |
| *** tvignaud has quit IRC | 12:47 | |
| *** jiriprox has joined #openstack-kolla | 12:47 | |
| *** jiriprox has quit IRC | 12:49 | |
| *** mandre_away is now known as mandre_mtg | 12:51 | |
| *** athomas has joined #openstack-kolla | 12:53 | |
| *** schwicht has quit IRC | 12:55 | |
| *** coolsvap has quit IRC | 12:56 | |
| *** tvignaud has joined #openstack-kolla | 12:58 | |
| *** tvignaud has quit IRC | 12:58 | |
| *** tvignaud has joined #openstack-kolla | 12:58 | |
| *** lucasxu has joined #openstack-kolla | 13:02 | |
| *** jrist has quit IRC | 13:02 | |
| *** clayton has quit IRC | 13:03 | |
| *** clayton has joined #openstack-kolla | 13:05 | |
| *** yangyapeng has joined #openstack-kolla | 13:05 | |
| *** ismc has quit IRC | 13:06 | |
| *** athomas has quit IRC | 13:06 | |
| *** sbezverk has quit IRC | 13:07 | |
| krtaylor | morning all | 13:09 |
| *** yangyapeng has quit IRC | 13:10 | |
| *** schwicht has joined #openstack-kolla | 13:12 | |
| serlex | Morning | 13:13 |
| *** Liuqing has joined #openstack-kolla | 13:17 | |
| *** yangyapeng has joined #openstack-kolla | 13:19 | |
| *** jtriley has joined #openstack-kolla | 13:23 | |
| *** rmart04 has joined #openstack-kolla | 13:33 | |
| *** zhubingbing has quit IRC | 13:34 | |
| *** rmart04 has quit IRC | 13:38 | |
| *** awiddersheim has joined #openstack-kolla | 13:40 | |
| *** ducttape_ has joined #openstack-kolla | 13:42 | |
| *** awiddersheim has quit IRC | 13:43 | |
| *** mdnadeem has quit IRC | 13:46 | |
| *** hongbin has joined #openstack-kolla | 13:47 | |
| *** Liuqing has quit IRC | 13:58 | |
| *** Teck7 has joined #openstack-kolla | 13:59 | |
| *** xinliang has quit IRC | 14:01 | |
| *** rfxn has quit IRC | 14:03 | |
| *** gaoyan has joined #openstack-kolla | 14:12 | |
| *** kiennt has joined #openstack-kolla | 14:12 | |
| *** xinliang has joined #openstack-kolla | 14:13 | |
| *** gaoyan has quit IRC | 14:15 | |
| *** mnasiadka has quit IRC | 14:22 | |
| *** spiette has quit IRC | 14:28 | |
| *** itlinux has joined #openstack-kolla | 14:28 | |
| *** eaguilar has quit IRC | 14:35 | |
| *** eaguilar has joined #openstack-kolla | 14:35 | |
| *** zhubingbing has joined #openstack-kolla | 14:36 | |
| *** eaguilar is now known as eaguilar|afk | 14:39 | |
| *** eaguilar|afk is now known as eaguilar | 14:43 | |
| *** zhangfei has joined #openstack-kolla | 14:44 | |
| *** jtriley has quit IRC | 14:57 | |
| *** jtriley has joined #openstack-kolla | 14:58 | |
| SamYaple | zhangfei: you have to be running the latest nova code. i dont recommend testing any master code without using a source based deploy | 14:59 |
| *** manheim has quit IRC | 15:00 | |
| *** dmsimard is now known as dmsimard|afk | 15:00 | |
| *** jrist has joined #openstack-kolla | 15:05 | |
| zhangfei | SamYaple: thanks, one silly question, how to run the latest nova code? build kolla will get the latest code? | 15:06 |
| SamYaple | zhangfei: yes, but you can't used a "binary" deploy | 15:07 |
| *** mattmceuen has joined #openstack-kolla | 15:07 | |
| *** jascott1 has joined #openstack-kolla | 15:09 | |
| zhangfei | yes, I am using "source", may try to rebuild nova tomorrow, nova-compute & nova-api, right? or all nova component? | 15:09 |
| *** lpetrut_ has joined #openstack-kolla | 15:11 | |
| *** stefan_bo has joined #openstack-kolla | 15:13 | |
| *** lpetrut has quit IRC | 15:14 | |
| *** eaguilar is now known as eaguilar|afk | 15:14 | |
| *** jascott1 has quit IRC | 15:14 | |
| *** eaguilar|afk is now known as eaguilar | 15:16 | |
| *** eaguilar is now known as eaguilar|afk | 15:16 | |
| SamYaple | zhangfei: technically the only one you need to rebuild is nova-api | 15:16 |
| SamYaple | however i dont know what other patches have been pulled in, so youll want to rebuild all of nova | 15:16 |
| *** lpetrut_ has quit IRC | 15:17 | |
| zhangfei | SamYaple: got it, thanks for suggestion, will have a try tomorrow | 15:17 |
| *** stefan_bo has quit IRC | 15:18 | |
| *** jtriley has quit IRC | 15:23 | |
| *** sbezverk has joined #openstack-kolla | 15:23 | |
| *** sbezverk_ has joined #openstack-kolla | 15:24 | |
| *** mmehan has joined #openstack-kolla | 15:24 | |
| *** sbezverk has quit IRC | 15:28 | |
| *** vhosakot has joined #openstack-kolla | 15:28 | |
| *** zhangfei has quit IRC | 15:29 | |
| *** eaguilar|afk has quit IRC | 15:29 | |
| *** Ally has joined #openstack-kolla | 15:30 | |
| Ally | Only way i can add new compute\storage node (ceph) is to run deploy -t to bootstrap osd'd on new node then run kolla-ansible reconfigure, is this correct? | 15:31 |
| Ally | Thot could run kolla-ansible upgrade to add new node but this always fails as ceph.conf etc not copied into containers | 15:32 |
| *** rhallisey has quit IRC | 15:37 | |
| tosika | hmm, at the moment all my deployment attempts fail during " Create cell0 mappings" | 15:42 |
| tosika | not sure if i misconfigured it or if it is broken | 15:43 |
| tosika | anybody else seeing this? | 15:43 |
| *** JoseMello has quit IRC | 15:45 | |
| *** livelace2 has joined #openstack-kolla | 15:48 | |
| *** janki has quit IRC | 15:48 | |
| *** ntpttr_laptop has joined #openstack-kolla | 15:50 | |
| openstackgerrit | sean mooney proposed openstack/kolla-ansible master: introduce playbook to ovs with dpdk https://review.openstack.org/408872 | 15:53 |
| *** krtaylor has quit IRC | 15:54 | |
| sean-k-mooney | inc0: ^ i need to rebuild all my images to fully test https://review.openstack.org/408872 but i think that version will support deploying ovs-dpdk as part of the deploy action | 15:54 |
| sean-k-mooney | inc0: im going to add the module config as a seperate patch on top. there is already too much in https://review.openstack.org/408872 so i dont want to keep makeing it bigger | 15:55 |
| *** tonanhngo has joined #openstack-kolla | 15:56 | |
| *** matrohon has quit IRC | 15:58 | |
| s-dean | how do i set the kolla_internal_vip_address interface it keeps defaulting to the network_interface which is not what i want | 16:05 |
| *** jtriley has joined #openstack-kolla | 16:10 | |
| *** kiennt has quit IRC | 16:15 | |
| sean-k-mooney | s-dean: you could try setting api_interface to something other then network_interface | 16:16 |
| sean-k-mooney | s-dean: but the path to kolla_internal_vip_address will generaly be determinded by your routing table | 16:16 |
| *** ntpttr_laptop__ has joined #openstack-kolla | 16:17 | |
| *** ntpttr_laptop has quit IRC | 16:17 | |
| sean-k-mooney | basically you should determin the what subnet range mapps to the interface you want to use and select a vip that is free in that range | 16:17 |
| inc0 | network_interface is not used for anything else than just to provide default for more detailed interface conf | 16:20 |
| inc0 | as sean-k-mooney said, you can specify separate api_interface or tunnel_interface (for things like vxlan) | 16:20 |
| s-dean | im trying to deploy this on a multi node environment, but im a bit confused, i understand that the VIP is meant to be a floating IP | 16:20 |
| inc0 | right, managed by keepalived | 16:20 |
| s-dean | but it keeps check the api interface | 16:20 |
| s-dean | the api interface is my management network right | 16:21 |
| inc0 | well it is floating IP in api_interface network | 16:21 |
| *** yangyapeng has quit IRC | 16:21 | |
| sean-k-mooney | s-dean: the api interface is the interface the openstack service listen on | 16:21 |
| inc0 | it will be terminated by haproxy and then haproxy will load balance request to actual API | 16:21 |
| *** yangyapeng has joined #openstack-kolla | 16:21 | |
| s-dean | right ok | 16:22 |
| s-dean | do i comment out network_interface | 16:22 |
| inc0 | well network interface is used for other stuff too | 16:23 |
| s-dean | considering its a default value for other variables | 16:23 |
| inc0 | vxlan tunnels, storage and sch | 16:23 |
| inc0 | such | 16:23 |
| inc0 | these things needs a network too;) | 16:23 |
| s-dean | im specifying my tunnel network | 16:23 |
| s-dean | i have tunnel net mgmt net and provider net | 16:23 |
| inc0 | well if you specify manually every network you're going to be using | 16:23 |
| *** krtaylor has joined #openstack-kolla | 16:23 | |
| inc0 | then you can safely comment network_interface | 16:24 |
| s-dean | i wish you guys had kept the terminology the same as in the docs | 16:24 |
| sean-k-mooney | s-dean: maybe it is better to start with describing the topology you would like and we can suggest how to achive that | 16:24 |
| s-dean | ok | 16:24 |
| s-dean | that would be of great help | 16:24 |
| s-dean | ok i have 2 bonds on my controller, 3 bonds on my network, and 2 bonds on my compute, i would like a provider network, for floating IP's, one management network for openstack services to communicate on, and on network for VM traffic | 16:25 |
| s-dean | one network for VM traffic | 16:25 |
| *** yangyapeng has quit IRC | 16:26 | |
| s-dean | i so from what you have said so far, | 16:26 |
| inc0 | right, then I'd say neutron_external_iface => net for floatings | 16:26 |
| s-dean | yup | 16:26 |
| inc0 | network_interface => mgmt | 16:26 |
| inc0 | and tunnel_interface => cross-vm traffic | 16:26 |
| inc0 | reason it's safer to specify network interface too is that as I said it's also used for example for cinder | 16:27 |
| inc0 | or ceph | 16:27 |
| *** ggillies_ has quit IRC | 16:27 | |
| s-dean | kolla_external_vip_interface ? | 16:27 |
| *** pcaruana has quit IRC | 16:27 | |
| inc0 | https://docs.openstack.org/kolla-ansible/latest/production-architecture-guide.html#network-configuration | 16:27 |
| s-dean | got that open right now | 16:28 |
| inc0 | external is different | 16:28 |
| serlex | Ally | 16:28 |
| inc0 | external_vip_interface is for scenario where you want people to have access to your APIs | 16:28 |
| sean-k-mooney | s-dean: kolla_external_vip_interface that is so you can use a seperate interface for external(internet) facing api calls vs internal(inter servce) api calls | 16:28 |
| inc0 | but not mgmt | 16:28 |
| inc0 | no openstack service will use external | 16:29 |
| s-dean | so kolla_internal_vip_address, this has to be a floating IP on my management network ? | 16:29 |
| inc0 | yes | 16:29 |
| sean-k-mooney | s-dean: yes that is the vip used by openstack services to comunicate | 16:29 |
| inc0 | and databases and stuff like that | 16:30 |
| s-dean | ok, and if i want to expose said service i use the kolla_external | 16:30 |
| *** ggillies has joined #openstack-kolla | 16:30 | |
| sean-k-mooney | s-dean: yes by default external and internal vips are the same | 16:30 |
| inc0 | right | 16:30 |
| Ally | hi serlex | 16:30 |
| inc0 | so under the hood it will create new floating in keepalived, new set of endpoints in haproxy | 16:31 |
| serlex | I believe you can add compute node with reconfigure, inc0 can you confirm? | 16:31 |
| inc0 | that will NAT to regular service APIs | 16:31 |
| serlex | to an existing running setup* | 16:31 |
| inc0 | compute node shouldn't be problem | 16:31 |
| *** slagle has quit IRC | 16:31 | |
| inc0 | but use deploy rather than reconfigugre | 16:31 |
| inc0 | we had conversation yesterday about moving rabbitmq from 1 node to 3 node cluster and that was funny | 16:32 |
| inc0 | but compute node should just work | 16:32 |
| serlex | there you go, thanks inc0...will it work with ceph-osd node? | 16:32 |
| inc0 | yeah I think so | 16:32 |
| serlex | I dont see why not, only thing you can't change with ceph is erasure profile, if you are using it | 16:33 |
| serlex | I think* | 16:33 |
| Ally | serlex, inc0 - thank u. I got it working using configure but its best to use deploy? i was always worried that it would be destructive | 16:33 |
| sean-k-mooney | s-dean: one of the resons for haveing two vips is so that we can have tls enabled on the external side but have it turned off internally so that we only pay the ssl cost form external requests | 16:35 |
| inc0 | Ally: we're very very careful to write our plays so it won't be destructive | 16:35 |
| inc0 | our plays should be idempotent' | 16:36 |
| inc0 | one thing you can do tho | 16:36 |
| inc0 | so kolla-ansible is just a shortcut for very long ansible-playbook command | 16:36 |
| inc0 | which is displayed as first thing in output | 16:36 |
| inc0 | if you use this command isntead of kolla-ansible, you get more flags you can use | 16:37 |
| inc0 | and you can for example limit playbook run to just a new node | 16:37 |
| inc0 | this will guarantee that it wont touch rest of the nodes | 16:37 |
| sean-k-mooney | depending on what you want to do you can also use --tags to have only a subset of the roles run | 16:38 |
| s-dean | thanks, i think ive got me head round that now, can i remove localhost from my multinode config ? it complaining that an interface is not found. | 16:40 |
| sean-k-mooney | s-dean: in the deployment section? | 16:40 |
| Ally | inc0 - much appreciated. when adding the combined compute\storage node i first ran kolls-ansible deploy -t ceph to boostrap the disk and then kolla-ansible reconfigure which added new node successfully. | 16:40 |
| *** zhubingbing has quit IRC | 16:40 | |
| s-dean | yeah | 16:41 |
| Ally | inc0 - so better to use ansible-playbook with additional flags to add this new node in one step? | 16:41 |
| s-dean | sorry i mean globals.yml | 16:41 |
| sean-k-mooney | s-dean: i think that is only need for the bootstrap role, inc0 it should be safe to comment out correct | 16:41 |
| s-dean | sorry im getting confused | 16:41 |
| sean-k-mooney | s-dean: your refering to [deployment] | 16:42 |
| sean-k-mooney | localhost ansible_connection=local | 16:42 |
| s-dean | yeah | 16:42 |
| inc0 | it's used just to do post-deploy | 16:42 |
| inc0 | as in, generate admin_openrc | 16:42 |
| s-dean | ok, is that all post-deploy does ? | 16:43 |
| inc0 | but what's the error? | 16:43 |
| inc0 | yes, that's all | 16:43 |
| sean-k-mooney | inc0: it allso gets picked up by the destroy playbook | 16:43 |
| *** dmsimard|afk is now known as dmsimard | 16:43 | |
| s-dean | its looking for an interface that doesn't exist on my deployment host, in this case bond0 | 16:44 |
| inc0 | I guess we have bug | 16:44 |
| inc0 | yeah you can just replace this entry with any node from cluster | 16:44 |
| sean-k-mooney | inc0: i found that out when i blew away one of my dev deployments for ovs-dpdk | 16:44 |
| inc0 | aw | 16:44 |
| inc0 | I guess we have all there;) | 16:44 |
| inc0 | hosts: all | 16:45 |
| inc0 | we should probably fix that | 16:45 |
| s-dean | so in the deployment section, i can replace localhost with controller | 16:45 |
| sean-k-mooney | ya i asked if it was intended behavior or not but got no answer | 16:45 |
| sean-k-mooney | s-dean: ya | 16:45 |
| sean-k-mooney | its ment to be the name of the host you are doing the deployment form | 16:45 |
| inc0 | but really it's just used for this admin generation | 16:46 |
| *** shardy has quit IRC | 16:46 | |
| s-dean | i have a separate deployment host from the controller, a VM | 16:46 |
| *** awiddersheim has joined #openstack-kolla | 16:46 | |
| s-dean | ok, I can probably generate the admin config from the passwords.yml | 16:46 |
| s-dean | that is generated? | 16:46 |
| sean-k-mooney | s-dean: you generate the password.yml before you do the deploy | 16:47 |
| s-dean | yeah, of course | 16:47 |
| sean-k-mooney | but yes you should just be able to copy the admin password form there | 16:47 |
| tosika | after deploy, you create the admin-openrc with kolla-ansible post-deploy | 16:48 |
| tosika | you're right | 16:48 |
| tosika | you can take the password from there | 16:48 |
| sean-k-mooney | not sure why it should need to lookup bond0 though. is that what you have set asl the network_interfaces in globales? | 16:48 |
| s-dean | yes | 16:48 |
| s-dean | on my deployment host, it has one interface, which is ens3 | 16:49 |
| *** eaguilar has joined #openstack-kolla | 16:49 | |
| inc0 | it shouldn't, that's why I'm asking what was real error;) | 16:49 |
| tosika | my deployment attempts still fail during nova cell setup | 16:49 |
| tosika | this time it is task 'Create base cell for legacy instances' | 16:50 |
| *** rhallisey has joined #openstack-kolla | 16:50 | |
| s-dean | fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "msg": "Please check the api_interface property - interface bond0 not found"} | 16:50 |
| *** awiddersheim has quit IRC | 16:51 | |
| inc0 | which task is it? | 16:51 |
| tosika | did someone else try a deploy using master-branch lately? | 16:52 |
| *** rhallisey has quit IRC | 16:52 | |
| *** rhallisey has joined #openstack-kolla | 16:52 | |
| *** serlex has quit IRC | 16:57 | |
| *** jtriley_ has joined #openstack-kolla | 16:58 | |
| *** tonanhngo has quit IRC | 16:58 | |
| *** Ally has quit IRC | 17:01 | |
| *** jtriley has quit IRC | 17:01 | |
| *** unicell has quit IRC | 17:01 | |
| sean-k-mooney | tosika: i hit that too | 17:03 |
| sean-k-mooney | that is why im currently rebuilding all my images | 17:03 |
| *** jtriley_ has quit IRC | 17:03 | |
| sean-k-mooney | im hoping that will fix it but maybe not | 17:04 |
| kolla-slack | <egonzalez> tosika, ubuntu binary or other distro? | 17:04 |
| sean-k-mooney | there is a kolla slack bridge ? | 17:04 |
| *** rwsu has quit IRC | 17:04 | |
| sean-k-mooney | also hi egonzalez | 17:05 |
| tosika | ubuntu, source | 17:05 |
| *** jtriley has joined #openstack-kolla | 17:05 | |
| kolla-slack | <egonzalez> sean-k-mooney yep, in kubernetes slack | 17:05 |
| *** tonanhngo has joined #openstack-kolla | 17:06 | |
| kolla-slack | <egonzalez> tosika, is rabbitmq cluster running properly? Check nova logs | 17:06 |
| tosika | sean-k-mooney: already dumped my images several times an retried with centos, binary and so on | 17:06 |
| tosika | no success until now | 17:06 |
| *** tonanhngo has quit IRC | 17:07 | |
| sean-k-mooney | tosika: mine are almost all rebuilt so ill see if i can reporduce and debug too. that error is currently blocking me testing ovs-dpdk deployment as part of the deploy action | 17:07 |
| sean-k-mooney | in my case i have not done a full rebuild in 5-6 weeks so its not a bad idea for me to rebuild anyway | 17:08 |
| kolla-slack | <egonzalez> Hrm, i've not seen any cell error for a while, on Friday they were working | 17:09 |
| sean-k-mooney | egonzalez i hit the same with master playbooks eairler today useing ubuntu source. as i said my images were a few weeks old so the issue is likely on the ansible side | 17:10 |
| tosika | hmm, no logs under /var/log/kolla on all hosts | 17:12 |
| kolla-slack | <egonzalez> tosika, /var/lib/docker/volumes/kolla_logs | 17:12 |
| tosika | thx | 17:13 |
| s-dean | does kolla place mariadb on the network node ? | 17:13 |
| tosika | nva-api logs that rootwrapper is not able to execute iptables-save | 17:15 |
| kolla-slack | <egonzalez> sean-k-mooney, a change in ansible master last week require a rebuild | 17:15 |
| sean-k-mooney | egonzalez ya i had assumed that there was a change in the kolla-toolbox or somthing along those lines | 17:16 |
| kolla-slack | <egonzalez> Tosika ^^ images require rebuilding, also ubuntu binary is broken until ubuntu packages nova change | 17:17 |
| sean-k-mooney | egonzalez we were seeing the issue on ubuntu source though | 17:17 |
| sean-k-mooney | egonzalez this is what seams to be failing https://github.com/openstack/kolla-ansible/blob/3a68aee3ad4ab5b79e867555b3fe6c5af69c1004/ansible/roles/nova/tasks/simple_cell_setup.yml#L17-L49 | 17:19 |
| sean-k-mooney | actully https://github.com/openstack/kolla-ansible/blob/3a68aee3ad4ab5b79e867555b3fe6c5af69c1004/ansible/roles/nova/tasks/simple_cell_setup.yml#L17-L28 | 17:19 |
| tosika | exactly | 17:19 |
| kolla-slack | <egonzalez> sean-k-mooney any error in logs? Currently gates succeed with cell creation tasks | 17:20 |
| sean-k-mooney | egonzalez im still rebuilding so cant check. tosika do you have any logs form the deploy | 17:21 |
| *** yangyapeng has joined #openstack-kolla | 17:21 | |
| tosika | this time it was task: "Create base cell for legacy instances" | 17:22 |
| tosika | settings were ubuntu, source, master | 17:22 |
| tosika | kolla-ansible (5.0.0.0b4.dev22) from git | 17:23 |
| kolla-slack | <egonzalez> Maybe this? http://logs.openstack.org/84/491684/1/check/gate-kolla-ansible-dsvm-deploy-ubuntu-binary-ubuntu-xenial-nv/2f12533/logs/kolla/nova/nova-api.txt.gz#_2017-08-08_07_02_13_539 | 17:23 |
| tosika | mesage was: | 17:23 |
| tosika | fatal: [controlhost -> controlhost]: FAILED! => {"changed": false, "cmd": ["docker", "exec", "nova_api", "nova-manage", "cell_v2", "create_cell"], "delta": "0:00:00.327543", "end": "2017-08-08 18:47:29.938833", "failed": true, "failed_when_result": true, "rc": 137, "start": "2017-08-08 18:47:29.611290", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} | 17:23 |
| tosika | adding -v to the deploy command does not give more information | 17:23 |
| tosika | where can i get more logs? | 17:24 |
| kolla-slack | <egonzalez> Logs for that task is in nova-api.log | 17:24 |
| *** eaguilar is now known as eaguilar|afk | 17:25 | |
| kolla-slack | <egonzalez> Tosika, likely is the same error in current ubuntu binary gates, if thars the case for source, rebuild with latest code | 17:25 |
| sean-k-mooney | egonzalez tosika just asked on the nova channel nova-manage cell_v2 create_cell is going direct to the db not calling the api | 17:26 |
| sean-k-mooney | so its the mysql logs that will likely have the output no? | 17:26 |
| kolla-slack | <egonzalez> But the task is failing in nova api container, check this link failing the same task in gates | 17:26 |
| *** yangyapeng has quit IRC | 17:26 | |
| kolla-slack | <egonzalez> http://logs.openstack.org/84/491684/1/check/gate-kolla-ansible-dsvm-deploy-ubuntu-binary-ubuntu-xenial-nv/2f12533/logs/kolla/nova/nova-api.txt.gz#_2017-08-08_07_02_13_539 | 17:26 |
| tosika | nova-api.log just complains about not being able to run iptables-save | 17:27 |
| tosika | 2017-08-08 19:26:36.546 7 CRITICAL nova [req-5667f013-a2a3-4273-a128-a09e86296507 - - - - -] ProcessExecutionError | 17:27 |
| tosika | : Unexpected error while running command. | 17:27 |
| tosika | Command: sudo nova-rootwrap /etc/nova/rootwrap.conf iptables-save -c | 17:27 |
| tosika | Exit code: 1 | 17:27 |
| tosika | Stdout: u'' | 17:27 |
| tosika | Stderr: u'iptables-save v1.6.0: Cannot initialize: Permission denied (you must be root)\n\n' | 17:27 |
| tosika | 2017 | 17:27 |
| sean-k-mooney | oh haha i know what caused this | 17:27 |
| sean-k-mooney | we recently droped privlages on nova-api | 17:28 |
| kolla-slack | <egonzalez> Thats what i said need image rebuild | 17:28 |
| tosika | but user is in sudoers and able to become root without password | 17:28 |
| SamYaple | the packages either need to be rebuilt or oyu need to rebuild your sources container | 17:28 |
| *** sambetts is now known as sambetts|afk | 17:29 | |
| SamYaple | you should never run binary packages for your master code, it will always be lagging | 17:29 |
| sean-k-mooney | SamYaple: so is https://github.com/openstack/kolla-ansible/commit/c18615efd54d57f89d1a81729099285ea1fca3a7 related or not | 17:30 |
| SamYaple | thats almost certainly whats causing it, yes sean-k-mooney | 17:30 |
| tosika | what do i need to do to rebuild those images? i thoug it would be sufficent to run "destroy --include-images --yes-i-really-really-mean-it" and the do a new deployment | 17:30 |
| SamYaple | tosika: if you are using ubuntu-binary the binary packages have not recieved the updated nova source code is my understanding | 17:31 |
| SamYaple | so there is nothing you can do with ubuntu-binary until the packages update | 17:31 |
| SamYaple | if you are using ubuntu-source, then a container rebuild will fix it | 17:31 |
| tosika | yeah, ok. but if I#m using 'source' the code should alwas be cloned from git, right? | 17:32 |
| sean-k-mooney | SamYaple: so does nova manage nolonger call iptables or does that neeed to be reverted. im rebuiling all my images currently from source but it takes years behind our proxies | 17:32 |
| *** eaguilar|afk is now known as eaguilar | 17:32 | |
| SamYaple | sean-k-mooney: the nova-metadata-api no longer pulls in old nova-net code which talked to iptables. the nova-api container should not need to modify iptables in anyway when running from nova master | 17:33 |
| vhosakot | inc0: if PTG same as mid-cycle summit? my manager is asking :) | 17:33 |
| vhosakot | inc0: is PTG same as mid-cycle summit? my manager is asking :) | 17:33 |
| inc0 | no, not exactly | 17:33 |
| sean-k-mooney | SamYaple: ah thanks that make sense | 17:33 |
| inc0 | midcycle is this low key "let's meet in a room" | 17:33 |
| inc0 | PTG is full fledged event for all projects | 17:34 |
| vhosakot | inc0: will there be a mid-cycle summit as well for which I need to ask travel funding apart from the PTG? :) | 17:34 |
| inc0 | it's much closer to design part of summit | 17:34 |
| sean-k-mooney | SamYaple: in that case ill just sit and wait for the rebuild to finish. i need to start using our caches when building... | 17:34 |
| inc0 | no midcycle | 17:34 |
| vhosakot | inc0: cool, thanks for the info. | 17:34 |
| sean-k-mooney | inc0: out of interest has your travel been approved to the ptg yet. im still waiting for mine | 17:35 |
| *** aagate has quit IRC | 17:35 | |
| inc0 | no, we have so much chaos around here that it'll probably get approved week or so before -.- | 17:35 |
| SamYaple | inc0: most teams use the PTG as the midcycle (as that was part of the original intention). does kolla have a midcycle and PTG? | 17:36 |
| inc0 | no, we don't have midcycle any more | 17:36 |
| SamYaple | oh ok | 17:36 |
| SamYaple | yea thats like most teams | 17:36 |
| SamYaple | ive heard of one or two that still do a mid-cycle, but its all remote | 17:36 |
| *** aagate has joined #openstack-kolla | 17:37 | |
| sean-k-mooney | the summit is more like a midcycle ceckin now instead with the PTG being the main planning event for kolla nova and neutron from what i have seen | 17:37 |
| *** sbezverk_ has quit IRC | 17:37 | |
| inc0 | I think PTG took what was good from both design summit and midcycles | 17:37 |
| SamYaple | indeed | 17:37 |
| vhosakot | yeah, my manager is like "so, you want to go to Colorado (PTG), Australia (summit) and Austin (may be kubeCon)?, where else?, do you want business class flight and chauffeur-driven cars too" :) | 17:37 |
| SamYaple | vhosakot: i mean if he is offering..... | 17:38 |
| vhosakot | hahaha | 17:38 |
| vhosakot | I wish :) | 17:38 |
| inc0 | I really like that we get 3 days of room and that's it. We're not bound by strict schedule and such | 17:38 |
| SamYaple | inc0: i really like getting a whole bunch of work done | 17:38 |
| sean-k-mooney | vhosakot: chauffeur too and from work would be nice haha | 17:38 |
| inc0 | brb | 17:38 |
| vhosakot | lol | 17:38 |
| vhosakot | I'm like "I can sit on lap in plane and share room" lol ;) | 17:39 |
| *** ducttape_ has quit IRC | 17:39 | |
| tosika | SamYaple, <egonzalez>: thx, i#m now rebuilding the images. I did not fully understand this until now | 17:39 |
| *** yangyapeng has joined #openstack-kolla | 17:40 | |
| * sean-k-mooney remind me never to rebuild alll imageage with out using caches again... | 17:40 | |
| sean-k-mooney | tosika: you should just need to rebuild the nova images to fix this issue | 17:41 |
| *** sbezverk has joined #openstack-kolla | 17:42 | |
| *** yangyapeng has quit IRC | 17:45 | |
| *** slagle has joined #openstack-kolla | 17:45 | |
| *** eaguilar is now known as eaguilar|afk | 17:46 | |
| *** aagate has quit IRC | 17:48 | |
| *** eaguilar|afk is now known as eaguilar | 17:48 | |
| *** aagate has joined #openstack-kolla | 17:49 | |
| *** eaguilar is now known as eaguilar|afk | 17:49 | |
| *** ducttape_ has joined #openstack-kolla | 17:49 | |
| *** eaguilar|afk is now known as eaguilar | 17:51 | |
| *** unicell has joined #openstack-kolla | 17:53 | |
| *** livelace2 has quit IRC | 17:55 | |
| *** tonanhngo has joined #openstack-kolla | 17:55 | |
| *** jrist has quit IRC | 17:57 | |
| *** tonanhngo_ has joined #openstack-kolla | 17:58 | |
| *** tonanhngo has quit IRC | 17:59 | |
| *** tonanhngo_ has quit IRC | 18:02 | |
| *** tonanhngo has joined #openstack-kolla | 18:02 | |
| *** itlinux has quit IRC | 18:04 | |
| s-dean | can someone just confirm, i got told that kolla does tls out of the box, does this cover every aspect of openstack, mariadb, rabbitmq, and all api's ? | 18:08 |
| *** gfidente is now known as gfidente|afk | 18:08 | |
| *** ducttape_ has quit IRC | 18:09 | |
| *** ducttape_ has joined #openstack-kolla | 18:09 | |
| *** tosika has quit IRC | 18:17 | |
| *** spiette has joined #openstack-kolla | 18:17 | |
| SamYaple | s-dean: no | 18:18 |
| SamYaple | s-dean: it covers only https at the load balancer and not all services are configured properly to talk https so youll need some overrides | 18:18 |
| SamYaple | for example in glance-api youll have to add "registry_host_protocol = https" | 18:19 |
| *** ddyer has joined #openstack-kolla | 18:19 | |
| SamYaple | as far as I know Kolla has no mechanisms to all TLS rabbitmq or mariadb at all, nor frontend loadbalancer to backend service with https | 18:20 |
| SamYaple | s/all/allow/ | 18:20 |
| s-dean | ahahahhaa, lovely | 18:20 |
| *** ducttape_ has quit IRC | 18:20 | |
| vhosakot | s-dean: no TLS for the api network. TLS secures public/external VIP by using https for the keystone public URL - https://github.com/openstack/kolla-ansible/blob/master/doc/advanced-configuration.rst#tls-configuration and https://github.com/openstack/kolla-ansible/blob/master/ansible/group_vars/all.yml#L277 | 18:21 |
| s-dean | i've been requested to TLS every openstack service, endpoint etc | 18:21 |
| SamYaple | s-dean: openstack-ansible has code for doing that in place. but Kolla and OpenStack-Ansible are very very different ways to deploy | 18:22 |
| s-dean | right ok | 18:22 |
| s-dean | somebody told me kolla could do it | 18:22 |
| s-dean | was someone from the nova channel | 18:22 |
| SamYaple | mostly in place I guess i should say. im not sure they have the mariadb tls bits | 18:22 |
| *** ducttape_ has joined #openstack-kolla | 18:22 | |
| s-dean | i've hand rolled this out, already with everything TLS'ed. just had a problem with nova-list timing out | 18:23 |
| s-dean | and now im back to square one | 18:23 |
| s-dean | shit | 18:24 |
| vhosakot | s-dean: yeah, doing per-service TLS is possible... I know neutron supports TLS thru LB/haproxy config tweaks. | 18:25 |
| SamYaple | s-dean: if it makes you feel better, ive done a full tls deploy in kolla, highly customized. but it was over a year ago when i was active in the project | 18:27 |
| SamYaple | s-dean: ive also hand rolled TLS | 18:27 |
| s-dean | This is a night mare, this one guy at my work will not leave it, he wants TLS everywhere. and thanks it does at least i know someone else has went through the pain of having to deploy this stuff with very little documentation. | 18:29 |
| s-dean | I might just try building my own docker containers, i duno, guess ill go read the openstack-ansible docs. | 18:30 |
| *** lpetrut_ has joined #openstack-kolla | 18:30 | |
| SamYaple | s-dean: so here is how i would address this. memcached has no support for https/tls/encryption/etc | 18:31 |
| SamYaple | memcached is basically a hard requirement for anything production iwth openstack | 18:31 |
| SamYaple | it needs to be on an isolated, non-routeable network for security (its the only security mechansim you can use for it) | 18:31 |
| s-dean | i was going to just firewall that off | 18:31 |
| SamYaple | great! | 18:31 |
| s-dean | using iptables | 18:31 |
| SamYaple | not so great | 18:31 |
| s-dean | lol | 18:31 |
| SamYaple | my point is you have a hard requirement on an un-tls-able service | 18:32 |
| SamYaple | now just roll rabbitmq and mariadb and all the backends into that | 18:32 |
| SamYaple | however you secure memcached, do it for the others | 18:32 |
| SamYaple | TLS up the front (the only thing the clients tlak to) and require netowkr security on the back | 18:32 |
| SamYaple | and if you use ceph, even better. cephs osd data replication network is entirely unencrypted too | 18:33 |
| SamYaple | not to mention the wierd bugs with rabbitmq TLS and openstack services | 18:33 |
| SamYaple | or mariadbs replication issues with rls | 18:33 |
| SamYaple | tls* | 18:33 |
| s-dean | have to upgrade it to OTP 19 | 18:33 |
| s-dean | I got rabbitmq,maria,keystone,glance and nova TLS'ed it was this new ocata release that ive been having problems with, nova just wouldnt play nice at all | 18:34 |
| SamYaple | my go to for these sistuations is "if someone has access to our internal network and can MitM our traffic internally... we have bigger issues" | 18:35 |
| SamYaple | TLS the front end and youre 99% the way to security | 18:35 |
| *** lpetrut_ has quit IRC | 18:35 | |
| s-dean | mate, all my networks are sitting behind a firewall, VM, MGMT, are blocked in my firewall from being accessed by anyone on my network other than me, the provider network is wide open as it should be | 18:36 |
| s-dean | but this guy thinks that its not enough | 18:36 |
| sean-k-mooney | SamYaple: if you really cant trsh your core network the only option you have left really is to interconnect each of the nodes via a vpn and route all your non encrypted traffic via that | 18:36 |
| s-dean | and wants all the communications internally, to be TLS'ed | 18:36 |
| SamYaple | "show me an attack vector" | 18:36 |
| s-dean | is that what you think i should say to him ? | 18:37 |
| SamYaple | if they can get your unencrypted traffic, they already own all your data as the passwords for all these services are in plain text in config files | 18:37 |
| SamYaple | s-dean: absolutely | 18:37 |
| SamYaple | s-dean: works for me | 18:37 |
| SamYaple | keep harping on the hard requirement for unencryptable sevice | 18:38 |
| s-dean | hes wasted around 4 weeks of my time. | 18:38 |
| SamYaple | purpose assinine solutions like sean-k-mooney just suggested | 18:38 |
| SamYaple | get him to do some work | 18:38 |
| s-dean | hes different department, dev | 18:38 |
| s-dean | im ops | 18:38 |
| sean-k-mooney | SamYaple: hehe i have actully got some really asks for encrypted tenant networks from costomers but so far no one has said they need the same for the openstack internal traffic | 18:39 |
| *** jascott1 has joined #openstack-kolla | 18:39 | |
| inc0 | it's nice "hurr durr I'm so security" | 18:40 |
| sean-k-mooney | SamYaple: the vpn sollution was actully also considerd for tunneling remote compute nodes back into centeralised contolers | 18:40 |
| inc0 | sean-k-mooney: building ipsec between sites is good idea | 18:41 |
| s-dean | the only option i can think of for a fast and secure VPN that has decent throughput would be wiregaurd | 18:41 |
| *** yangyapeng has joined #openstack-kolla | 18:41 | |
| inc0 | s-dean: or have hardware appliance for that | 18:41 |
| inc0 | I'm sure vhosakot would have options for you:P | 18:41 |
| inc0 | truth be told, keystone needs memcache | 18:42 |
| inc0 | memcache holds tokens, so if you sniff on traffic to it, you get full access to infra | 18:42 |
| s-dean | the stupid thing is, I rebuilt my company network because there was security concerns, installed 2 Pfsense Firewalls, one INT one EXT and vlan'ed and segmented the entire network. and he still wants TLS everywhere | 18:43 |
| inc0 | memcache doesn't support tls | 18:43 |
| SamYaple | s-dean: i wasnt insulting you, i was saying that solution is silly for the issue at hand | 18:43 |
| vhosakot | yeah, per=segment combined with per-service full security is a grey area IMO :) | 18:43 |
| s-dean | im in a agreement with you | 18:43 |
| vhosakot | s-dean: are you building a public cloud? | 18:43 |
| inc0 | so if you MinM for memcache traffic, you're screwed | 18:43 |
| s-dean | private | 18:43 |
| inc0 | only way to remove MinM potential is total lockout of mgmt network | 18:44 |
| inc0 | which means you don't need tls for other stuff in it | 18:44 |
| inc0 | that's it | 18:44 |
| s-dean | which i have done, using pfsense. | 18:44 |
| s-dean | but this guy still thinks TLS everywhere | 18:45 |
| inc0 | the only "hacker" full TLS would protect you from is hacker that doesn't know openstack and doesn't know it needs memcache | 18:45 |
| SamYaple | inc0: to be fair, these days there is very little that is vulnerable in memcache since now most of the objects get encrypted before storing | 18:45 |
| SamYaple | but yea, like i said if an attack can MitM in your core network, youve got bigger issues | 18:46 |
| SamYaple | why dump the traffic when you have the passwords | 18:46 |
| *** yangyapeng has quit IRC | 18:46 | |
| vhosakot | sean-k-mooney: wow, api-traffic in VPN tunnel... I like the idea.. it is secure for sure :) | 18:47 |
| inc0 | vhosakot: but to protect here you'd need mesh of tunnels | 18:47 |
| sean-k-mooney | vhosakot: haha secure but not nessisairaly scalable or fast | 18:47 |
| s-dean | it would increase your overhead far to much | 18:47 |
| sean-k-mooney | inc0: or have the vpn tunnel relay the trafic | 18:48 |
| inc0 | we're assuming that anything that leaves network adapter can be sniffed | 18:48 |
| sean-k-mooney | *server | 18:48 |
| inc0 | well, then you can MinM between node and vpn server | 18:48 |
| vhosakot | inc0: right, we need like an admin tunnel, service/API tunnel, storage tunnel, sounds tough ;) | 18:48 |
| inc0 | vhosakot: what I'm saying is you need separate tunnel to evert node in cluster | 18:48 |
| inc0 | on every node | 18:49 |
| sean-k-mooney | inc0: openvpn suports client to client networking where the server routes the traffic between the clientes and its only decrypeted on the openvpn server itself while routeing | 18:49 |
| s-dean | right, think im done with chasing the TLS dragon, might give this a go tomorrow. https://www.wireguard.com/performance/ | 18:49 |
| inc0 | so you need n! tunnels | 18:49 |
| s-dean | openvpn does not have the greatest of throughput | 18:49 |
| vhosakot | one-node-to-every-other-node mesh... fun ;) | 18:50 |
| *** mgoddard has quit IRC | 18:50 | |
| sean-k-mooney | inc0: you can do it with n tunnesl with openvpn but its n*n for ipsec. anyway vpn should realy only be used for inter site traffic not within the same datacenter | 18:50 |
| inc0 | right | 18:51 |
| inc0 | we're in crazy town here;) | 18:51 |
| s-dean | I swear working in the software industry makes you lose your mind. | 18:51 |
| sean-k-mooney | inc0: so looking at https://blog.couchbase.com/memcached-security/ it seams memchace support tls now | 18:51 |
| *** eaguilar is now known as eaguilar|afk | 18:52 | |
| SamYaple | sean-k-mooney: how did you get that from the article? | 18:52 |
| sean-k-mooney | inc0: actully no it support sasl never mind | 18:52 |
| s-dean | sean-k-mooney: I was about to say | 18:53 |
| sean-k-mooney | SamYaple: i misread the wiki articl on sasl | 18:53 |
| SamYaple | yea i think there was some work in openstack to support memcached sasl | 18:53 |
| inc0 | can ks use redis as cache? | 18:53 |
| inc0 | nvm | 18:54 |
| inc0 | redis can't tls too;) | 18:54 |
| inc0 | I guss everyone assumes that encryption cost on cache is missing purpose | 18:55 |
| inc0 | cache is supposed to be fast | 18:55 |
| sean-k-mooney | inc0: well if you have a local copy of memcache and just use tls between each memcache instance it should be fine | 18:56 |
| sean-k-mooney | inc0: we have had aes-ni for years in xeon cpu and most of our competetors too | 18:56 |
| inc0 | so each ks uses local memcache exclusively? | 18:56 |
| SamYaple | thats not how memcached is used by keystone! | 18:57 |
| inc0 | I know | 18:57 |
| SamYaple | :P | 18:57 |
| inc0 | but we can make it | 18:57 |
| s-dean | do other openstack services not access memcache ? | 18:57 |
| SamYaple | no it wont serve a purpose inc0 | 18:57 |
| inc0 | I know | 18:57 |
| SamYaple | s-dean: all of them do, mostly via keystone_authtoken | 18:57 |
| inc0 | but it will solve the TLS issue:P | 18:57 |
| sean-k-mooney | im sugesting that each service would connect to a loacl memcaced and that they are clustered using tls but local access would not need tls | 18:57 |
| s-dean | that sounds like a great idea | 18:58 |
| SamYaple | s-dean: technically they dont have to, nothing REQUIRES memcached, but y ou need it if you like it to perform at all | 18:58 |
| s-dean | right | 18:58 |
| inc0 | SamYaple: I think what's happening is services access ks and ks uses memcache | 18:58 |
| inc0 | so if you have local memcache for every ks node | 18:58 |
| sean-k-mooney | SamYaple: you know the other option is to not store stuff in plain texted in memcache in the frist place | 18:58 |
| inc0 | and it will access it over 127.0.0.1 | 18:59 |
| inc0 | each ks will have it's own local cache so it should actually work | 18:59 |
| SamYaple | inc0: no the middleware talks directly to keystone | 18:59 |
| SamYaple | sean-k-mooney: blame nova-consoleauth. i think thats the only one that still does it | 18:59 |
| *** vhosakot has quit IRC | 18:59 | |
| SamYaple | inc0: let me say that again. keystone middleware (keystone_authtoken) talks directly to memcached before keystone | 19:00 |
| SamYaple | thats why we have to pass around the shared secret | 19:00 |
| inc0 | well then...memcache per node? | 19:00 |
| sean-k-mooney | SamYaple: your should be able to handel that in oslo.cache without needing nova chages though if it uses oslo cache right. | 19:00 |
| SamYaple | inc0: no, eveeryone needs the same list of memcached servers otherwise the cache doesnt work | 19:00 |
| *** vhosakot has joined #openstack-kolla | 19:00 | |
| inc0 | :( | 19:00 |
| SamYaple | sean-k-mooney: im not disagreeing :) | 19:00 |
| sean-k-mooney | SamYaple: hehe ill add it to the ever growing list of things i wish was magically fixed with openstack lol | 19:01 |
| s-dean | is keystone the most important service to protect with TLS in openstack > | 19:03 |
| s-dean | ? | 19:04 |
| SamYaple | s-dean: i would say all client endpoints are the most important | 19:04 |
| SamYaple | anything on the backend is much less important given the amount of plain text passwords we have in each openstack file | 19:04 |
| s-dean | when you use the terminology client, you are referring to users? | 19:05 |
| *** vhosakot has quit IRC | 19:05 | |
| SamYaple | s-dean: more or less, yea. anyone interacing with openstack. TLS all those points of interaction | 19:05 |
| SamYaple | really that just means HTTPS on the load balancer | 19:05 |
| s-dean | right. | 19:06 |
| *** eaguilar|afk has quit IRC | 19:08 | |
| *** jamesbenson has joined #openstack-kolla | 19:09 | |
| *** vhosakot has joined #openstack-kolla | 19:10 | |
| sean-k-mooney | not to re architect all the thing but looking at oslo.cacche the only backend that seams to support ssl is mongo db so maybe you could use mongo db with the in-memory storage engine if you really needed to encrypte the cache connection | 19:12 |
| sean-k-mooney | still likely eaiser to encrypte stuff before putting it in the cache with a shared secrete | 19:13 |
| SamYaple | sean-k-mooney: i really dont know why there isn't a oslo.cache encrypt_shared_key=XXX option | 19:14 |
| sean-k-mooney | SamYaple: proably cause no one asked for it yet | 19:15 |
| SamYaple | sean-k-mooney: ive asked for it! | 19:15 |
| *** manheim has joined #openstack-kolla | 19:16 | |
| SamYaple | but to be fair, i also havent implemented it | 19:16 |
| *** stefan_bo has joined #openstack-kolla | 19:16 | |
| SamYaple | so i cant blame them | 19:16 |
| *** jtriley has quit IRC | 19:16 | |
| sean-k-mooney | is memcache the only thing that cant be secured with ssl today that openstack uses by defualt | 19:17 |
| s-dean | From what i have read, yes. every other component can be secure in one way or another | 19:17 |
| sean-k-mooney | im excluding ceph currently as you could used another a different storage solution that would support that if you cant secure the osd traffic. | 19:18 |
| SamYaple | sean-k-mooney: i believe so | 19:18 |
| sean-k-mooney | hum maybe we should see if that is someting we can adress in Queens. | 19:19 |
| *** stefan_bo has quit IRC | 19:20 | |
| *** manheim has quit IRC | 19:20 | |
| SamYaple | sean-k-mooney: honestly, a simple shared secret encrypt string is all it would take | 19:20 |
| *** robellison has joined #openstack-kolla | 19:20 | |
| SamYaple | ive just been so busy i havent had a chanceto dig into any of the oslo projects | 19:20 |
| sean-k-mooney | ya and its not like we dont already copy the db password to all the controlers which are the only nodes that run memcached so shared secreate in config is the same as the db passwords we already have | 19:22 |
| *** eaguilar has joined #openstack-kolla | 19:22 | |
| *** matrohon has joined #openstack-kolla | 19:22 | |
| SamYaple | well not *quite*. a misconfiguration of networking might allow people ot inject into memcached even if memcached can't return ot them with some spoofing | 19:23 |
| SamYaple | but for the most part, yea i get your point | 19:23 |
| *** itlinux has joined #openstack-kolla | 19:25 | |
| *** ducttap__ has joined #openstack-kolla | 19:26 | |
| sean-k-mooney | SamYaple: well if everything was encrytped and they didn not have the secreate, it would be decrypted as garbage so you sould also need to store a checksum to detect if the value was correct before returning it | 19:26 |
| SamYaple | sean-k-mooney: i would assume that protection is already there since bit flips and what not | 19:27 |
| SamYaple | either it is valid or not | 19:27 |
| s-dean | is it easier to TLS openstack after you have installed it | 19:28 |
| s-dean | got it working etc | 19:28 |
| *** gfidente|afk is now known as gfidente | 19:28 | |
| sean-k-mooney | SamYaple: well im assuming that you were just encrypting the value not the key so you would want a hash to ensure it is the same thing you put in | 19:28 |
| *** ducttape_ has quit IRC | 19:29 | |
| *** jamesbenson has quit IRC | 19:29 | |
| sean-k-mooney | s-dean: am it depend on if you are jsut doing external tls or not. if it just external tls it should just be changes to haproxy. if its internal also im not sure | 19:29 |
| SamYaple | dont forget updating the endpoints.... | 19:30 |
| sean-k-mooney | well with kolla in general once you get your images built and you get a inventory file and globals that work its pretty easy to tear everyting down and redeploy | 19:31 |
| *** ducttape_ has joined #openstack-kolla | 19:31 | |
| sean-k-mooney | so maybe get to that point and then enable tls. but i have not enabled tls so really not sure what would be easier | 19:32 |
| SamYaple | sean-k-mooney: it was on my todo list, just never got around to it | 19:32 |
| SamYaple | you understand | 19:32 |
| sean-k-mooney | SamYaple: ? the oslo cache stuff? if so yes | 19:33 |
| SamYaple | no the tls everything | 19:34 |
| SamYaple | well and the oslo cache stuff | 19:34 |
| sean-k-mooney | if tls also yes i have never had time to test out the internal and externall tls code in kolla | 19:34 |
| s-dean | I successfully, encrypted mariadb, rabbit, keystone, and glance. but nova just didn't want to work, how important is nova's services can i leave that service unencrypted ? | 19:34 |
| s-dean | still got the configs | 19:34 |
| *** ducttap__ has quit IRC | 19:34 | |
| sean-k-mooney | s-dean: nova is normally the most used service as it is used to procision all compute resoruces. | 19:35 |
| SamYaple | s-dean: to be completely honest, i would not encrypt nova or glance since in Ocata you are still running with eventlet | 19:35 |
| SamYaple | eventlet has pretty awful https support | 19:35 |
| SamYaple | in Pike you should be able to deploy them both as WSGI apps behind apache2/nginx/uwsgi with https support proper | 19:35 |
| s-dean | i did notice a massive slow down, so you recommend just keystone | 19:35 |
| sean-k-mooney | well placement can be run under wsgi right? | 19:35 |
| s-dean | yup | 19:35 |
| SamYaple | sean-k-mooney: *must* be run | 19:35 |
| SamYaple | s-dean: yea placement is ok | 19:36 |
| s-dean | placement i thought was under apache | 19:36 |
| SamYaple | s-dean: placement and keystone, that would be it | 19:36 |
| SamYaple | s-dean: its still a WSGI app | 19:36 |
| SamYaple | it can be run apache2/nginx/uwsgi | 19:36 |
| SamYaple | keystone is WSGI as well | 19:36 |
| SamYaple | i personally run keystone with uwsgi | 19:36 |
| sean-k-mooney | the nova api can also be run under apache with uwsgi in pike too right | 19:37 |
| SamYaple | sean-k-mooney: yea all services can | 19:37 |
| SamYaple | well most services can. there was a community goal for it | 19:37 |
| sean-k-mooney | right | 19:37 |
| sean-k-mooney | s-dean: so are you currently standing up an ocata cluster | 19:38 |
| s-dean | i hand rolled it out first, then got told that openstack-ansible, and kolla did TLS out the box | 19:38 |
| s-dean | but yes | 19:39 |
| SamYaple | some tls, not all of it | 19:39 |
| sean-k-mooney | SamYaple: one thing that would be really nice to add to kolla would be letencrypt support for ha proxy terminated external tls | 19:39 |
| s-dean | i got to the dashboard | 19:39 |
| s-dean | think i left neutron in the end, cause i was so frustrated. deploying kolla was alot easier, less headache | 19:40 |
| *** yangyapeng has joined #openstack-kolla | 19:42 | |
| SamYaple | sean-k-mooney: all my openstack at home is automated lets-encrypted :) | 19:42 |
| SamYaple | openstack.yaple.net | 19:42 |
| s-dean | would you say kolla is a superior way to deploy openstack ? | 19:42 |
| sean-k-mooney | SamYaple: hehe i get a 503 is that my office network or something exploded on your side | 19:43 |
| sean-k-mooney | s-dean: people on this irc may be a little biased | 19:44 |
| sean-k-mooney | s-dean: in comparison to devstack yes | 19:44 |
| SamYaple | sean-k-mooney: im in the middle of an upgrade right now. keystone.yaple.net is the only thing up | 19:44 |
| SamYaple | oh thats another thing. i only have 1 port exposed on my firewall.... 443! | 19:44 |
| SamYaple | all my stuff is sni'd | 19:44 |
| SamYaple | all the services... sni! | 19:45 |
| SamYaple | single ip address | 19:45 |
| s-dean | how long you guys been working on openstack ? | 19:45 |
| SamYaple | s-dean: ive been around since right after folsom was released | 19:45 |
| sean-k-mooney | SamYaple: are you useing kolla-ansible/k8s at home or someting custom | 19:45 |
| SamYaple | sean-k-mooney: i hand roll these days | 19:46 |
| s-dean | right wow, that must be pretty far back because ive never heard of that release | 19:46 |
| sean-k-mooney | s-dean: about 4 years started with hevana | 19:46 |
| *** jtriley has joined #openstack-kolla | 19:46 | |
| SamYaple | i would use kolla-ansible but it needs some changes to work with the LOCI images and those aren't popular | 19:46 |
| s-dean | and it was the first release x) | 19:46 |
| SamYaple | s-dean: its all been alphabetically named | 19:47 |
| *** yangyapeng has quit IRC | 19:47 | |
| sean-k-mooney | SamYaple: i assume your using somting based on your kolla-salt stuff? | 19:47 |
| SamYaple | austin bexar bexar cactus diablo essex folsom | 19:47 |
| s-dean | nice, do you get payed for your contribution or is it volunteer work at openstack ? | 19:47 |
| SamYaple | sean-k-mooney: im actually doing saltstack openstack deploy work for another company now, unreleated to that code. im hoping to make the new stuff public yea | 19:48 |
| sean-k-mooney | grizzly hevana icehouse juno kilo liberty mitaka netwon ocata and now pike | 19:48 |
| SamYaple | s-dean: suprisingly i hve never contributed to openstack code that was used by the company i was employed at | 19:48 |
| s-dean | support for openstack and salt is very low | 19:48 |
| s-dean | i looked at it | 19:48 |
| SamYaple | so you could say its volunteer work | 19:48 |
| s-dean | nice | 19:48 |
| SamYaple | all my kolla work was 100% my time | 19:49 |
| sean-k-mooney | s-dean: i work at intel but we done sell or support openstack directly | 19:49 |
| *** robellison has quit IRC | 19:50 | |
| s-dean | wow, im dwarfed by the big leagues, haha. my business is a startup. | 19:50 |
| SamYaple | that explains the "tls everything". lack of experience and prioritization | 19:51 |
| SamYaple | sorry s-dean, im calling it. youre going under | 19:51 |
| *** robellison has joined #openstack-kolla | 19:51 | |
| *** robellison has quit IRC | 19:51 | |
| s-dean | haha | 19:52 |
| s-dean | there is definitely a lack of experience and prioritization. | 19:53 |
| sean-k-mooney | s-dean: one the annowing thing about working at intel is no direct acess to the internet for anything. the good thing is that if anyone cance acess my openstack cloud its ITs fault for not locking down the devleopment labs firewall enough | 19:54 |
| *** eaguilar is now known as eaguilar|afk | 19:54 | |
| *** eaguilar|afk is now known as eaguilar | 19:54 | |
| inc0 | you are in startup and you already have different org branches for ops and dev? | 19:54 |
| *** eaguilar is now known as eaguilar|afk | 19:54 | |
| SamYaple | inc0: AS IT SHOULD BE | 19:55 |
| s-dean | ahhhh classic, yeah. | 19:55 |
| s-dean | 3 man OPS team. | 19:55 |
| inc0 | fuck no, I think being dev in ops team was best learning for both | 19:55 |
| inc0 | you don't get this "tls everywhere" bs | 19:55 |
| SamYaple | inc0: its not bs! its the right way to do it | 19:56 |
| s-dean | oh how i wish i could merge the two, but i cant code for shit. | 19:56 |
| SamYaple | it really is | 19:56 |
| inc0 | I believe in devops | 19:56 |
| inc0 | really, I think if devs would have oncall duty more our software would be better | 19:56 |
| *** matrohon has quit IRC | 19:57 | |
| s-dean | yeah, once things settle down, im gona try learn some go | 19:57 |
| SamYaple | inc0: i dont trust most devs near production | 19:57 |
| s-dean | xD # | 19:57 |
| s-dean | that makes me laugh | 19:57 |
| *** eaguilar|afk is now known as eaguilar | 19:57 | |
| inc0 | right, I'm dev and I don't trust myself | 19:57 |
| SamYaple | an organzation where operations controlls what devs work on, that is an organization that works well | 19:58 |
| *** eaguilar is now known as eaguilar|afk | 19:58 | |
| s-dean | SamYaple: can i just enable debugging on production | 19:58 |
| inc0 | but also I've seen ops trying to make their life easier by writing tools | 19:58 |
| SamYaple | s-dean: you should probably leave the debug logs on 24/7, yea | 19:58 |
| inc0 | in a most horrid way possible | 19:58 |
| SamYaple | s-dean: its the best way to troubleshoot | 19:58 |
| SamYaple | inc0: hence operations telling dev what do to | 19:59 |
| SamYaple | "need this thing, go code monkey it, here is a shitty bash script" | 19:59 |
| sean-k-mooney | speaking of devops stuff is there an issue with gathering facts in ansible a second time? | 19:59 |
| inc0 | sean-k-mooney: well, shouldn't be I guess | 20:00 |
| inc0 | if stuff changed during playbook run, you're in for a ride anyway | 20:00 |
| vhosakot | I'm dev and do on-call many times. I think if I'm DevOps, OpsDev, DevDev or OpsOps ;) | 20:00 |
| sean-k-mooney | im testing the ovs-dpdk as part of deploy but one of the ip adress needed for neutron is not available untill after the ovs-dpdk role runs | 20:01 |
| *** eaguilar|afk is now known as eaguilar | 20:01 | |
| s-dean | wish things where run different, but im bottom of the rung and also the youngest in my team. got to have "experience" to make any decisions in business | 20:01 |
| inc0 | lol | 20:01 |
| inc0 | yeah, aren't we all like that | 20:01 |
| sean-k-mooney | hehe i got pulled into debuging kolla-ansible in a wind farm last month because the guy working on it was on vacation | 20:02 |
| sean-k-mooney | that was fun | 20:02 |
| SamYaple | sean-k-mooney: just do like me and job hop until you come _in_ as most senior experience | 20:03 |
| inc0 | lol kolla-ansioble runs a windfarm? we should have a "business run by kolla bingo" | 20:03 |
| inc0 | "radiotelescope - bingo!" | 20:03 |
| sean-k-mooney | inc0: yep we have been working to get kolla ansible chosen as the deployment tool of chich for the eu funded virtuwind project | 20:04 |
| sean-k-mooney | they have a test deployment scheduled in denmak later this month i think | 20:04 |
| *** jamesbenson has joined #openstack-kolla | 20:05 | |
| inc0 | put a server in a wind turbine | 20:05 |
| vhosakot | s-dean: after you are experienced, there will be different problems ;) | 20:05 |
| inc0 | you get both power and cooling at the same time | 20:05 |
| *** jemcevoy has joined #openstack-kolla | 20:06 | |
| s-dean | im aware | 20:06 |
| sean-k-mooney | inc0: basically, the project is deploying the turbing monitoring and control systems on openstack . we got them to adopt both openstack and kolla to deploy it | 20:06 |
| inc0 | cool stuff | 20:07 |
| vhosakot | sean-k-mooney: cool, do you have a link or pic I can tweet? :) | 20:07 |
| s-dean | that is pretty cool | 20:07 |
| s-dean | so plenty of openstack jobs then :P | 20:07 |
| vhosakot | somebody was saying my twitter account looks like an advertising board for kolla :) | 20:07 |
| inc0 | http://ska-sdp.org <- this is Kolla too | 20:07 |
| sean-k-mooney | haha am im not sure this is the site http://www.virtuwind.eu/ | 20:07 |
| jemcevoy | How can I apply a patch to the horizon spice console to add the ctrl-alt-del button? https://bugs.launchpad.net/openstack-ansible/icehouse/+bug/1423669 and https://access.redhat.com/solutions/1288163 | 20:08 |
| openstack | Launchpad bug 1423669 in spice-html5 (Ubuntu) "CTRL+ALT+DEL Button missing" [Undecided,New] | 20:08 |
| sean-k-mooney | inc0: thats the giant telescope yes? | 20:08 |
| inc0 | yp | 20:08 |
| s-dean | sweet | 20:08 |
| sean-k-mooney | jemcevoy: with icehouse i dont think we can even build icehouse images | 20:09 |
| *** jamesbenson has quit IRC | 20:09 | |
| sean-k-mooney | jemcevoy: but in generall you can build from source from a patches local github repo or use a template override to apply the patch as part of kolla build | 20:09 |
| jemcevoy | It is just adding the button to call ctrl-alt-del so Window can login... This is the redhat solution... <button type="button" onclick="sendCtrlAltDel()"> | 20:10 |
| jemcevoy | Send Ctrl-Alt-Delete | 20:10 |
| jemcevoy | </button> sean-k-mooney | 20:10 |
| SamYaple | jemcevoy: the proper solution is to use novnc! | 20:11 |
| SamYaple | burn spice to the ground | 20:11 |
| sean-k-mooney | SamYaple: actully i had to swap to spice to work around a weird latency issue i had with no vnc | 20:11 |
| sean-k-mooney | is novnc still unmaintainted | 20:12 |
| SamYaple | sean-k-mooney: its 1: not unmaintained, and 2: spice literally calls itself alpha and beta | 20:12 |
| jemcevoy | I just tried to connect to the container to see where that file is and got the error: docker exec -it nova_html5proxy bash " nova - no matching entries in passwd file | 20:12 |
| SamYaple | the latest novnc commit was 4 days ago | 20:12 |
| sean-k-mooney | SamYaple: which one was not maintained was it xvnc | 20:13 |
| SamYaple | sean-k-mooney: spice html5 was over a year ago | 20:13 |
| SamYaple | sean-k-mooney: yea that sounds right | 20:13 |
| SamYaple | https://github.com/novnc/noVNC | 20:13 |
| SamYaple | https://github.com/SPICE/spice-html5 | 20:13 |
| sean-k-mooney | which ever one devstack used to use 2 or 3 cycles ago i rember the swaped there default but i cant remember what the swaped to/from | 20:13 |
| SamYaple | i know which one i would choose... | 20:13 |
| inc0 | oh joy http://www.iflscience.com/environment/unbelievably-massive-volcanic-engine-found-hiding-under-washington/ | 20:14 |
| inc0 | gotta love living near active volcanoes | 20:14 |
| sean-k-mooney | inc0: i do here hawaii is quite nice :) | 20:15 |
| jemcevoy | inc0: a yellowstone super volcano in our neighborhood... yikes | 20:16 |
| vhosakot | jemcevoy: why do you need console access in the first place? DHCP/dnsmasq issues? :) | 20:16 |
| sean-k-mooney | jemcevoy: if yellowstone erupts i dont think there is anywere that would not notice the effects | 20:16 |
| sean-k-mooney | maybe mars | 20:16 |
| s-dean | all: have a good night,evening or morning where ever you are. im off home, its getting late. mabey catch you guys in here tomorrow. | 20:17 |
| *** lucasxu has quit IRC | 20:17 | |
| *** robellison has joined #openstack-kolla | 20:17 | |
| jemcevoy | vhosakot: Just to build a Windows image or update the gold image... This customer has like 130GB image once once all the apps are installed and it is painful to export from ceph and then import into glance | 20:19 |
| sean-k-mooney | inc0: so ya runinng deploy twice works with the dpdk playbook. i need to add gater facts after again to pick up the tunnel ip since we move it form the interface to the ovs bridge | 20:19 |
| inc0 | sean-k-mooney: add setup task then | 20:19 |
| sean-k-mooney | inc0: yep just resetting my environment and ill test that. | 20:20 |
| jemcevoy | SamYaple: how hard it it to switch to vnc? | 20:20 |
| sean-k-mooney | jemcevoy: its basically just a reconfigure and set the default back to vnc in the global.yml | 20:21 |
| sean-k-mooney | jemcevoy: you have to manually clean up the spice container yourself however | 20:21 |
| vhosakot | sean-k-mooney: you can add a new task with "gather_facts: yes" | 20:22 |
| sean-k-mooney | jemcevoy: assuming you have the continers already built | 20:22 |
| jemcevoy | and some nova.conf change too I bet | 20:22 |
| vhosakot | jemcevoy: ah ok, you're building custom images. | 20:22 |
| jemcevoy | I'll ask the when I can run the reconfigure | 20:23 |
| sean-k-mooney | vhosakot: i was thinking of adding a task to wait for the tunnel ip to show up then have that notify a handeler to gater facts again. | 20:23 |
| jemcevoy | I just built a local repo from centos rpms | 20:23 |
| jemcevoy | not source | 20:23 |
| inc0 | sean-k-mooney: sounds good | 20:24 |
| vhosakot | sean-k-mooney: yeah, triggering a handler to gather facts the second time after waiting is a good idea. | 20:28 |
| sean-k-mooney | vhosakot: is gather_facts enough or should i call setup? | 20:29 |
| SamYaple | jemcevoy: with kolla? its jsut an option | 20:29 |
| SamYaple | jemcevoy: it requires you to hard reboot your instance though to refresh the libvirt.xml | 20:30 |
| jemcevoy | sean-k-mooney: Will a reconfigure deploy the vnc container to the 3 controllers or will I need to do a kolla deploy? | 20:30 |
| jemcevoy | SamYaple: Can't I just restart the libvirt container? | 20:31 |
| vhosakot | sean-k-mooney: yes, you can use the setup module to filter the gathered facts the second time - https://stackoverflow.com/a/34487639 | 20:31 |
| SamYaple | jemcevoy: nope. thats unrelated | 20:31 |
| sean-k-mooney | jemcevoy: am good questing i think it did a deploy then a reconfigure when i went form vnc to spice | 20:31 |
| SamYaple | jemcevoy: nova generates the libvirt.xmls per instance. to get it to generate one with vnc instead of spice requires a hard reboot | 20:32 |
| SamYaple | to regen the xml file | 20:32 |
| jemcevoy | SamYaple: So I will need to reboot each of the three controllers one at a time to keep gallera alive... right.... | 20:33 |
| SamYaple | no no no | 20:33 |
| SamYaple | you misunderstand | 20:33 |
| SamYaple | nova reboot --hard <instance> | 20:33 |
| SamYaple | the instnace needs to be hard rebooted, you shouldnt need to restart the controllers | 20:34 |
| jemcevoy | Ohhh you mean the VMs... | 20:34 |
| SamYaple | yea | 20:34 |
| SamYaple | but specifically, you need to hard reboot them through nova | 20:34 |
| jemcevoy | Thanks... It is the libvirt XML for the instance that need to be regenerated... | 20:36 |
| jemcevoy | gotcha | 20:36 |
| SamYaple | yep | 20:36 |
| *** ducttape_ has quit IRC | 20:39 | |
| *** manheim has joined #openstack-kolla | 20:40 | |
| *** ducttape_ has joined #openstack-kolla | 20:42 | |
| *** manheim has quit IRC | 20:44 | |
| *** ducttape_ has quit IRC | 20:46 | |
| *** awiddersheim has joined #openstack-kolla | 20:47 | |
| *** eaguilar has quit IRC | 20:48 | |
| *** krtaylor has quit IRC | 20:51 | |
| *** awiddersheim has quit IRC | 20:51 | |
| *** sbezverk has quit IRC | 20:53 | |
| *** manheim has joined #openstack-kolla | 20:55 | |
| *** manheim has quit IRC | 20:58 | |
| *** manheim has joined #openstack-kolla | 20:58 | |
| *** manheim has quit IRC | 20:59 | |
| *** manheim has joined #openstack-kolla | 20:59 | |
| *** manheim has quit IRC | 21:01 | |
| *** schwicht has quit IRC | 21:01 | |
| *** ducttape_ has joined #openstack-kolla | 21:05 | |
| *** robellison has quit IRC | 21:07 | |
| *** robellison has joined #openstack-kolla | 21:08 | |
| *** robellison has quit IRC | 21:08 | |
| Reepicheep | so my keystone token table is filling up in ocata, https://bugs.launchpad.net/kolla/+bug/1632811 | 21:10 |
| openstack | Launchpad bug 1632811 in kolla "Keystone token table filling up" [High,In progress] - Assigned to Mathias Ewald (mewald) | 21:10 |
| Reepicheep | do you know if we can change the token provider to Fernet on a running system | 21:12 |
| SamYaple | Reepicheep: changing to fernet from uuid (or whatever you are using) is absolutely possible | 21:13 |
| SamYaple | it will be an outage of the keystone service though | 21:13 |
| SamYaple | basically youll want to generate and stage the fernet keys first, make the config update and restart all the keystone services | 21:14 |
| SamYaple | at that point you *may* need to restart some other services if they fail to retrieve new tokens properly | 21:14 |
| SamYaple | but you shouldnt need to, if you do thats a bug | 21:14 |
| *** jtriley has quit IRC | 21:15 | |
| inc0 | SamYaple: btw did you publish this overlay over LOCI to be deployable by kolla-ansible? | 21:15 |
| SamYaple | nope i have not | 21:18 |
| inc0 | :( | 21:18 |
| SamYaple | well i was only talking about pushing the dockerfile for the *images* | 21:18 |
| SamYaple | to really use them kolla-ansible needs to change | 21:19 |
| inc0 | right, and keeping compatibility between both LOCI and Kolla images won't be trivial | 21:19 |
| SamYaple | what are you talking about? there is no issue doing that... | 21:20 |
| inc0 | well we'd need to host config code in both repos | 21:20 |
| SamYaple | you just need ot move the config files out of the baked in image | 21:20 |
| SamYaple | what "both repos"? | 21:20 |
| inc0 | there are no config files baked into image...there are scripts | 21:20 |
| inc0 | config files are mounted... | 21:20 |
| Reepicheep | Thanks SamYaple, I'm reading about that now | 21:20 |
| SamYaple | inc0: you dont drop in paste.ini or policy.json. those are baked in | 21:20 |
| SamYaple | or rootwrap for that matter | 21:21 |
| SamYaple | those are all configs | 21:21 |
| SamYaple | if its in /etc its a config | 21:21 |
| *** schwicht has joined #openstack-kolla | 21:21 | |
| inc0 | hmm I remmeber seeing changes for thaqt | 21:22 |
| *** sbezverk has joined #openstack-kolla | 21:22 | |
| SamYaple | ok. so here is where im at. basically you can build the kolla base image. use that base for LOCI images. then run those LOCI images with kolla-ansible *IF* kolla ansible owns every config file in /etc/ itself | 21:23 |
| SamYaple | (and it pushed in extend_start.sh which is not a config file) | 21:23 |
| SamYaple | otherwise i have to generate a bunch of images with the config files baked in, then toss in an extend_start.sh every once in a while | 21:24 |
| SamYaple | but honestly, kolla-ansible is very close to being image agonositc, bring your own image | 21:24 |
| inc0 | I know, there are only few things that are assumed in image | 21:24 |
| inc0 | extended_start being one | 21:25 |
| SamYaple | well no, extend_start is specifically not required. but things like set_configs, yes thats required | 21:25 |
| SamYaple | however, thats where teh LOCI builds from kolla base thing comes in | 21:25 |
| inc0 | https://github.com/openstack/kolla-ansible/blob/master/ansible/roles/nova/tasks/config.yml#L115 we have stuff like that | 21:25 |
| inc0 | not exactly what you mean but close | 21:25 |
| inc0 | extended start is required for bootstrap and stuff | 21:26 |
| inc0 | while you could rewrite kolla-ansible to run this code, we don't do it | 21:26 |
| SamYaple | inc0: but if its mounted in at runtime (where i think it belongs anyway) then ther eis no problem | 21:26 |
| inc0 | well mounting code on runtime is different discussion. What I'm saying is that it's not that easy today | 21:27 |
| inc0 | and I don't thing extended_start will ever be moved to kolla-ansible | 21:27 |
| inc0 | that would be enormous breach of ABI | 21:27 |
| SamYaple | which ive never heard anyone outside of this channel tlak about in any fashion, just saying | 21:28 |
| inc0 | well, our images are being used outside of kolla-ansible | 21:28 |
| SamYaple | are they using all the set_config stuff? | 21:28 |
| inc0 | no idea, maybe | 21:28 |
| inc0 | can you guarantee they're not? | 21:29 |
| SamYaple | i dont have to? i dont care | 21:29 |
| SamYaple | but hey. i have no expectations for kolla to be anything other than kolla in kollas own little ecosystem | 21:29 |
| SamYaple | im ok with that | 21:29 |
| inc0 | I know you are, what I'm saying is that we can't break this | 21:29 |
| SamYaple | you absolutely can | 21:29 |
| SamYaple | you just have t odeprecate it | 21:29 |
| inc0 | well, maybe someday, not in Queens at least | 21:30 |
| SamYaple | again, you aren't telling me something i didnt know. ive given up on changing anyones mind in Kolla | 21:30 |
| SamYaple | i was just doing fun things with the LOCI images | 21:30 |
| SamYaple | you also have to remember, i have no intention of building rabbitmq or mariadb images | 21:31 |
| SamYaple | if kolla ever wants to consume anything upstream they must inject code into the container anyway | 21:31 |
| *** manheim has joined #openstack-kolla | 21:33 | |
| inc0 | yeah I was playing with this and it's not horrible | 21:33 |
| *** tonanhngo has quit IRC | 21:33 | |
| SamYaple | well my offer always stands. you want to move forward to image agnostic and upstream images AND have community by in, ill do the work | 21:34 |
| SamYaple | the more choice the better in my opinion | 21:34 |
| inc0 | and I agree with you, what I'm saying is I don't want to cause chaos and break our users in the process | 21:35 |
| SamYaple | im 99% sure thats what deprecation cycles are for | 21:35 |
| SamYaple | leaving extend start in the images forever doesnt break anything. | 21:35 |
| inc0 | problem with that is if we commit to that, we're commited | 21:35 |
| SamYaple | tautologys are tautologys | 21:36 |
| inc0 | lol | 21:36 |
| inc0 | you know what I mean | 21:36 |
| SamYaple | yea i do | 21:36 |
| SamYaple | my point is im not going to fight that fight, but im happy to do the work | 21:36 |
| inc0 | it's a lot of work, often non trivial and arch-changing | 21:36 |
| inc0 | while keeping people happy | 21:37 |
| SamYaple | in this case, not so much | 21:37 |
| SamYaple | its just some extra template files and thats it | 21:37 |
| inc0 | well, I'm talking about extended start, set_configs and stuff like that | 21:37 |
| SamYaple | youre arguing the unknown people who use the kolla images without ansible or k8s might break | 21:37 |
| SamYaple | im saying dont remove set_configs or extended_start | 21:37 |
| SamYaple | just mount over them | 21:37 |
| inc0 | well not all of them are unknown | 21:37 |
| inc0 | anyway, it won't happen in Queens for sure | 21:38 |
| *** tonanhngo has joined #openstack-kolla | 21:38 | |
| SamYaple | im doubtful it will ever happen, but my offer to do work will remain | 21:38 |
| inc0 | kk, thanks | 21:38 |
| *** tonanhngo has quit IRC | 21:43 | |
| *** yangyapeng has joined #openstack-kolla | 21:43 | |
| *** jascott1 has quit IRC | 21:45 | |
| *** jascott1 has joined #openstack-kolla | 21:45 | |
| *** jascott1 has quit IRC | 21:46 | |
| *** jascott1 has joined #openstack-kolla | 21:46 | |
| *** yangyapeng has quit IRC | 21:48 | |
| *** jascott1 has quit IRC | 21:49 | |
| *** jascott1 has joined #openstack-kolla | 21:50 | |
| *** tonanhngo has joined #openstack-kolla | 21:53 | |
| *** jascott1 has quit IRC | 21:54 | |
| *** schwicht has quit IRC | 21:57 | |
| *** tonanhngo has quit IRC | 21:57 | |
| *** tonanhngo has joined #openstack-kolla | 21:59 | |
| openstackgerrit | sean mooney proposed openstack/kolla-ansible master: introduce playbook to ovs with dpdk https://review.openstack.org/408872 | 22:00 |
| *** awiddersheim has joined #openstack-kolla | 22:01 | |
| *** kolla-slack has quit IRC | 22:01 | |
| *** kolla-slack has joined #openstack-kolla | 22:01 | |
| sean-k-mooney | inc0: ^ that should now deploy ovs-dpdk all in one go as part of standard deploy. ill added the kernel module auto loading as a followup patch tommorow | 22:01 |
| *** ntpttr_laptop__ is now known as ntpttr_laptop | 22:01 | |
| *** tonanhngo has quit IRC | 22:03 | |
| *** tonanhngo has joined #openstack-kolla | 22:05 | |
| *** awiddersheim has quit IRC | 22:07 | |
| *** tonanhngo has quit IRC | 22:09 | |
| inc0 | cool sean-k-mooney | 22:11 |
| *** tonanhngo has joined #openstack-kolla | 22:11 | |
| inc0 | sean-k-mooney: why not making dpdk part of regular deploy play? | 22:12 |
| sean-k-mooney | inc0: i have i just also left the independent playbook too | 22:12 |
| sean-k-mooney | i can delete that in next version if you like | 22:13 |
| inc0 | let's remove independent one as it's duplication | 22:13 |
| inc0 | yeah | 22:13 |
| sean-k-mooney | cool will do i just did not get around to deleting it. | 22:13 |
| *** manheim has quit IRC | 22:15 | |
| *** tonanhngo has quit IRC | 22:16 | |
| *** tonanhngo has joined #openstack-kolla | 22:17 | |
| *** tonanhngo has quit IRC | 22:22 | |
| *** tonanhngo has joined #openstack-kolla | 22:24 | |
| *** tonanhngo has quit IRC | 22:27 | |
| *** tonanhngo has joined #openstack-kolla | 22:28 | |
| *** schwicht has joined #openstack-kolla | 22:30 | |
| *** jascott1 has joined #openstack-kolla | 22:41 | |
| *** yangyapeng has joined #openstack-kolla | 22:44 | |
| *** calbers has quit IRC | 22:45 | |
| *** calbers has joined #openstack-kolla | 22:48 | |
| *** yangyapeng has quit IRC | 22:48 | |
| *** vhosakot has quit IRC | 22:49 | |
| *** krtaylor has joined #openstack-kolla | 22:51 | |
| *** schwicht has quit IRC | 23:05 | |
| *** jascott1 has quit IRC | 23:06 | |
| *** rhallisey has quit IRC | 23:06 | |
| *** pbourke has quit IRC | 23:07 | |
| *** pbourke has joined #openstack-kolla | 23:09 | |
| *** ducttape_ has quit IRC | 23:19 | |
| *** rhallisey has joined #openstack-kolla | 23:20 | |
| *** ducttape_ has joined #openstack-kolla | 23:25 | |
| *** jamesbenson has joined #openstack-kolla | 23:28 | |
| *** hongbin has quit IRC | 23:31 | |
| *** jamesbenson has quit IRC | 23:32 | |
| *** gfidente is now known as gfidente|afk | 23:39 | |
| *** yangyapeng has joined #openstack-kolla | 23:45 | |
| *** mattmceuen has quit IRC | 23:45 | |
| *** yangyapeng has quit IRC | 23:50 | |
| *** slagle has quit IRC | 23:59 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!