Wednesday, 2017-05-24

*** powerd has joined #openstack-kolla00:01
*** yingjun has joined #openstack-kolla00:03
*** chas has quit IRC00:05
*** salv-orlando has quit IRC00:16
*** dave-mccowan has quit IRC00:17
*** Pavo has joined #openstack-kolla00:18
bmaltaisOne thing I find strange is that on my "storage only" node there is no container with a name containing ceph00:19
*** Pavo has quit IRC00:22
*** itlinux has joined #openstack-kolla00:24
*** huzhengchuan_ has joined #openstack-kolla00:26
*** huzhengchuan has joined #openstack-kolla00:27
*** salv-orlando has joined #openstack-kolla00:29
*** powerd has quit IRC00:40
*** m1dev has quit IRC00:40
*** chas has joined #openstack-kolla00:42
openstackgerritOpenStack Proposal Bot proposed openstack/kolla-kubernetes master: Updated from global requirements  https://review.openstack.org/46439300:46
*** chas has quit IRC00:46
*** dave-mccowan has joined #openstack-kolla00:51
*** cuongnv has joined #openstack-kolla00:52
*** tovin07_ has joined #openstack-kolla00:54
*** dave-mccowan has quit IRC00:55
*** eaguilar has joined #openstack-kolla00:56
*** duonghq has joined #openstack-kolla00:59
*** chas has joined #openstack-kolla01:03
*** itlinux has quit IRC01:06
*** chas has quit IRC01:08
*** yuanying_ has joined #openstack-kolla01:13
*** yuanying has quit IRC01:13
*** salv-orlando has quit IRC01:17
*** salv-orlando has joined #openstack-kolla01:20
*** chas has joined #openstack-kolla01:24
*** Pavo has joined #openstack-kolla01:25
*** chas has quit IRC01:28
*** jascott1 has joined #openstack-kolla01:31
*** caowei has joined #openstack-kolla01:32
duonghqmorning guys01:32
*** itlinux has joined #openstack-kolla01:35
*** jascott1 has quit IRC01:35
openstackgerritMerged openstack/kolla-kubernetes master: Updated from global requirements  https://review.openstack.org/46439301:36
openstackgerritMerged openstack/kolla-kubernetes master: Remove the invariable value in string.format()  https://review.openstack.org/46701001:37
openstackgerritjimmygc proposed openstack/kolla-ansible master: Add vSphere support to nova_compute  https://review.openstack.org/44952901:37
*** Pavo has quit IRC01:42
openstackgerritjimmygc proposed openstack/kolla-ansible master: Add Glance Swift backend support  https://review.openstack.org/45205901:42
*** chas has joined #openstack-kolla01:44
*** salv-orlando has quit IRC01:45
openstackgerritjimmygc proposed openstack/kolla-ansible master: Make glance filesystem_store_datadir configurable.  https://review.openstack.org/45209401:45
openstackgerritjimmygc proposed openstack/kolla-ansible master: Add vmware support to ceilometer  https://review.openstack.org/46284201:47
bmaltaisAny one has ceph experience with kolla-ansible?01:48
bmaltaisI am looking for info on making a basic ceph 3 storage node working under kolla-ansible01:49
*** chas has quit IRC01:49
bmaltaisSo far no luck. The guide at https://github.com/openstack/kolla-ansible/blob/master/doc/ceph-guide.rst is a bit unclear01:50
*** itlinux has quit IRC01:52
*** caowei has quit IRC01:54
*** caowei has joined #openstack-kolla01:57
*** yuanying_ has quit IRC01:57
*** yuanying has joined #openstack-kolla02:01
openstackgerritshaofeng cheng proposed openstack/kolla-ansible master: Add VMware DataStore support to cinder  https://review.openstack.org/45213102:04
*** chas has joined #openstack-kolla02:06
*** caowei has quit IRC02:06
*** caowei has joined #openstack-kolla02:07
bmaltaisI noticed something odd. instruction for ceph state to add node name to [storage] section...02:10
bmaltaisBut only adding the storage node name there does not deploy ceph containers on them...02:10
*** chas has quit IRC02:10
bmaltaisNodes running ceph also need to be added under [controller]02:10
*** spsurya has joined #openstack-kolla02:11
bmaltaisPerhaps they also need to be under [network] for ceph to work?02:11
*** zhurong has joined #openstack-kolla02:11
zhubingbingmoring duonghq02:11
openstackgerritAnthony Lin proposed openstack/kolla-kubernetes master: Script to Deploy Kolla Kubernetes AIO  https://review.openstack.org/46431202:12
duonghqsup zhubingbing02:13
openstackgerritDuong Ha-Quang proposed openstack/kolla-ansible master: Specify 'become' to necessary tasks (general roles)  https://review.openstack.org/39868202:21
*** eanylin has quit IRC02:22
openstackgerritDuong Ha-Quang proposed openstack/kolla-ansible master: Specify 'become' for only necessary tasks (default roles)  https://review.openstack.org/39868402:22
openstackgerritDuong Ha-Quang proposed openstack/kolla-ansible master: Specify 'become' for only neccesary tasks (all other roles)  https://review.openstack.org/39868502:24
*** chas has joined #openstack-kolla02:26
*** chas has quit IRC02:30
openstackgerritjimmygc proposed openstack/kolla-ansible master: Fix fwaas options  https://review.openstack.org/46388202:34
sdakeinc0 you about02:35
sdakezhubingbing you about02:35
sdakesup duonghq02:35
zhubingbinghi sdake02:35
*** stefan_bo has joined #openstack-kolla02:35
sdakezhubingbing my rack is about online02:35
sdakezhubingbing i need totake out my pcie ssd and reinstall centos02:35
zhubingbing;)02:35
sdakeand then you wlil be ready to go02:35
zhubingbingok02:36
duonghqhey sdake02:37
*** stefan_bo has quit IRC02:39
*** sgrasley1 has quit IRC02:44
*** masber has joined #openstack-kolla02:45
*** sgrasley has joined #openstack-kolla02:46
*** chas has joined #openstack-kolla02:47
*** iniazi_ has quit IRC02:51
*** chas has quit IRC02:52
*** iniazi_ has joined #openstack-kolla02:56
*** ducttap__ has quit IRC03:01
*** eanylin has joined #openstack-kolla03:01
*** ducttape_ has joined #openstack-kolla03:04
*** chas has joined #openstack-kolla03:08
*** huzhengchuan has quit IRC03:09
*** huzhengchuan_ has quit IRC03:09
*** itlinux has joined #openstack-kolla03:10
*** chas has quit IRC03:12
*** cliles has quit IRC03:15
*** huzhengchuan has joined #openstack-kolla03:17
*** huzhengchuan_ has joined #openstack-kolla03:17
*** jamesbenson has joined #openstack-kolla03:19
*** caowei has quit IRC03:25
*** chas has joined #openstack-kolla03:29
*** eaguilar has quit IRC03:29
spsuryamorning peeps03:32
*** chas has quit IRC03:34
duonghqmorning spsurya03:34
*** krtaylor has joined #openstack-kolla03:39
*** caowei has joined #openstack-kolla03:47
*** ducttape_ has quit IRC03:48
*** chas has joined #openstack-kolla03:50
*** chas has quit IRC03:55
openstackgerritChen proposed openstack/kolla master: Add "Vitrage" to "Openstack services" in README.rst  https://review.openstack.org/46322803:58
*** ChanServ changes topic to "support: ask.openstack.org | New to Kolla: docs @ http://docs.openstack.org/developer/kolla/ | | Kolla IRC meetngs on Wednesdays @ 16:00 UTC - see agenda @ https://goo.gl/OXB0DL - IRC channel is *LOGGED* @ http://goo.gl/3mzZ7b"04:07
-openstackstatus- NOTICE: Sufficient free space has been reclaimed that jobs are passing again; any POST_FAILURE results can now be rechecked.04:07
*** itlinux has quit IRC04:08
*** vhosakot has joined #openstack-kolla04:09
*** chas has joined #openstack-kolla04:11
*** zhurong has quit IRC04:11
*** vhosakot has quit IRC04:11
*** zhurong has joined #openstack-kolla04:15
*** chas has quit IRC04:15
openstackgerritDuong Ha-Quang proposed openstack/kolla-ansible master: Specify 'become' for only necessary tasks (default roles)  https://review.openstack.org/39868404:18
*** lucasxu has joined #openstack-kolla04:20
openstackgerritDuong Ha-Quang proposed openstack/kolla-ansible master: Specify 'become' for only neccesary tasks (all other roles)  https://review.openstack.org/39868504:31
*** chas has joined #openstack-kolla04:31
*** chas has quit IRC04:36
*** jamesbenson has quit IRC04:50
*** chas has joined #openstack-kolla04:52
*** jaosorior_away is now known as jaosorior04:55
*** chas has quit IRC04:57
*** lucasxu has quit IRC05:10
*** skramaja has joined #openstack-kolla05:11
*** chas has joined #openstack-kolla05:13
*** chas has quit IRC05:18
*** chas has joined #openstack-kolla05:31
*** huzhengchuan_ has quit IRC05:36
*** huzhengchuan_ has joined #openstack-kolla05:41
*** mewald has joined #openstack-kolla05:42
*** yingjun has quit IRC05:43
*** spsurya has quit IRC05:45
*** spsurya has joined #openstack-kolla05:49
*** stefan_bo has joined #openstack-kolla05:54
*** unicell has joined #openstack-kolla06:07
*** chas has quit IRC06:13
*** huzhengchuan_ has quit IRC06:16
*** huzhengchuan has quit IRC06:16
*** haplo37 has quit IRC06:18
*** haplo37 has joined #openstack-kolla06:20
*** gfidente has quit IRC06:21
*** huzhengchuan has joined #openstack-kolla06:21
*** huzhengchuan_ has joined #openstack-kolla06:22
*** chas has joined #openstack-kolla06:24
*** jmccarthy has joined #openstack-kolla06:26
*** korzen has joined #openstack-kolla06:26
*** mandre|away is now known as mandre06:35
*** huzhengchuan has quit IRC06:38
*** huzhengchuan_ has quit IRC06:38
*** mnasiadka has joined #openstack-kolla06:45
*** huzhengchuan_ has joined #openstack-kolla06:47
*** huzhengchuan has joined #openstack-kolla06:47
*** caowei has quit IRC06:47
*** matrohon has joined #openstack-kolla06:52
*** unicell has quit IRC06:59
*** manheim_ has joined #openstack-kolla07:02
*** manheim_ has quit IRC07:02
*** unicell has joined #openstack-kolla07:05
*** unicell has quit IRC07:06
*** manheim_ has joined #openstack-kolla07:06
*** jroll has quit IRC07:12
*** manheim_ has quit IRC07:13
*** shardy has joined #openstack-kolla07:14
*** athomas has joined #openstack-kolla07:15
*** pcaruana has joined #openstack-kolla07:16
*** oanson has quit IRC07:36
*** oanson has joined #openstack-kolla07:37
*** powerd has joined #openstack-kolla07:42
*** mnasiadka has left #openstack-kolla07:44
*** mnasiadka has joined #openstack-kolla07:44
*** egonzalez has joined #openstack-kolla07:45
*** korzen has quit IRC07:50
mewaldAnybody else have problems building neutron-server on ubuntu? https://gist.github.com/mewald1/ae2dc609352eaa0927963a0500b0c58007:55
*** sbezverk_ has joined #openstack-kolla07:56
*** lpetrut has joined #openstack-kolla07:58
openstackgerritEduardo Gonzalez proposed openstack/kolla master: DNM: test master branch  https://review.openstack.org/45610807:59
*** sbezverk has quit IRC07:59
egonzalezmewald, ubuntu binary gates suffer that problem, not sure if is an issue on our side or ubuntu repositories08:09
*** mgoddard has joined #openstack-kolla08:09
hrwmorning08:10
hrwhttp://people.linaro.org/~marcin.juszkiewicz/kolla/ has results of my recent builds on aarch64, ppc64le, x86-64 architectures. Logs are available08:10
hrwnice easy to read html table08:11
*** sambetts|afk is now known as sambetts08:20
mewaldegonzalez: I was able to fix it by applying this patch to upstream: https://gist.github.com/mewald1/08f88f7cacf8b126e20af8f562557d1d The "allow-downgrades" should not be necessary when building from scratch.08:21
*** caowei has joined #openstack-kolla08:24
*** pbourke has quit IRC08:29
*** pbourke has joined #openstack-kolla08:30
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: glance: use proper package name for python-oslo.vmware  https://review.openstack.org/46752108:31
hrwegonzalez: that patch gets glance built for debian08:32
egonzalezhrw, roger, atm all kolla gates are busted due a mariadb issue, trying to find whats happening but no clue for now08:34
hrwegonzalez: ok.08:34
hrwegonzalez: I am looking at results table and going image by image with small fixes08:34
*** duonghq has quit IRC08:35
*** dasTor_ has quit IRC08:37
*** dasTor has joined #openstack-kolla08:37
*** duonghq has joined #openstack-kolla08:38
*** korzen has joined #openstack-kolla08:39
*** dciabrin has quit IRC08:44
*** Serlex has joined #openstack-kolla08:56
*** dciabrin has joined #openstack-kolla08:59
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: use proper package name for python-oslo.vmware  https://review.openstack.org/46752109:01
hrwfound two more places09:01
*** daidv has joined #openstack-kolla09:02
hrw~curse rebuilds09:04
hrwgit merge; kolla == rebuilds of base, openstack-base ;(09:06
*** gfidente has joined #openstack-kolla09:07
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible master: Fix libgalera path  https://review.openstack.org/46753909:07
openstackgerritEduardo Gonzalez proposed openstack/kolla stable/ocata: [DNM] test ocata branch  https://review.openstack.org/45651409:10
*** rmart04 has joined #openstack-kolla09:23
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: Add an option to get clean logs  https://review.openstack.org/46754609:29
hrwegonzalez: this 'clean logs' stuff is handy for people who do rebuilds over and over and use --logs-dir option09:29
*** salv-orlando has joined #openstack-kolla09:35
openstackgerritSam Betts proposed openstack/kolla-kubernetes master: Combine ironic-inspector deployments into one pod  https://review.openstack.org/46727209:36
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible master: Fix libgalera path  https://review.openstack.org/46753909:37
*** spsurya has quit IRC09:47
*** salv-orlando has quit IRC09:48
*** dciabrin has quit IRC09:49
*** tovin07_ has quit IRC10:03
*** matrohon has quit IRC10:04
*** zhurong has quit IRC10:04
openstackgerritDuong Ha-Quang proposed openstack/kolla-ansible master: Specify 'become' for only necessary tasks (default roles)  https://review.openstack.org/39868410:05
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: dragonflow: install proper package on Debian/Ubuntu  https://review.openstack.org/46756810:06
openstackgerritSam Betts proposed openstack/kolla-kubernetes master: Combine ironic-inspector deployments into one pod  https://review.openstack.org/46727210:07
Serlexhi, is ceph generally deployed on the controller or compute nodes?10:08
egonzalezSerlex, mons on controller and osds in storage nodes10:10
*** cuongnv has quit IRC10:11
SerlexThanks egonzalez.10:12
*** ArminderSingh- has quit IRC10:13
sean-k-mooneyo/10:22
egonzalezsean-k-mooney, cannot find the root cause of the galera issue, found the libgalera is in other path, but seems mariadb is not recognizing wsrep variables10:23
sean-k-mooneyegonzalez: i was just going to ask if ye had any luck with the gate issue10:24
*** jroll has joined #openstack-kolla10:25
egonzalezI dont understand why mariadb removed/changed the packages in a stable release10:26
*** korzen has quit IRC10:27
sean-k-mooneywell actully looking back at the 10.x folders in that repo i dont see the package we were installing unless we go back to the 5.5 branch10:28
sean-k-mooneyall of the 10.x branches are missing the Mariadb-galara-server rpm10:28
sean-k-mooneythey are still packaged on ubuntu though http://nyc2.mirrors.digitalocean.com/mariadb/repo/10.0/ubuntu/pool/main/m/mariadb-10.0/ hence why the ubuntu jobs are fine10:29
*** stefan_bo has quit IRC10:29
openstackgerritSam Betts proposed openstack/kolla-kubernetes master: Combine ironic-inspector deployments into one pod  https://review.openstack.org/46727210:32
egonzalezhrm, 34 hours mariadb-galera-server was present in their repos with version MariaDB* 10.0.30, now installs 10.0.31 http://paste.openstack.org/show/610533/10:32
*** daidv has quit IRC10:35
openstackgerritPaul Bourke (pbourke) proposed openstack/kolla-ansible master: Add vhost to outward rabbitmq for Murano  https://review.openstack.org/45420610:42
openstackgerritPaul Bourke (pbourke) proposed openstack/kolla-ansible master: Add a new 'outward' rabbitmq instance  https://review.openstack.org/45373810:42
*** sgrasley has quit IRC10:45
pbourkecores please ack https://review.openstack.org/#/c/463798/10:50
openstackgerritSam Betts proposed openstack/kolla-kubernetes master: Combine ironic-inspector deployments into one pod  https://review.openstack.org/46727210:51
egonzalezsean-k-mooney, mariadb-galera-server packages should be in 10.0 repos as it was merged into 10.1, but from what mariadb guys told me galera package is not released at the same time. we can stick the repo to 10.0.30 instead of global 10.0 https://yum.mariadb.org/10.0.30/centos6-amd64/rpms/11:02
hrwcaowei: can we talk about dragonflow images?11:03
hrwcaowei: I want to bump libzmq1 -> libzmq3-dev to get them buildable on debian and ubuntu11:03
sean-k-mooneyegonzalez: ah cool i check a https://yum.mariadb.org/10.1/centos6-amd64/rpms/ and a few other and did not see it so i assumed it was nologer being made.11:09
sean-k-mooneyshould i rebase my patch to just point to https://yum.mariadb.org/10.0.30/centos6-amd64/rpms so and skip the name change?11:09
sean-k-mooneyegonzalez: also the 10.1.x 10.2.x and 10.3.x release all are missing the mariadb-galera-server packages given that some of those have been released since 2016 did they say if the mariadb-galera-server packages were going to be released going forward11:12
*** rhallisey has joined #openstack-kolla11:13
egonzalezsean-k-mooney, yep, mariadb-galera-server is merged into mariadb-server from 10.1, we install 10.011:13
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: dragonflow: install proper package on Debian/Ubuntu  https://review.openstack.org/46756811:14
sean-k-mooneyany reason not to got to 10.1 or later?11:14
egonzalezdunno, maybe we should update to a newer version11:14
egonzalezrdo delorean have 10.1 iirc11:15
hrwsean-k-mooney: on the other way: why use version outside of distribution channels?11:15
bmaltaisMorning11:15
openstackgerritzhubingbing proposed openstack/kolla master: Fix rally can't verifier error  https://review.openstack.org/46759111:15
sean-k-mooneyhrw: they were really broken that why we stopped using them11:16
zhubingbinghi guys11:16
bmaltaisegonzalez: Do you have experience with ceph? I was able to deploy a basic environment with ceph but I noticed something strange.11:16
hrwin past distros got created to give some trusted place to grab binaries from. nowadays I feel that software development goes more and more into 'lets grab some random binary from net and hope for best'11:16
sean-k-mooneyegonzalez: the latest stable release is 10.2.6 so maybe we should bump to 10.211:16
bmaltaisegonzalez: According to the documentation you only need to add ceph storage node to the [storage] section... But that does not work11:16
bmaltaisI added my 3rd node as a storage node only in multinode and ceph never installed on it11:17
egonzalezbmaltais, some more steps are needed https://github.com/openstack/kolla-ansible/blob/master/doc/ceph-guide.rst#ceph-in-kolla11:17
sean-k-mooneyhrw: well i know that the distro binaries were causing test failures in the gate and the upstream biniars were adotpted to resolve those issues11:17
hrwsean-k-mooney: did someone reported bug in distros?11:18
bmaltaisegonzalez: That is the guide I follower ;-) but it is not clear. Just adding the new node under storage did not work11:18
sean-k-mooneyhrw: yep im pretty sure inc0 did11:18
hrwsean-k-mooney: asking cause yum.mariadb.org repo does not cover all architectures covered by kolla11:18
sean-k-mooneyhrw: the arm biniaries are missing right?11:19
bmaltaisegonzalez: I had to add it to all thew other sections and install all the containers for it to finally work. And make more globals.yml changes also.11:19
sean-k-mooneyit has ppc11:19
hrwsean-k-mooney: and while I do not have to care about x86-64 much I do about aarch6411:19
hrwsean-k-mooney: aarch64 are missing, yes11:19
openstackgerritzhubingbing proposed openstack/kolla master: Fix rally create verifier error  https://review.openstack.org/46759111:19
egonzalezi think we should stick to the distros for version compatibility11:19
sean-k-mooneywell we could try using 10.1 form rdo if that works11:19
hrwsean-k-mooney: we already do11:20
egonzalezbmaltais, to add a new ceph node, need to include in the inventory/ enable_ceph at globals and label each disk is going to be used as osd (parted /dev/sdb -s -- mklabel gpt mkpart KOLLA_CEPH_OSD_BOOTSTRAP 1 -1)11:20
sean-k-mooneyill disable the maria db repo in base and lets see how the gate like it11:20
hrwsean-k-mooney: it will break badly due to base_arch == x86_64 checks in images11:21
bmaltaisegonzalez: Did that and it is working... but I am puzzled as to why the new ceph node also need to be part of [controller], [compute], etc to properly deploy.11:21
hrwsean-k-mooney: external repos use MariaDB names while distros goes for mariadb names. and it messes yum11:21
sean-k-mooneyhrw: ok i can update that too11:22
bmaltaisegonzalez: If you only make it part of [storage] then ceph containers are never installed on it... is it normal?11:22
egonzalezbmaltais, osd go in storage group, and mons on controller11:22
hrwsean-k-mooney: base, kolla-toolbox, mariadb, openstack-base images11:22
sean-k-mooneywhy would kolla-toolbox or openstack-base need to be updated11:23
sean-k-mooneythe binary names are the same11:23
hrwbmaltais: and you need odd number of OSD nodes11:23
sean-k-mooneythe packages are different but the provide the same executables11:23
bmaltaisegonzalez: I see.11:24
hrwsean-k-mooney: for yum MariaDB != mariadb ;(11:24
*** korzen has joined #openstack-kolla11:24
bmaltaishrw: And a minimum of 3 ;-)11:24
hrwbmaltais: on Linaro Developer Cloud we have OSD on compute nodes and MON on controllers11:25
egonzalezbmaltais, 3 is for HA to make mons quorum, but can deploy an allinone with ceph too11:25
hrwsean-k-mooney: it is possible that it will just work when mariadb.repo gets dropped11:25
sean-k-mooneyhrw: yes but that is just the package name. the only place a yum install of mariadb is done should be in the mariadb container11:26
bmaltaisegonzalez: I wanted to test survivability so I went with 3 ;-) Thank you for the info and the help. I will experiment more with the setup to test.11:26
hrwsean-k-mooney: agree. that would be best11:26
sean-k-mooneyhrw: ill build locally and ill see what happens.11:26
hrwsean-k-mooney: kolla-toolbox wants to have mariadb client.11:27
sean-k-mooneymy only concern really with disableing the mariadb repo would be oracale linux. ok well ill see what breaks when i build11:28
hrwsean-k-mooney: and openstack-base only wants devel headers11:28
sean-k-mooneyhrw: ah yes it proably needs those for the python modules11:29
hrwyep11:29
openstackgerritPaul Bourke (pbourke) proposed openstack/kolla-ansible master: Add a new 'outward' rabbitmq instance  https://review.openstack.org/45373811:30
hrwsean-k-mooney: builds MySQL-python one11:30
openstackgerritPaul Bourke (pbourke) proposed openstack/kolla-ansible master: Add vhost to outward rabbitmq for Murano  https://review.openstack.org/45420611:30
*** stefan_bo has joined #openstack-kolla11:35
*** mnasiadka has left #openstack-kolla11:37
*** ipsecguy_ is now known as ipsecguy11:40
*** cduchesne_ has quit IRC11:48
*** zeus has quit IRC11:49
*** cduchesne has joined #openstack-kolla11:50
sean-k-mooneyhrw: egonzalez there are conflits between percona and the delorean mariadb packages so percona would have to go too if we drop the mariadb repo11:52
*** gfidente has quit IRC11:52
*** zeus has joined #openstack-kolla11:53
sean-k-mooneyi assume we are using percona for the inodb/xtraDB features11:53
*** zeus is now known as Guest4163811:53
hrwsean-k-mooney: yep11:56
hrw(as to conflict)11:56
hrwsean-k-mooney: in short: we have 4 mariadb packages: centos, rdo, percona, mariadb.org11:56
sean-k-mooneywell percona we are just useing the inodb engine but we are using the main server binary from mariadb.org11:57
hrwsean-k-mooney: I only build ;D11:58
sean-k-mooneywe know the base centos rpm has or had issue with openstack in the past ill see if rdo ship a xtraDB binary but the best fix for the gate right now is probaly to just pin mariadb.org to 10.0.30 for now and revisit this in general later11:59
* hrw off for some12:00
bmaltaisegonzalez: You are correct. osd does get deployed to the node that is only under [storage]. I must have goofed with the initialization of the disk when I tried yesterday.12:01
bmaltaisegonzalez: All ceph disk need to be re-formated everytime a full rebuild of the environment is done. Just doing a kolla-ansible destroy is not enough for ceph.12:01
sean-k-mooneybmaltais: if you do multiple deploys you need to reinitalise the dies each time the ceph bootstrap tasks run as the change the partion lables12:05
sean-k-mooneybmaltais: baically if you did kolla-ansible deploy and then destroy and then deploy again you need to repartiion your ceph osd disks12:06
bmaltaissean-k-mooney: That is what I figured late last night unfortunately ;-) Part of learning by trial and error.12:07
bmaltaissean-k-mooney: I could not get the ods container running so I gave that a shot and it worked after ;-) Doh12:08
sean-k-mooneybmaltais: i hit the same issue. i suggested extending the destroy to undo the lablel changes so that it would work on deploy again but i never had time and it was actully more work then i had taught12:08
bmaltaissean-k-mooney: I think I will use my experience to add to the ceph guide to provide more info for future users that might run into this.12:09
sean-k-mooneydocs changes are always welcome12:09
bmaltaissean-k-mooney: The guide could be clearer as some of the steps are actually not helping to make things work... especially the part that config two partition on /dev/sdb12:10
sean-k-mooneythere are a few other ceph bugs that should be documented12:10
sean-k-mooneyand fixed12:10
sean-k-mooneyare you planning to use cache teirs?12:10
bmaltaisIsean-k-mooney: It just overwrite the 1st partition instead of adding a 2nd one (I think)...12:10
bmaltaissean-k-mooney: Did not implement that yet, Building the environment one lego brick at a time.12:11
sean-k-mooneythe ceph cache support in kolla does not set the total size fo the cache teir so if you use it it will never flush the data to the backing store12:12
bmaltaissean-k-mooney: In the end the only partition needed was "parted /dev/sdb -s -- mklabel gpt mkpart KOLLA_CEPH_OSD_BOOTSTRAP 1 -1"12:12
sean-k-mooneyafter kolla deploys you have to manually set that yourself12:12
sean-k-mooneybmaltais: yep that makes sense if you are not useing external journals that should be all that is needed12:12
bmaltaissean-k-mooney: Good to know if I implement caching12:13
bmaltaissean-k-mooney: So the guide could use an "example for a simple environment" section.12:13
bmaltaissean-k-mooney: Like a 3 node ceph with no external storage and cache12:13
bmaltaissean-k-mooney: Just to get users going with a basic 3 node ceph for testing.12:14
sean-k-mooneyi think it used to have that at one poit12:14
openstackgerritzhubingbing proposed openstack/kolla master: Fix rally create verifier error  https://review.openstack.org/46759112:14
openstackgerritsean mooney proposed openstack/kolla master: pin mariadb version for rpm distros  https://review.openstack.org/46736012:15
*** dciabrin has joined #openstack-kolla12:17
sean-k-mooneyegonzalez: hrw inc0 https://review.openstack.org/467360 seams to build locally fine ill ping you when the ci finishes if it fixes the gate jobs12:17
*** ducttape_ has joined #openstack-kolla12:18
*** krtaylor has quit IRC12:24
*** gfidente has joined #openstack-kolla12:27
*** gfidente has quit IRC12:27
*** gfidente has joined #openstack-kolla12:27
*** rwellum has joined #openstack-kolla12:28
*** daidv has joined #openstack-kolla12:31
*** lucasxu has joined #openstack-kolla12:31
*** zhurong has joined #openstack-kolla12:35
*** zhurong has quit IRC12:36
openstackgerritPaul Bourke (pbourke) proposed openstack/kolla-ansible master: Add dev mode for Murano  https://review.openstack.org/46760812:39
*** dave-mccowan has joined #openstack-kolla12:41
*** ducttape_ has quit IRC12:47
*** lucasxu has quit IRC12:50
*** fguillot has joined #openstack-kolla12:54
*** ducttape_ has joined #openstack-kolla13:11
*** eaguilar has joined #openstack-kolla13:11
*** Pavo has joined #openstack-kolla13:12
*** ducttape_ has quit IRC13:12
*** eanylin has quit IRC13:18
*** lucasxu has joined #openstack-kolla13:20
*** stefan_bo has quit IRC13:20
*** caowei has quit IRC13:21
*** jtriley has joined #openstack-kolla13:24
*** eaguilar has quit IRC13:29
*** emccormick has quit IRC13:29
*** vhosakot has joined #openstack-kolla13:30
*** huzhengchuan has quit IRC13:33
*** huzhengchuan_ has quit IRC13:33
*** eanylin has joined #openstack-kolla13:33
hawihi. quick question: i wonder if it possible to have mariadb outside kolla installation for openstack?13:34
*** eaguilar has joined #openstack-kolla13:36
egonzalezhawi, is possible, but never tried13:37
vhosakotsean-k-mooney: I'm digging gate logs FYI.13:38
egonzalezhawi, at first glance, enable_mariadb: no and configure database_address, database_user, database_port13:38
hawiegonzalez: what is needed for it, enable_mariadb="no" and.... aaaa, I see, superb13:38
egonzalezhawi, ansible should create user/db/permissions for each service13:39
openstackgerritzhubingbing proposed openstack/kolla master: Fix rally create verifier error  https://review.openstack.org/46759113:39
egonzalezhawi, so, database_user should be admin13:39
egonzalezhawi, maybe something else is needed, or need to change something in the playbooks, as said, never tried13:40
vhosakothrw: cool aarch64, ppc64le kolla images in http://people.linaro.org/~marcin.juszkiewicz/kolla/ :)13:41
hawiegonzalez: thanks, will try out13:43
openstackgerritSam Betts proposed openstack/kolla-kubernetes master: Combine ironic-inspector deployments into one pod  https://review.openstack.org/46727213:43
openstackgerritzhubingbing proposed openstack/kolla master: Fix rally create verifier error  https://review.openstack.org/46759113:46
*** krtaylor has joined #openstack-kolla13:47
*** Pavo has quit IRC13:48
*** Pavo has joined #openstack-kolla13:49
*** skramaja has quit IRC13:53
*** ddyer has quit IRC13:53
*** ddyer has joined #openstack-kolla13:57
*** eaguilar has quit IRC13:59
hrwvhosakot: status you mean? thanks14:09
hrwvhosakot: helps to see where issues are14:09
vhosakothrw: and, also how failed images are highlighted in red :)14:10
hrwvhosakot: table is based on https://fedora.juszkiewicz.com.pl/syscalls.html one14:10
vhosakothrw: very cool, we need to have similar build boards internally at my company :)14:11
*** iniazi_ has quit IRC14:11
hrwvhosakot: give me moment14:13
vhosakothrw: cool14:14
*** korzen has quit IRC14:15
openstackgerritJuan Badia Payno proposed openstack/kolla master: WIP - Modified fluentd Image  https://review.openstack.org/46721914:17
*** chas has quit IRC14:18
*** chas has joined #openstack-kolla14:18
*** chas has quit IRC14:19
hrwvhosakot: https://github.com/hrw/misc-scripts/tree/kolla/generate-html-table-with-multiarch-results14:21
hrwvhosakot: may not be easiest to catch how it works but should not be compolicated14:22
vhosakothrw: cool, thanks for the links! :)14:22
hrwvhosakot: and if you plan to reuse it then please host css/js instead of reusing my server14:22
vhosakothrw: yep, if we plan to use, I'll let you know and will ping you :)  cool work really14:23
*** hongbin has joined #openstack-kolla14:34
*** ducttape_ has joined #openstack-kolla14:35
*** ducttape_ has quit IRC14:36
*** lucasxu has quit IRC14:36
*** lucasxu has joined #openstack-kolla14:37
*** ducttape_ has joined #openstack-kolla14:37
*** ducttape_ has quit IRC14:42
*** ducttape_ has joined #openstack-kolla14:42
*** itlinux has joined #openstack-kolla14:45
*** iniazi_ has joined #openstack-kolla14:46
*** glycerine has joined #openstack-kolla14:49
*** lucasxu has quit IRC14:50
*** lucasxu has joined #openstack-kolla14:52
*** srwilkers has joined #openstack-kolla15:03
hrwargh. mariadb container is building for over 3h now15:07
sean-k-mooneyhrw: locally?15:08
hrwsean-k-mooney: aarch64 box15:09
sean-k-mooneyhrw: is it retrying or just running slow?15:09
hrwslow15:09
hrwINFO:kolla.image.build.mariadb:Setting up mariadb-server-10.1 (10.1.23-8) ...15:09
hrwsean-k-mooney: all my builds are with -r115:09
hrwsean-k-mooney: for me if image does not builds then it is broken so why repeat15:10
openstackgerritSam Betts proposed openstack/kolla-kubernetes master: Combine ironic-inspector deployments into one pod  https://review.openstack.org/46727215:10
sean-k-mooneyok still sound like it has hung or something has it been stuck on that line for 3 hours?15:10
sean-k-mooneyhrw: if you are behind a proxy like we are it can just be that it was flaky for a sec and the conection to donwload a package droped15:11
hrwsean-k-mooney: it fetched all packages. postinstall script of mariadb-server is acting weird probably15:12
sean-k-mooneyit just funny to me that you are having different mariadb issues when mariadb is also broken in the gate. i also notice that you are not use the same mariadb version on aarch64 as x86-64/ppc6415:16
hrwsean-k-mooney: aarch64 is still quite fresh architecture ;(15:16
*** manheim_ has joined #openstack-kolla15:17
sean-k-mooneywell ye are using a newer shinier version so isnt that a good thing :)15:17
hrwsean-k-mooney: machine I use has first aarch64 cpu ever so it is quite old at same time ;d15:18
sean-k-mooneyso  Cortex-A53?15:19
hrwsean-k-mooney: iirc X-Gene 1 was custom design15:20
*** sgrasley has joined #openstack-kolla15:20
*** mewald has quit IRC15:21
sean-k-mooneyoh wait aarch64 and arm64 are 2 different things hum15:21
hrwsean-k-mooney: same15:22
hrwaarch64 is architecture name, arm64 is kernel name, armv8a is instruction set name15:22
*** shardy has quit IRC15:23
sean-k-mooneyhehe intel has the same issues with x86 there are so many different names for x86-64 depending on if you men the architecture/kernel/instruction set15:23
*** ducttap__ has joined #openstack-kolla15:23
*** ducttape_ has quit IRC15:23
*** salv-orlando has joined #openstack-kolla15:24
hrwsean-k-mooney: x86 is complete mess ;d15:27
*** ducttape_ has joined #openstack-kolla15:28
hrwaodh 3.0.0 is ocata and aodh 4.0.0 is pike, right?15:28
sean-k-mooneyare you talking about kolla image version of aodh source version15:29
hrwupstream aodh versions. not kolla image tags15:29
*** salv-orlando has quit IRC15:30
hrwhm. no. 3.0.0 was newton. october 2016. so 4.0.0 is ocata (feb 2017)15:30
*** ducttap__ has quit IRC15:31
*** spsurya has joined #openstack-kolla15:31
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: aodh: disable for Debian/binary combo  https://review.openstack.org/46765415:32
*** itlinux has quit IRC15:33
egonzalezhrw, aodh ocata is 4.0.0 https://review.openstack.org/#/c/428218/3/deliverables/ocata/aodh.yaml15:33
*** tvignaud is now known as tvignaud_pto15:34
*** duonghq has quit IRC15:35
*** duonghq has joined #openstack-kolla15:35
openstackgerritMartin André proposed openstack/kolla master: Define blocks around macros  https://review.openstack.org/46765515:35
*** duonghq_ has joined #openstack-kolla15:36
*** itlinux has joined #openstack-kolla15:36
duonghqevening guys15:36
kfox1111morning.15:36
spsuryakfox1111: morning15:39
spsuryasup duonghq15:39
kfox1111spsurya: morning. :)15:39
hrwhm. looks like I have to automate my kolla builders ;D15:40
*** jamesbenson has joined #openstack-kolla15:42
egonzalezany from k8s can take a look at this bug? https://bugs.launchpad.net/kolla-ansible/+bug/169301415:42
openstackLaunchpad bug 1693014 in kolla-ansible "kolla-ansible / kolla-kubernetes neutron/tasks/config.yml mismatch" [Undecided,New]15:42
openstackgerritDuong Ha-Quang proposed openstack/kolla-ansible master: Specify 'become' for only necessary tasks (default roles)  https://review.openstack.org/39868415:42
egonzalezafik, genconfig is not used anymore to generate templates15:43
*** korzen has joined #openstack-kolla15:46
kfox1111egonzalez: looking....15:46
openstackgerritSam Betts proposed openstack/kolla-kubernetes master: Combine ironic-inspector deployments into one pod  https://review.openstack.org/46727215:47
*** manheim_ has quit IRC15:48
sdakekfox1111 already looked15:48
*** manheim_ has joined #openstack-kolla15:49
kfox1111sdake: k. yeah, I think the dev docs just need updating to do kolla-kubernetes genconfig instead of kolla-ansible.15:49
*** salv-orlando has joined #openstack-kolla15:52
*** itlinux has quit IRC15:52
*** manheim_ has quit IRC15:53
vhosakotsean-k-mooney: just rechecked https://review.openstack.org/#/c/467360/, patch is great, except don't know why ubuntu xenial fails to build neutron images in gate -->  http://logs.openstack.org/60/467360/2/check/gate-kolla-dsvm-build-ubuntu-binary-ubuntu-xenial-nv/45a5fd2/console.html#_2017-05-24_14_36_57_104937 (looks unrelated to your patch set).15:54
egonzalezvhosakot, ubuntu binary is having issues with deps, not sure if is our business or ubuntu repo issues15:56
*** lucasxu has quit IRC15:56
vhosakotegonzalez: ah cool, thnaks15:56
egonzalezvhosakot, mewal was able to fix locally with this change https://gist.github.com/mewald1/08f88f7cacf8b126e20af8f562557d1d15:56
vhosakotI'll start reviewing more again :)15:57
vhosakotegonzalez: cool15:57
egonzalezbut I dont think -allow-downgrades is a good way to fix that15:57
rwellumWe have meeting today?15:57
*** salv-orlando has quit IRC15:57
sean-k-mooneyvhosakot: ya i saw that the neutron server image failed but i dont know either15:58
vhosakotdon't like to just review to increase my +1/+2 numbers in stackalytics, when I review, I'll make sure that I try to understand the code/bug/bp :)15:58
spsuryarwellum: https://wiki.openstack.org/wiki/Meetings/Kolla15:58
vhosakotsean-k-mooney: ah cool :)15:58
spsuryahope so15:58
duonghqt-2min15:58
*** unicell has joined #openstack-kolla15:58
rwellumCheers spsurya - I have the schedule but no-one had mentioned it yet that I saw :)15:59
spsuryarwellum: :)15:59
sean-k-mooneyvhosakot: on the mariadb container one of the topics at the ptg was to reduce or stop pulling biniars from random places on the internet16:00
*** korzen has quit IRC16:00
vhosakotcool, random upstream packages on the internet, I fear them :)16:01
vhosakotsean-k-mooney: ^^16:01
sean-k-mooneyvhosakot: we might want to bring back up the quetion of getting the required version into rdo and ubuntu could archives16:01
vhosakotsean-k-mooney: yep16:01
*** manheim_ has joined #openstack-kolla16:02
sean-k-mooneyvhosakot: personnally i prefer the upstream packages to distro in this case but if percona and mariadb in rdo/cloud archive work it would be better16:02
vhosakotsean-k-mooney: right, rdo/ubuntu cloud archive is not so random and safe, agreed :)16:02
sean-k-mooneywell the ubuntu cloud archive pulling in libvirt 2.5 did break the world but at least all the pointy objects are in one place and we know who to shout at :)16:03
*** rcrit has joined #openstack-kolla16:07
rcritI'm trying to containerize a new openstack service and am working on the source target. Building it fails with:16:07
rcritStep 3 : ADD novajoin-archive /novajoin-source16:07
rcritlstat novajoin-archive: no such file or directory16:08
rcritI assume I need to put a tarball somewhere?16:08
jaosoriorrcrit: might wanna point to the review's URL16:08
sean-k-mooneyyou need to define an entry in the kolla-build config for the base image of the service16:08
rcritjaosorior, bah, it looks like it is right there in the docs16:09
rcritsean-k-mooney, thanks, should have checked those first :-(16:09
jaosoriorI thought I had added it tot he config16:09
egonzalezrcrit, the file to copy is novajoin-base-archive instead of novajoin-archive16:10
vhosakotrcrit: for source builds, the tar's path is in  kolla/common/config.py -->  https://github.com/openstack/kolla/blob/master/kolla/common/config.py#L525-L52816:10
hrwsean-k-mooney: I may found out why mariadb-server halts. 'logger' ;(16:10
sean-k-mooneyegonzalez: in generall isnt it <foldername>-archive where <foldername> is the folder you create under the docker directory16:11
jaosorioroh, right, it should be novajoin-base-archive as egonzalez says https://review.openstack.org/#/c/466268/9/kolla/common/config.py16:12
*** salv-orlando has joined #openstack-kolla16:12
*** manheim_ has quit IRC16:13
sean-k-mooneyjaosorior: so out of interest what is novajoin? is it cells v2 related?16:13
jaosoriorit's not16:13
jaosoriorit's a vendordata plugin that registers nova instances into FreeIPA16:14
sean-k-mooneyi read the title of https://github.com/openstack/novajoin and it confuses me16:14
*** Serlex has quit IRC16:15
jaosoriorso, novajoin would get the request that an instance was created, it will register it in FreeIPA, get an OTP for it, pass the OTP via metadata to the instance, and a cloud-init script via user-data. And the node should automatically enroll to FreeIPA.16:15
sean-k-mooneyjaosorior: whats an IPA server (ironic python agent?)16:16
jaosoriorso being enrolled, it could requests stuff such as certs from the CA, or secrets and such.16:16
rcritthanks guys, it is buildin gnow16:16
rcritIPA is an identity management service16:16
jaosoriorhaha damn, IPA is quite overused :D16:16
rcritfreeipa.org16:16
sean-k-mooneyrcrit: jaosorior ya just googleing FreeIPA now16:16
sean-k-mooneyjaosorior: is it a plugin to nova?16:17
jaosoriorsean-k-mooney: it's a bundle of services that includes LDAP, kerberos, dogtag (for secret storage and is a CA as well) amongst other things16:17
jaosoriorsean-k-mooney: novajoin is a plugin for nova. FreeIPA is it's own project.16:17
rcritsort of. nova no longer allows plugins16:17
jaosoriorrcrit would probably explain it way better than me16:17
sean-k-mooneyok so plugins are handeled differently in kolla then services16:17
rcritit instead defines a REST mechanism to get data from external sources to inject into an instance metadata16:17
sean-k-mooneyjaosorior: plugins follow this workflow https://github.com/openstack/kolla/blob/master/doc/image-building.rst#plugin-functionality16:18
jaosoriorsean-k-mooney: so, I didn't really follow that since it doesn't really need much from nova (only json requests coming from it and rabbitmq notifications)16:19
Reepicheepany issues with upgrading to ansible 2.3.x?  I have been holding off now it is actually in the CentOS repos16:20
jaosoriorsean-k-mooney: it doesn't even need openstack-nova-common anymore16:20
*** chas has joined #openstack-kolla16:20
sean-k-mooneyjaosorior: yes but for plugins we do not create seperate containers. the are added to the service that loads the plugin via a template override16:20
Reepicheepactually it is in the epel repo16:21
rcritplugin vs service is a bit fuzzy here. It has its own listen port, uses its own config, etc. It is a standalone service that is only called by nova16:21
*** jmccarthy has left #openstack-kolla16:21
sean-k-mooneyrcrit: ah so its a seperate process and not a plugin in the same sense as the neutron or horizon plugins16:22
*** manheim_ has joined #openstack-kolla16:22
rcritcorrect16:22
rcritI don't believe nova has plugins any more (they had a hooks mechanism that was deprecated some time ago)16:22
sean-k-mooneyrcrit: the scheduler is plugable16:23
rcritok, fair point16:23
rcritI have blinders on when it comes to some services :-)16:23
*** pcaruana has quit IRC16:24
sean-k-mooneyjaosorior: rcrit so is the intent to create teh new contaienr as a child of https://github.com/openstack/kolla/tree/master/docker/nova or as a sibling16:24
*** chas has quit IRC16:25
openstackgerritPaul Bourke (pbourke) proposed openstack/kolla master: Make rabbitmq log dir configurable  https://review.openstack.org/45373516:25
*** manheim_ has quit IRC16:26
sean-k-mooneyjaosorior: rcrit bassically im asking will it inherit from nova-base or from openstack-base?16:26
vhosakotrcrit: in neutron world, every neutron plugin is a service, isn't it?16:26
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla-ansible master: Enable multinode gate  https://review.openstack.org/46600716:26
rcritsean-k-mooney, openstack-base16:27
*** eaguilar has joined #openstack-kolla16:27
*** manheim_ has joined #openstack-kolla16:27
sean-k-mooneyvhosakot:  not quite. they done expose there own tcp port for the rest services the extend the main neutron api16:27
rcritreview is https://review.openstack.org/#/c/466268/16:27
sean-k-mooneyvhosakot: the ml2 dirvers also are plugins that are part of the neutron-server process16:28
rcritit works fine with a binary build, confirming it works with source build as well now16:28
vhosakotsean-k-mooney: plugins are not seen in like "ps aux" or "ps -ef" ?16:28
sean-k-mooneyvhosakot: correct they are not seen for nova or horizon16:29
vhosakotsean-k-mooney: ah nova and horizon, cool16:29
sean-k-mooneyvhosakot: for novajoin it sounds like it would be seen as its a seperate process16:29
*** faridda has joined #openstack-kolla16:31
*** eaguilar has quit IRC16:31
rcritwoo source containers built, thanks for all the help16:31
vhosakotsean-k-mooney: ah cool16:31
*** lpetrut has quit IRC16:31
*** ansiwen has joined #openstack-kolla16:32
*** manheim_ has quit IRC16:33
*** sambetts is now known as sambetts|afk16:34
sean-k-mooneyvhosakot: if it was part of the nova-api or compute agents then it could use https://github.com/openstack/kolla/blob/master/doc/image-building.rst#plugin-functionality but in there case ya they will need a seperate container and role/tasks to deploy and configure it. nova always finds a way to be special lol16:34
vhosakotsean-k-mooney: cool, thanks for the info.16:36
*** egonzalez has quit IRC16:36
*** manheim_ has joined #openstack-kolla16:37
*** unicell has quit IRC16:48
*** eaguilar has joined #openstack-kolla16:50
*** lucasxu has joined #openstack-kolla16:51
*** emccormick has joined #openstack-kolla16:52
*** salv-orlando has quit IRC16:53
jamesbensonI'm deploying kolla with 3 controllers and it seems to not get past "Waiting for MariaDB service to be ready". The docker images say they are up and running though, thoughts?16:53
hrwdocker--16:53
hrwhalf day wasted because debian container on centos != debian container on debian16:54
sean-k-mooneyhrw: yes there be dragons there. and kernel abi breakages but mainly dragons16:56
*** salv-orlando has joined #openstack-kolla16:57
sean-k-mooneyhrw: for what its worth it works bettern now everything uses systemd then it did when you had different init systems too16:57
hrwsean-k-mooney: next step: VM containers16:57
sean-k-mooneyvm contaienrs as in clear/hyper/hyperv containers16:58
hrwsean-k-mooney: everything uses systemd. except containers which use whateverdevsinventedonthatdayinit16:58
hrwdumbinit, tinyinit, MyNameIsChrisInit16:59
sean-k-mooneyya :( i dislike systemd mailly because it has dbus as a singel point of failure16:59
hrwsean-k-mooney: so use sysvinit like everyone before systemd.17:00
*** kolla-slack has quit IRC17:00
hrwI am 41y old and tired of reinventing the wheel over and over and over again17:00
*** kolla-slack has joined #openstack-kolla17:00
sean-k-mooneyi dont dislike it enought to not use it i just dont like that if dbus dies i cant reboot my system without ipmi or pressing the power button17:01
*** salv-orlando has quit IRC17:02
*** manheim_ has quit IRC17:02
*** lucasxu has quit IRC17:03
hrw;D17:04
* hrw off17:04
*** mgoddard has quit IRC17:05
*** powerd has quit IRC17:06
*** lucasxu has joined #openstack-kolla17:07
rwellumDoes anyone know how to clean up the dockers library safely? /var/lib/docker... I ask because as I deploy and redeploy in my Centos VM - it rapidly fills up. I thought that shouldn't really happen unless the containers change.17:07
duonghqnight guys, cya17:10
*** duonghq has quit IRC17:10
jamesbensonAny HA recommendations?17:10
*** mewald has joined #openstack-kolla17:12
vhosakotrwellum: the only successful way I know to cleanup is:  stop all containers, delete all containers, delete all volumes, rm -rf /var/lib/docker, reboot, "rm -rf /var/lib/docker" once more again after reboot.17:13
sean-k-mooneyjamesbenson: it makes you life harder until something goes wrong. but for any aspect in particalar?17:14
jamesbensonI'd like to have 3 controllers/networks, but mariadb doesn't seem to like that17:15
*** sbezverk_ has quit IRC17:15
sean-k-mooneyjamesbenson: oh that should work fine. a deployment like that currently17:15
sean-k-mooneyjamesbenson: what was the error?17:16
jamesbensonmariadb : Waiting for MariaDB service to be ready fails17:16
sean-k-mooneydid you run the precheks on the nodes before deploying?17:16
jamesbensonyep, all passed17:16
sean-k-mooneyok i have hit that in the past on 2 ocations 1 i had a process bound to the mariadb port on the host the other the mariadb contianer was contiually rebooting17:17
sean-k-mooneydid the mariadb contaienr start correctly?17:17
jamesbensonit restarts every few seconds17:18
sean-k-mooneyok then that is similar to what happend to me. what release are you using?17:18
sean-k-mooneyis it master or a stable branch17:18
jamesbensonstable17:19
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla-ansible master: Enable multinode gate  https://review.openstack.org/46600717:19
*** unicell has joined #openstack-kolla17:20
jamesbensonthanks sean-k-mooney for the help17:20
*** gfidente has quit IRC17:22
jamesbensonkolla-ansible (4.0.1)17:22
*** athomas has quit IRC17:22
sean-k-mooneyjamesbenson: i cant rembere of the top of my head why this failed in the past but i would try running docker logs on the mariadb container17:22
jamesbensonyeah, I did, but nothing good there, just setting permissions...17:23
sean-k-mooneydid you build the conteiner youself in the last few days17:23
jamesbensonno I pulled it17:23
*** jaosorior has quit IRC17:24
sean-k-mooneyok and is it centos or ubuntu?17:24
jamesbensonubuntu and I'm deploying 4.0.017:24
jamesbensonubuntu source17:25
sean-k-mooneyok cool i was wondering if you were hitting the galara issue we were having in the gate17:25
*** rmart04 has quit IRC17:25
sean-k-mooneyhttps://github.com/openstack/kolla/blob/4.0.1/docker/base/MariaDB.repo on the rpm side use the same repo as master and would not build correctly currently17:25
*** bmace__ has quit IRC17:26
sean-k-mooneyjamesbenson: one thing you could try is to modify the run command in /etc/kolla/mariadb/config.json from /usr/bin/mysqld_safe to bash17:28
*** manheim has quit IRC17:28
sean-k-mooneythen when the contaier start you can docker exec into it and run /usr/bin/mysqld_safe yourself and see if there is a more detailed error17:28
rwellumvhosakot: ah that's messy - especially with the reboot.17:30
*** AzazeL_ has joined #openstack-kolla17:30
sean-k-mooneyrwellum: is it filling up because of logs or are you rebuiling the images when redeploying?17:31
trozetmandre: hi17:31
*** mewald has quit IRC17:32
sean-k-mooneyrwellum: if your using kolla-ansible destroy the only thing that i can tink of that would be growing would be images if you are rebuilind between deploys17:33
rwellumsean-k-mooney: actually this is kolla-kubernetes17:33
*** eaguilar has quit IRC17:34
*** logan- has quit IRC17:34
rwellumsean-k-mooney: it's a good qn.  I'm basically following the deployment guide - is it rebuilding or pulling the images? Not sure...17:34
sean-k-mooneyrwellum: oh well i guess it should be the same though if you destroy all the pods and the volumes then it should remove everythign form the previous build except atign the cached images locally on the worker node17:35
*** logan- has joined #openstack-kolla17:35
sean-k-mooneyrwellum: i havent used kolla-kubernetes but i think the charts were referecing docker hub by default17:36
rwellumOk so not building then?17:37
jamesbensonhttp://paste.openstack.org/raw/610588/17:37
jamesbensonlooks like it couldn't connect to my other controllers17:37
rwellumsean-k-mooney: https://www.irccloud.com/pastebin/FeTdgZoA/17:38
*** sbezverk has joined #openstack-kolla17:39
sean-k-mooneyrwellum: well for a start you are using devicemapper which means your performance is going to suck17:40
sean-k-mooneyrwellum: if you run docker images do you see alot of image with <none> if so docker rmi $(docker images | grep "<none>" | awk '{print $3}') will remove all the image you dont need17:41
rwellumI think once I clean up, I haven't found any dangling volumes etc - all seems clean. But that directory keeps filling up.17:42
rwellumInteresting re devicemapper - will have to ask sbezverk or sdake if there's a reason for this.17:42
sean-k-mooneyjamesbenson: are do you have a proxy and can you ping the controler ips form inside the container?17:43
sean-k-mooneyrwellum: devicemapper can be ok if its directly on hardware but if its useing a loopback block device its horrible17:43
sean-k-mooneyyou are better off with overlayfs,btrfs or aufs. i use btrfs personally17:44
openstackgerritBernard Maltais proposed openstack/kolla-ansible master: Add a complete example of configuring a basic 3 nodes Ceph environment. This should help new users get started quickly by having all of the needed configuration to accomplish the task in one place.  https://review.openstack.org/46769517:45
rwellumsean-k-mooney: loopback block device is exactly what I am using :)17:45
vhosakotrwellum: I've seen some resources (lock files, pid files, sockets) getting cleanup only after reboot... I see error like "cannot delete /var/lib/docker, file blah in used by docker.service" even even after I've stopped/deleted all the containers and volumes.17:45
sean-k-mooneyrwellum: docker defaults to creating a loopback device with devicemapper on ubuntu i think when not using the docker repo17:45
rwellumvhosakot: ack - it's a good point and it's not hard to reboot a vm :)17:46
rwellumsean-k-mooney: ack - ok will dig in a little - maybe change how I'm doing things here.17:46
sbezverkrwellum: btrfs is way to go ;)17:47
sean-k-mooneysdake: and inc0 both have many feeling on the topic of devicemapper and loopback device hehe17:47
vhosakotdevicemapper is apparently one of the top searched word in google in 201617:48
rwellumWell I noticed my deployment seems very slow, but hadn't considered this as a bottle neck until now.17:49
*** salv-orlando has joined #openstack-kolla17:49
rwellumvhosakot: dang one reboot and I went to 98% free capacity.17:49
rwellum:)17:49
vhosakotrwellum: :)17:50
sean-k-mooneyrwellum: if you looked in lsof there was proablly lots of files pending deltion17:50
sean-k-mooneyrwellum: i do all or allmost all my dev in an openstack vm on a cloud i deployed with kolla17:51
sean-k-mooneyrwellum: works pretty well17:51
jamesbensonsean-k-moooney: no proxy; it'll be a few minutes, I just destroyed.... I know with 1 controller, everything worked fine.. I'm slowly building up complexity to what we normally have in production.17:52
rwellumsean-k-mooney: Yeah that's where I want to get to.17:53
sean-k-mooneyjamesbenson: makes sense. it is likely that iptables/ufw is blocking it or there is a routing error or something along those lines17:53
sean-k-mooneyrwellum: eventually i need to learn to upgrade a deployment in kolla17:53
sean-k-mooneyup to now i have had the freedom to just redeploy an new cloud every 6 months or so17:54
rwellumsean-k-mooney: yeah makes sense.17:54
rwellumunder 5 minutes too :)17:54
sean-k-mooneywell if you are using ceph and have to partition all the drives and other stuff its a little longer but it still pretty fast17:55
*** salv-orlando has quit IRC17:56
*** sgrasley has quit IRC17:59
*** mgoddard has joined #openstack-kolla17:59
jamesbensonI'm using MaaS in our datacenter so it takes even longer, but still a heck of a lot faster than fuel and other methods....18:00
jamesbenson(We used to use fuel)18:00
sean-k-mooneyif you have maas you were not tempted by juju for your openstack deployment?18:01
sean-k-mooneyi prefer kolla but juju and maas do work well together18:02
jamesbensonhad little luck with juju in the past, so trying to go completely openstack route now...18:02
*** powerd has joined #openstack-kolla18:03
jamesbensonbut previously we only had ipmi for deploying the OS, so it's a big improvement ;-)18:03
sean-k-mooneyi have used bifrost in the past im debating about using kolla with ironic at home i had kolla deployed at home untill last weekend when i rewired everything decided to start from scratch but havent decided what to deploy yet18:04
jamesbensononce kolla is up, we'll probably go back to testing juju, but need a new production up18:05
openstackgerritsean mooney proposed openstack/kolla master: enable the kolla build system to use out of tree docker files  https://review.openstack.org/46613218:07
*** powerd has quit IRC18:08
* inc0 is afk most of the time, conference18:10
*** salv-orlando has joined #openstack-kolla18:10
openstackgerritBernard Maltais proposed openstack/kolla-ansible master: Adding 2nd option to shutdown the mariadb containers.  https://review.openstack.org/46688118:10
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla-ansible master: Enable multinode gate  https://review.openstack.org/46600718:16
*** eaguilar has joined #openstack-kolla18:16
bmaltaisWhat are you guys using to edit .rst files? Is there a good editor out there?18:17
*** egonzalez has joined #openstack-kolla18:17
sean-k-mooneybmaltais: i just use nano then i render the docks after or have the gate do it if im feeling lazy and think its right18:18
sean-k-mooneybmaltais: rst are basically a subset of markdown so sublime text or notepad++ works ok but i have not found a good editor that would live render the html version or anything like that18:21
*** mgoddard has quit IRC18:21
rwellumbmaltais: emacs :) does .rst .md beautifully. Renders them, compiles them into html. Even handles the git commit to gerrit...18:21
sean-k-mooneyrwellum: really?18:21
sean-k-mooneyi would have to learn emacs though...18:21
*** chas has joined #openstack-kolla18:21
bmaltaisrwellum: Thanks, will look into it.18:21
*** rmart04 has joined #openstack-kolla18:21
* sean-k-mooney is conflicted with this news18:22
*** salv-orlando has quit IRC18:22
sean-k-mooneyrwellum: is there a way to use emacs without learning 50 billion keyboard shortcuts18:22
sean-k-mooneyi have come to understand that keyboard shortcuts and macros are kindof its thing18:23
rwellumsean-k-mooney: sure it does see: https://prnt.sc/fbnrgk18:25
*** chas has quit IRC18:26
rwellumsean-k-mooney: emacs has a package (called helm :) ) - that almost completely eliminates learning key strokes.18:26
sean-k-mooneyhelm as in k8s helm? i would totally not be suprised if you told me there was a kolla-k8s plugin for emacs that allowed it to be used as a ui for kolla18:27
bmaltaissean-k-mooney: Funny!18:28
rwellumsean-k-mooney: no it's not k8s-helm - just naming overload. But you're correct - I'm sure someone will product that at some point. There's a vagrant plugin for example.18:28
rwellum:)18:28
sean-k-mooneyemacs is one of those things that has plugins for everything but i like nice simple editor like nano that just you know edit documents. that or i will use a full ide like intelij idea18:28
egonzalezsean-k-mooney atom also render rst18:29
sean-k-mooneyegonzalez: ya though to be honest the raw rst is pretty readable. i read them on github far more often then i go to the docs site18:30
bmaltaisrwellum: Trying emac right now. What option do you use to render the .rst?18:30
rwellumsean-k-mooney: I believe emacs has a nano plugin, also has a vim-mode :)18:30
sean-k-mooneyrwellum: i have been trying to avoid trying it to stay out of the vim emacs wars hence one of the reason i use nano18:31
sean-k-mooneyits an unobtrsive light weight editor that tends not to offend anyone18:31
sean-k-mooneyegonzalez: does atom work in the terminal?18:32
rwellumbmaltais: I am not 100% sure. I have a package called markdown-mode installed, but either that is doing .rst natively or emacs does. Gimme a sec - I'll figure it out.18:32
sean-k-mooneyrwellum: but ya https://prnt.sc/fbnrgk is pretty nice18:32
*** manheim has joined #openstack-kolla18:32
egonzalezsean-k-mooney only  gui AFIK18:33
rwellumsean-k-mooney: then you do C-c c c - and it compiles into html, then you can open that html in emacs and view it. Any errors are displayed and will walk you to the points of failure in the file.18:33
sean-k-mooneyrwellum: and the macros/keyboard short cuts start :)18:35
sean-k-mooneyegonzalez: im going to call it a day am https://review.openstack.org/#/c/467360/ seams to be ok now if you think it make sense to pin to 10.0.30 as the mariadb guys suggested18:36
rwellumsean-k-mooney: well compile in emacs has been C-c c for 30+ years :) But yeah - you can use 'helm'. M-x compile... - it will complete the command for you.18:36
*** manheim has quit IRC18:37
rwellumbmaltais: looks like it's rst-mode - which is built into latest emacs - can you try M-x rst-mode - see if it works?18:37
rwellumsean-k-mooney: I recently added Wolfram-Alpha into emacs. Works great :)18:38
*** srwilkers has quit IRC18:39
openstackgerritTim Rozet proposed openstack/kolla master: [WIP] Adds opendaylight specific neutron-server container  https://review.openstack.org/46771218:40
bmaltaisrwellum: New to emacs... but pasting that in a scratch buffer does nothing... Using gnu emacs on windows... so possibly missing a plugin18:41
*** jascott1 has joined #openstack-kolla18:42
rwellumbmaltais: i'd have to see what 'nothing' is - reports an error or just does nothing. But emacs on windows I don't know well.18:42
bmaltaisrwellum: I think my issue is that I don't know where to type "M-x rst-mode" in emacs... starting from scratch with emacs. Should look at a youtube intro I guess ;-)18:44
*** jascott1 has quit IRC18:46
sean-k-mooneyinc0: im going to reviert the ovs-dpdk container into a seperate container instead of the template override so that it can be eailly used with https://review.openstack.org/#/c/459179/18:47
inc0sean-k-mooney: makes sense18:48
sean-k-mooneyinc0: the role could still be external or in contrib or seperate from neutron in main when i get back to it but a seperate image in kolla just seams cleanest to me18:50
rwellumbmaltais: M-x - is usually Esc-x18:50
inc0sean-k-mooney: I have different plan for ansible piece18:50
inc0I'll tell you about it after I'm back from OSTS:)18:51
bmaltaisxrest18:51
sean-k-mooneyinc0: well for ansible i think we have many options18:51
sean-k-mooneyinc0: but cool chat to you when your back. hope your having fun18:52
bmaltaisrwellum: It is installes. C-c c c failed since I am on windows and rst2html is not installed ;-)18:52
*** itlinux has joined #openstack-kolla18:52
rwellumOh right yeah - :)18:53
*** Guest41638 is now known as zeus18:57
*** zeus has quit IRC18:57
*** zeus has joined #openstack-kolla18:57
inc0ok, https://review.openstack.org/#/c/466007/69 is ready for reviews18:59
inc0kolla-ansible cores, please pay attention to this one, quite bit overhaul to our gates19:00
*** itlinux has quit IRC19:01
*** Pavo has quit IRC19:11
*** rmart04 has quit IRC19:13
vhosakotinc0: "bit" ;)19:15
vhosakotinc0: great great work!  will the new gate workflow be documented?19:15
*** itlinux has joined #openstack-kolla19:15
vhosakotinc0: I'm planning to add upgrade gates with duonghq_ after June 6th.19:16
vhosakotinc0: I'll be on vacation tomorrow until June 5th and will return on June 6th FYI.19:16
*** stefan_bo has joined #openstack-kolla19:17
*** stefan_bo has quit IRC19:22
*** itlinux has quit IRC19:23
vhosakotrwellum: haha, yeah.. I've seen and used vim in emacs... lol  I'm a vim guy19:24
*** mewald has joined #openstack-kolla19:25
*** manheim has joined #openstack-kolla19:25
rwellumvhosakot: search for 'modern emacs' on CEC - see what you get :)19:27
*** mewald1 has joined #openstack-kolla19:28
*** manheim has quit IRC19:30
*** lpetrut has joined #openstack-kolla19:31
*** mewald has quit IRC19:31
vhosakotrwellum: cool, found your doc.. very cool doc :)19:32
openstackgerritMerged openstack/kolla master: pin mariadb version for rpm distros  https://review.openstack.org/46736019:32
*** dasTor_ has joined #openstack-kolla19:33
openstackgerritEduardo Gonzalez proposed openstack/kolla master: Revert "pin mariadb version for rpm distros"  https://review.openstack.org/46772919:33
openstackgerritEduardo Gonzalez proposed openstack/kolla master: Revert "pin mariadb version for rpm distros"  https://review.openstack.org/46772919:33
*** dasTor has quit IRC19:35
*** rmart04 has joined #openstack-kolla19:35
*** rmart04 has quit IRC19:36
*** powerd has joined #openstack-kolla19:38
*** itlinux has joined #openstack-kolla19:39
*** manheim has joined #openstack-kolla19:39
*** itlinux has quit IRC19:39
*** spsurya has quit IRC19:40
rwellumvhosakot: :)19:40
hawiweird. if i say no central logging and set up external elasticsearch server, even then elastic is being deployed to controllers, why? or maybe i got it wrong19:41
*** eaguilar has quit IRC19:42
*** powerd has quit IRC19:42
*** manheim has quit IRC19:43
trozetsean-k-mooney: hi19:43
*** faridda has quit IRC19:48
*** openstackgerrit has quit IRC19:48
*** salv-orlando has joined #openstack-kolla19:49
*** aagate has joined #openstack-kolla19:49
*** manheim has joined #openstack-kolla19:54
*** lucasxu has quit IRC19:55
*** lucasxu has joined #openstack-kolla19:56
*** itlinux has joined #openstack-kolla19:57
*** manheim has quit IRC19:59
*** manheim has joined #openstack-kolla20:02
*** sgrasley has joined #openstack-kolla20:06
*** manheim has quit IRC20:06
egonzalezhawi what   are is globals.yml config? by default elastic is  only deployed when  central logging or freezer are enabled20:06
hawiaaa, I do have freezer20:08
hawithat explains it, thanks20:08
*** jrich523 has quit IRC20:12
*** jrich523 has joined #openstack-kolla20:12
*** sgrasley has quit IRC20:15
*** itlinux has quit IRC20:18
*** powerd has joined #openstack-kolla20:19
*** blallau has joined #openstack-kolla20:19
*** chas has joined #openstack-kolla20:22
*** powerd has quit IRC20:23
*** chas has quit IRC20:26
*** blallau has quit IRC20:26
*** openstackgerrit has joined #openstack-kolla20:28
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Support OSprofile usage  https://review.openstack.org/45562820:28
*** eaguilar has joined #openstack-kolla20:28
*** powerd has joined #openstack-kolla20:30
*** eaguilar has quit IRC20:36
*** manheim has joined #openstack-kolla20:40
*** manheim has quit IRC20:45
*** itlinux has joined #openstack-kolla20:47
*** manheim has joined #openstack-kolla20:47
*** salv-orlando has quit IRC20:49
*** lucasxu has quit IRC20:49
jamesbensonsean-k-mooney: Still there?20:51
*** manheim has quit IRC20:51
*** lucasxu has joined #openstack-kolla20:53
*** manheim has joined #openstack-kolla20:53
*** itlinux has quit IRC20:57
*** manheim has quit IRC20:57
openstackgerritTim Rozet proposed openstack/kolla master: Adds opendaylight specific neutron-server container  https://review.openstack.org/46771220:58
openstackgerritMerged openstack/kolla-ansible master: Fix haproxy restart while genconfig  https://review.openstack.org/46304020:58
*** sayantan_ has joined #openstack-kolla20:58
*** krtaylor has quit IRC20:59
openstackgerritAndrew Bruce proposed openstack/kolla-kubernetes master: Remove kolla-ansible dependencies  https://review.openstack.org/46776720:59
openstackgerritAndrew Bruce proposed openstack/kolla-kubernetes master: openvswitch not required to be kernel module  https://review.openstack.org/46776921:05
trozetinc0: hi21:06
*** gfidente has joined #openstack-kolla21:06
openstackgerritAndrew Bruce proposed openstack/kolla-kubernetes master: permit rabbitmq-mgmt to select rabbitmq as endpoint  https://review.openstack.org/46777121:07
*** glycerine has quit IRC21:07
*** fguillot has quit IRC21:11
*** lucasxu has quit IRC21:17
*** salv-orlando has joined #openstack-kolla21:19
*** dave-mccowan has quit IRC21:20
*** srwilkers has joined #openstack-kolla21:20
inc0hey trozet21:23
inc0I'm semi-afk21:23
inc0but what's up?21:24
*** sbezverk has left #openstack-kolla21:28
*** manheim has joined #openstack-kolla21:29
*** salv-orlando has quit IRC21:30
*** jtriley has quit IRC21:30
*** sbezverk has joined #openstack-kolla21:31
*** manheim has quit IRC21:34
*** manheim has joined #openstack-kolla21:36
*** lpetrut has quit IRC21:40
*** manheim has quit IRC21:41
*** mewald1 has quit IRC21:41
*** ducttape_ has quit IRC21:42
*** ducttape_ has joined #openstack-kolla21:45
openstackgerritAndrew Bruce proposed openstack/kolla-kubernetes master: Remove kolla-ansible dependencies  https://review.openstack.org/46776721:45
*** manheim has joined #openstack-kolla21:47
*** sbezverk_ has joined #openstack-kolla21:49
*** sbezverk has quit IRC21:49
*** manheim has quit IRC21:51
*** powerd has quit IRC21:52
*** jamesbenson has quit IRC21:53
*** imcsk8_ has quit IRC21:54
*** ducttape_ has quit IRC21:56
*** ducttape_ has joined #openstack-kolla21:57
openstackgerritAndrew Bruce proposed openstack/kolla-kubernetes master: 3-second curl call to validate rabbitmq-mgmt  https://review.openstack.org/46778922:01
*** salv-orlando has joined #openstack-kolla22:01
*** imcsk8 has joined #openstack-kolla22:02
*** vhosakot has quit IRC22:06
*** salv-orl_ has joined #openstack-kolla22:07
*** salv-orlando has quit IRC22:11
*** rwsu has quit IRC22:15
*** rhallisey has quit IRC22:18
*** salv-orl_ has quit IRC22:19
*** manheim has joined #openstack-kolla22:20
*** chas has joined #openstack-kolla22:23
*** manheim has quit IRC22:25
*** manheim has joined #openstack-kolla22:25
*** eanylin has quit IRC22:26
*** chas has quit IRC22:27
*** eaguilar has joined #openstack-kolla22:28
*** huzhengchuan has joined #openstack-kolla22:35
*** sbezverk_ has quit IRC22:35
*** ducttape_ has quit IRC22:39
*** huzhengchuan has quit IRC22:42
openstackgerritBernard Maltais proposed openstack/kolla-ansible master: Added more details regarding next step after running an option proposed.  https://review.openstack.org/46688122:42
*** itlinux has joined #openstack-kolla22:46
*** manheim has quit IRC22:48
*** krtaylor has joined #openstack-kolla22:50
*** rwellum has quit IRC22:57
*** manheim has joined #openstack-kolla22:58
*** eaguilar has quit IRC23:02
*** manheim has quit IRC23:02
*** egonzalez has quit IRC23:04
*** powerd has joined #openstack-kolla23:06
*** manheim has joined #openstack-kolla23:11
*** itlinux has quit IRC23:11
*** manheim has quit IRC23:15
*** sbezverk has joined #openstack-kolla23:16
*** huzhengchuan_ has joined #openstack-kolla23:16
*** eaguilar has joined #openstack-kolla23:19
*** stefan_bo has joined #openstack-kolla23:19
*** eaguilar has quit IRC23:20
*** eaguilar has joined #openstack-kolla23:20
*** manheim has joined #openstack-kolla23:21
*** huzhengchuan_ has quit IRC23:23
*** stefan_bo has quit IRC23:23
*** yuanying has quit IRC23:25
*** manheim has quit IRC23:26
*** eaguilar has quit IRC23:27
*** manheim has joined #openstack-kolla23:36
*** huzhengchuan_ has joined #openstack-kolla23:36
*** manheim has quit IRC23:40
*** huzhengchuan has joined #openstack-kolla23:40
*** salv-orlando has joined #openstack-kolla23:41
*** sbezverk has quit IRC23:46
*** manheim has joined #openstack-kolla23:47
*** powerd has quit IRC23:51
*** manheim has quit IRC23:52
*** salv-orlando has quit IRC23:54

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!