*** wolverineav has quit IRC | 00:02 | |
*** Sravan has joined #openstack-kolla | 00:04 | |
*** Sravan has quit IRC | 00:17 | |
*** Sravan has joined #openstack-kolla | 00:24 | |
*** Sravan has quit IRC | 00:27 | |
*** wolverineav has joined #openstack-kolla | 00:28 | |
*** wolverineav has quit IRC | 00:35 | |
*** Sravan has joined #openstack-kolla | 00:37 | |
*** andrein has quit IRC | 00:38 | |
*** Sravan has quit IRC | 00:41 | |
*** iniazi has quit IRC | 00:52 | |
*** iniazi has joined #openstack-kolla | 00:54 | |
*** faridda has joined #openstack-kolla | 01:02 | |
*** tolisbar has quit IRC | 01:04 | |
*** tolisbar has joined #openstack-kolla | 01:16 | |
*** faridda has quit IRC | 01:23 | |
*** faridda has joined #openstack-kolla | 01:27 | |
*** Sravan has joined #openstack-kolla | 01:48 | |
*** Sravan has quit IRC | 01:52 | |
openstackgerrit | Jon Schlueter proposed openstack/kolla master: Drop redundant rpms from nova-base package list https://review.openstack.org/643214 | 02:01 |
---|---|---|
openstackgerrit | Jon Schlueter proposed openstack/kolla master: Drop redundant rpms from nova-base package list https://review.openstack.org/643214 | 02:01 |
*** tolisbar has quit IRC | 02:07 | |
openstackgerrit | Taeha Kim proposed openstack/kolla master: Add RHEL subscription registration https://review.openstack.org/495148 | 02:08 |
*** wolverineav has joined #openstack-kolla | 02:10 | |
*** hongbin has joined #openstack-kolla | 02:15 | |
*** wolverineav has quit IRC | 02:15 | |
*** faridda has quit IRC | 02:27 | |
*** faridda has joined #openstack-kolla | 02:31 | |
*** tolisbar has joined #openstack-kolla | 02:50 | |
*** dave-mccowan has joined #openstack-kolla | 02:50 | |
*** dave-mccowan has quit IRC | 02:54 | |
*** faridda has quit IRC | 03:13 | |
*** faridda has joined #openstack-kolla | 03:13 | |
*** skramaja has joined #openstack-kolla | 03:33 | |
*** hongbin has quit IRC | 03:36 | |
*** faridda has quit IRC | 03:39 | |
*** faridda has joined #openstack-kolla | 03:41 | |
*** faridda has quit IRC | 03:44 | |
*** faridda has joined #openstack-kolla | 03:45 | |
*** faridda has quit IRC | 03:47 | |
*** Sravan has joined #openstack-kolla | 03:50 | |
*** zhangfei has joined #openstack-kolla | 04:20 | |
*** janki has joined #openstack-kolla | 04:35 | |
*** iniazi has quit IRC | 04:52 | |
*** Sravan has quit IRC | 04:53 | |
*** Sravan has joined #openstack-kolla | 04:55 | |
*** Sravan has quit IRC | 05:06 | |
*** iniazi has joined #openstack-kolla | 05:07 | |
*** Sravan has joined #openstack-kolla | 05:07 | |
*** Sravan has quit IRC | 05:37 | |
*** jbadiapa has quit IRC | 06:03 | |
*** jbadiapa has joined #openstack-kolla | 06:04 | |
*** wolverineav has joined #openstack-kolla | 06:18 | |
*** ivve has joined #openstack-kolla | 06:37 | |
*** wolverin_ has joined #openstack-kolla | 06:42 | |
*** shardy has quit IRC | 06:44 | |
*** wolverineav has quit IRC | 06:45 | |
*** Luzi has joined #openstack-kolla | 06:46 | |
*** wolverin_ has quit IRC | 06:49 | |
*** shardy has joined #openstack-kolla | 06:52 | |
*** rgogunskiy has joined #openstack-kolla | 06:57 | |
*** iniazi has quit IRC | 07:00 | |
*** hamdyk has joined #openstack-kolla | 07:16 | |
*** iniazi has joined #openstack-kolla | 07:16 | |
*** wolverineav has joined #openstack-kolla | 07:27 | |
*** wolverineav has quit IRC | 07:31 | |
*** gkadam_ has joined #openstack-kolla | 07:50 | |
*** luksky has joined #openstack-kolla | 07:54 | |
*** hamzaachi has joined #openstack-kolla | 07:55 | |
*** hamzaachi has quit IRC | 07:56 | |
*** hamzaachi has joined #openstack-kolla | 07:57 | |
egonzalez | hi, can we get this merged? spsurya maybe? https://review.openstack.org/#/c/642984/ | 07:59 |
spsurya | egonzalez: sure | 08:00 |
*** hamzaachi has quit IRC | 08:02 | |
*** tolisbar has quit IRC | 08:06 | |
*** pcaruana has joined #openstack-kolla | 08:07 | |
*** pcaruana has quit IRC | 08:11 | |
*** tolisbar has joined #openstack-kolla | 08:13 | |
mgoddard | klindgren_: have you optimised ansible? e.g. increased the number of forks, enable pipelining, etc? | 08:17 |
*** iniazi has quit IRC | 08:19 | |
mgoddard | klindgren_: tried running kolla-ansible on a decent bare metal host? | 08:23 |
*** pcaruana has joined #openstack-kolla | 08:23 | |
openstackgerrit | caoyuan proposed openstack/kolla-ansible master: Add ceilometer_ipmi container into ceilometer role https://review.openstack.org/565057 | 08:24 |
*** hamzaachi has joined #openstack-kolla | 08:24 | |
*** hamzaachi has quit IRC | 08:25 | |
*** hamzaachi has joined #openstack-kolla | 08:26 | |
*** tolisbar1 has joined #openstack-kolla | 08:26 | |
*** tolisbar has quit IRC | 08:28 | |
*** tolisbar1 is now known as tolisbar | 08:28 | |
*** hjensas has quit IRC | 08:35 | |
*** iniazi has joined #openstack-kolla | 08:40 | |
*** tolisbar has quit IRC | 08:41 | |
mgoddard | seems the lower-constraints job is having some issues. Can't reproduce locally though | 08:56 |
*** dougsz has joined #openstack-kolla | 08:59 | |
*** k_mouza has joined #openstack-kolla | 09:17 | |
openstackgerrit | Merged openstack/kolla-ansible master: Remove redundant copying of Fluentd input files https://review.openstack.org/641612 | 09:19 |
mandre | jschlueter: I did something similar some time ago and hooked it up with CI https://review.openstack.org/#/c/580993/ | 09:21 |
mandre | jschlueter: see the output with a recent job http://logs.openstack.org/60/642460/1/check/kolla-build-centos-binary/cd315fb/logs/packages-info.txt.gz | 09:21 |
mandre | jschlueter: all the oslo stuff is definitely good candidate to move to the base openstack image | 09:22 |
*** sshnaidm is now known as sshnaidm|rover | 09:25 | |
openstackgerrit | Merged openstack/kolla-ansible master: Support customising Fluentd formatting https://review.openstack.org/641613 | 09:26 |
mandre | jschlueter: at the time I played a bit with the idea of an httpd-base image to stop installing httpd and the (big) dependencies in each container needing it but never finished this work | 09:27 |
mandre | jschlueter: https://review.openstack.org/#/c/583361/ if you want to take a look | 09:27 |
*** andrein has joined #openstack-kolla | 09:27 | |
mandre | the main issue is that we'll end up with http in some of the containers that don't need it because of how we layer things | 09:28 |
*** tolisbar has joined #openstack-kolla | 09:37 | |
hrw | morning | 09:40 |
hrw | https://review.openstack.org/#/c/612681/ anyone? I am tired of 'git review -x 612681' each time I work on kolla patches | 09:41 |
mandre | hrw: looking | 09:41 |
hrw | thx mandre | 09:41 |
mandre | and good morning :) | 09:42 |
*** iniazi has quit IRC | 09:44 | |
*** pcaruana has quit IRC | 09:48 | |
*** iniazi has joined #openstack-kolla | 09:50 | |
mandre | hrw: left a couple of comments | 09:51 |
hrw | thx | 09:52 |
hrw | mandre: good catch with rabbitmq | 09:52 |
hrw | thx | 09:52 |
mandre | hrw: oh and release notes | 09:53 |
hrw | hm. have to dig them as wrote them... | 09:54 |
*** hjensas has joined #openstack-kolla | 09:57 | |
openstackgerrit | Marcin Juszkiewicz proposed openstack/kolla master: Move to Debian 'buster' https://review.openstack.org/612681 | 10:23 |
hrw | mandre: addressed comments | 10:24 |
*** serhatd has joined #openstack-kolla | 10:25 | |
jschlueter | mandre: ack and thanks, I also found that targetcli at least downstream OSP is currently pulling in libX11 ... | 10:27 |
hrw | http://logs.openstack.org/81/612681/9/check/openstack-tox-lower-constraints/c136850/job-output.txt.gz - wth it is? | 10:38 |
mgoddard | hrw: yeah, I'm looking into it. Seems to be due to infra switching images to bionic | 10:44 |
*** wolverineav has joined #openstack-kolla | 10:49 | |
*** k_mouza has quit IRC | 10:52 | |
*** k_mouza has joined #openstack-kolla | 10:53 | |
*** iniazi has quit IRC | 10:53 | |
*** wolverineav has quit IRC | 10:54 | |
*** iniazi has joined #openstack-kolla | 10:54 | |
hrw | mgoddard: thanks | 10:55 |
hrw | mgoddard: https://review.openstack.org/#/c/612681/ waits for you to give +2 again ;d | 10:56 |
mgoddard | hrw: done | 10:59 |
serhatd | Hello | 11:02 |
serhatd | I am trying to deploy multi-master k8s cluster with magnum but it did not succeed. single master deployments works well. | 11:02 |
serhatd | I guess something wrong related to etcd. "--etcd-servers" parameter for kubernetes-apiserver points to http://127.0.0.1:2379. Should not it be load balancer's IP instead of localhost? | 11:02 |
serhatd | openstack release is queens | 11:03 |
serhatd | Has anyone faced this issue before? | 11:03 |
mnasiadka | serhatd: probably it would be better to ask on #openstack-containers, or investigate which configuration variable is responsible for etcd-servers pointing to 127.0.0.1 ;) | 11:05 |
hrw | mgoddard: thanks | 11:09 |
hrw | zuul failures anyway | 11:10 |
*** shardy has quit IRC | 11:11 | |
*** zbr has quit IRC | 11:16 | |
*** zhangfei has quit IRC | 11:18 | |
openstackgerrit | Surya Prakash (spsurya) proposed openstack/kolla-ansible master: Update programming language mentioned in setup.cfg https://review.openstack.org/643297 | 11:18 |
openstackgerrit | Surya Prakash (spsurya) proposed openstack/kolla master: Update programming language mentioned in setup.cfg https://review.openstack.org/643298 | 11:18 |
*** henriqueof has joined #openstack-kolla | 11:25 | |
*** gfidente has joined #openstack-kolla | 11:30 | |
*** rgogunskiy has quit IRC | 11:31 | |
*** pcaruana has joined #openstack-kolla | 11:32 | |
*** rgogunskiy has joined #openstack-kolla | 11:33 | |
*** Sravan has joined #openstack-kolla | 11:40 | |
*** Sravan has quit IRC | 11:45 | |
*** dave-mccowan has joined #openstack-kolla | 11:45 | |
*** luksky has quit IRC | 11:50 | |
*** rgogunskiy has quit IRC | 11:55 | |
*** rgogunskiy has joined #openstack-kolla | 11:56 | |
*** tolisbar has quit IRC | 11:57 | |
*** skramaja_ has joined #openstack-kolla | 11:57 | |
*** skramaja has quit IRC | 11:58 | |
*** zbr has joined #openstack-kolla | 11:58 | |
openstackgerrit | Mark Goddard proposed openstack/kolla master: Bump setuptools requirement to 21.0.0 https://review.openstack.org/643308 | 12:09 |
mgoddard | hrw: mandre: spsurya: ^^^ should fix the lower-constraints job failures | 12:09 |
hrw | thx | 12:10 |
mgoddard | it was a pain to diagnose that | 12:10 |
hrw | mgoddard: thanks for doing it | 12:10 |
hrw | debugging is painful thing | 12:10 |
*** skramaja has joined #openstack-kolla | 12:11 | |
*** skramaja_ has quit IRC | 12:11 | |
hrw | I have one thing to debug on my list for over a year. still keep it below 'let make coffee' or 'let play civ vi' for as long as possible | 12:11 |
*** luksky has joined #openstack-kolla | 12:18 | |
spsurya | mgoddard: reviewed | 12:23 |
mgoddard | spsurya: thanks | 12:23 |
openstackgerrit | Marcin Juszkiewicz proposed openstack/kolla master: Bump setuptools requirement to 21.0.0 https://review.openstack.org/643308 | 12:38 |
hrw | dropped .0 | 12:39 |
hrw | hm. commit message should also drop .0 | 12:39 |
*** hamdykhader has joined #openstack-kolla | 12:49 | |
*** faridda has joined #openstack-kolla | 12:50 | |
*** hamdyk has quit IRC | 12:53 | |
*** JangwonLee_ has quit IRC | 12:55 | |
*** JangwonLee_ has joined #openstack-kolla | 12:56 | |
*** ktibi has joined #openstack-kolla | 12:58 | |
*** jchhatbar has joined #openstack-kolla | 13:00 | |
*** janki has quit IRC | 13:00 | |
*** iniazi has quit IRC | 13:00 | |
*** skramaja has quit IRC | 13:01 | |
*** ktibi has quit IRC | 13:07 | |
*** ktibi has joined #openstack-kolla | 13:07 | |
*** jchhatbar has quit IRC | 13:10 | |
*** janki has joined #openstack-kolla | 13:10 | |
*** andrein has quit IRC | 13:11 | |
*** iniazi has joined #openstack-kolla | 13:12 | |
*** ktibi has quit IRC | 13:30 | |
*** rgogunskiy has quit IRC | 13:32 | |
*** faridda has quit IRC | 13:37 | |
*** mjturek has joined #openstack-kolla | 13:37 | |
*** mjturek has quit IRC | 13:38 | |
*** ktibi has joined #openstack-kolla | 13:42 | |
*** JangwonLee_ has quit IRC | 13:43 | |
*** mjturek has joined #openstack-kolla | 13:46 | |
*** iclon has joined #openstack-kolla | 13:49 | |
*** faridda has joined #openstack-kolla | 13:56 | |
*** janki has quit IRC | 13:58 | |
*** faridda has quit IRC | 14:00 | |
openstackgerrit | Scott Solkhon proposed openstack/kolla-ansible master: Support separate Swift storage networks https://review.openstack.org/642758 | 14:00 |
*** serhatd has quit IRC | 14:02 | |
openstackgerrit | Pierre Riteau proposed openstack/kolla-ansible stable/queens: Create /volumes when using external ceph https://review.openstack.org/643087 | 14:03 |
openstackgerrit | Pierre Riteau proposed openstack/kolla-ansible stable/queens: Fix failure to mount cephfs https://review.openstack.org/643088 | 14:03 |
*** faridda has joined #openstack-kolla | 14:05 | |
*** hjensas has quit IRC | 14:06 | |
*** faridda has quit IRC | 14:07 | |
*** andrein has joined #openstack-kolla | 14:14 | |
*** faridda has joined #openstack-kolla | 14:18 | |
*** faridda has quit IRC | 14:19 | |
*** hjensas has joined #openstack-kolla | 14:20 | |
openstackgerrit | Merged openstack/kolla-ansible master: Update programming language mentioned in setup.cfg https://review.openstack.org/643297 | 14:21 |
openstackgerrit | Merged openstack/kolla-ansible master: ubuntu: update configuration Stein UCA https://review.openstack.org/627954 | 14:30 |
mgoddard | seems we have another CI issue - ubuntu binary horizon & nova-placement-api images failing to build | 14:36 |
*** gfidente has quit IRC | 14:39 | |
*** dougsz has quit IRC | 14:45 | |
*** faridda has joined #openstack-kolla | 14:48 | |
*** wolverineav has joined #openstack-kolla | 14:50 | |
*** faridda has quit IRC | 14:51 | |
*** wolverineav has quit IRC | 14:54 | |
*** gfidente has joined #openstack-kolla | 14:55 | |
*** faridda has joined #openstack-kolla | 14:55 | |
hrw | ~curse devs using single core cpus... | 14:57 |
hrw | "/usr/bin/python2 /usr/bin/pycompile -p python-pymysql" is taking 100% of single cpu | 14:57 |
hrw | @$@#$!! | 14:58 |
*** faridda has quit IRC | 14:58 | |
mnasiadka | mgoddard: well, I guess for the next couple of weeks every day will have some surprises :) | 15:05 |
mgoddard | mnasiadka: seems so. So much for stability near a release. Instead everyone merges their changes and the world breaks :) | 15:06 |
mnasiadka | mgoddard: yes, that's why OpenStack is usable ~2 months after the release and everybody releases .0.1 (instead of GA) :) | 15:07 |
mgoddard | I think I have the horizon issue. Not sure how it crept in. Seems we're currently running python5 - a weird mix of 2 and 3 | 15:07 |
*** hamdykhader has quit IRC | 15:09 | |
mnasiadka | python5... | 15:09 |
*** hjensas has quit IRC | 15:12 | |
*** iniazi has quit IRC | 15:17 | |
openstackgerrit | Mark Goddard proposed openstack/kolla master: Fix lower-constraints and kolla-build-ubuntu-binary jobs https://review.openstack.org/643308 | 15:23 |
*** iniazi has joined #openstack-kolla | 15:28 | |
openstackgerrit | Mark Goddard proposed openstack/kolla master: Add some irrelevant files to build jobs https://review.openstack.org/643368 | 15:29 |
*** dougsz has joined #openstack-kolla | 15:31 | |
erolg | please review https://review.openstack.org/#/c/642749/ | 15:32 |
openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: Add lower-constraints.txt to irrelevant-files https://review.openstack.org/643369 | 15:32 |
hrw | at fosdem there was a talk about python5 or perl11 - forgot details | 15:35 |
* hrw off | 15:41 | |
*** hamzy has quit IRC | 15:43 | |
openstackgerrit | Luigi Toscano proposed openstack/kolla master: Sahara: install also the split plugins https://review.openstack.org/643378 | 15:55 |
openstackgerrit | Taeha Kim proposed openstack/kolla master: WIP: Fedora support for python3 package testing https://review.openstack.org/624838 | 15:58 |
openstackgerrit | Taeha Kim proposed openstack/kolla master: Enable support for fedora distro in kolla https://review.openstack.org/635512 | 16:00 |
*** Luzi has quit IRC | 16:03 | |
openstackgerrit | Merged openstack/kolla-ansible master: Support separate Swift storage networks https://review.openstack.org/642758 | 16:19 |
*** gfidente has quit IRC | 16:22 | |
*** ivve has quit IRC | 16:25 | |
*** hamzaachi has quit IRC | 16:25 | |
*** hamzaachi has joined #openstack-kolla | 16:27 | |
*** hamzaachi_ has joined #openstack-kolla | 16:30 | |
*** hamzaachi has quit IRC | 16:32 | |
*** luksky has quit IRC | 16:41 | |
*** gfidente has joined #openstack-kolla | 16:44 | |
*** gkadam_ has quit IRC | 16:50 | |
*** ktibi has quit IRC | 16:53 | |
*** wolverineav has joined #openstack-kolla | 16:59 | |
*** wolverineav has quit IRC | 17:02 | |
*** wolverin_ has joined #openstack-kolla | 17:02 | |
*** sshnaidm|rover is now known as sshnaidm|afk | 17:02 | |
*** wolverin_ has quit IRC | 17:02 | |
*** wolverineav has joined #openstack-kolla | 17:02 | |
hrw | wth... my devel aarch64 machine does 3 distros binary/source in 3h. my desktop not finished openstack-base in 3h | 17:04 |
mgoddard | hrw: network connection? | 17:05 |
hrw | no, build time | 17:05 |
mgoddard | hrw: yeah, but do they have different internet links | 17:06 |
mgoddard | ? | 17:07 |
hrw | mgoddard: 250/20 Mbps was enough always | 17:07 |
hrw | mgoddard: the bottleneck is pycompile this time | 17:07 |
*** hamzy has joined #openstack-kolla | 17:09 | |
mgoddard | now we're seeing this in the ansible ubuntu source job: "Nova requires QEMU version 2.8.0 or greater." | 17:13 |
mgoddard | spsurya: we're having CI issues, please hold off rechecking | 17:14 |
spsurya | mgoddard: noted | 17:15 |
mgoddard | spsurya: affecting kolla, not kolla-ansible (yet...) | 17:15 |
*** Sravan has joined #openstack-kolla | 17:19 | |
*** Sravan has quit IRC | 17:20 | |
*** Sravan has joined #openstack-kolla | 17:20 | |
hrw | mgoddard: qemu??? which job? | 17:21 |
mgoddard | hrw: http://logs.openstack.org/08/643308/3/check/kolla-ansible-ubuntu-source/70f151a/primary/logs/kolla/nova/nova-compute.txt.gz#_2019-03-14_16_00_26_458 | 17:22 |
*** Sravan has quit IRC | 17:22 | |
hrw | thx | 17:22 |
hrw | mgoddard: looks like another issue rather than version of qemu | 17:23 |
hrw | "Cannot update service status on host "primary" since it is not registered.: ComputeHostNotFound_Remote: Compute host primary could not be found." | 17:24 |
mgoddard | hrw: http://logs.openstack.org/08/643308/3/check/kolla-ansible-ubuntu-source/70f151a/primary/logs/kolla/libvirt/libvirtd.txt.gz | 17:24 |
mgoddard | I think we removed qemu somewhere along the way :/ | 17:24 |
mgoddard | hrw: yeah, no qemu in nova-libvirt anymore | 17:25 |
hrw | qemu-system-common should be there | 17:26 |
*** Sravan has joined #openstack-kolla | 17:26 | |
hrw | nova-libvirt.log:INFO:kolla.common.utils.nova-libvirt:Step 4/7 : RUN apt-get update && apt-get -y install --no-install-recommends ceph-common ebtables libvirt-clients libvirt-daemon-system openvswitch-switch pm-utils python3-cephfs python3-rados python3-rbd qemu qemu-block-extra qemu-efi dmidecode xen-utils && apt-get clean && rm -rf /var/lib/apt/lists/* | 17:28 |
hrw | recent ubuntu-binary-nova-libvirt has whole set of qemu packages | 17:29 |
hrw | but I do not have x86 images handy | 17:30 |
mgoddard | hrw: it definitely has something, but not on the path | 17:32 |
mgoddard | dpkg-query -L qemu-system-x86 | 17:33 |
mgoddard | dpkg-query: package 'qemu-system-x86' is not installed | 17:33 |
mgoddard | Use dpkg --info (= dpkg-deb --info) to examine archive files, | 17:33 |
mgoddard | and dpkg --contents (= dpkg-deb --contents) to list their contents. | 17:33 |
mgoddard | maybe deps have changed to no longer automatically pull in the x86 emulator | 17:33 |
hrw | mgoddard: dpkg -l|grep qemu | 17:34 |
mgoddard | hrw: thanks - I'm a deb noob :) | 17:35 |
hrw | mgoddard: more in rpm land? | 17:35 |
mgoddard | hrw: what's rpm? | 17:35 |
mgoddard | hrw: just kidding :) | 17:35 |
hrw | I once called it "Really Poorly Maintained" - https://marcin.juszkiewicz.com.pl/2006/10/17/rpm-is-really-poorly-maintained/ | 17:36 |
mgoddard | 16 qemu packages in images on dockerhub, 2 in a freshly built image | 17:36 |
hrw | mgoddard: do you have qemu-system-common there? | 17:37 |
mgoddard | hrw: nope, qemu, qemu-block-extra | 17:37 |
hrw | hm. have to check then | 17:37 |
hrw | 18:37 (1s) hrw@puchatek:~$ docker run -it -u root kolla/ubuntu-binary-nova-libvirt:master bash | 17:38 |
hrw | let's see in official image | 17:38 |
mgoddard | it's in there | 17:39 |
hrw | bingo! | 17:39 |
hrw | i A libvirt-daemon Recommends qemu-kvm | qemu (>= 0.9.1) | 17:39 |
hrw | just recommends... | 17:39 |
hrw | and we do not install recommends | 17:40 |
hrw | let me make a patch which will install qemu in nova-libvirt | 17:40 |
mgoddard | hrw: it needs to be added to https://review.openstack.org/#/c/643308/ | 17:41 |
hrw | k | 17:41 |
*** k_mouza has quit IRC | 17:41 | |
mgoddard | hrw: we already include qemu in that image | 17:41 |
hrw | hm. yep | 17:41 |
mgoddard | should we add qemu-kvm? | 17:42 |
hrw | there is no such anymore | 17:42 |
hrw | qemu-kvm is virtual | 17:42 |
mgoddard | I think we need to list the emulator packages we want | 17:43 |
hrw | no | 17:43 |
hrw | qemu -> qemu-system -> qemu-system-* -> qemu-system-common | 17:43 |
hrw | dependencies chain | 17:43 |
*** priteau has joined #openstack-kolla | 17:45 | |
*** gfidente is now known as gfidente|afk | 17:46 | |
mgoddard | hrw: nope: http://paste.openstack.org/show/747807/ | 17:46 |
hrw | hm. I fetched master tag.. | 17:47 |
klindgren_ | mgoddard, yes we have pipelining enabled, we are running 40 forks, we even tried running mitogen for the deployment. We tried running in "free" strategy, but that fails pretty horribly with kolla_ansible. Right now when we deploy it seems like a single ansible thread is taking up 100% cpu, seems like this is happening when its processing "skips", which there is a ton of them. Do you have any additional ideas? I know I am not the only one hav | 17:48 |
klindgren_ | ing deployment time issues, I saw someone else post about multiple hour long deployments. | 17:48 |
hrw | 18:47 (1s) hrw@puchatek:~$ docker pull kolla/ubuntu-binary-nova-libvirt:8.0.0 | 17:48 |
hrw | Error response from daemon: manifest for kolla/ubuntu-binary-nova-libvirt:8.0.0 not found | 17:48 |
mgoddard | hrw: that's the tag when building locally | 17:48 |
mgoddard | image on dockerhub has 16 qemu packages | 17:49 |
hrw | yep. just built on other machine | 17:49 |
hrw | fsck | 17:49 |
mgoddard | klindgren_: that's worrying to hear. What kind of machine are you running on? | 17:49 |
hrw | Description: fast processor emulator, dummy package | 17:49 |
hrw | FUCKING ubuntu so-called-developers... | 17:49 |
*** ktibi has joined #openstack-kolla | 17:50 | |
klindgren_ | Its running in a container on a 2x hex core 3ghz server with 128gb ram. | 17:52 |
mgoddard | klindgren_: you'd hope that would be enough... | 17:52 |
klindgren_ | With mitogen, we can probably increase forks, but it defaults to 32 forks, as ram usage was significantly lower under mitogen | 17:52 |
mgoddard | klindgren_: tried with host networking in the container? | 17:53 |
mgoddard | klindgren_: I think we need to try to find some very simple representative playbooks that show this problem | 17:54 |
klindgren_ | it's running with host networking in privileged mode | 17:54 |
hrw | so now we need to install: qemu-system + qemu-kvm (on ubuntu) | 17:54 |
*** ktibi has quit IRC | 17:55 | |
mgoddard | klindgren_: some benchmarks of simple tasks running in ansible, then add kolla-ansible-like things | 17:55 |
hrw | and debian too | 17:55 |
hrw | #@$@%@ | 17:55 |
hrw | mgoddard: will edit patch | 17:55 |
mgoddard | hrw: just tried, that brings in lots | 17:56 |
hrw | good | 17:58 |
mgoddard | hrw: did a diff. we now have extra packages: qemu-data, qemu-gui, and one missing: qemu-user | 17:58 |
hrw | qemu-gui? | 17:59 |
hrw | weird | 18:00 |
mgoddard | yeah | 18:00 |
mgoddard | sorry, qemu-system-gui | 18:00 |
hrw | no such here | 18:01 |
hrw | ipxe-qemu ipxe-qemu-256k-compat-efi-roms qemu qemu-block-extra:amd64 qemu-kvm qemu-slof qemu-system qemu-system-arm qemu-system-common qemu-system-data qemu-system-mips qemu-system-misc qemu-system-ppc qemu-system-s390x qemu-system-sparc qemu-system-x86 | 18:02 |
hrw | mgoddard: qemu-system-arm recommends qemu-system-gui. and we do not install recommends | 18:05 |
*** k_mouza has joined #openstack-kolla | 18:08 | |
*** k_mouza has quit IRC | 18:13 | |
*** wolverineav has quit IRC | 18:16 | |
*** hamzaachi_ has quit IRC | 18:16 | |
hrw | mgoddard: do you have updated patch? or should I send? | 18:16 |
*** ivve has joined #openstack-kolla | 18:17 | |
*** wolverineav has joined #openstack-kolla | 18:19 | |
openstackgerrit | Marcin Juszkiewicz proposed openstack/kolla master: Fix lower-constraints and kolla-build-ubuntu-binary jobs https://review.openstack.org/643308 | 18:20 |
hrw | let it burn | 18:20 |
*** wolverineav has quit IRC | 18:21 | |
*** wolverineav has joined #openstack-kolla | 18:22 | |
*** jonaspaulo has joined #openstack-kolla | 18:23 | |
*** faridda has joined #openstack-kolla | 18:24 | |
* hrw off | 18:27 | |
*** wolverineav has quit IRC | 18:28 | |
*** wolverineav has joined #openstack-kolla | 18:29 | |
*** gmann is now known as gmann_afk | 18:51 | |
*** faridda has quit IRC | 18:56 | |
*** priteau has quit IRC | 18:56 | |
*** faridda has joined #openstack-kolla | 18:58 | |
*** faridda has quit IRC | 19:07 | |
*** faridda has joined #openstack-kolla | 19:07 | |
*** wolverineav has quit IRC | 19:07 | |
*** priteau has joined #openstack-kolla | 19:16 | |
*** priteau has quit IRC | 19:21 | |
*** dougsz has quit IRC | 19:27 | |
*** iniazi has quit IRC | 19:29 | |
*** faridda has quit IRC | 19:31 | |
*** faridda has joined #openstack-kolla | 19:34 | |
*** absubram has joined #openstack-kolla | 19:35 | |
*** faridda has quit IRC | 19:36 | |
*** wolverineav has joined #openstack-kolla | 19:42 | |
*** luksky has joined #openstack-kolla | 19:43 | |
*** wolverineav has quit IRC | 19:43 | |
*** wolverineav has joined #openstack-kolla | 19:44 | |
*** faridda has joined #openstack-kolla | 19:45 | |
*** Sravan has quit IRC | 19:47 | |
*** faridda has quit IRC | 19:47 | |
*** wolverineav has quit IRC | 19:47 | |
*** wolverineav has joined #openstack-kolla | 19:51 | |
*** wolverineav has quit IRC | 19:52 | |
*** wolverineav has joined #openstack-kolla | 19:53 | |
*** gfidente|afk is now known as gfidente | 19:57 | |
*** faridda has joined #openstack-kolla | 20:08 | |
*** faridda has quit IRC | 20:10 | |
*** Sravan has joined #openstack-kolla | 20:17 | |
*** Sravan has quit IRC | 20:22 | |
*** priteau has joined #openstack-kolla | 20:26 | |
mgoddard | hrw: ubuntu source ansible job failed :( | 20:39 |
mgoddard | hrw: http://logs.openstack.org/08/643308/4/check/kolla-ansible-ubuntu-source/eee856d/primary/logs/kolla/nova/nova-compute.txt.gz#_2019-03-14_19_11_31_709 | 20:39 |
mgoddard | I put something in the nova channel, let's see if they bite | 20:39 |
*** priteau has quit IRC | 20:43 | |
openstackgerrit | Mark Goddard proposed openstack/kolla master: Fix lower-constraints and kolla-build-ubuntu-binary jobs https://review.openstack.org/643308 | 20:43 |
*** hamzy has quit IRC | 20:45 | |
*** mjturek has quit IRC | 20:53 | |
*** wolverineav has quit IRC | 21:00 | |
*** wolverineav has joined #openstack-kolla | 21:01 | |
*** Sravan has joined #openstack-kolla | 21:10 | |
*** Sravan has quit IRC | 21:15 | |
*** itlinux has joined #openstack-kolla | 21:17 | |
*** Sravan has joined #openstack-kolla | 21:18 | |
*** andrein has quit IRC | 21:20 | |
hrw | mgoddard: let me joing nova channel just in case | 21:21 |
hrw | mgoddard: got any help? | 21:31 |
*** wolverineav has quit IRC | 21:36 | |
*** wolverineav has joined #openstack-kolla | 21:37 | |
hrw | mgoddard: https://review.openstack.org/643467 | 21:44 |
hrw | fun. looks like 3 patches for the same ;d | 21:48 |
*** iniazi has joined #openstack-kolla | 21:54 | |
*** unicell has joined #openstack-kolla | 22:09 | |
*** alistarle has joined #openstack-kolla | 22:11 | |
alistarle | Hi, have you seen that kolla-ansible is broken in python3 since 10 month from this commit to a wrong filemode openning ? https://github.com/openstack/kolla-ansible/commit/1db352f007e79f33d969361fa4997b60ef04e9a6 | 22:12 |
alistarle | I have written the fix, but unit test are kind of tricky in that part of the code :/ | 22:13 |
hrw | alistarle: send patch for review, add note about tests and let someone help? | 22:17 |
* hrw off | 22:17 | |
*** pcaruana has quit IRC | 22:21 | |
*** Sravan has quit IRC | 22:29 | |
*** wolverineav has quit IRC | 22:31 | |
*** dteselkin has quit IRC | 22:33 | |
openstackgerrit | Victor Coutellier proposed openstack/kolla-ansible master: Fix action_plugins python3 compatibility https://review.openstack.org/643479 | 22:35 |
alistarle | hrw: done, hope I did not break everything in the CI | 22:35 |
*** dteselkin has joined #openstack-kolla | 22:35 | |
*** absubram has quit IRC | 22:36 | |
alistarle | I will write the test tomorrow | 22:36 |
hrw | np | 22:36 |
hrw | alistarle: and thanks | 22:36 |
*** ivve has quit IRC | 22:36 | |
*** wolverineav has joined #openstack-kolla | 22:36 | |
*** tolisbar has joined #openstack-kolla | 22:38 | |
openstackgerrit | Victor Coutellier proposed openstack/kolla-ansible master: Fix action_plugins python3 compatibility https://review.openstack.org/643479 | 22:39 |
*** gmann_afk is now known as gmann | 22:40 | |
*** gfidente has quit IRC | 22:50 | |
*** wolverineav has quit IRC | 23:04 | |
*** wolverineav has joined #openstack-kolla | 23:08 | |
*** Sravan has joined #openstack-kolla | 23:11 | |
*** luksky has quit IRC | 23:11 | |
*** wolverineav has quit IRC | 23:11 | |
*** wolverineav has joined #openstack-kolla | 23:12 | |
*** Sravan has quit IRC | 23:14 | |
*** alistarle has quit IRC | 23:15 | |
*** Sravan has joined #openstack-kolla | 23:15 | |
*** Sravan has quit IRC | 23:25 | |
*** Sravan has joined #openstack-kolla | 23:31 | |
*** wolverineav has quit IRC | 23:41 | |
*** wolverineav has joined #openstack-kolla | 23:42 | |
*** Sravan has quit IRC | 23:42 | |
*** Sravan has joined #openstack-kolla | 23:43 | |
*** k_mouza has joined #openstack-kolla | 23:46 | |
*** wolverineav has quit IRC | 23:46 | |
*** wolverineav has joined #openstack-kolla | 23:49 | |
*** k_mouza has quit IRC | 23:51 | |
*** wolverineav has quit IRC | 23:53 | |
*** wolverineav has joined #openstack-kolla | 23:55 | |
*** Sravan has quit IRC | 23:56 | |
*** wolverineav has quit IRC | 23:56 | |
*** wolverineav has joined #openstack-kolla | 23:56 | |
*** wolverineav has quit IRC | 23:58 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!